The Bash

The Bash is probably the most important shell and scripting language in the world as it is included with almost any operating system (excl. MS Windows). The automation that can be achieved with very predictable results is great even though the complexity of the scripts is somehow limited. Doing calculations using the Bash is not so straight-forward for example.

Personalized Configuration Files – The Difference between .bashrc And bash_profile

Both files are configuration files for the bash. You can enter aliases, change the look of the prompt or add additional scripts that make work with the Bash easier. You can change bash_profile just the same way as .bashrc. The difference is that .bashrc is called at each start of an interactive shell (such as a terminal); bash_profile is run only at the run of a startup of a login shell. So if you want to something to be executed only once, put it in the bash_profile. If you want the change to happen every time you start a terminal or a console, bashrc is the place to go. For further information see http://wiki.ubuntuusers.de/Bash/bashrc (German) and http://en.wikipedia.org/wiki/Unix_shell#Configuration_files_for_shells. Also check man bash.

Shell Parameter Expansion

Shell parameter expansion is good to reduce usage of sed and grep in bash scripts! Information on this feature can be found in the section 'Shell Parameter Expansion' of the official manual or on:

Bash Standard Environment Variables

Source (German): http://www.linux-community.de/Internal/Artikel/Print-Artikel/LinuxUser/2007/05/Bequem-eingerichtet

Variable Meaning
CDPATH search path for the command cd
EDITOR standard text editor
HISTFILE history file, usually ~/.bash_history
HISTSIZE maximum number of commands in the history
HOME home directory of the user
HOSTNAME host name of the computer
LANG language for program output, dates etc. if none of the following LC_ variables is set. You can get a list of all defined language variables with the command locale.
LC_ALL country settings such as C or de; this disables LANG and all other LC_ variables.
LC_MESSAGES language for program and error messages
LC_TIME date format
LOGNAME login name of the user
MAIL path to the system mailbox of the user
MANPATH search path for manual pages
PATH search path for executables
PS1 look of the standard shell prompt; for non-privileged users, this is usually \u@\h:\w\$ and for root \h:\w\$
PWD print working directory - the path you are currently in
SHELL complete path of the currently used shell (/bin/bash)
TERM terminal setting such as xterm or vt100
TZ time zone such as CET, MET or UTC

Working on the Bash

Working on the Bash usually includes making use of standard unix tools such as cut, tee, cp, mv, rm, mkdir and many more. Together with piping (symbol |) they offer almost unlimited possibilities for the information flow and stream modification you may think of.

The Command cut

Imagine, you've got the following information:

part1:http://www.megaupload.com/?d=VTAUTDGT
part2:http://www.megaupload.com/?d=QP0W5P71
part3:http://www.megaupload.com/?d=XDQ8492K
part4:http://www.megaupload.com/?d=AZ26NQ6I
part5:http://www.megaupload.com/?d=8S8H6GA3

All you want are the links! so the "partX:" is anoying. So simply use the following command on the Bash to get only the URLs back:

cat << EOF | cut -d ":" -f 2 -f 3
part1:http://www.megaupload.com/?d=VTAUTDGT
part2:http://www.megaupload.com/?d=QP0W5P71
part3:http://www.megaupload.com/?d=XDQ8492K
part4:http://www.megaupload.com/?d=AZ26NQ6I
part5:http://www.megaupload.com/?d=8S8H6GA3
EOF

The output is:

http://www.megaupload.com/?d=VTAUTDGT
http://www.megaupload.com/?d=QP0W5P71
http://www.megaupload.com/?d=XDQ8492K
http://www.megaupload.com/?d=AZ26NQ6I
http://www.megaupload.com/?d=8S8H6GA3

Bash Scripts Collection

Bash guide for Beginners: good!
http://www.vias.org/linux-knowhow/wrapnt_bash_guide_for_beginners193.html

Functions for the Bash Shell

Here is an example: The function 'Super Kill' which takes one argument, the name scan the process table for. Killing processes with this function is extremely easy:

skill ()
{
    kill -9 `ps ax | grep $1 | grep -v grep | awk '{print $1}'`
}

For example, skill firefox will terminate all the programs with firefox in their name.

Filter out Comment Lines in a Configuration File

Configuration files often contain loads of lines that are comments for all the options possible + empty lines. To get a bare output of the configuration settings actually set, (non-empty lines not starting with a #), run:

cat ~/.gnupg/gpg.conf | grep -v "^#" | grep -v "^$"

Where ~/.gnupg/gpg.conf is an example for a configuration file in this case.

iterate over the lines of a large file

http://www.somebits.com/weblog/tech/bash-read-command.html

Instead of

for f in `cat list`; do
  ls -l "$f"
done

use

cat list | while read f; do
  ls -l "$f"
done

This is important for large files and for files containing lines with spaces!

Move a Selection of Files

unfortunately the move files command mv is not pipe compatible.... therefore you must take another way to move selected files: tar up the files in that directory to standard output and pipe standard output to a different directory where another copy of tar will untar them and then remove the orignals

An example: Transfer all *.png files from /f1 to /f2:

tar -cf - ./*.png | (cd /f2; tar -xvf -); rm /f1/*.png

Redirect stdout AND stderr to the same File

APPLICATION 2>&1 | tee -a LOG_FILE

awk

AWK is a programming language that is designed for processing text-based data. (To generate lists for evaluation.)

http://en.wikipedia.org/wiki/AWK | http://de.wikipedia.org/wiki/Awk

documentation and tutorials:

sed

A filter to apply textual transformations.

http://de.wikipedia.org/wiki/Sed_%28Unix%29

usage and examples

  • http://en.wikipedia.org/wiki/Sed#Usage
  • http://www.cs.utk.edu/~vose/c-stuff/sed_tutorial.html

Code for a terminal application that constantly overwrites its previous output – for progress bars etc.

http://mydebian.blogdns.org/?p=246

while :; do
    echo -n -e "\r$(date)"
    sleep 1
done

The trick consists of three parts:

  1. -n: to tell echo not to output a new line character "\n" at the end of the line.
  2. -e: we tell echo to translate any escaped sequence to the corresponding control character.
  3. \r: the return carriage control character; this way, we always print in the same line over and over.

A more elaborate example on how to print a progress bar with cp is shown down on http://mydebian.blogdns.org/?p=246.

Comments