Linux Blog

Shell Scripting 101

Filed under: Shell Script Sundays — at 2:27 am on Sunday, August 5, 2007

In my last shell scripting article I made a script to manipulate images with imagemagick. The script was only five lines long and has saved me endless hours of manual work. The script can be adapted to do anything repetitive on any types of files.

Since this is my first official shell scripting post I am going to introduce some common basic shell scripting. This is intended for people that want to learn the basics of shell scripting, so those that have experience already may want to skip this post.

Basic applications used in this tutorial
In order to teach a little about shell scripting, I need to use a couple of programs other than bash its self. The ones that I will use in this post are:
This program displays a line of text
Cat likes to read files and print what it finds out.
Search through files with an expression to find a line that matches the pattern. Sounds complicated but really is not.
A paging utility is used to display data. Its kind of like opening a file in notepad Windows except with most utilities more can be done.
less is probably my favorite pager. There are many of these including tail and more. Less is more is the tag line of this pager because it does more than less but is the same thing.

Basic standard output redirecting
Often abbreviated as stdout or stderr for standard out and standard error.
This is normally used to take the output of one application and direct it somewhere. You can overwrite or append to the stdout. The symbol to do this is > to overwrite and >> to append.
stdin example:

echo "This is the new text" > File_To_Be_Written
echo "This is appended" >> File_To_Be_Written

“This is the new text” (without quotes) is redirected to File_To_Be_Written and overwrites the file.

“This is appended” is appended to the end of the file that was just created. If the file did not exist before hand, the file would have been created.

Redirecting Errors
Redirecting errors is achieved the same way as redirecting stdout except it has a 2 in front of the greater than symbol.
stderror example:

echo "This is the new text" 2> File_To_Be_Written
echo "This is appended" 2>> File_To_Be_Written

To append to stderror just think “To redirect to stderror is to redirect with two” Meaning, to redirect stderror is just to redirect with two in front. Cheesy, but its easy to remember.

Piping is a common technique I normally use in all of my scripts. The way this works is to send the stdout of one program to the stdin of another.

cat Big_File | grep search | less

This basically reads the Big_File and then grep’s the file for lines that have search in them. It then pipes that output into the paging utility less.

A list is just a sequence of pipes that are separated by control operators. Basically
what that means is you have operators that do specific things depending on return codes

This operator puts the command into the background, gives a return status of 1 and moves on to the next task.

owen@amd:~$ echo "tails" > heads
owen@amd:~$ cat heads & echo "done"
[1] 15385
[1]   Done                    cat heads

The ; operator waits until the process is finished before moving onto the next task.

owen@amd:~$ cat heads; echo "done"

command1 && command2
command2 will only be executed if command1 returns an exit status of zero

owen@amd:~$ cat heads && echo "done"

Note this works because cat has returned the correct exit status

owen@amd:~$ cat eads && echo "done"
cat: eads: No such file or directory

Notice that this does not work because cat did not return the correct exit status.

command1 || command2
command2 will be executed if command1 returns a non zero return code.

owen@amd:~$ cat heads || echo "Did not work"
Did not work is not printed because cat exited properly
owen@amd:~$ cat eads || echo "Did not work"
cat: eads: No such file or directory
Did not work

Did not work was printed because the eads file does not exist.

Throwing it all together:

echo "This is whats in the output file" > output;
echo "Now that has done, lets add something to the end" >> output;
cat output > normal_output && echo "This is not the error log. Press Q to exit this pager and view the error log" >> normal_output;
cat put >> normal_output 2> error_output;
cat normal_output | less ; cat error_output  | less

This short example basically echos some text to the output file. It then adds some more to the end of the output file. Next it reads from the output file into the normal_output file. If that executes successfully it goes on to echo some instructions to the normal_output file. The script moves on to read the put file (which does not exist) and directs the error of that command to a error_output file. After that, it reads the normal output and sends it to the less pager. The first prompt has instructions for ending the pager, and once the pager has been stopped it reads the error log.

Variables are used to temporarily store data in memory. To read a variable you can just just the echo command. An example of setting and reading a bash variable is below:

owen@amd:~$ variable='This is my variable'
owen@amd:~$ echo $variable
This is my variable

Flow Control
There are ways of controlling how a script is going to work. This is the logic part of the script, the intelligence if you will. There are many different ways to control flow with bash. The following topics should get you started:
Bash If / Then / Else statements & Cases
Bash Loops For, While and Until
Creating menus in Bash with select

Where to from here?
If you followed this example and understand everything so far then you have got your feet wet in shell scripting. You should now know a little on how to read/write files, redirect output and use conditional operators.
If you would like more help there are plenty of resources on-line and probably on the computer you are using to try the examples. Check the man pages for more help on the utility and “man bash” for bash help. If your looking for more tutorials go to Google and type “bash shell scripting tutorial”.
If you have a specific question that needs answering leave a comment or contact me.