Linux Blog

Shell Script Flow Control – my most refered to articles

Filed under: Shell Script Sundays — at 2:38 pm on Sunday, November 9, 2008

This week I do not have time to write a full fledged article on shell scripting, so I am going to some flow control and logic articles that I wrote that I read the most.

One script that I continuously refer to is one of the first shell scripting articles I wrote titled “When Photoshop Fails.” The reason I refer to this article is because it describes a couple of looping techniques and a loop that works with spaces in filenames. For those interested but do not want to read the whole thing here is the loop in short:

find * -iname “*” | while read i; do echo “$i”; done

The next article I’m linking to is the one I wrote on loops: it outlines some of the basic looping techniques. What I fail to mention in this article is that seq can be used to generate sequences of numbers for the [in list] section. The following is how you would create a loop to loop from 1 to 10 echoing out each number:

for i in `seq 1 10`; do echo $i; done;

Select Statements in Bash is exactly what the title implies. How to implement select statements in bash scripting. Also known as switch’s and case’s in other languages many will be familiar with this sort of logic. Even though I do not refer to it as often as some of the other articles I think it is worth of a mention in this list.

Decision making using if statements is something that every shell scripter is bound to come across. While not exactly the most in depth article on bash if then else statements it is a good start for any one wanting to learn more.

I love dialog’s and it just so happens that dialog and xdialog do a great job of making dialogs for your shell scripts. Creating Dialogs with Dialog has some basic uses of dialog. While it is not exactly logic or flow control, it can be used to prompt the user for input or just to pretty things up a little. It just so happens that it has an X front end called xdialog which does a good job too.

This is not all of the articles I refer to, but its the ones I wrote. If any one else has a list of articles they refer to that are bookmarked that I wrote or not I’d be interested to see them. Just paste them in a comment.

Putting it all together

Filed under: Shell Script Sundays — at 12:40 pm on Sunday, October 28, 2007

Since the start of this column I’ve shown many techniques that can be used in Shell Scripts. Today I will show an example of what can be done when it is all put together.

To start off, I recently made a photo tagging script for a friend of mine. The purpose of this script is to basically gather information about the pictures such as the month, year and who are in the photo. It also allows people to upload new pictures. The information is stored in a MySQL database and photos are kept in one folder as a unique hash of the file.

The script that I am going to show this week basically hashes the filenames and creates the SQL code that is needed to import the records into the database.

The full script source code can be found here.

The first thing the script does is use getopts to put the options into variables. For more information on how to do this can be found in my blog post on Creating Script Parameters With getopts.

### Get passed options.
while getopts “:s:d:f:” ARG; do
case “${ARG}” in
s) dirSource=”${OPTARG}”;;
d) dirName=”${OPTARG}”;;
f) sqlDump=”${OPTARG}”;;

Once the options have been put into variables the following is ran:

The only thing that is needed for this script to run is a path. All that the above does is make sure that a path was specified. If not it calls the function usage and exits the script.
Here is the usage function:

usage () {
echo -e “\nUsage: $0 -s [ -d -f ]\n”
echo -e “\t-s: specifies the source directory. (required)”
echo -e “\t-d: specify the destination.”
echo -e “\t-f: create an sql dump to import into MySQL.\n”

The main loop is very simular to my When Photoshop Fails article as it uses find. Except it finds all jpg files regardless of case. It does this by using [j,J][p,P][g,G] as the file mask.

find “${dirSource}” -name *.[j,J][p,P][g,G] | while read sourceFile; do

The following checks to make sure that dirName is not empty. The dirname command takes a file and will extract the directory name from it. So, if a directory to put the moved files in wasn’t given at run time, the directory that the new picture gets put in is the same as the old one.

### We don’t want to overwrite this var by mistake.
if [ -z "${dirName}" ]; then
dirName=`dirname “${sourceFile}”`

This is the last part of the loop. It uses md5sum to create a hash of the file is piped to awk which is used to print the actual hash. It then moves the file to the directory that was created from above. Create SQL and cleanVar are both functions that are called to do another task.

fileHash=`md5sum -b “${sourceFile}” | awk ‘{print $1}’`
mv “${sourceFile}” “${dirName}/${fileHash}.jpg”
createSQL; cleanVar

The createSQL function is used in this script to take the hash and insert it into the table that the PHP web app uses. At the moment the script just creates the SQL. The importing has to be done manually.

createSQL () {
if [ -n "${sqlDump}" ]; then
echo “INSERT INTO \`pictures\` (\`img_name\`) VALUES(‘${fileHash}’);” >> “${sqlDump}”

The cleaning function unsets fileHash which is the hash of the file that was just copied. It also does a check to see if the $dirName variable needs to be unset. It does this by checking to see if the dirName is the same as the directory name of the source file and unsets $dirName if it is.

### Clean up variables before next loop (precautionary).
cleanVar () {
unset fileHash

### We don’t want to clean this var unless we have to.
if [ "${dirName}" == "$(dirname ${sourceFile})" ]; then
unset dirName

Its a pretty simple script and uses a lot of what I’ve gone over in previous posts. Heres the script in action (the directory that the pictures are to be copied to must exist.)

owen@the-linux-blog:$ ls NewPics Pictures
owen@the-linux-blog:$ ls Pictures/
1.jpg 2.jpg 3.jpg 4.jpg 5.jpg
owen@the-linux-blog:$ ls NewPics/
owen@the-linux-blog:$ ./ -s Pictures/ -d NewPics -f SQLFile.sql
owen@the-linux-blog:$ ls NewPics Pictures SQLFile.sql
owen@the-linux-blog:$ ls Pictures/
1.jpg 2.jpg 3.jpg 4.jpg 5.jpg
owen@the-linux-blog:$ ls NewPics/
294e8b2c5a9c4d61de67a166cd8e8e29.jpg 9b3270fc78fdf3920d0e44197da52944.jpg
6ef86ca6dcfb2467b46ed9d929fd4070.jpg f353aa857522586d08ad7956090c8566.jpg
owen@the-linux-blog:$ cat SQLFile.sql
INSERT INTO `pictures` (`img_name`) VALUES(’8aa35639bc0d3e7abc23c0b70acce08d’);
INSERT INTO `pictures` (`img_name`) VALUES(‘f353aa857522586d08ad7956090c8566′);
INSERT INTO `pictures` (`img_name`) VALUES(’294e8b2c5a9c4d61de67a166cd8e8e29′);
INSERT INTO `pictures` (`img_name`) VALUES(’6ef86ca6dcfb2467b46ed9d929fd4070′);
INSERT INTO `pictures` (`img_name`) VALUES(’9b3270fc78fdf3920d0e44197da52944′);

If you have a use for the script or this has helped you some please let me know, I always like to hear from people that find my posts useful. Until next time: Happy Shell Scripting!

When Photoshop Fails

Filed under: General Linux,Shell Script Sundays — at 9:50 pm on Monday, January 15, 2007

I was recently assigned the task of reducing the quality of over 1000 images. My first instinct was to use photoshop’s batch functions to complete the task. After playing around with it for a little while it became apparent that you cannot save files for web with spaces in. Photoshop had failed me. My next idea was to use a bash script to loop over every file and process the image. This was sure to work. Since I have prior experience with image processing I decided to use ImageMagick to complete the task. The command to adjust the quality is:

convert (FILENAME) -quality 50 (FILENAME)

This command is useless on its own so using a for loop I came up with:

for i in $(ls); do
convert $i -quality 50 $i

This script would have worked for me if there were no spaces in the filenames. Because there were it took each filename that had spaces and ran the command on each part of the file name. Unfortunately this was not going to work.

After googling for a while it became apparent that I was not the only person to have the problem of spaces in filenames with for loops. The solution I found was to use the find command and a while loop. Below is working script that successfully completed the task in no time:

find * -iname “*” | \
while read I; do
convert “$I” -quality 50 “$I”

This script could be easily modified to take advantage of ImageMagick’s many other functions. For example it could be used to batch resize a folder of images to make thumbnails by changing the command to

convert “$I” -resize 200 “$I”

Or it could be used to overlay text onto an image with the following command:

convert “$I” -gravity southeast -annotate +5+10 “” -fill black “$I”

As shown above ImageMagick is really powerful image editing software which can easily be used with the bash shell to process thousands of images with no trouble.