Linux Blog

Reworking Shell Scripts

Filed under: Shell Script Sundays — TheLinuxBlog.com at 7:02 am on Sunday, August 24, 2008

To me shell Scripts are all about automation, their primary purpose is to get stuff done. If you have a one liner, snippet or other script you use on a regular basis, have you thought about how you could rework it for it to become more in handy?

Lets take last weeks snippet from this column. It was a simple one liner to reconnect to a host. Now, I knew when I posted this article that it was a helpful snippet of code. Now, how can this script be adapted to be a neat utility that we use on a regular basis? Over the next few week’s we’ll find out.

The first thing that I will note on is that this script or shell snippet is a pain to remember. Does a script save you time if you can’t remember how it works? Is it worth the hassle? Not exactly. So, in order to make this snippet a little better the first thing we are going to do is add something that it needs: parameters. Adding parameters to shell scripts is actually easy, much easier than adding parameters to some other languages that we wont mention. although this script does not use it getopts can be used. I’ve covered how to do this with getopts in other posts. Just do a site search (located at the bottom of the right bar for getopts.)

So, here is the modified script that automatically reconnects to a host by using ping and SSH:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#!/bin/bash
 
# Sleep Time Default: 15 seconds
STIME=15
 
# Set a default user up
USERNAME=LinuxBlog
 
#usage function
usage () {
echo -e "Usage: $0 host [user] [Sleep Time]"
}
 
# display usage if no host is specified
[ -z $1 ] && { usage && exit 1; }
 
# set the variables
[ $1 ] && { HOST=$1; }
[ $2 ] && { USERNAME=$2; }
[ $3 ] && { STIME=$3; }
 
# trying:
echo -e "host: $HOST \nuser: $USERNAME \ndelay: $STIME"
 
while ! ping -W 1 -c 1 $HOST 2>&1 >/dev/null; do true; done && echo "Successful waiting: $STIME secs"; sleep $STIME; ssh $USERNAME@$HOST

Now that you have that done, all you need to do is give the file a name (I called mine ssh_auto) and put it in a folder in your path. Use the filename and parameters defined in the script to connect to the host.

The next shell scripting article I demonstrate how you can further rework shell scripts to better suit your needs.

Easy script to upload to an ftp server

Filed under: Shell Script Sundays — Kaleb at 11:43 am on Sunday, February 10, 2008

This weeks Shell Script Sunday article is a guest post by Kaleb. Kaleb has been helping me out a by writing for me since I have a project for The Linux Blog that I’d like to complete and can’t dedicate as much time to writing right now. So if you get a chance visit his site and drop some comments.

Hello it’s Kaleb from http://kpstuff.servebeer.com again and today I have come to tell you about a little shell script that I wrote. It’s a script that allows for extremely easy and simple uploads to an ftp server.

The script is actually quite simple utilizing just standard bash scripting along with ncftp which is just an ftp client with some special features. So you need to install ncftp in order to use this script.

Gentoo:

emerge -av ncftp

Arch Linux:

pacman -Sy ncftp

Ubuntu:

apt-get install ncftp

After you install that you need to edit the upload.sh file to fit your needs. There are a few variables you need to change such as your username and password for the server, and the server itself. It is fairly simple to configure.

Set the user variable to your username

user="kaleb"

Set the passwd variable to your password

passwd="kalebspass"

Then you need to set the server variable to your server you want to connect to

server="kpstuff.servebeer.com"

Be sure to not precede the url with something like http:// or ftp://

Finally you should set the default directory on the server for your file to be placed into.

DIR="/home/$user"

Use $user for your username

Also if your system is odd you may need to make sure that /usr/bin/ncftpput exists. if it does not you will need to find out where on your system ncftp is and change the variable FTPCOM to suit your system.

Now all you need to do is run “chmod +x upload.sh” or “chmod 755 upload.sh” to make it executable and you are finished.

Usage for this script is quite simple. all you need to do is:

./upload.sh <FILE>

Making sure to replace FILE with the filename of the file you want to upload

And if you want to temporarily send the file to a different directory then you specified in the file.

./upload.sh <FILE> <DIR>

Remember to replace with the file you want to upload and replace

with the directory you want to upload to.

You can obtain the script at this address http://kpstuff.servebeer.com/~admin/scripts/upload.sh . And that is it thank you very much for your time and I hope that this script helps you out and if not at least gives you ideas.

Putting it all together

Filed under: Shell Script Sundays — TheLinuxBlog.com at 12:40 pm on Sunday, October 28, 2007

Since the start of this column I’ve shown many techniques that can be used in Shell Scripts. Today I will show an example of what can be done when it is all put together.

To start off, I recently made a photo tagging script for a friend of mine. The purpose of this script is to basically gather information about the pictures such as the month, year and who are in the photo. It also allows people to upload new pictures. The information is stored in a MySQL database and photos are kept in one folder as a unique hash of the file.

The script that I am going to show this week basically hashes the filenames and creates the SQL code that is needed to import the records into the database.

The full script source code can be found here.

The first thing the script does is use getopts to put the options into variables. For more information on how to do this can be found in my blog post on Creating Script Parameters With getopts.

### Get passed options.
while getopts ":s:d:f:" ARG; do
case "${ARG}" in
s) dirSource="${OPTARG}";;
d) dirName="${OPTARG}";;
f) sqlDump="${OPTARG}";;
esac
done

Once the options have been put into variables the following is ran:

The only thing that is needed for this script to run is a path. All that the above does is make sure that a path was specified. If not it calls the function usage and exits the script.
Here is the usage function:

usage () {
echo -e "\nUsage: $0 -s [ -d -f ]\n"
echo -e "\t-s: specifies the source directory. (required)"
echo -e "\t-d: specify the destination."
echo -e "\t-f: create an sql dump to import into MySQL.\n"
}

The main loop is very simular to my When Photoshop Fails article as it uses find. Except it finds all jpg files regardless of case. It does this by using [j,J][p,P][g,G] as the file mask.

find "${dirSource}" -name *.[j,J][p,P][g,G] | while read sourceFile; do

The following checks to make sure that dirName is not empty. The dirname command takes a file and will extract the directory name from it. So, if a directory to put the moved files in wasn’t given at run time, the directory that the new picture gets put in is the same as the old one.

### We don't want to overwrite this var by mistake.
if [ -z "${dirName}" ]; then
dirName=`dirname "${sourceFile}"`
fi

This is the last part of the loop. It uses md5sum to create a hash of the file is piped to awk which is used to print the actual hash. It then moves the file to the directory that was created from above. Create SQL and cleanVar are both functions that are called to do another task.

fileHash=`md5sum -b "${sourceFile}" | awk '{print $1}'`
mv "${sourceFile}" "${dirName}/${fileHash}.jpg"
createSQL; cleanVar

The createSQL function is used in this script to take the hash and insert it into the table that the PHP web app uses. At the moment the script just creates the SQL. The importing has to be done manually.

createSQL () {
if [ -n "${sqlDump}" ]; then
echo "INSERT INTO \`pictures\` (\`img_name\`) VALUES('${fileHash}');" >> "${sqlDump}"
fi
}

The cleaning function unsets fileHash which is the hash of the file that was just copied. It also does a check to see if the $dirName variable needs to be unset. It does this by checking to see if the dirName is the same as the directory name of the source file and unsets $dirName if it is.

### Clean up variables before next loop (precautionary).
cleanVar () {
unset fileHash
 
### We don't want to clean this var unless we have to.
if [ "${dirName}" == "$(dirname ${sourceFile})" ]; then
unset dirName
fi
}

Its a pretty simple script and uses a lot of what I’ve gone over in previous posts. Heres the script in action (the directory that the pictures are to be copied to must exist.)

owen@the-linux-blog:$ ls
JPG-hasmove.sh  NewPics  Pictures
owen@the-linux-blog:$ ls Pictures/
1.jpg  2.jpg  3.jpg  4.jpg  5.jpg
owen@the-linux-blog:$ ls NewPics/
owen@the-linux-blog:$ ./JPG-hasmove.sh -s Pictures/ -d NewPics -f SQLFile.sql
owen@the-linux-blog:$ ls
JPG-hasmove.sh  NewPics  Pictures  SQLFile.sql
owen@the-linux-blog:$ ls Pictures/
1.jpg  2.jpg  3.jpg  4.jpg  5.jpg
owen@the-linux-blog:$ ls NewPics/
294e8b2c5a9c4d61de67a166cd8e8e29.jpg  9b3270fc78fdf3920d0e44197da52944.jpg
6ef86ca6dcfb2467b46ed9d929fd4070.jpg  f353aa857522586d08ad7956090c8566.jpg
8aa35639bc0d3e7abc23c0b70acce08d.jpg
owen@the-linux-blog:$ cat SQLFile.sql
INSERT INTO `pictures` (`img_name`) VALUES('8aa35639bc0d3e7abc23c0b70acce08d');
INSERT INTO `pictures` (`img_name`) VALUES('f353aa857522586d08ad7956090c8566');
INSERT INTO `pictures` (`img_name`) VALUES('294e8b2c5a9c4d61de67a166cd8e8e29');
INSERT INTO `pictures` (`img_name`) VALUES('6ef86ca6dcfb2467b46ed9d929fd4070');
INSERT INTO `pictures` (`img_name`) VALUES('9b3270fc78fdf3920d0e44197da52944');

If you have a use for the script or this has helped you some please let me know, I always like to hear from people that find my posts useful. Until next time: Happy Shell Scripting!