Linux Blog

Halloween Pumpkin

Filed under: General Linux — TheLinuxBlog.com at 12:15 am on Friday, October 30, 2009

For Halloween this year every department at work was asked to carve or decorate a pumpkin. Since I’m in IT I thought I’d take the task of decorating our pumpkin with something IT related that would scare the crap out of everyone. Most of our end users are actually Windows users, so this pumpkin should be especially scary to them.

I introduce the B.S.O.Dkin, yes it’s a B.S.O.D pumpkin

B.S.O.D Pumpkin.

B.S.O.D Pumpkin.

I’d like to say it looks better in person, but it really doesn’t. I don’t really have an artsy knack for carving or painting pumpkins or I could have done something a lot better.

I guess my “Halloween Costume” is a blue Mexican wrestlers mask, blue t-shirt, and jeans to complete the B.S.O.D theme. With as much hate as I’ve been giving Microsoft recently I’m going to have to keep my fingers crossed for no kernel panics. Karma might bite me.

Parse ifconfig data with shell scripts

Filed under: Shell Script Sundays — TheLinuxBlog.com at 2:25 pm on Sunday, June 8, 2008

This week in TheLinuxBlogs.com’s Shell Script Sundays article I’m going to show you how you can use basic UNIX commands to parse networking data. As always there are a number of different methods of achieving this, and I am in no way saying that this is absolutely the way you must do it, or the best way. Its just an example of how you can use shell scripts to your advantage.

Firstly most know that Linux uses the ifconfig command to get information about networking interfaces. If you issue the ifconfig followed by the interface name you get information just about that interface as follows:

# /sbin/ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:0E:35:7F:E2:98 inet addr:192.168.2.13 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::20e:35ff:fe7f:e298/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1146 errors:0 dropped:39 overruns:0 frame:0 TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23748601 (22.6 MiB) TX bytes:507899 (495.9 KiB) Interrupt:11 Base address:0x4000 Memory:fceff000-fcefffff

This information is not in the best format to parse (it has also been distorted by my blogging software.) To solve this problem we are going to search for the whitespaces at the beginning of each line and replaces them with commas. By doing this:

# /sbin/ifconfig eth1 | sed 's/          /,/'
eth1 Link encap:Ethernet HWaddr 00:0E:35:7F:E2:98 ,inet addr:192.168.2.13 Bcast:192.168.2.255 Mask:255.255.255.0 ,inet6 addr: fe80::20e:35ff:fe7f:e298/64 Scope:Link ,UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ,RX packets:1344 errors:0 dropped:39 overruns:0 frame:0 ,TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 ,collisions:0 txqueuelen:1000 ,RX bytes:23809630 (22.7 MiB) TX bytes:507899 (495.9 KiB) ,Interrupt:11 Base address:0x4000 Memory:fceff000-fcefffff

That gives us a nice comma after every line. In order to grab fields from this line the tr command can be used to replace spaces with pipes.

#/sbin/ifconfig eth1 | sed ‘s/ /,/’ | tr [:space:] \|
eth1||||||Link|encap:Ethernet||HWaddr|00:0E:35:7F:E2:98|||,inet|addr:192.168.2.13||Bcast:192.168.2.255||Mask:255.255.255.0|,inet6|addr:|fe80::20e:35ff:fe7f:e298/64|Scope:Link|,UP|BROADCAST|RUNNING|MULTICAST||MTU:1500||Metric:1|,RX|packets:1765|errors:0|dropped:39|overruns:0|frame:0|,TX|packets:1|errors:0|dropped:0|overruns:0|carrier:0|,collisions:0|txqueuelen:1000||,RX|bytes:23941275|(22.8|MiB)||TX|bytes:507899|(495.9|KiB)|,Interrupt:11|Base|address:0x4000|Memory:fceff000-fcefffff|||

Now that the fields are all delimited properly, lets use the cut command to grab a line from this. Since I am interested in the RX and TX bytes I’m going to grab data from line 8 by using the cut command as follows:

#/sbin/ifconfig eth1 | sed 's/          /,/' | tr [:space:] \| | cut -d , -f 8
RX|bytes:24014818|(22.9|MiB)||TX|bytes:507899|(495.9|KiB)|

That gave us a nice line of output which is easy to parse even further by using the cut command. You will notice the fields are delimited by a pipe (the | character) and are not always consistent since we replaced all spaces with a pipe. Take a look at the first two fields RX|bytes: This means that to get the RX bytes in bytes we need to cut yet again. Since I’m not to bothered about Bytes and the largest number is delimeted in fields 3 and 4 I will concentrate on those.

#/sbin/ifconfig eth1 | sed 's/          /,/' | tr [:space:] \| | cut -d , -f 8 | cut -d \| -f 3-4
(23.0|MiB)

This is a nice RX MiB output yet it has one last problem, the pipe between the characters. Sed can be used to replace this and any other characters if you wish. Just issue a sed find and replace like this:

#/sbin/ifconfig eth1 | sed 's/          /,/' | tr [:space:] \| | cut -d , -f 8 | cut -d \| -f 3-4 | sed 's/|/ /'
(23.0 MiB)

That looks good for now. If you would like more information on how to parse data regarding this post or any other you can always leave me a comment and I’ll try my best to help. Especially if we can post the results on TheLinuxBlog in another Shell Script Sundays Article. Thanks for reading The Linux Blog and come back soon!

Linux ipconfig

Filed under: General Linux,Linux Software — TheLinuxBlog.com at 7:32 am on Thursday, December 13, 2007

I’m both proud and annoyed that there is no Linux ipconfig command:

owen@linux-blog:~$ ipconfig
-bash: ipconfig: command not found

I’m proud to be different but it annoys me because I like to see standards between operating systems. Luckily the Linux ipconfig utility is easy to remember. On most Linux distributions it is ifconfig. I’m not exactly sure why its called ifconfig but I remember it as the word “if” or “InterFace”. If you try to execute the command from your shell you might get this error:

owen@linux-blog:~$ ifconfig
-bash: ifconfig: command not found

If this happens don’t be alarmed the problem is probably that you are logged into your regular user account and not root. To run as root I would use sudo, or su to switch to the root user and try again. There is a trick to get ifconfig to work with a regular user but no changes to the configuration will be able to be made. The path to ifconfig and trick success rate varies by distribution, some high security distributions will not let you do this trick:

owen@linux-blog:~$ /sbin/ifconfig
eth0      Link encap:Ethernet  HWaddr 00:10:B5:70:B0:79
          inet addr:192.168.1.100  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1609257 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1640883 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:236511428 (225.5 Mb)  TX bytes:422972120 (403.3 Mb)
          Interrupt:11 Base address:0xee00
 
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:34516 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34516 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:13210707 (12.5 Mb)  TX bytes:13210707 (12.5 Mb)

For an IPConfig that is a lot of data. If you know the interface that you want the data for, you can just specify the name with the command like so:

owen@linux-blog:~$ /sbin/ifconfig eth1

There is a lot more to this command then first meets the eye. I’ll go into detail about networking and the Linux ipconfig utility in another blog post. Thanks for reading, show me some love by leaving me a comment!

Putting it all together

Filed under: Shell Script Sundays — TheLinuxBlog.com at 12:40 pm on Sunday, October 28, 2007

Since the start of this column I’ve shown many techniques that can be used in Shell Scripts. Today I will show an example of what can be done when it is all put together.

To start off, I recently made a photo tagging script for a friend of mine. The purpose of this script is to basically gather information about the pictures such as the month, year and who are in the photo. It also allows people to upload new pictures. The information is stored in a MySQL database and photos are kept in one folder as a unique hash of the file.

The script that I am going to show this week basically hashes the filenames and creates the SQL code that is needed to import the records into the database.

The full script source code can be found here.

The first thing the script does is use getopts to put the options into variables. For more information on how to do this can be found in my blog post on Creating Script Parameters With getopts.

### Get passed options.
while getopts ":s:d:f:" ARG; do
case "${ARG}" in
s) dirSource="${OPTARG}";;
d) dirName="${OPTARG}";;
f) sqlDump="${OPTARG}";;
esac
done

Once the options have been put into variables the following is ran:

The only thing that is needed for this script to run is a path. All that the above does is make sure that a path was specified. If not it calls the function usage and exits the script.
Here is the usage function:

usage () {
echo -e "\nUsage: $0 -s [ -d -f ]\n"
echo -e "\t-s: specifies the source directory. (required)"
echo -e "\t-d: specify the destination."
echo -e "\t-f: create an sql dump to import into MySQL.\n"
}

The main loop is very simular to my When Photoshop Fails article as it uses find. Except it finds all jpg files regardless of case. It does this by using [j,J][p,P][g,G] as the file mask.

find "${dirSource}" -name *.[j,J][p,P][g,G] | while read sourceFile; do

The following checks to make sure that dirName is not empty. The dirname command takes a file and will extract the directory name from it. So, if a directory to put the moved files in wasn’t given at run time, the directory that the new picture gets put in is the same as the old one.

### We don't want to overwrite this var by mistake.
if [ -z "${dirName}" ]; then
dirName=`dirname "${sourceFile}"`
fi

This is the last part of the loop. It uses md5sum to create a hash of the file is piped to awk which is used to print the actual hash. It then moves the file to the directory that was created from above. Create SQL and cleanVar are both functions that are called to do another task.

fileHash=`md5sum -b "${sourceFile}" | awk '{print $1}'`
mv "${sourceFile}" "${dirName}/${fileHash}.jpg"
createSQL; cleanVar

The createSQL function is used in this script to take the hash and insert it into the table that the PHP web app uses. At the moment the script just creates the SQL. The importing has to be done manually.

createSQL () {
if [ -n "${sqlDump}" ]; then
echo "INSERT INTO \`pictures\` (\`img_name\`) VALUES('${fileHash}');" >> "${sqlDump}"
fi
}

The cleaning function unsets fileHash which is the hash of the file that was just copied. It also does a check to see if the $dirName variable needs to be unset. It does this by checking to see if the dirName is the same as the directory name of the source file and unsets $dirName if it is.

### Clean up variables before next loop (precautionary).
cleanVar () {
unset fileHash
 
### We don't want to clean this var unless we have to.
if [ "${dirName}" == "$(dirname ${sourceFile})" ]; then
unset dirName
fi
}

Its a pretty simple script and uses a lot of what I’ve gone over in previous posts. Heres the script in action (the directory that the pictures are to be copied to must exist.)

owen@the-linux-blog:$ ls
JPG-hasmove.sh  NewPics  Pictures
owen@the-linux-blog:$ ls Pictures/
1.jpg  2.jpg  3.jpg  4.jpg  5.jpg
owen@the-linux-blog:$ ls NewPics/
owen@the-linux-blog:$ ./JPG-hasmove.sh -s Pictures/ -d NewPics -f SQLFile.sql
owen@the-linux-blog:$ ls
JPG-hasmove.sh  NewPics  Pictures  SQLFile.sql
owen@the-linux-blog:$ ls Pictures/
1.jpg  2.jpg  3.jpg  4.jpg  5.jpg
owen@the-linux-blog:$ ls NewPics/
294e8b2c5a9c4d61de67a166cd8e8e29.jpg  9b3270fc78fdf3920d0e44197da52944.jpg
6ef86ca6dcfb2467b46ed9d929fd4070.jpg  f353aa857522586d08ad7956090c8566.jpg
8aa35639bc0d3e7abc23c0b70acce08d.jpg
owen@the-linux-blog:$ cat SQLFile.sql
INSERT INTO `pictures` (`img_name`) VALUES('8aa35639bc0d3e7abc23c0b70acce08d');
INSERT INTO `pictures` (`img_name`) VALUES('f353aa857522586d08ad7956090c8566');
INSERT INTO `pictures` (`img_name`) VALUES('294e8b2c5a9c4d61de67a166cd8e8e29');
INSERT INTO `pictures` (`img_name`) VALUES('6ef86ca6dcfb2467b46ed9d929fd4070');
INSERT INTO `pictures` (`img_name`) VALUES('9b3270fc78fdf3920d0e44197da52944');

If you have a use for the script or this has helped you some please let me know, I always like to hear from people that find my posts useful. Until next time: Happy Shell Scripting!