Linux Blog

tweeting from the command line

Filed under: Shell Script Sundays — TheLinuxBlog.com at 12:25 pm on Sunday, June 29, 2008

This is a subject that has been covered time and time again but I don’t think that it will hurt one more time. Twitter is a very popular “Microblogging” site where you can constantly change your status to let those who “follow” you know what you are doing. Since I just signed up for twitter for The Linux Blog I figured I’d write this post on how I update my twitter feed. While I’m at it I might as well invite you over to my feed URL: http://twitter.com/linuxblog

So here is the script:

#!/bin/bash
echo “Enter Tweet: “;
read inputline; TWEET=”$inputline”;
curl -u user:password -s -F status=”$inputline” http://twitter.com/statuses/update.xml http://twitter.com/account/end_session

This is a very basic twitter script, it does no error checking and probably doesn’t escape characters properly. None the less it works. The part that gets input from the shell is the following line:

read inputline; TWEET=”$inputline”;

If you’d like more information on how this works read this article: Shell Script to get user input

Curl is used to send the data to Twitter, to view curl tutorials and how-to’s visit the Curl Man Page which has a wealth of information at the bottom.

Until next time, happy tweeting!

Bash Aliases

Filed under: General Linux,Linux Software,Quick Linux Tutorials — TheLinuxBlog.com at 12:23 pm on Wednesday, June 25, 2008

Have you ever wanted to make a command for something that did not exist? Perhaps modify the functionality of a particular application to something more suitable? I know I have. For example, and I know that there is probably a better way to do this, but on certain Linux machines (such as servers), I like to clear the output before exiting. While I can type the command:

clear; exit

this still leaves me with a line at the top of the screen. Some distributions clear this automatically, for those that don’t an alias can be used.The basic principal for a bash alias is easy. You set an alias up and then use that alias instead of the command.
It appears that aliases have precedence over any already existing applications in the path so it becomes handy if you wish to override a command or perform a task before launching a built in command. I’m sure that this option can be changed if needed.

Now you know what aliases are here is how to use an alias to override the exit command in bash.

alias exit=”clear; exit > /dev/null 2> /dev/null”

Aliases with parameters can get tricky, the best workaround I have found is to write a shell script and put it in your local bin directory.

Parse ifconfig data with shell scripts

Filed under: Shell Script Sundays — TheLinuxBlog.com at 2:25 pm on Sunday, June 8, 2008

This week in TheLinuxBlogs.com’s Shell Script Sundays article I’m going to show you how you can use basic UNIX commands to parse networking data. As always there are a number of different methods of achieving this, and I am in no way saying that this is absolutely the way you must do it, or the best way. Its just an example of how you can use shell scripts to your advantage.

Firstly most know that Linux uses the ifconfig command to get information about networking interfaces. If you issue the ifconfig followed by the interface name you get information just about that interface as follows:

# /sbin/ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:0E:35:7F:E2:98 inet addr:192.168.2.13 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::20e:35ff:fe7f:e298/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1146 errors:0 dropped:39 overruns:0 frame:0 TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23748601 (22.6 MiB) TX bytes:507899 (495.9 KiB) Interrupt:11 Base address:0×4000 Memory:fceff000-fcefffff

This information is not in the best format to parse (it has also been distorted by my blogging software.) To solve this problem we are going to search for the whitespaces at the beginning of each line and replaces them with commas. By doing this:

# /sbin/ifconfig eth1 | sed ‘s/ /,/’
eth1 Link encap:Ethernet HWaddr 00:0E:35:7F:E2:98 ,inet addr:192.168.2.13 Bcast:192.168.2.255 Mask:255.255.255.0 ,inet6 addr: fe80::20e:35ff:fe7f:e298/64 Scope:Link ,UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ,RX packets:1344 errors:0 dropped:39 overruns:0 frame:0 ,TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 ,collisions:0 txqueuelen:1000 ,RX bytes:23809630 (22.7 MiB) TX bytes:507899 (495.9 KiB) ,Interrupt:11 Base address:0×4000 Memory:fceff000-fcefffff

That gives us a nice comma after every line. In order to grab fields from this line the tr command can be used to replace spaces with pipes.

#/sbin/ifconfig eth1 | sed ‘s/ /,/’ | tr [:space:] \|
eth1||||||Link|encap:Ethernet||HWaddr|00:0E:35:7F:E2:98|||,inet|addr:192.168.2.13||Bcast:192.168.2.255||Mask:255.255.255.0|,inet6|addr:|fe80::20e:35ff:fe7f:e298/64|Scope:Link|,UP|BROADCAST|RUNNING|MULTICAST||MTU:1500||Metric:1|,RX|packets:1765|errors:0|dropped:39|overruns:0|frame:0|,TX|packets:1|errors:0|dropped:0|overruns:0|carrier:0|,collisions:0|txqueuelen:1000||,RX|bytes:23941275|(22.8|MiB)||TX|bytes:507899|(495.9|KiB)|,Interrupt:11|Base|address:0×4000|Memory:fceff000-fcefffff|||

Now that the fields are all delimited properly, lets use the cut command to grab a line from this. Since I am interested in the RX and TX bytes I’m going to grab data from line 8 by using the cut command as follows:

#/sbin/ifconfig eth1 | sed ‘s/ /,/’ | tr [:space:] \| | cut -d , -f 8
RX|bytes:24014818|(22.9|MiB)||TX|bytes:507899|(495.9|KiB)|

That gave us a nice line of output which is easy to parse even further by using the cut command. You will notice the fields are delimited by a pipe (the | character) and are not always consistent since we replaced all spaces with a pipe. Take a look at the first two fields RX|bytes: This means that to get the RX bytes in bytes we need to cut yet again. Since I’m not to bothered about Bytes and the largest number is delimeted in fields 3 and 4 I will concentrate on those.

#/sbin/ifconfig eth1 | sed ‘s/ /,/’ | tr [:space:] \| | cut -d , -f 8 | cut -d \| -f 3-4
(23.0|MiB)

This is a nice RX MiB output yet it has one last problem, the pipe between the characters. Sed can be used to replace this and any other characters if you wish. Just issue a sed find and replace like this:

#/sbin/ifconfig eth1 | sed ‘s/ /,/’ | tr [:space:] \| | cut -d , -f 8 | cut -d \| -f 3-4 | sed ‘s/|/ /’
(23.0 MiB)

That looks good for now. If you would like more information on how to parse data regarding this post or any other you can always leave me a comment and I’ll try my best to help. Especially if we can post the results on TheLinuxBlog in another Shell Script Sundays Article. Thanks for reading The Linux Blog and come back soon!

Wakoopa For Linux

Filed under: General Linux,Linux Software — TheLinuxBlog.com at 12:01 am on Monday, March 31, 2008

I stumbled across Jakes blog post over at: http://blogs.howtogeek.com/jatecblog/posts/software-tracker-for-linux. Until this point I had never heard of the Wakoopa service. It seems like a really good idea. It is sort of the Alexa for software applications. Naturally I left a comment showing interest in an open source Wakoopa and shortly after received an e-mail from Jake.

Here it is:

Hello Owen, 

First I'd like to clarify that I don't actually have a need for the
application tracker... it would be purely for fun. That said, I would love if
you would be willing to create this. Here is the idea I have envisioned in
more detail but do not have the skills to create:

1) The process list is purged every so often to generate a log file.
2) The log file is periodically sent to a server. It is cleared after each
time it is uploaded.
3) The server then has an application which goes through and sorts out process
names and so forth and presents them as user reader data (much like Wakoopa) 

I think that this would be the easiest way, but I'd love to hear your
suggestions. If you were to make this I think it would be used and loved by many, as well as being useful.

Now that he has broken it down like that it seems like it would be pretty easy to implement. The only thing that I can see being a little bit complicated is determining what processes are running and how long they have been running for. I hopefully have a short shell script up for next Sundays column and have some sort of prototype. There should be nothing new in this script that I haven’t covered before on this blog, except possibly the sort command. Other commands I plan to know I will probably use are ps or top, cat and echo. There will probably be lots of loops and conditional if’s. The good thing about this idea is that if I write a shell script to do this some one will be able to translate it into another language. The real part where I would like to spend the majority of my time would be in the web interface. I expect that this will be written in PHP but I am unsure of the database technology that will be used since the recent happenings with MySQL.

So when this open source Wakoopa prototype is finished how many people do you think will use this service? Would you use it? What do you think an acceptable update time is? Any one have any other questions / input?

Using wc and How To Count Table Rows

Filed under: Shell Script Sundays — TheLinuxBlog.com at 1:07 pm on Sunday, March 9, 2008

I made this little script to check how many packages were available on the web from the Cygwin Package Repository located at http://www.cygwin.com/packages

Its a one liner but it does its job well.

CYGLIST=$(curl http://www.cygwin.com/packages/ | grep \<tr | grep ball | wc -l); echo $CYGLIST;

All the above is doing is creating a variable called CYGLIST that is the result of grabbing the cygwin.com/packages/ page, grepping all of the TR’s that also have the word “ball” in it (for the image) and then using the wc -l (L) command to count how many results are found. Then the list is echoed out.

wc is a very useful command for printing newline, word and byte counts. This is a good example of how to use wc to count lines in a shell script. wc can also be used to print all of these values in one line of a file.  The syntax is below:

bash-3.1# wc file.txt
9  20 184 file.txt

The above shows the number of lines in the file.txt, it shows how many words are in the file and also how many bytes. In my first example wc uses the -l switch to display the number of lines. This script can also be used with a little bit of bash math to calculate how many items are in an HTML list. I’m working on a script that automatically does this, when its finished I will be sure to post it here on The Linux Blog.

PHP Script To Log Into cPanel

Filed under: Shell Script Sundays — TheLinuxBlog.com at 3:47 am on Sunday, February 24, 2008

Earlier this week I made a script that logs into cPanel to check statistics. Basically if you have a webhost that runs cPanel and you wish to log into cPanel for some reason then this script is for you. Once you are logged in you can basically do anything you would want to do. For example my specific use was to log into my cPanel nightly and parse some data provided by AWStats. But with some modification this script could automate anything you can do by hand.

Since this is more of a web project for me I decided to write my cPanel login script in PHP. I found a PHP class to login here. curl is used to fetch the URL’s and I parse the data using PCRE regular expressions. The statistics code is still very basic but I thought I would post it for those interested and what better place then The Linux Blog’s Shell Script Sundays column?

Onto the script.

It consists of three scripts each with their own purpose in run time. They are as follows:

cPanel.php – This script does all of the dirty work in connecting to cPanel and fetching the pages. I modified this from the original a little
class.mysql.php – Just a generic data base handler. MySQL configuration information is stored in here.
login.php – This is the script that starts off the process. I named it login.php instead of index.php so that I do not have it run as the default page in my web browser. login.php also does all of the parsing of the data and is where the data gets inserted into the database.

To run the script edit login.php and then you can either put it in your PHP powered web server directory or run it from the command line by doing:

php login.php

The output should be as follows:

Num: 0 Date: 2454521 uniques: X visits: X visits per visitor: (Xvisits/visitor) pages: X pages per visitor: (XPages/Visit) hits: X hits per visitor: (XHits/Visit) bandwidth: X GB bandwidth per visitor: (XMB/Visit)<br/>

Feel free to modify this as you wish. If any questions can be answered I’d be happy to do so. I’d like to hear what people are using this for too, so drop a comment!

Download the PHP cPanel Login Script

Easy script to upload to an ftp server

Filed under: Shell Script Sundays — Kaleb at 11:43 am on Sunday, February 10, 2008

This weeks Shell Script Sunday article is a guest post by Kaleb. Kaleb has been helping me out a by writing for me since I have a project for The Linux Blog that I’d like to complete and can’t dedicate as much time to writing right now. So if you get a chance visit his site and drop some comments.

Hello it’s Kaleb from http://kpstuff.servebeer.com again and today I have come to tell you about a little shell script that I wrote. It’s a script that allows for extremely easy and simple uploads to an ftp server.

The script is actually quite simple utilizing just standard bash scripting along with ncftp which is just an ftp client with some special features. So you need to install ncftp in order to use this script.

Gentoo:

emerge -av ncftp

Arch Linux:

pacman -Sy ncftp

Ubuntu:

apt-get install ncftp

After you install that you need to edit the upload.sh file to fit your needs. There are a few variables you need to change such as your username and password for the server, and the server itself. It is fairly simple to configure.

Set the user variable to your username

user=”kaleb”

Set the passwd variable to your password

passwd=”kalebspass”

Then you need to set the server variable to your server you want to connect to

server=”kpstuff.servebeer.com”

Be sure to not precede the url with something like http:// or ftp://

Finally you should set the default directory on the server for your file to be placed into.

DIR=”/home/$user”

Use $user for your username

Also if your system is odd you may need to make sure that /usr/bin/ncftpput exists. if it does not you will need to find out where on your system ncftp is and change the variable FTPCOM to suit your system.

Now all you need to do is run “chmod +x upload.sh” or “chmod 755 upload.sh” to make it executable and you are finished.

Usage for this script is quite simple. all you need to do is:

./upload.sh <FILE>

Making sure to replace FILE with the filename of the file you want to upload

And if you want to temporarily send the file to a different directory then you specified in the file.

./upload.sh <FILE> <DIR>

Remember to replace <FILE> with the file you want to upload and replace <DIR> with the directory you want to upload to.

You can obtain the script at this address http://kpstuff.servebeer.com/~admin/scripts/upload.sh . And that is it thank you very much for your time and I hope that this script helps you out and if not at least gives you ideas.

Shell Script to get user input

Filed under: Shell Script Sundays — TheLinuxBlog.com at 2:06 am on Sunday, January 27, 2008

Creating a shell script to get input is rather easy. Shell scripts prompting input are generally more user friendly too. In this article I’ll show you how to read input from the bash shell. Take the example below:

#!/bin/bash

echo "Shell Script To Get User Input";

while read inputline
do
what="$inputline"
echo $what;

if [ -z "${what}" ];
then
exit
fi

done

All it does is echo’s the introduction “Shell Script To Get User Input” and then goes right to the bash read input loop. The next line makes a variable so that we can echo it out and also check if its empty with the if [ -z part. If the script is empty we exit, if not we loop around another time.

This is a very basic example but it can easily be modified so that you can use bash to grab user input. If you have any trouble with this script drop me a comment and I’ll be happy to help you out.

How to End or Exit a Shell Script

Filed under: Shell Script Sundays — TheLinuxBlog.com at 9:32 pm on Sunday, January 20, 2008

This week I apologize for the late entry to this column and for the crappy title. But this blog is as much for me to learn as it is for others so this week I’m going to post something that I couldn’t remember how to do correctly. Ending a shell script. Yes, laugh all you want folks, I forgot how to get out of a shell script loop without pressing CTRL+C. I tried the following in the if statement:

end
die
quit

The last one I tried was obviously the correct one which is exit. I’m hoping that this happens to every one and not just me. I seem to do it all the time, I just can’t remember the commands for certain things. There are a couple in PHP that I can never remember.

Clear browser history from command prompt

Filed under: General Linux,Quick Linux Tutorials — TheLinuxBlog.com at 3:36 pm on Thursday, January 17, 2008

I found this link a while back on clearing the history in IE7 from the command prompt

Clear IE7 browsing history from the command line

Although not specifically Linux related it is interesting and I thought users who sometimes use IE7 may be interested in this. It also provoked me to ask the question “How do I clear FireFox history from command line.”

Well, it is a bit simpler then clearing your history in IE7. The only thing you need to know is your profile name, and even then you don’t really have to because you can guess.

Firstly there are a couple of files that you should be aware of they are: cookies.txt, downloads.rdf, history.dat and one folder full of files: <profile>/Cache/

Now to clear your downloads list, your cookies list, your history or your cache you basically just remove the corresponding file. So, lets take a look at the command. We’ll assume my randomly generated profile name is called linux-blog

owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/cookies.txt
owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/Cache/*
owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/downloads.rdf
owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/history.dat

The above need to be ran individually. Make sure that FireFox is closed before the above is ran, or some pages may not load correctly. If any one is interested I’ll write a shell script to make this easier.

FUDCon 2008

Filed under: The Linux Blog News — TheLinuxBlog.com at 12:07 pm on Monday, January 14, 2008

Unfortunately I did not get to attend FUDCon 2008 in Raleigh this weekend. By the time I had traveled from Charlotte to Raleigh it was already half way finished. I also did not feel so good on Sunday morning after riding 15 miles and going out on Saturday night. I’ll be writing some new posts soon. Next week I’ll be sure to have a Shell Script Sundays article up.

« Previous PageNext Page »