Linux Blog

Graphical Shell Scripting

Filed under: Shell Script Sundays — TheLinuxBlog.com at 4:07 pm on Sunday, December 16, 2007

From all of my other previous shell scripting articles you can see that shell scripting is a very good way to get a task done or to automate. This is great but some times a little bit of that three letter acronym “GUI” is a nice touch. Well my friends, if you’ve been reading my articles and following my dialog examples then you are in luck. If not don’t worry, you can view all of the articles in the Shell Script section to the right.

The program to make your GUI’s for your shell scripts is the exact same thing as dialog except its graphical. Its called Xdialog. Once installed you basically use it the same way as dialog. If your thinking about upgrading a script thats written in dialog you might want to think twice because some window managers will display the windows differently.

Check out these Xdialog examples and corresponding screenshots:

Xdialog

Xdialog -yesno “Do you like Xdialog?” 5 50 && echo “Thats nice” || echo “Yea, some times its better to stick to CLI”

Xdialog ExampleXdialog Example

Fetching Online Data From Command Line

Filed under: Shell Script Sundays — TheLinuxBlog.com at 6:12 pm on Sunday, December 2, 2007

Shell Scripts can come in handy for processing or re-formatting data that is available from the web. There are lots of tools available to automate the fetching of pages instead of downloading each page individually.

The first two programs I’m demonstrating for fetching are links and lynx. They are both shell browsers, meaning that they need no graphical user interface to operate.

Curl is a program that is used to transfer data to or from a server. It supports many protocols, but for the purpose of this article I will only be showing the http protocol.

The last method (shown in other blog posts) is wget. wget also fetches files from many protocols. The difference between curl and wget is that curl by default dumps the data to stdout where wget by default writes the file to the remote filename.

Essentially the following do the exact same thing:

owen@linux-blog-:~$ lynx http://www.thelinuxblog.com -source > lynx-source.html
owen@linux-blog-:~$ links http://www.thelinuxblog.com -source > links-source.html
owen@linux-blog-:~$ curl http://www.thelinuxblog.com > curl.html

Apart from the shell browser interface links and lynx also have some differences that may not be visible to the end user.
Both lynx and links re-format the code received into a format that they understand better. The method of doing this is -dump. They both format it differently so which ever one is easier for you to parse I would recommend using. Take the following:

owen@linux-blog-:~$ lynx -dump http://www.thelinuxblog.com > lynx-dump.html
owen@linux-blog-:~$ links -dump http://www.thelinuxblog.com > links-dump.html
owen@linux-blog-:~$ md5sum links-dump.html
8685d0beeb68c3b25fba20ca4209645e links-dump.html
owen@linux-blog-:~$ md5sum lynx-dump.html
beb4f9042a236c6b773a1cd8027fe252 lynx-dump.html

The md5 indicates that the dumped HTML is different.

wget does the same thing (as curl, links -source and lynx -source) but will create the local file with the the remote filename like so:

owen@linux-blog-:~$ wget http://www.thelinuxblog.com
–17:51:21– http://www.thelinuxblog.com/
=> `index.html’
Resolving www.thelinuxblog.com… 72.9.151.51
Connecting to www.thelinuxblog.com|72.9.151.51|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html][ <=> ] 41,045 162.48K/s

17:51:22 (162.33 KB/s) – `index.html’ saved [41045]

owen@linux-blog-:~$ ls
index.html

Here is the result md5sum on all of the files in the directory:

owen@linux-blog-:~$ for i in $(ls); do md5sum $i; done;
a791a9baff48dfda6eb85e0e6200f80f curl.html
a791a9baff48dfda6eb85e0e6200f80f index.html
8685d0beeb68c3b25fba20ca4209645e links-dump.html
a791a9baff48dfda6eb85e0e6200f80f links-source.html
beb4f9042a236c6b773a1cd8027fe252 lynx-dump.html
a791a9baff48dfda6eb85e0e6200f80f lynx-source.html

Note: index.php is wget’s output.
Where ever the sum matches, the output is the same.

What do I like to use?
Although all of the methods (excluding dump) produce the same results I personally like to use curl because I am familiar with the syntax. It handles variables, cookies, encryption and compression extremely well. The user agent is easy to change. The last winning point for me is that it has a PHP extension which is nice to avoid using system calls to the other methods.