Linux Blog

Linux Blog new year roundup

Filed under: The Linux Blog News — TheLinuxBlog.com at 10:16 pm on Monday, December 31, 2007

I’d just like to say I’ve been busy over the past week or two with some projects at work and with the holiday season so I haven’t had the time to dedicated to this blog.

Its been a good year at the Linux Blog, I’ve increased traffic since I’ve started and have been beginning to see more activity with comments. I have a couple of goals for the upcoming year.

Firstly I’d like to make some money with Google ads. This isn’t specifically related to the Linux Blog but every little counts. I want to do this because I think that this site has some value to those that use it and I would like to be rewarded for my hard work.

Another goal I have is to write more posts than I did this year. It shouldn’t be hard to do since I stopped writing for a long time in 2007. From about Jan to July. I have a long list of posts to write but I need to know what topics people want to read about.

I have a list of articles to write for the Shell Scripting Sundays column but haven’t got enough ideas for the year yet. I may be able to get a few more if I split some of the more intense ones up but I’d rather keep them simple. If any one has any ideas for articles please let me know.

Thats going to be all for this year, I hope you enjoyed reading my blog and continue to visit me in the new year. For those that do

Happy New Year!

Graphical Shell Scripting

Filed under: Shell Script Sundays — TheLinuxBlog.com at 4:07 pm on Sunday, December 16, 2007

From all of my other previous shell scripting articles you can see that shell scripting is a very good way to get a task done or to automate. This is great but some times a little bit of that three letter acronym “GUI” is a nice touch. Well my friends, if you’ve been reading my articles and following my dialog examples then you are in luck. If not don’t worry, you can view all of the articles in the Shell Script section to the right.

The program to make your GUI’s for your shell scripts is the exact same thing as dialog except its graphical. Its called Xdialog. Once installed you basically use it the same way as dialog. If your thinking about upgrading a script thats written in dialog you might want to think twice because some window managers will display the windows differently.

Check out these Xdialog examples and corresponding screenshots:

Xdialog
 
Xdialog -yesno "Do you like Xdialog?" 5 50 && echo "Thats nice" || echo "Yea, some times its better to stick to CLI"

Xdialog ExampleXdialog Example

Comments Are Back!

Filed under: The Linux Blog News — TheLinuxBlog.com at 4:03 am on Saturday, December 8, 2007

Ok, I’ve decided to add comments back to The Linux Blog. The idea behind commends is that people post comments for help, advice, questions, comments on the article or just to be nice. Before this was not happening so I turned them off. Now, I’ve re-added them back hoping that people will actually comment.

The spam problem has been fixed and we should not see any spammy comments since posters now have to be approved.

I’ll leave them on for a while and see how it does. In other news I’ve been writing like mad, trying to get some good articles written.

On the list of stuff to write are a couple of Shell Scripting articles, one about IP Soft Phones for Linux, Battery Life & Optimization, virtualization. I also have some tutorials that I would really like to write to help people out with WordPress and other web applications that run on open source software such as MediaWiki.

If you have any questions, or would like to request something, now you can actually just comment , so go ahead, leave a comment

Making your scripts user and sysadmin friendly

Filed under: Shell Script Sundays — TheLinuxBlog.com at 11:08 pm on Sunday, November 18, 2007

When designing a shell script it is important to make them easy to use but also to make it easily automated for deployment. One example of this that comes to mind is the NVIDIA installer. It has command line options to allow for deployment but also gives a nice interface for the end user.

To implement this “dialog” can be used for the user interface and “getopts” can be used for the command line options. The script may look something like:

#help function
help () {
echo "Linux Blog - getopts and dialog example";
echo "Usage:";
echo -e "\t -h shows this help";
echo -e "\t -a [y/n][other] ANSWER (Yes, No or Other)";
}
 
#show dialog to get the answers
showDialog () {
dialog --yesno "Do you want to enter y?" 5 50 && \
ANS="Yes was entered using dialog" ||\
ANS="No was entered using dialog"
showAnswer;
}
 
#actually show the answer
showAnswer() {
echo $ANS;
}
 
#check answer for command line processing
checkAns() {
if [ "${OPT1}" == "y" ]
then
ANS="Yes sent by getopts";
elif [ "${OPT1}" == "n" ]
then
ANS="No was sent getopts";
else
ANS="This: $OPT1 was sent by getopts";
fi
#call showAnswer
showAnswer;
}
 
#get the options
while getopts "a:h" ARG;
do case "${ARG}" in
a) OPT1="${OPTARG}";;
h) HELP="TRUE";;
esac;
done
 
#see if help was entered
if [ "${HELP}" ]
then
#display help and quit
help;
exit;
fi
#if the options are empty
if [ -z "${OPT1}" ]
then
showDialog;
else
checkAns;
fi

Keep this getopts and dialog post in mind next time your shell scripting. It will take a little extra time to implement but the result will be a user and sysadmin friendly script.

Creating Dialogs with Dialog

Filed under: Shell Script Sundays — TheLinuxBlog.com at 5:48 pm on Sunday, October 21, 2007

Have you ever seen those pretty dialogs used in Shell Scripts such as the Slackware installation, the slackpkg program or even the NVIDIA driver installer? Well, my friends to display dialog boxes from shell scripts is very easy with… you guessed it – Dialog.

First of all, there are many different types of dialogs that you can create they are as follows: calendar, checklist, fselect, gauge, infobox, inputbox, menu, msgbox (message), password, radiolist, tailbox, tailboxbg, textbox, timebox, and yesno (yes/no).

This blog post is intended to be a primer on using dialog. More examples will be posted in future blog posts in the Shell Script Sunday’s column.

The simplest form of a dialog in a shell script is probably the msgbox. All this really does is displays text. To display text in a dialog you would do the following:

owen@the-linux-blog:$ dialog --msgbox "Hello from the Linux Blog!" 5 50

The numbers after the text in quotes are the widths and heights of the box. The minimum height that I like to use is 5. The width doesn’t really matter as long as it is big enough. It is good to keep the box sizes standard across a whole script because it gets annoying with constantly resizing boxes.
If the text in a message box is too long it will auto wrap around and give you a type of scroll bar. As follows:

owen@the-linux-blog:$ dialog --msgbox "Hello from The Linux Blog. This text is so long it wraps it to a New Line" 5 50

Dialogs can be canceled. Clicking Ok or pressing enter/return returns “true” and pressing escape or Ctrl+C returns a false.
The simple shell scripting syntax shown in Shell Scripting 101

is used for this:

owen@the-linux-blog:$ dialog --msgbox "Dialog Exit Example" 5 50 && echo "ok" || echo "false"

Another simple dialog example is the Yes/No box. The syntax for this is exactly the same as the msgbox example except instead of using –msgbox, –yesno is used. The difference between a msgbox and a yesno box is that there a two buttons. It is pretty obvious as to what they are labeled, but for those in the back, I’ve included an example and some screen shots anyway.

owen@the-linux-blog:$ dialog --yesno "Are you learning anything from this blog" 5 50 && echo "Yes, thanks Owen." || echo "No, Write some better Linux Related Posts"

The Linux Blog - Dialog Example - Yes / No

Thats about all I have time for this week. Check back next week!

Some Linux Blog News

Filed under: The Linux Blog News — TheLinuxBlog.com at 5:25 pm on Wednesday, October 10, 2007

Firstly I would like to welcome a new writer to The Linux Blog his name is Andrew. Those of you who paid attention will have noticed that he wrote last Sundays shell scripting article. This was a huge favor for me since I was at the beach and did not have any internet access. His biography is in the works but in the mean time I would like everyone to give Andrew a nice warm Linux Blog welcome.

And now for another apology.

I’m sorry that I haven’t posted much here recently, I have been very busy. I do however have lots of half written articles and many more articles to write. If there is a particular topic that any one would like me to blog about, send me an email to owen -at- thelinuxblog.com.

- Owen

Using BASH to sort a book collection. ISBN Data Mining – Part 1

Filed under: General Linux,Shell Script Sundays — TheLinuxBlog.com at 2:47 am on Sunday, September 16, 2007

Many problems can be solved with a little bit of shell scripting.
This week I plan to show you a script that does a little data mining from Barnes and Noble.
I have a lot of books and wanted cataloged information on them. Each book has a unique identifier called an ISBN. So I collected all of my ISBN numbers and a simple loop that wrapped around a script a friend of mine made to find basic information.
Here is his script:

#!/bin/bash
ISBN="$1"
 
function fetchInfo () {
### Using barnesandnoble.com to fetch info...
lynx -source "http://search.barnesandnoble.com/booksearch/isbninquiry.asp?ISBN=${ISBN}" |\
tr -d '[:cntrl:]' | sed 's/>\n
 
### Parsing book title.
if [ "${lineArray[0]}" == "
echo "4|Title: ${lineArray[*]}" | sed 's/<[^>]*>//g;s/ ([^)]*)//g'### Parsing book author.
elif [ "$(echo ${lineArray[*]} | grep "id=\"contributor\"")" ]; then
echo "3|Author(s): ${lineArray[*]}" | sed 's/by //;s/>[^>]*>//g'
 
### Parsing additional data.
elif [ "${lineArray[0]}" == "
[ "$(echo ${lineArray[*]} | grep -ve "bullet" -ve "title")" ]; then
echo "1|${lineArray[*]}" | sed 's/<[^>]*>//g;s/:/: /;s/  / /'
fidone | sort -ur | awk -F\| '{print $2}' | grep ":"
 
}
 
if [ "${#ISBN}" -ge "10" ]; then
fetchInfo
fi

The script should be called as followed (assuming the script name is eBook.sh):

sh eBook.sh ISBNNUMBER

The first step is to see if the ISBN is greater than 10 characters, if it is it goes to the fetchInfo() function.
It then takes the given ISBN number and searches the barnsandnoble.com site for any matches. To do this lynx is used, the -source option tells lynx to output the source instead of using browsing mode. The output of lynx is piped to tr and sed. tr is used to delete all line breaks from the source, the sed expression adds a line break at the end of each HTML tag. The while loop loops over each line from from the piped lynx, tr and sed.
Within the loop is where anything from the output of the search page can be pulled out. This script pulls out the book title, the author and additional data.

I formatted my ISBN’s in a text list and used the following loop to fetch information on my books and save them with the ISBN as the file name.

for i in $(cat list.txt); do sh eBook.sh $i > $i.txt; done;

In the next issue I plan to expand on this to format data in an even more presentable manor.
Used applications
tr, lynx, sed, awk, sort, grep

Bringing The Internet Up After Failure

Filed under: Shell Script Sundays — TheLinuxBlog.com at 9:58 pm on Sunday, September 9, 2007

This Shell Script Sunday is a short one but don’t let that fool you to the power of the shell. This script I wrote earlier in the week due to power spikes at the office. All of our equipment would stay powered on due to UPS’s but unfortunately something with the ISP was not staying on. Once the brownout occurred our router box would still have an IP and seem to be working but it wouldn’t. We had our suspicions about what piece of equipment it was but had no power to fix it. I would renew the IP from the ISP bring the public interface down by using eth0 down and then eth0 up but this was not successful. To fix it from the router I had to actually reset the network. This worked, but we have some services running at the office that I like to access from home. So to fix the problem I wrote a one liner to reset the network if the connection goes down.

ping -c 1 OurISP.com 2> /dev/null > /dev/null && echo > /dev/null || sudo /etc/rc.d/network restart

The techniques in this script are covered in Shell Scripting 101. All this does is ping OurISP.com one time and output the error & standard output to /dev/null. If the ping was successful it does nothing and if the ping failed then it restarts the network. To get it to repeat at an interval I just set it up as a cron job. This did the trick and I now do not have to worry about brownouts.

Shell Scripting 101

Filed under: Shell Script Sundays — TheLinuxBlog.com at 2:27 am on Sunday, August 5, 2007

In my last shell scripting article I made a script to manipulate images with imagemagick. The script was only five lines long and has saved me endless hours of manual work. The script can be adapted to do anything repetitive on any types of files.

Since this is my first official shell scripting post I am going to introduce some common basic shell scripting. This is intended for people that want to learn the basics of shell scripting, so those that have experience already may want to skip this post.

Basic applications used in this tutorial
In order to teach a little about shell scripting, I need to use a couple of programs other than bash its self. The ones that I will use in this post are:
echo
This program displays a line of text
cat
Cat likes to read files and print what it finds out.
grep
Search through files with an expression to find a line that matches the pattern. Sounds complicated but really is not.
less
A paging utility is used to display data. Its kind of like opening a file in notepad Windows except with most utilities more can be done.
less is probably my favorite pager. There are many of these including tail and more. Less is more is the tag line of this pager because it does more than less but is the same thing.

Basic standard output redirecting
Often abbreviated as stdout or stderr for standard out and standard error.
This is normally used to take the output of one application and direct it somewhere. You can overwrite or append to the stdout. The symbol to do this is > to overwrite and >> to append.
stdin example:

echo "This is the new text" > File_To_Be_Written
echo "This is appended" >> File_To_Be_Written

“This is the new text” (without quotes) is redirected to File_To_Be_Written and overwrites the file.

“This is appended” is appended to the end of the file that was just created. If the file did not exist before hand, the file would have been created.

Redirecting Errors
Redirecting errors is achieved the same way as redirecting stdout except it has a 2 in front of the greater than symbol.
stderror example:

echo "This is the new text" 2> File_To_Be_Written
echo "This is appended" 2>> File_To_Be_Written

To append to stderror just think “To redirect to stderror is to redirect with two” Meaning, to redirect stderror is just to redirect with two in front. Cheesy, but its easy to remember.

Pipes
Piping is a common technique I normally use in all of my scripts. The way this works is to send the stdout of one program to the stdin of another.
Example:

cat Big_File | grep search | less

This basically reads the Big_File and then grep’s the file for lines that have search in them. It then pipes that output into the paging utility less.

Lists
A list is just a sequence of pipes that are separated by control operators. Basically
what that means is you have operators that do specific things depending on return codes

&
This operator puts the command into the background, gives a return status of 1 and moves on to the next task.
Example:

owen@amd:~$ echo "tails" > heads
owen@amd:~$ cat heads & echo "done"
tails
[1] 15385
done
[1]   Done                    cat heads

;
The ; operator waits until the process is finished before moving onto the next task.
Example:

owen@amd:~$ cat heads; echo "done"
tails
done

&&
command1 && command2
command2 will only be executed if command1 returns an exit status of zero
Example:

owen@amd:~$ cat heads && echo "done"
tails
done

Note this works because cat has returned the correct exit status

owen@amd:~$ cat eads && echo "done"
cat: eads: No such file or directory

Notice that this does not work because cat did not return the correct exit status.

||
command1 || command2
command2 will be executed if command1 returns a non zero return code.
Example:

owen@amd:~$ cat heads || echo "Did not work"
tails
Did not work is not printed because cat exited properly
owen@amd:~$ cat eads || echo "Did not work"
cat: eads: No such file or directory
Did not work

Did not work was printed because the eads file does not exist.

Throwing it all together:

echo "This is whats in the output file" > output;
echo "Now that has done, lets add something to the end" >> output;
cat output > normal_output && echo "This is not the error log. Press Q to exit this pager and view the error log" >> normal_output;
cat put >> normal_output 2> error_output;
cat normal_output | less ; cat error_output  | less

This short example basically echos some text to the output file. It then adds some more to the end of the output file. Next it reads from the output file into the normal_output file. If that executes successfully it goes on to echo some instructions to the normal_output file. The script moves on to read the put file (which does not exist) and directs the error of that command to a error_output file. After that, it reads the normal output and sends it to the less pager. The first prompt has instructions for ending the pager, and once the pager has been stopped it reads the error log.

Variables
Variables are used to temporarily store data in memory. To read a variable you can just just the echo command. An example of setting and reading a bash variable is below:

owen@amd:~$ variable='This is my variable'
owen@amd:~$ echo $variable
This is my variable

Flow Control
There are ways of controlling how a script is going to work. This is the logic part of the script, the intelligence if you will. There are many different ways to control flow with bash. The following topics should get you started:
Bash If / Then / Else statements & Cases
Bash Loops For, While and Until
Creating menus in Bash with select

Where to from here?
If you followed this example and understand everything so far then you have got your feet wet in shell scripting. You should now know a little on how to read/write files, redirect output and use conditional operators.
If you would like more help there are plenty of resources on-line and probably on the computer you are using to try the examples. Check the man pages for more help on the utility and “man bash” for bash help. If your looking for more tutorials go to Google and type “bash shell scripting tutorial”.
If you have a specific question that needs answering leave a comment or contact me.

Recent changes to The Linux Blog.

Filed under: The Linux Blog News — TheLinuxBlog.com at 10:39 pm on Sunday, July 29, 2007

This article is mostly just updates on the site.

Server Changes
I have recently moved the site to a new server. This was a fairly simple task which took longer than it should because of DNS issues.

Now that it is on the new Linux based web host all should be dandy. The fact that I have more flexibility over the old host is an added bonus.

Bugs

Some problems that surfaced after I moved the site have just been resolved in a matter of minutes ago.

The biggest problem the site had was that the detailed/archive WordPress URL’s were not getting processed correctly. I was aware of the problem last night but was too tired to fix it. The fix was simple. I had to place the .htaccess file in the directory. The problem was that when I used lftp to mirror the old code, it didn’t download the hidden .htaccess file. I should have checked this before I updated the DNS but I guess something will always go wrong and I’m glad it was a simple fix. When I figure out how to mirror a directory in lftp including hidden files I will be sure to let everyone know. After scanning the help for the mirror command it didn’t jump out at me, but maybe thats just because I’m tired.
The URL’s that were affected by the bug were:

http://www.thelinuxblog.com/2007/07/29/shell-script-sundays/

http://www.thelinuxblog.com/2007/07/28/phones-meet-linux/

New Category Created
Not only did I fix this little problem, I also created the Shell Script Sundays section and moved the related posts to that category. The When Photoshop Fails article was posted on a Monday but I did most of the writing on Sunday, so I believe that since its mostly a shell scripting article it is worthy of this category.

After writing a paragraph in the description section of WordPress, I realized that it doesn’t actually show up anywhere on the site. I’m unsure if it shows up in the RSS feeds or not but I’m going to post it here anyway for the web browsers.

Shell Script Sundays Description
This section of the site is dedicated to Linux shell scripts. Twice a month I will post a nifty shell script that will perform a certain task. Most of the scripts will be written in Bash or the Korn shell and occasionally a CLI PHP or Perl script may surface. Some scripts will be more advanced than others and some will require additional software to be installed. This section will show how powerful scripting can be and I hope it educates people on how to shell script with Linux.

More to come!

Expect a Linux related post within the next few days. I have hundreds of topics to choose from, but I am always willing to take peoples suggestions on what to blog about. So if there is a particular topic that interests you, just contact us and we’ll do our best to cater to your needs.

Ciao,

- Owen.

« Previous Page