Linux Blog

Searching for multiple strings with grep

Filed under: Shell Script Sundays — TheLinuxBlog.com at 9:36 pm on Sunday, August 23, 2009

Sometimes when using grep it’s nice to be able to search for more than one string in a file. It’s actually pretty easy to do, if you can remember the simple syntax. Basically, you pipe the terms together in double quotes and escape the pipe.

grep "gnome\|kde" install.log

The above example will search for gnome or KDE. It works for more more than one term, and also works with the invert match -v option, to exclude lines that include gnome or kde.

Todays article extra short, and late, but better late and short than none at all I guess.

Sequences with seq

Filed under: Quick Linux Tutorials,Shell Script Sundays — TheLinuxBlog.com at 9:28 pm on Sunday, July 19, 2009

I’m going to keep this one short since time I’ve had a serious lack of time recently. If you need to free up some time by putting repetitive commands into loops, you can use the command seq.

Basically, you can use any generic loop such as the while or for loop. Here are two examples:

for i in `seq 1 10`; do echo $i; done;
 
seq 1 10 | while read i; do echo $i; done;

These both count and echo from 1 to ten. Replace the echo $i; with whatever it is you want to do. You can use sleep, to sleep for a number of seconds, and any other logic you wish here.

A Manpage Template for your Scripts

Filed under: Shell Script Sundays — TheLinuxBlog.com at 11:44 am on Sunday, June 21, 2009

So, you just finished that killer script and the first thing you want to do is write that man page for it right? Not exactly? What do you mean?
Writing a man page isn’t that hard and will give your users an idea of how to use a script.

Here is a basic template for creating your own man pages:

.\" This is a comment
.\" Contact Owen@thelinuxblog.com
.TH man 1 "21 June 2009" ".1" "Man Page Example"
.SH NAME
Man \- Example Man Page
.SH SYNOPSIS
Example Man Page
.SH DESCRIPTION
This is an example of a man page.
.SH OPTIONS
No options for this man page
.SH SEE ALSO
man(1), groff(1), man.config(5)
.SH BUGS
No known bugs for this template, except you might want to replace the quotes if you copy from my blog.
.SH AUTHOR
Owen (owen@thelinuxblog.com)

When you’re done with the gruelling task of writing documentation (which your users won’t read or thank you for) just plop it the correct man section in /usr/share/man
That’s all there is to it!

Coppermine Photo gallery Upload Script

Filed under: Shell Script Sundays — TheLinuxBlog.com at 9:15 pm on Sunday, May 31, 2009

This week I bring you a script that I helped Kaleb (who has written posts here before) write. Well, I got him started with it, using curl and he rolled with it and finished it up. Here is the script:

#!/bin/bash
# Script to Upload to http://kporter.homelinux.com/~kaleb/cpg14x
# Written by Kaleb Porter May 23 2009
# with help of www.thelinuxblog.com
# email: porterboy55@gmail.com
# if you wish to use this code for something else please give me credit
 
IMAGE="$1"
URL="http://kporter.homelinux.com/~kaleb/cpg14x/upload.php"
DA=`date '+%d%b%y-%N'`
# If the user does not specify a file or url
if [ -z "$1" ]; then
echo "You did not give a file to upload"
echo "Takeing a screenshot in 3 Seconds..."
sleep 3
scrot $DA.png
IMAGE=$DA.png
fi
FI=`echo "$IMAGE" | grep '^[a-z]*://'`
AL=2
FIUP=`curl -s -F control=phase_1 -F blaa=continue -F file_upload_array[]=@$IMAGE $URL | grep unique_ID | awk -F\" '{print $6}'`
URLUP=`curl -s -F control=phase_1 -F blaa=continue -F URI_array[]=$IMAGE $URL | grep unique_ID | awk -F\" '{print $6}'`
 
#Get the title for the image from the user and change all the spaces to "%20"
echo "Enter a title for the image"
read TITLE1
TITLE=`echo $TITLE1 | sed 's/ /\%20/g'`
 
#Get the Description for the image from the user and change the spaces to "%20"
echo "Enter a discription"
read DES1
DES=`echo $DES1 | sed 's/ /\%20/g'`
 
#Get the keywords for the image from the user and change the spaces to "%20"
echo "Enter keywords (separated by spaces)"
read KEY1
KEY=`echo $KEY1 | sed 's/ /\%20/g'`
 
if [ -z "$FI" ]; then
 
UNIQUE_ID=$FIUP
#echo "Choose the NUMBER value for the album  you want"
#curl -s -F control=phase_2 -F unique_ID=$UNIQUE_ID $URL | awk '/name="album"/{disp=1} {if (disp==1) print} /<\/select>/{disp=0}' | grep 'value="[0-9]"' | sed 's/<option//' | sed 's/<\/option>//' | sed 's/>//'
#read AL
curl -o /dev/null -d "control=phase_2&unique_ID=$UNIQUE_ID&album=$AL&title=$TITLE&caption=$DES&keywords=$KEY&blaa=continue" $URL
exit 0
else
 
# If the image is from a URL
UNIQUE_ID=$URLUP
#echo "Choose the NUMBER value for the album  you want"
#curl -s -F control=phase_2 -F unique_ID=$UNIQUE_ID $URL | awk '/name="album"/{disp=1} {if (disp==1) print} /<\/select>/{disp=0}' | grep 'value="[0-9]"' | sed 's/<option//' | sed 's/<\/option>//' | sed 's/>//'
#read AL
curl -o /dev/null -d "control=phase_2&unique_ID=$UNIQUE_ID&album=$AL&title=$TITLE&caption=$DES&keywords=$KEY&blaa=continue" $URL
exit 0
fi

If there are any questions you can pretty much read the Shell Script Sundays column and figure out everything you need to know. Now that the upload script works, and tries to take a screenshot with scrot, the next step is a check to see if scrot exists, if it doesn’t a check for import, if not an error message.

It really does amaze me at the capabilities of the shell. Especially how mashable it is and how you can combine it with pretty much anything, this script is a great example of combining the power of the shell with the intrawebs. Well, I hoped you learned something, and as always if you have any questions, you know where the comment box is.

– Owen.

Last 50 Characters of Each line

Filed under: Shell Script Sundays — TheLinuxBlog.com at 4:18 pm on Sunday, May 24, 2009

I got a question from a user called Bastiaan. He had found my site while searching for ‘cut from end of line Linux’ and landed on the Using cut – shellscript string manipulation article. I haven’t received a lot of feedback on it, but am happy with the feedback I have and the amount of visits it gets. As I’ve said before if no one else reads The Linux Blog I still use it as a reference, so I am glad people are finding it useful. Anyways, Bastiaan’s problem was he works in a University and has a file with A LOT of DNA records in it. He needed to grab the last 50 characters of each line, regardless of the line length. After some correspondence we came up with a solution.

I have experience in doing this sort of thing in other languages such as PHP but not bash. Here is what I came up with for bash:

cat find.txt | while read i; do echo $i | \
cut -b $((`echo $i | wc -c` - 50))-; done;

While this was really quick to write it is not the most efficient way in the world. It has to read each line, echo it out, calculate the length of the line, subtract 50 from it. Again, does the job but not very gracefully.

Bastiaan then had told me he reversed the whole file and then was processing that with cut. I have heard of tac, to reverse entire files, but not had never heard of rev. Using rev I assumed that he was running something like the following:

rev file.txt > rev_file.txt
cat rev_file.txt | cut -c -50 | rev

That will get you the last 50 characters from each line (well, really the first 50 of a reversed file) That works pretty good so the final solution was to try to stream line it a little bit so that it could be done in one step.

rev file.txt | cut -c -50 | rev > out.txt

So there you have it, if you’re looking to use cut to “cut” characters from the end of the line, the above will cut 50 characters off of the end. Obviously you can remove the last “> out.txt” to get the output on the screen.

Hope this helps some one, and thanks to Bastiaan for the question!

Bulk Editing Text Files

Filed under: Shell Script Sundays — TheLinuxBlog.com at 1:00 am on Sunday, May 10, 2009

A Co-Worker wanted to edit a number of files in a directory that contained a lot of files. Each file that needed to be edited contained a function that needed to be replaced. Since it was production data we did not want to do a backup and run a sed find and replace for all files and risk screwing something up we decided to use vi to edit a list of files. Here is what I came up with to do that:

vi `grep function\_name * -n |cut -d : -f 1 | uniq`

If it were me, I would not have wanted to type sed find and replaces and would have done something like this because I’m lazy and I like to live on the edge:

 grep function\_name * -n | cut -d : -f 1 | uniq | while read i; do cp $i $i-bak; sed 's/function_name/new_function_name/g' $i-bak > $i; done;

Rather than editing them with vi it makes a -bak file, and uses sed to replace function_name with new_function_name. It does this from the bak file into the original. Some may think it’s kind of scary not making a backup, but I figure the -bak file should be enough depending on the operation. Make a backup if you value your data though.

Screen Script, Part Deux

Filed under: Shell Script Sundays — TheLinuxBlog.com at 6:09 pm on Sunday, May 3, 2009

So, the last script for handling the screen didn’t quite work out as I had liked so I have stopped using it. Basically if you try to SCP something, it would flip out because there was no terminal. Also, if you were trying to do anything in X after logging in with SSH you would have to detach your screen session first, which could get annoying. I’ve came up with this snippet, its rather ugly but is intended to be run after you log in if you want to start a screen session. I’d like a solution that logs me in with this sort of dialog selection but also allows X and scp transfers to work if a selection is not made. Perhaps it could be done on a timer. Anyhow, here is the script:

screen -r $(dialog --stdout --menu Test 20 50 10 `echo $(screen -ls | grep \( | sed -e 's/\t/,/g' | cut -d , -f 2- | awk -F ',' '{print $1,$2}' | awk -F '.' '{print $1,$3}' | cut -d-f 1,3)`)

It errors if no selection was made but that can be changed pretty easily by adding some if logic. Perhaps I’ll tweak it and repost, or perhaps I’ll find a better solution. Who knows.

Reattach Screen Script

Filed under: Shell Script Sundays — TheLinuxBlog.com at 2:02 pm on Sunday, April 12, 2009

A friend of mine who happens to be an avid screen user sent me this snippet below:

### Reattach to a screen if one exists ###
if [[ $TERM != 'screen' ]] ; then
if [[ `screen -list | grep -v "No" | awk '$2 { print }' | wc -l` == 0 ]] ; then
screen
else
screen -dr
fi
fi

What this handy snippet does is looks for a screen session, if it finds one it detaches the running screen, and reattaches it(-dr) if it isn’t lucky enough to find one, then it just starts a session up for you. Its rather handy to put in your .bashrc file to auto launch a screen session. The only thing I have modified for my use is replacing -dr for -x to enable me to reattach the screen without detaching the session I may have had open on another terminal. It works pretty well, although when you open a new “screen” CTRL-a + c, the tab doesn’t show up on the other sessions until you change to it, or cycle through them. It isn’t a big deal and could even be a local configuration issue. Anyway, enjoy this snippet and as always let me know if you found it useful.

Basic Scripting Snippets Repository

Filed under: Shell Script Sundays — TheLinuxBlog.com at 11:04 pm on Sunday, April 5, 2009

Everyone has their own favorite snippets of code that they use. With command line fu you can share your snippets and look at snippets that other people have posted. I’ve known about it for quite some time and frequently check it out, it’s really shaping up nicely.


Command-Line-Fu is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you’ve been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on and discussed – digg-esque voting is also encouraged so the best float to the top.

I don’t think that you should delete your snippets file, just in-case you don’t have the intrawebs but I think you could get something out of browsing through the highest rated if nothing else.

APC Access Temperature Query and Conversion. (2 of 2)

Filed under: Shell Script Sundays — TheLinuxBlog.com at 6:55 pm on Sunday, March 29, 2009

This second part of the script APC Access Temperature Query Script and its been a long time coming. Basically this script is the part that runs as a cron and will e-mail me if the temperature goes over a certain threshold. Once it returns to normal it e-mails me again. It has the option to send a text message to me via my SMS gateway, but it is commented out.

#!/bin/bash
 
temp=$(/home/linux/bin/temp f)
threshold=76
 
if [ "`echo \"$temp > $threshold\" | bc`" == 1 ]; then
echo $(date +%s) $temp >> /home/linux/thermal-over.log
echo "High Temp";
 
if [ "$(cat temp.txt)" == "norm" ]; then
echo "Sending E-Mail, High Temp";
echo "Current Temperature Is: $(/home/linux/bin/temp f)" | mail -s "Thermal Overload" owen@linuxblog                #echo "Current Temp Is: $(/home/linux/bin/temp f)" | mail -s "Thermal Overload" mynumber@cingularme.com
echo "high" > temp.txt
fi
elif [ "`echo \"$temp < $threshold\" | bc`" == 1 ]; then
echo "Low Temp";
 
if [ "$(cat temp.txt)" == "high" ]; then
echo "Temp Resumed, Sending E-Mail";
echo $(date +%s) Resumed at: $temp | mail -s "Thermal Normal" owen@linuxblog
echo "norm" > temp.txt
fi
 
fi
 
echo $(date +%s) $temp >> /home/linux/thermal.log

When I first wrote the script, I did not do any temperature checking. I found out that I needed to when I came back one morning with a bunch of emails that I needed to delete. Its pretty simple to figure out, temp.txt holds a value that is either norm or high. It gets switched when the temperature changes, this will in turn stop it from e-mailing me repeatedly. Once the temperature drops it flips it back. It will still e-mail if your temperature fluctuates between 75 and 77 which can be annoying, but you can adjust the threshold with the variable and set it to what you need. Thankfully our chiller has been fixed and I no longer have to worry about the temperature, but it still runs on a cron just in case.

APC Access Temperature Query and Conversion. (1 of 2)

Filed under: Shell Script Sundays — TheLinuxBlog.com at 10:54 pm on Sunday, February 1, 2009

This week I present to you; dearest reader part of a script I wrote to monitor the temperature on APC brand UPS’s. The script requires the apcupsd package to be installed and properly configured.

Here is the script:

#!/bin/bash
 
f () {
echo $(echo "($(/usr/sbin/apcaccess | grep ITEMP | cut -d : -f 2 | cut -d \  -f 2)*1.8)+32" | bc);
}
 
c () {
echo "$(/usr/sbin/apcaccess | grep ITEMP | cut -d : -f 2 | cut -d \  -f 2)"
}
 
case "$1" in
c)
c
;;
f)
f
;;
b)
echo "$(/usr/sbin/apcaccess | grep ITEMP | cut -d : -f 2 | cut -d \  -f 2)" C
echo $(echo "($(/usr/sbin/apcaccess | grep ITEMP | cut -d : -f 2 | cut -d \  -f 2)*1.8)+32" | bc) F;
;;
*)
echo $"Usage $0 {f|c|b}"
;;
esac

The script uses basic bash, grep, cut and bc. It requires only one input, and that is f, c or b. f for Fahrenheit, c for Celsius and b for both. The second part of the script is a cron, with some more basic bash. It write a log and e-mails me if the temperature goes over a certain threshold, and it e-mails me again once the temperature has been resumed. I will post the cron script next week as it is a major portion. The tricky part of the cron was making it e-mail me only once.

Until next time, Happy Scripting!

APCUPSD

« Previous PageNext Page »