Linux Blog

Bulk Editing Text Files

Filed under: Shell Script Sundays — TheLinuxBlog.com at 1:00 am on Sunday, May 10, 2009

A Co-Worker wanted to edit a number of files in a directory that contained a lot of files. Each file that needed to be edited contained a function that needed to be replaced. Since it was production data we did not want to do a backup and run a sed find and replace for all files and risk screwing something up we decided to use vi to edit a list of files. Here is what I came up with to do that:

vi `grep function\_name * -n |cut -d : -f 1 | uniq`

If it were me, I would not have wanted to type sed find and replaces and would have done something like this because I’m lazy and I like to live on the edge:

grep function\_name * -n | cut -d : -f 1 | uniq | while read i; do cp $i $i-bak; sed ‘s/function_name/new_function_name/g’ $i-bak > $i; done;

Rather than editing them with vi it makes a -bak file, and uses sed to replace function_name with new_function_name. It does this from the bak file into the original. Some may think it’s kind of scary not making a backup, but I figure the -bak file should be enough depending on the operation. Make a backup if you value your data though.

rsync to smb share

Filed under: General Linux — TheLinuxBlog.com at 4:30 pm on Thursday, September 11, 2008

If you happen to have a SMB share with a lot of disk space laying around, then you may have considered backing up to it. There is more than one methods that you could back up to a SMB share but this article will show how to rsync to a smb share. This blog post assumes that you have successfully set up your SMB share and have installed RSync.
(Read on …)

htaccess allow from

Filed under: General Linux — TheLinuxBlog.com at 10:32 am on Tuesday, September 9, 2008

htaccess allow from gives you the ability to allow (or deny) specific IP’s or domain names from a directory on your server. To do this the syntax is quite simple. Using VIM or nano open up the .htaccess file in the directory that you want to restrict access to. You need to add the following:

Order Deny,Allow
Deny from all
Allow from 127.0.0.1

This allows access from your local host and the IP address you specify. Using .htaccess you can also allow by host name. This is useful if you wish to allow or deny a friend access to a directory. (note: it will also work if you have them in your hosts file)

Order Deny,Allow
Deny from all
Allow from LinuxBlog
Allow from .thelinuxblog.com

Using htaccess to allow from your LAN is also pretty easy. You use your CIDR address (ip/subnet) to do this try something like this (changing to match your LAN):

Order Deny,Allow
Deny from all
Allow from 192.168.1.1/24

I run into htaccess allow problems a lot, and hope that this will clear the air up for me. htaccess can be very handy if you do not want to keep turning your firewall on and off, but do not want your directories wide open. Just remember, if you want to stop everyone except those you choose to access your apache web directories, use htaccess allow from!

VMWare: “Unable to build the vmnet module”

Filed under: General Linux,Linux Software — TheLinuxBlog.com at 10:49 am on Monday, July 21, 2008

If you run into the following problem:

VMware Server is installed, but it has not been (correctly) configured
for the running kernel. To (re-)configure it, invoke the
following command: /usr/local/bin/vmware-config.pl.

and then try to issue the vmware-config.pl command and get something similar to the following:

/tmp/vmware-config1/vmnet-only/bridge.c: In function ‘VNetBridgeUp’:
/tmp/vmware-config1/vmnet-only/bridge.c:949: error: implicit declaration of function ‘sock_valbool_flag’
make[2]: *** [/tmp/vmware-config1/vmnet-only/bridge.o] Error 1
make[1]: *** [_module_/tmp/vmware-config1/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.25.10-47.fc8-i686′
make: *** [vmnet.ko] Error 2
make: Leaving directory `/tmp/vmware-config1/vmnet-only’
Unable to build the vmnet module.

Then try to use the VMWare any patch from: http://groups.google.com/group/vmkernelnewbies/files
I had used the patch before to get my VMWare Server up and running but did not realize that you had to use the patch after kernel upgrade or your VMWare server will no longer work.

Who knew?

Slackware 12 – MPlayer Setup in 3 Easy Steps

Filed under: General Linux — TheLinuxBlog.com at 9:46 am on Thursday, September 6, 2007

I made a quick installer script that will install MPlayer 1.0rc1 with the essential codecs pack and the Firefox plugin on a new installation of Slackware 12. With this script you should be able to get MPlayer set up on Slackware 12 in 3 easy steps.

1. Download The MPlayer Installer that I made (Right click Save Link As).
2. Change to root user and run the MPlayer Installer OR run the MPlayer Installer with sudo as followed:

bash-3.1$ su root
Password:
bash-3.1# sh mplayer_setup.sh

OR

sudo sh mplayer_setup.sh

3. If everything installed smoothly you can remove the mplayer directory and all files inside it. If it failed then all you need to do is (as root) removepkg on all of the .tgz files and do a rm /usr/lib/codecs/*.

Firefox should be restarted after the script has run. Test Firefox by going to various media sites, one I like to use is Apple Trailers (Because of the quality and speed). If you could contact me and let me know if the installer worked or failed I would appreciate it.

Line by line breakdown of the installer:

mkdir mplayer; cd mplayer;

Creates the mplayer directory and moves into it.

wget http://www2.linuxpackages.net/packages/Slackware-12.0/X11/MPlayer/mplayer-1.0rc1-i486-1goa.tgz; installpkg mplayer-1.0rc1-i486-1goa.tgz

Gets the mplayer-1.0rc1-i486 package from linuxpackages.net then installs the package

wget http://www2.linuxpackages.net/packages/Slackware-12.0/Library/mplayerplug-in/mplayerplug-in-3.45-i686-1amg.tgz; installpkg mplayerplug-in-3.45-i686-1amg.tgz

Gets the mplayer plug-in for Firefox and installs it

wget http://www4.mplayerhq.hu/MPlayer/releases/codecs/essential-20061022.tar.bz2

Downloads the essential codecs pack from mplayerhq.hu

tar xvjf essential-20061022.tar.bz2; mv essential-20061022/* /usr/lib/codecs; rm -rf essential-20061022

extracts the essential codecs package to one of the directories MPlayer looks for codecs in and removes the directory.

Decision making in Bash If / Then / Else statements and Cases

Filed under: Shell Script Sundays — TheLinuxBlog.com at 3:44 pm on Sunday, August 5, 2007

There comes a time in every shell script where a decision has to be made.

To make a decision in bash the following if then else syntax can be used:

if [ condition ]
then

statements

[ elif [ condition ]

then

statements ]

[ else

statements ]

fi

Anything in non bold [ brackets ] is optional, and they should not be typed but the bold in them is required. Anything in bold must be typed. Statements are generally other commands or flow control statements.

To give an example of how to use bash if then else statements in the real world take the following scenario:

A system administrator has a strict habit of firing people that have too many .png files. He checks the systems regularly and makes sure that nobody has too many. The following script will display a message depending on the number of .png’s that are in the directory.

#!/bin/bash
gif_count=$(ls | grep .png | wc -l)
echo “Number of PNG’s: $gif_count”
if [ $gif_count -lt 10 ]
then
echo “He will be happy, you have less than 10 files”
elif [ $gif_count -gt 10 ] && [ $gif_count -lt 20 ]
then
echo “Consider deleting some files”
else
echo “you have too many files, DELETE SOME BEFORE HE FINDS OUT!”;
fi

Using Cases.

Cases are similar to if statements in that they match a expression and perform a task if the pattern is matched.

bash case syntax:

case expression in

pattern1 ) statements ;;

pattern2 ) statements ;;

esac

This is fairly simple and some people find this easier than doing if statements for simple logic. Take the following real world example:

The system administrator has recently gone on a bigger power trip than before. Since people got wise about using png’s and started saving images in other file formats he is now monitoring png’s gif’s and jpg’s. To combat the problem, you can use a case to count how many files you have of each type. (This is intended as an example, there are many ways to accomplish this task, this is just to demonstrate how cases work)

#!/bin/bash
#set all variables to 0
png=0
gif=0
jpg=0

# start loop
for wc in $(ls); do

case “$wc” in
*png ) let png=$png+1 ;;
*gif ) let gif=$gif+1 ;;
*jpg ) let jpg=$jpg+1 ;;
esac

# end loop
done

echo “Png’s $png”;
echo “gif’s $gif”;
echo “jpg’s $jpg”;

There you have it, two ways to make basic decisions in bash. Just figure out what you want to do then use an if then else, or a case statement to do the logic. I myself prefer if statements over cases as they make more sense to me and I find it easier to perform logic within ifs.

Programs used in this post
ls, echo, grep, wc