Linux Blog

Remove lines that are in another file

Filed under: Shell Script Sundays — TheLinuxBlog.com at 6:52 pm on Sunday, February 7, 2010

Remove lines from a file that exist in another fileI had an issue this week where I needed to remove lines from one file if they existed in another file. Looking back it was frustrating as such a task should be simple.

I tried all sorts of things. Differencing the two files and using grep to grab the lines I wanted. Whatever I tried just did not produce the expected results. Thanks to a buddy I found the solution which ended up being to sort the two files before using diff.

Example:
Assuming two files exist, File_1 and File_2. File_1 containing lines with a, b, c and. File_2 containing b and d. If we want to remove b and d from File_1 because they exist in File_2 you could use something like the this:

owen@linuxblog:~$ cat File_1.txt
a
b
c
d
owen@linuxblog:~$ cat File_2.txt
b
d

owen@linuxblog:~$ diff File_1.txt File_2.txt | grep \< | cut -d \  -f 2
a
c

That’s all fine and dandy until File_2.txt contains the same lines in a different order. Running the same command produces different results. See Below:

owen@linuxblog:~$ cat File_2.txt
d
b

owen@linuxblog:~$ diff File_1.txt File_2.txt | grep \< | cut -d \  -f 2
a
b
c

The solution as noted above is to use sort before hand and then difference them:

owen@linuxblog:~$ sort File_1.txt >> File_1-sorted; sort File_2.txt >> File_2-sorted;
owen@linuxblog:~$ diff File_1-sorted File_2-sorted | grep \< | cut -d \  -f 2
a
c

Obviously the example has been simplified, when dealing with thousands of lines the sort could take a while. With that said I’m sure there are more efficient ways to achieve the same results. I wouldn’t doubt there being a command better suited to do this. Have at it in the comments.

Suspend Scripts for the Toshiba Tecra M2

Filed under: Quick Linux Tutorials,Shell Script Sundays — TheLinuxBlog.com at 12:15 am on Sunday, March 30, 2008

As you may know if you are a regular reader I own a Toshiba Tecra M2. One of the things that annoyed me was I had to turn the brightness up every time my computer came out of standby mode. A fix for this is to adjust the brightness every time the computer comes out of standby mode.

The script is intended to be run under cron. I have mine set up to suspend after 5 minutes of the lid being closed.

if [ $(cat /proc/acpi/button/lid/LID/state | sed 's/state:      //') == "closed" ]; then
VAR=$(cat /proc/acpi/toshiba/lcd | sed ‘s/brightness:              //’ grep -v levels);
sudo su -c “echo mem > /sys/power/state”;
if [ $VAR -eq 1 ]; then
ACTION=ADD;
elif [ $VAR -eq 7 ]; then
ACTION=SUB;
else
ACTION=ADD;
fi;
if [ $ACTION == "ADD" ]; then
VAR=$(($VAR + 1));
else
VAR=$(($VAR – 1));
fi;
sudo su -c “echo brightness:$(echo $VAR) > /proc/acpi/toshiba/lcd”;
fi;

I run this with the following cron entry:

*/5 * * * * sh hibernate.sh

The script first checks the current brightness. If the brightness is currently 1 or 7 it adjusts the mathematic operation so that when the laptop is opened the brightness is adjusted. Basically if the brightness is one, it adds one. If the brightness is 7 or any other value it subtracts one. This is currently working out quite well for me. I don’t know how useful this is to any body else, unless you happen to have a Toshiba that is doing the same thing but it should give you a good overall idea of how to perform basic mathematic operations in bash.

Checking your battery life from the shell

Filed under: General Linux,Linux Hardware,Quick Linux Tutorials — TheLinuxBlog.com at 8:46 am on Tuesday, December 11, 2007

I’ve often thought to my self “Wouldn’t knowing how much battery life I had be nice.”

Now when a window manager is open, this isn’t a problem. But when your just in a terminal it can be a bit of a problem. If you don’t get the gist of it heres an example, if I’m just in a terminal writing an article, or programming in VIM on battery life. Now I no longer have to execute the following command and guess how much life I have left:

cat /proc/acpi/battery/BAT1/state

I simply run the command:

yacpi

It tells me all kinds of information about my power usage, what the current temperature is, am I plugged in, what CPU governor I’m using. Its a really neat tool. I would recommend for everyone to install it. Worst case scenario it can be used when the system is undergoing maintenance, or if you left your box sitting in the other room and need to check the battery live via SSH. Check out the screenshot:

YACPI Screenshot

To install it it will vary by distribution. I’m sure Debian has it in its XXXX number of packages which means Ubuntu probably has it too. To install on other distributions such as Gentoo or Slackware you’ll need the source. You can acquire the source from here: http://freshmeat.net/redir/yacpi/55486/url_homepage/yacpi
You’ll also need libacpi which can be found here: http://freshmeat.net/projects/libacpi/?branch_id=70062
Make sure you compile libacpi first, or the make will fail for yacpi.

Thats it, once you download the package and install or download and compile the source you’re good to go. Have a good time checking the battery life from the shell. Try doing that in DOS!