Linux Blog

CORRECTION – Using BASH to sort a book collection. ISBN Data Mining – Part 1

Filed under: Shell Script Sundays — TheLinuxBlog.com at 8:00 am on Sunday, January 25, 2015

This may be cheating but I consider it a break from the download cleanup script.

Amazingly I got a comment out of the blue from an article I wrote in 2007 about ISBN Data Mining. The comment, stated the fact that the script didn’t work. I did a little investigating and was able to find out why. I figured it was just old and didn’t work but that was not the case. Apparently when I formatted my posts for code, a while back it appears that some of the formatting got a bit fubar.

Luckily for me and Gabe I was able to find an old copy:

Here is his script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#!/bin/bash
ISBN="$1"
 
function fetchInfo () {
  ### Using barnesandnoble.com to fetch info...
  lynx -source "http://search.barnesandnoble.com/booksearch/isbninquiry.asp?ISBN=${ISBN}" |\
   tr -d '[:cntrl:]' | sed 's/></>\n</g' | while read -a lineArray; do
 
  ### Parsing book title.
  if [ "${lineArray[0]}" == "<h1" ]; then
   echo "4|Title: ${lineArray[*]}" | sed 's/<[^>]*>//g;s/ ([^)]*)//g'
 
  ### Parsing book author.
 elif [ "$(echo ${lineArray[*]} | grep "id=\"contributor\"")" ]; then
  echo "3|Author(s): ${lineArray[*]}" | sed 's/by //;s/<[^>]*>//g'
 
  ### Parsing additional data.
  elif [ "${lineArray[0]}" == "<li" ] &&
     [ "$(echo ${lineArray[*]} | grep -ve "bullet" -ve "title")" ]; then
   echo "1|${lineArray[*]}" | sed 's/<[^>]*>//g;s/:/: /;s/  / /'
  fi
 
  done | sort -ur | awk -F\| '{print $2}' | grep ":"
 
}
 
if [ "${#ISBN}" -ge "10" ]; then
   fetchInfo
fi

The script should be saved to a file and called as ./isbn.sh . Amazingly after all of these years it still works, I guess that’s one of the beauties of shell scripting. Here is the output:

owen@linuxblog:~$ isbn.sh 1593275676
Title:  How Linux Works: What Every Superuser Should Know    by  Brian Ward

Here is ISBN Data Mining – Part 2 although, I cannot guarantee that it works after 8 years.

Auto Clean-up Downloaded Files – Part III

Filed under: Shell Script Sundays — TheLinuxBlog.com at 8:00 am on Sunday, January 18, 2015

In Part 2, we added some read prompts to read which directory to run the script in and used some bash if/then/else statements to do some basic input validation. This week by using the creating script parameters with getopts article we’ll enhance the script a little to remove the echo from the example to allow the user to delete the files if they choose, defaulting to not remove files.
(Read on …)

Auto Clean-up Downloaded Files – Part II

Filed under: Shell Script Sundays — TheLinuxBlog.com at 8:00 am on Sunday, January 11, 2015

Last week I showed a one liner that could be used to remove duplicate files from your downloads folder. Using previous Shell Script Sunday articles, over the next few weeks we’ll add some additional functionality to make it a little more functional.

On its own the snippet is not that useful. The script will only run in the current directory. Adding a prompt to ask the user what directory to run in, or defaulting to the current directory would be a nice addition. Using the Shell Script to Get User Input article you’ll see that adding some prompts with read is pretty easy. Next we’ll use some bash if/then/else statements to read over that input to check for blank input and a check to make sure that it is a valid directory, exiting if it is not.
(Read on …)

Auto Clean-up Downloaded Files

Filed under: Shell Script Sundays — TheLinuxBlog.com at 11:37 am on Sunday, January 4, 2015

This week I went through my downloads folder, cleaning up erroneous files. In light of that I’ll share a quick tip on how to clean up the multiple copies of files that inevitably pile up. The issue is, when you save a file from firefox or Chrome, the next time you download the file, it just makes another copy with (1) or (2). I have a number of these on multiple systems, so hit the jump for a quick snippet, and explanation.
(Read on …)

xrandr – Set Primary Monitor

Filed under: Linux Hardware,Linux Software,Shell Script Sundays — Owen at 11:04 pm on Sunday, October 27, 2013

I had an issue with my dual monitor setup where my primary monitor was my second, but only in X. Rearranging the monitors in Gnome preferences did nothing to solve the problem. While not exactly a shell script, here is a one-liner to change your primary monitor with xrandr.

#!/bin/bash
xrandr --output DVI-0 --primary

The above uses xrandr to set the primary to DVI-0. I put this in my ~/bin folder, chmod’d and set it to start when Gnome starts. Problem solved!

Cleaning Up Files & Directories

Filed under: Shell Script Sundays — TheLinuxBlog.com at 8:30 am on Sunday, August 19, 2012

While not really a shell script as such, this is shell related and  it just so happens to be Sunday.

Find Empty Directories:

find . -type d -empty > empty_folders.txt

Find Empty Files

find . -type f -empty > empty_files.txt

The issue with these two commands is that within my jumble of unorganized mess I have an abundance of version control repositories (mostly SVN) and tons of source code with empty, but needed directories. Fear not if you have the same problem, you can filter them out by using:

grep -v empty_files.txt svn

Duplicate File Removal

There are multiple ways to remove duplicate files. I used to think a  recursive md5sum script was the way to go but that was before I found the utility fdupes.
Install via your package manager if it’s available and use recursively like:

fdupes -r [directory] > duplicates.txt

With fdupes you also have the ability to symlink or delete files, although because of the amount of source code, I’d rather review it manually.


With these utilities I was able to merge copies of photos remove a bunch of old files and empty directories and am one step closer to being digitally organized. If you have the same problem, hopefully they will help you too. If you have any cleanup or organization tools, please comment and let me know.

tee time!

Filed under: Shell Script Sundays — TheLinuxBlog.com at 3:19 pm on Sunday, June 10, 2012

This weekend was South East Linux Fest, although I’m bummed I couldn’t make it I did learn something new from taking an LPIC level 1 self assessment. What I learned about was the application tee.

The question was something like:

“What command outputs to stdout and also writes to a file”

My answer was “echo” but that was the wrong answer. The correct answer was tee.

I’d seen it before in either Slackware or Debian’s packages list but forgot totally about it. It’s a pretty handy utility to have in your arsenal if you ask me. Usage would be something like this:

date | tee date.txt

this will echo the current date out, and also write it to date.txt. Of course, there are plenty of useful options which can be viewed in the man page which is linked below.

Enjoy! Happy Shell Scripting!

Spring Cleaning

Filed under: General Linux,Shell Script Sundays — TheLinuxBlog.com at 6:30 am on Sunday, April 1, 2012

It’s that time of year, tree’s, plants and animals doing there thing. The time of the year where other wildlife and beings start cleaning up since the weather is nice. I guess it’s time for a post on some spring cleaning for the Linux folk

We will start off with a classic, sure to give you a clean start:
rm -rf .
Best done from root, just remember to press CTRL+c and/or reboot as quickly as possible when you realize what you have done.

On a serious note, I had a bunch of annoying hidden files in backup directories I wanted to get rid of. This did the trick.

find . -iname ".*"
find . -iname ".*" | wc -l
find . -iname ".*" | while read i ; do echo rm "$i" >> possibly_remove; done;

The above is conservative. ‘chmod 755 possibly_remove’, verify there are no files in there you actually want, you are in the correct directory then ‘./possibly_remove’ and you’re golden. Mmm. Spring freshness.

Since I like to live on the wild side, I run it without creating a file of files to delete that can be executed like this:

find . -iname ".*" | while read i ; do rm "$i"; done;

This will also work to cleanup nasty files that may have been accumulating a while that may be have left behind. It can be used to find and delete all Thumbs.db files by doing this:

find . -iname "thumbs.db" | while read i ; do rm "$i"; done;

The above is pretty careless, in most cases it probably wouldn’t hurt.

That is all the spring cleaning I have done, except for some random fsck’ing’ that was long overdue.

Automated Scanning with the Shell – UPDATE

Filed under: Shell Script Sundays — TheLinuxBlog.com at 10:00 am on Sunday, December 18, 2011

I wrote a little script a while back that would help to automate scanning from the shell.  Mark posted some suggestions that I’ll be implementing in this post. (Read on …)

Virtual Box Clone Script

Filed under: Shell Script Sundays — TheLinuxBlog.com at 8:30 am on Sunday, February 27, 2011

I was in need for a way to clone virtual box vms so I wrote a quick bash script to clone them. After writing it, I realized that I could just export a VM, and then import it. There are limitations such as no snapshot support and probably a bunch I’ve never thought about. Either way, hit the jump to see the outcome.

(Read on …)

Bash Script Renaming Files

Filed under: Shell Script Sundays — TheLinuxBlog.com at 3:15 am on Sunday, February 6, 2011

I recently needed to rename a bunch of files. I didn’t want to do it manually because it would take too long and I don’t like copying+pasting that many times. This is where shell-scripting comes in handy. (Read on …)

Next Page »