Linux Blog

Last 50 Characters of Each line

Filed under: Shell Script Sundays — TheLinuxBlog.com at 4:18 pm on Sunday, May 24, 2009

I got a question from a user called Bastiaan. He had found my site while searching for ‘cut from end of line Linux’ and landed on the Using cut – shellscript string manipulation article. I haven’t received a lot of feedback on it, but am happy with the feedback I have and the amount of visits it gets. As I’ve said before if no one else reads The Linux Blog I still use it as a reference, so I am glad people are finding it useful. Anyways, Bastiaan’s problem was he works in a University and has a file with A LOT of DNA records in it. He needed to grab the last 50 characters of each line, regardless of the line length. After some correspondence we came up with a solution.

I have experience in doing this sort of thing in other languages such as PHP but not bash. Here is what I came up with for bash:

cat find.txt | while read i; do echo $i | \
cut -b $((`echo $i | wc -c` - 50))-; done;

While this was really quick to write it is not the most efficient way in the world. It has to read each line, echo it out, calculate the length of the line, subtract 50 from it. Again, does the job but not very gracefully.

Bastiaan then had told me he reversed the whole file and then was processing that with cut. I have heard of tac, to reverse entire files, but not had never heard of rev. Using rev I assumed that he was running something like the following:

rev file.txt > rev_file.txt
cat rev_file.txt | cut -c -50 | rev

That will get you the last 50 characters from each line (well, really the first 50 of a reversed file) That works pretty good so the final solution was to try to stream line it a little bit so that it could be done in one step.

rev file.txt | cut -c -50 | rev > out.txt

So there you have it, if you’re looking to use cut to “cut” characters from the end of the line, the above will cut 50 characters off of the end. Obviously you can remove the last “> out.txt” to get the output on the screen.

Hope this helps some one, and thanks to Bastiaan for the question!

Does a room without Windows have doors?

Filed under: General Linux — TheLinuxBlog.com at 5:29 pm on Friday, May 15, 2009

I was e-mailed an interesting project from the folks that run AyeTea (Pronounced IT) that I thought people might be interested in. Here is the description they sent:

A Room Without Windows is a new project set to launch on the 1st of June 2009.  Scheduled to last for 31 days, the basis of the project is to take long term Windows users and deprive them of their familiar software.  Our IT lab rats will have one month to find open source software that will replace the function of their Windows based machines.

This is an interesting concept and one that I think will succeed. It will either open or close doors by letting people try out Linux and those that like it will stick with it, and those that don’t will go back to using whatever they were using before, which seems to be Windows. There are two “lab rats” which will be experimented on, I’ll definitely tune into see how it goes.

As far as learning, and replacing everything they do with an open source application, it should not really be too hard. There are replacements for just about everything with the exception of perhaps very popular large cad engines, but again, they’re IT folk so what are they going to be using cad for? Anyway, letting Windows go is sometimes the best way to use Linux and learn. Hopefully they’ll do ok.

So Derek and Blair, you have a DOS background, and you’re going to be using Linux for a month, it should bring back memories. Just don’t be hating if it turns out badly, and try not to lose your job over this.

Full details of the project are available here: http://www.ayetea.com/announcing-a-room-without-windows.html

Homemade Bench top Power Supply

Filed under: Linux Hardware,The Linux Blog News — TheLinuxBlog.com at 1:06 am on Tuesday, March 31, 2009

Homemade Bench Top Power Supply

I’ve always had an interest for electronics and recently I’ve been exploring my interests more. Last week I sorted through my tub of parts and placed them in individual draws. It took a good while to sort everything but I think it was worth it. I’ve seen power supplies built from PC power supplies before so I thought I’d build one my self. Thing is, I never really got around to it.

Yesterday I was feeling rather ambitious and decided to make a bench top power supply for small electronics. All the sites I found I have lost, so I kind of made it up as I went along. Most of them used ATX power supplies that are readily available, but I opted for the easy way out and used an AT with a hard on/off switch. At first this was the only reason I used it, but there are more advantages to using a AT over an ATX power supply for an external power supply. Firstly, it was cheap, well free actually. I took it from a PC that I had modified some time ago. I have a box full of AT power supplies in storage that I’ll get to some time and replace it. But I won’t be using the PC it came out of for a while, mostly because I have toaster ovens that are faster. Another reason it is better than a ATX is it has less voltages. The only voltages listed are 12v, 5v, -5v (7v) and GND. They vary in amps but are sufficient for what I will be using it for. It made it easy not to screw it up since there wasn’t many wires.

To make it was really easy. I took the top off. Drilled 4 holes in the case and inserted the insulated terminal, checking to make sure they didn’t ground out on the case. Cut most of the cables, leaving a couple of molex’s hanging out just in case I need them. I then soldered the remaining wires to a terminal by voltage (Yellow +12, Red +5, Red +/-5, Black GND.) It might not be the prettiest of them all, but I think it will do its job well.

DFD Today

Filed under: The Linux Blog News — TheLinuxBlog.com at 12:48 pm on Wednesday, March 25, 2009

No, not Dataflow Diagram. Document freedom day. Undoubtedly you’re are aware that it is today, given the amount of press it has got and what a good cause. I was thinking about how I could participate in Document Freedom Day. A few things came to mind. The first was was:

“How about I save all my documents in .odt, that will teach them.”

No go on that one, I already do that because I’m too lazy to save into .doc. I also call the “open source” people out on it when they ask me to save as .doc because it “works on Windows”.

“I could translate all the .doc’s and .docx’s on the file server to an open format.”

Well, my wife just lost her job, I don’t need to lose mine too, although it would be hilarious and it would raise awareness.

Too bad, I’ll have to do nothing this year, perhaps next year I can join the celebration if there is a “Team” in my area. They should really make it a Friday. I’d totally go out and down a few beers in the name of document freedom.

Why I Love Open Source Software

Filed under: General Linux — TheLinuxBlog.com at 10:32 pm on Thursday, April 10, 2008

Have you been looking for a piece of software that does exactly what you want it to? Perhaps its a tool for a client, or an application that would just make your life easier. On a daily basis I always am thinking of new things that I would like my favorite applications to do.

I am going to be writing within the next couple of days about my favorite CHM tools. One of the useful ones I use is a great little tool and it gets the job done, but it doesn’t do EXACTLY what I want. This is why I love open source. I can simple grab the source and change it if I need to. That is the freedom you are given.

Well, I’m not exactly the best C coder in the world but, given time if the application is that critical to me I can make the changes. I can get help from communities when needed and read free information on the web all day long to help me get the job done. If I can’t figure out how to do it in the language the application was written in its not a problem, I can analyze the source code and possibly find a work around. One powerful work around for the CHM application is the Shell.

This is one of the reasons I love open source software. There are many others, feel free to chip in and say why you love open source!

Using wc and How To Count Table Rows

Filed under: Shell Script Sundays — TheLinuxBlog.com at 1:07 pm on Sunday, March 9, 2008

I made this little script to check how many packages were available on the web from the Cygwin Package Repository located at http://www.cygwin.com/packages

Its a one liner but it does its job well.

CYGLIST=$(curl http://www.cygwin.com/packages/ | grep \<tr | grep ball | wc -l); echo $CYGLIST;

All the above is doing is creating a variable called CYGLIST that is the result of grabbing the cygwin.com/packages/ page, grepping all of the TR’s that also have the word “ball” in it (for the image) and then using the wc -l (L) command to count how many results are found. Then the list is echoed out.

wc is a very useful command for printing newline, word and byte counts. This is a good example of how to use wc to count lines in a shell script. wc can also be used to print all of these values in one line of a file.  The syntax is below:

<p align="left">bash-3.1# wc file.txt
9  20 184 file.txt

The above shows the number of lines in the file.txt, it shows how many words are in the file and also how many bytes. In my first example wc uses the -l switch to display the number of lines. This script can also be used with a little bit of bash math to calculate how many items are in an HTML list. I’m working on a script that automatically does this, when its finished I will be sure to post it here on The Linux Blog.

Recursive MD5 Sum Script

Filed under: Shell Script Sundays — TheLinuxBlog.com at 12:08 am on Sunday, December 9, 2007

This week I made this shell script to search one level deep and MD5 all of the files. I did this because I had multiple images and I wanted to see what images were the same so that I could merge them together. Its a pretty simple script & the output is the same as md5suming a file except there is more than one sum generated.

#MD5 Files in the directories
md5Dir () {
echo $directory;
for x in $(ls -1 $directory); do
md5sum $directory'/'$x;
done;
}
#Lists The Directories
for i in $(ls | grep active); do
directory=$i;
md5Dir;
done;

It only does one level deep but thats good enough for now. I am going to make it search recursively depending on the levels given by the user. I would also like to make it display files that are the same at the end.

It gets the job done for small directories, but if I wanted to run it on large multiple directories with lots of files in them I would definitely redirect the output to a file because it can be quite overwhelming. To run it just copy the code into a file and do the following:

sh [filename]

I hope this helps some one who is trying to MD5 multiple files in different directories!

Video Card Failure.

Filed under: Linux Hardware — TheLinuxBlog.com at 11:59 pm on Thursday, November 29, 2007

Linux Blog - Video Card

At the moment I am a little unhappy with my Linux Box. It was making funny sounds (more than normal) the other night so I decided to turn it off. When I turned it back on, it was making even more racket. Turns out that my video card has been toasted, literally. The picture on the left shows the damage. Its pretty evident that the fan stopped spinning and burned up.

So much for my 128MB GeForce FX 5200.

I have an older GeForce laying around somewhere but I’ll have to use the legacy drivers. Until I find the card don’t fear I’ll be continuing to blog on my laptop. Which isn’t the best machine to use, but it will get the job done. Also from this experience I have thought of an blog post to write: dealing with hardware failure.

Slackware 12 HAL fix

Filed under: General Linux — TheLinuxBlog.com at 9:15 am on Wednesday, September 5, 2007

HAL is short for Hardware Abstraction Layer. Its job is to make hardware work with minimal user interaction.
Unfortunately HAL on Slackware 12 does not work right out of the box.
While playing around trying to get HAL to work I was getting weired error messages such as:

File "/usr/bin/hal-device-manager", line 7, in
import pygtk
ImportError: No module named pygtk

and

A security policy in place prevent this sender from sending this message to this recipient, see message bus configuration file (rejected message had interface "org.freedesktop.Hal.Device.Volume" member "Mount error name "(unset" destination "org.freedesktop.Hal")

After doing some research I found that all that is needed to fix this to add your user name to the plugdev group in /etc/group
plugdev:x:83:youruser
If you have multiple users that need access to HAL then add all of those user names to the /etc/group file while your at it. Seperate them with commas as followed:
plugdev:x:83:userone,usertwo
For more information on the HAL project check out the HAL project page.

Living Without Windows

Filed under: General Linux — TheLinuxBlog.com at 8:58 pm on Monday, January 8, 2007

Some may have heard the quote:
“In a world without fences, who needs Windows and Gates?” Although this phrase is clever and slightly funny in reality people do need to use and rely on Windows to do their business on a daily basis.

Like it or not, Windows is here to stay. Given how frustrating it may be, it is not going away any time soon. I have heard so many people complaining about how much they dislike Windows and explain how it is a badly coded unstable operating system but they still choose to run it.
If it were such a bad operating system why do so many people use it? And why is Microsoft still in business? The answer is clear – There is no better alternative for every situation. Some people make claims that Linux is better. Is it? Do these people use it on a daily basis and know that it is in fact a better operating system? It may be a better operating system for one purpose but it is not better in every situation. An example of this could be in the medical field. Imagine the staff are very familiar with Windows and know how to operate it to complete their job. Now imagine your life depended on the staff being able to do their job. Is Linux still a better choice for the given situation?

Linux may be a little harder to set up and use but once a user is familiar with Linux it rewards them with the flexibility they need to get the job done. Tasks that are possible with Linux are not always as simple to accomplish with Windows without pricey third party software.

So how can I live without Windows?

There are two excuses people often use when not running Linux. One is hardware support and the other is software. Below I offer some solutions to these two problems.

Software

Open Source Alternative
A lot of times if the application one relies on doesn’t work on Linux an alternative piece of software is available. If searching freshmeat.net and sourceforge.net yields no results. Asking on forums may also spark some interest with your application need.

Use Cross Platform Software
As a casual Windows user I like to use software that works on Windows, Linux and BSD variants.
Below are some application categories and solutions.

Development: Zend Studio (Java) Commercial
Development: JEdit (Java) Open Source
Development: Eclipse (Java) Open Source
Mind Mapping: Freemind (Java) Open Source
Project Management: Gantt (Java) Open Source
Graphics: Gimp (C++) Open Source
Graphics: InkScape (C++) Open Source
3D Graphics: Blender3D (C++) Open Source
Instant Messaging: Gaim (C++) Open Source
Instant Messaging: aMSN (C++) Open Source
Office: OpenOffice (C++ & Java) Open Source

Web Applications
Most web applications are cross platform and are very good at the task at hand. Some may be free while others come with a subscription fee. They may be not be as fast as desktop equivalents and response time may vary but web applications are becoming more popular.

Run your Windows applications on Linux
If you do not with to find an alternative piece of software there are some options that may be appealing to you.

Wine
Using a little setup time and a handy toolkit called Wine (Wine is not an emulator) one can run many applications on the Linux desktop without ever touching windows. Except possibly for copying important DLL files.

Cross Over Office
Cross over office is a commercial piece of software which uses wine. It eases the installation process for many supported applications.

Virtual Machines
Virtual Machines are perfect for the more modern faster machines out there. By using the commercial VMWare server (now free) the Open Source Xen or QEMU machine emulators one can run a virtual copy of windows inside an X11 Session. If your machine is powerful enough it may even be faster then an underpowered separate computer.
VMWare Server: http://www.vmware.com/products/server/
Xen: http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
QEMU: http://www.qemu.org

Another Physical PC
Nobody said that you couldn’t use a Windows box. This obviously provides the most compatibility with the software you may own. Personal Computers continue to drop in price and a reasonably equipped machine that is capable of running an older version of windows can be purchased for a reasonable price.
Potential Problems and Solutions
Space
Purchase a smaller PC
Keyboard / Mouse / Monitor
If you don’t want a separate keyboard, mouse and monitor on your desk you could purchase a KVM and use your current setup.
Use VNC. By using VNC and a LAN connection you can effectively use the computer as if you were right in front of it. I do not recommend this method over wireless connections, as the quality of the VNC session may not be very good.

Hardware

One drawback of using Linux is a lack of hardware support. Although most major hardware is supported often one may run across some strange hardware that is not supported. Often this is also used as an excuse to run Windows.

NDIS Wrapper
If you have a wireless card that has no native driver, it is probably possible to get it working with NDIS Wrapper. This allows one to use Windows wireless drivers under Linux. A friend of mine has successfully used this approach to get his Dell Inspiron wireless working.

Laptop Support
I once owned a Compaq Evo 1000v which I had successfully installed Slackware on. The hardware support seemed buggy at the time, but Linux did run on it. I traded it with a local PC store owner for a smaller underpowered Toshiba Portégé 4010 which I now have a working installation.

Peripherals
Often the techie type PC users have a lot of gadgets. Depending on how many other users have this gadget there may or may not be a driver available. If there is not, all hope is not lost.

VMWare
Sometimes with certain type devices VMWare can be used. I successfully used this method to use Linux with a Parallax PIC micro controller, which is programmable by a serial connection. All that needs to be done here is some toying with the VMWare settings.

Buy new hardware
If you have a need for a particular device and support is available for similar devices. But not your particular piece of hardware, sometimes it is advisable to purchase new hardware.

Develop a solution
If buying new hardware is not an option due to funds or no other available hardware and searching yielded no results. Posting to forums may help. If other users have this hardware maybe some one out there with the skills necessary could start a project. With a project started more owners of this hardware will find the project and may be able to chip-in.

Ignore the problem
If you are really lazy like me you may just choose to wait and do nothing about the hardware support. In some rare cases this approach will work. Ignoring the lack of hardware support and just using Linux anyway is sometimes a better approach. For example, my roommate refused to use Linux on his main PC because there was no support for his second on board Ethernet so he used Windows. What baffled me was that he wasn’t even using the second Ethernet card under windows. Some time passed and a new kernel got released, low and behold a driver for his controller is now included. He finally installs Linux and went through all of the trouble of backing up files and the installation routine again. This could have been avoided if Linux was used to begin with.
If the hardware doesn’t cause any system problems then the only problem is that the device simply doesn’t work.

Conclusion
The best way to learn Linux is to become surrounded with it.
It is easy to use Linux as a primary operating system and for those interested in Information Technology and a lot can be learned.

So which is better for me?
I could say that Windows is less practical for me because it does not always provide the flexibility I need. Constant issues with stability and drivers not working also become a problem when it is not possible for me to repair them.

After using Linux as my primary operating system for a while it has become clear how much users are restrained by using Windows. For example I have become accustomed to the Xfce 4 keyboard shortcuts and I create workspaces for different groups of applications I have running to perform a certain task. With my Windows PC at work there is no way for me to organize my windows in a way that makes sense to me. I find my self closing all applications and opening them one by one to reorder them in a way that makes sense. My productivity is higher when I am working on a Linux Box. Therefore I would say that Linux is my preferred choice.