Linux Blog

Using SSH as a Proxy

Filed under: General Linux,Quick Linux Tutorials — TheLinuxBlog.com at 2:46 pm on Tuesday, April 15, 2008

A helpful reader left a comment on this blog about using SSH as a Socks Proxy. Here is how to do it.

ssh -D <source port> <host>

Thats it, once your logged in you are good to go.

Now, the problem I ran into was making Firefox use this proxy. I found a great extension called SwitchProxy which can be installed from the extensions site. Once installed, you can easily switch between proxies. This is really useful to use while at a coffee shop.

Check out the screenshot of the toolbar that it installs (click for a larger image):

Using SSH as a proxy on Linux

It makes it really easy to turn the proxy on or off. One day if there is interest I might try to extend this to establish the SSH connection.

Facts About Selenium

Filed under: General Linux,Linux Software — TheLinuxBlog.com at 9:22 pm on Friday, March 28, 2008

Selenium is a chemical element. What you may not know is that Selenium is also a powerful testing tool for web applications. Selenium runs its tests directly in a browser, just like real users do. It is cross platform and the developers plan to have it for the iPhone but thats another story. Selenium can run in one of two modes Core and Remote Control (RC). The RC method has a way of using distributed computing much like the way Samba allows cross compiling over multiple cpu’s. There is an IDE for Selenium that can be used to easily learn Selenium.

Enough Facts About Selenium already!

What exactly can you do with Selenium?

Well, the answer is simple. Pretty much anything that you can do with a browser Selenium can do. Its primary purpose is for developers to use as a quality assurance tool. For QA purposes you can create a test cases, run them and verify that the end result is what you expected. You can create test cases in Java, Ruby, Python, Perl, PHP or .NET.

I had a little trouble getting the Selenium driver for PHP installed so here is the how to:

sudo su
pear channel-update pear.php.net
pear install Testing_Selenium-beta
pear install PHPUnit
exit

The above allowed me to communicate with the Selenium Remote Control that I downloaded with PHP. Here is an example from their website that I have modified so that it works:

<?php

set_include_path(get_include_path() . PATH_SEPARATOR . ‘./PEAR/’);
require_once ‘Testing/Selenium.php’;
require_once ‘PHPUnit/TestCase.php’;

class GoogleTest
{
private $selenium;

public function setUp()
{
$this->selenium = new Testing_Selenium(“*firefox”, “http://www.google.com”);
$this->selenium->start();
}

public function tearDown()
{
$this->selenium->stop();
}

public function testGoogle()
{
$this->selenium->open(“/”);
$this->selenium->type(“q”, “hello world”);
$this->selenium->click(“btnG”);
$this->selenium->waitForPageToLoad(10000);
$this->testCase(“/Google Search/”, $this->selenium->getTitle());
echo “<hr>”;
$this->testCase(“/Yeahh Search/”, $this->selenium->getTitle());

}

public function testCase($regEx, $string) {

preg_match($regEx, $string, $matches);

print_r($matches);

}

}

$google = new GoogleTest();

$google->setUp();
$google->testGoogle();
$google->tearDown();
?>

Before attempting to run this you must make sure that you downloaded Selenium RC and that it is running. Selenium runs on Java so make sure that Java is installed download selenium from here, unzip and run the following in the directory that it is extracted to:

cd selenium-remote-control-1.0-beta-1
cd selenium-server-1.0-beta-1
java -jar selenium-server.jar -interactive

Once this is running you can start scripting with PHP to get Selenium to do anything that you want. Once you are done make sure that you exit the Selenium server by running the “exit” at the prompt.

I’m sorry about the format of this post, its been a while since I have used Selenium and I’m quite excited about it. If you have any questions about Selenium post them here and I will try to answer them for you.

Using wc and How To Count Table Rows

Filed under: Shell Script Sundays — TheLinuxBlog.com at 1:07 pm on Sunday, March 9, 2008

I made this little script to check how many packages were available on the web from the Cygwin Package Repository located at http://www.cygwin.com/packages

Its a one liner but it does its job well.

CYGLIST=$(curl http://www.cygwin.com/packages/ | grep \<tr | grep ball | wc -l); echo $CYGLIST;

All the above is doing is creating a variable called CYGLIST that is the result of grabbing the cygwin.com/packages/ page, grepping all of the TR’s that also have the word “ball” in it (for the image) and then using the wc -l (L) command to count how many results are found. Then the list is echoed out.

wc is a very useful command for printing newline, word and byte counts. This is a good example of how to use wc to count lines in a shell script. wc can also be used to print all of these values in one line of a file.  The syntax is below:

bash-3.1# wc file.txt
9  20 184 file.txt

The above shows the number of lines in the file.txt, it shows how many words are in the file and also how many bytes. In my first example wc uses the -l switch to display the number of lines. This script can also be used with a little bit of bash math to calculate how many items are in an HTML list. I’m working on a script that automatically does this, when its finished I will be sure to post it here on The Linux Blog.

SSH Tunnel

Filed under: Linux Software,Quick Linux Tutorials — TheLinuxBlog.com at 11:23 pm on Thursday, March 6, 2008

Today I had the need to access my development web server at my office network. I have a firewall that runs SSH but doesn’t my firewall does not forward the port for the web server. So, in an emergency situation I was able to use an Linux with SSH and Tunnel into my network on port 80. Since I have a server running on port 80 on this computer I could not use this port. I chose port 8080 to use for the local port and forward it to my web server on the firewalled network. I did this by doing the following:

ssh <SSH HOST> -L 8080:192.168.1.X:80

Using SSH Tunneling I was able to then browse to http://localhost:8080 on this computer and successfully view the contents of my web server. So, any time you need to connect to a port on a computer behind a NAT firewall that you have access to SSH on is just use a SSH Tunnel and then use the -L Switch with the port on your local machine first, then the destination address and port. Simple!

PHP Script To Log Into cPanel

Filed under: Shell Script Sundays — TheLinuxBlog.com at 3:47 am on Sunday, February 24, 2008

Earlier this week I made a script that logs into cPanel to check statistics. Basically if you have a webhost that runs cPanel and you wish to log into cPanel for some reason then this script is for you. Once you are logged in you can basically do anything you would want to do. For example my specific use was to log into my cPanel nightly and parse some data provided by AWStats. But with some modification this script could automate anything you can do by hand.

Since this is more of a web project for me I decided to write my cPanel login script in PHP. I found a PHP class to login here. curl is used to fetch the URL’s and I parse the data using PCRE regular expressions. The statistics code is still very basic but I thought I would post it for those interested and what better place then The Linux Blog’s Shell Script Sundays column?

Onto the script.

It consists of three scripts each with their own purpose in run time. They are as follows:

cPanel.php – This script does all of the dirty work in connecting to cPanel and fetching the pages. I modified this from the original a little
class.mysql.php – Just a generic data base handler. MySQL configuration information is stored in here.
login.php – This is the script that starts off the process. I named it login.php instead of index.php so that I do not have it run as the default page in my web browser. login.php also does all of the parsing of the data and is where the data gets inserted into the database.

To run the script edit login.php and then you can either put it in your PHP powered web server directory or run it from the command line by doing:

php login.php

The output should be as follows:

Num: 0 Date: 2454521 uniques: X visits: X visits per visitor: (Xvisits/visitor) pages: X pages per visitor: (XPages/Visit) hits: X hits per visitor: (XHits/Visit) bandwidth: X GB bandwidth per visitor: (XMB/Visit)<br/>

Feel free to modify this as you wish. If any questions can be answered I’d be happy to do so. I’d like to hear what people are using this for too, so drop a comment!

Download the PHP cPanel Login Script

Easy script to upload to an ftp server

Filed under: Shell Script Sundays — Kaleb at 11:43 am on Sunday, February 10, 2008

This weeks Shell Script Sunday article is a guest post by Kaleb. Kaleb has been helping me out a by writing for me since I have a project for The Linux Blog that I’d like to complete and can’t dedicate as much time to writing right now. So if you get a chance visit his site and drop some comments.

Hello it’s Kaleb from http://kpstuff.servebeer.com again and today I have come to tell you about a little shell script that I wrote. It’s a script that allows for extremely easy and simple uploads to an ftp server.

The script is actually quite simple utilizing just standard bash scripting along with ncftp which is just an ftp client with some special features. So you need to install ncftp in order to use this script.

Gentoo:

emerge -av ncftp

Arch Linux:

pacman -Sy ncftp

Ubuntu:

apt-get install ncftp

After you install that you need to edit the upload.sh file to fit your needs. There are a few variables you need to change such as your username and password for the server, and the server itself. It is fairly simple to configure.

Set the user variable to your username

user=”kaleb”

Set the passwd variable to your password

passwd=”kalebspass”

Then you need to set the server variable to your server you want to connect to

server=”kpstuff.servebeer.com”

Be sure to not precede the url with something like http:// or ftp://

Finally you should set the default directory on the server for your file to be placed into.

DIR=”/home/$user”

Use $user for your username

Also if your system is odd you may need to make sure that /usr/bin/ncftpput exists. if it does not you will need to find out where on your system ncftp is and change the variable FTPCOM to suit your system.

Now all you need to do is run “chmod +x upload.sh” or “chmod 755 upload.sh” to make it executable and you are finished.

Usage for this script is quite simple. all you need to do is:

./upload.sh <FILE>

Making sure to replace FILE with the filename of the file you want to upload

And if you want to temporarily send the file to a different directory then you specified in the file.

./upload.sh <FILE> <DIR>

Remember to replace <FILE> with the file you want to upload and replace <DIR> with the directory you want to upload to.

You can obtain the script at this address http://kpstuff.servebeer.com/~admin/scripts/upload.sh . And that is it thank you very much for your time and I hope that this script helps you out and if not at least gives you ideas.

Clear browser history from command prompt

Filed under: General Linux,Quick Linux Tutorials — TheLinuxBlog.com at 3:36 pm on Thursday, January 17, 2008

I found this link a while back on clearing the history in IE7 from the command prompt

Clear IE7 browsing history from the command line

Although not specifically Linux related it is interesting and I thought users who sometimes use IE7 may be interested in this. It also provoked me to ask the question “How do I clear FireFox history from command line.”

Well, it is a bit simpler then clearing your history in IE7. The only thing you need to know is your profile name, and even then you don’t really have to because you can guess.

Firstly there are a couple of files that you should be aware of they are: cookies.txt, downloads.rdf, history.dat and one folder full of files: <profile>/Cache/

Now to clear your downloads list, your cookies list, your history or your cache you basically just remove the corresponding file. So, lets take a look at the command. We’ll assume my randomly generated profile name is called linux-blog

owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/cookies.txt
owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/Cache/*
owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/downloads.rdf
owen@linux-blog:~$ rm .mozilla/firefox/linux-blog.default/history.dat

The above need to be ran individually. Make sure that FireFox is closed before the above is ran, or some pages may not load correctly. If any one is interested I’ll write a shell script to make this easier.

General Linux Kill Process

Filed under: General Linux,Linux Software,Quick Linux Tutorials — TheLinuxBlog.com at 6:38 am on Tuesday, January 15, 2008

Killing a process with Linux is an easy task. As always there is more than one way to do it. There are graphical process managers that can be used to aid in killing a process on Linux. The first method I’ll demonstrate may work depending on your window manager. Either way you can set it up to work the same way if you like it.

The name of the program is xkill. My XFCE has a shortcut of CTRL+ALT+ESC but this may not be the case for every version of XFCE. Basically you press this keyboard shortcut and you get a skull and crossbones. Once you get that you can click on the window of the process you’d like to kill and it kills it.

To use Linux to kill a process from the command line, you can use one of two commands that are pretty standard throughout all Linux Distros the kill, and killall commands. The only real hard part is figuring out what process to kill. To figure out what process I want to kill I use the following command:

owen@linux-blog:~$ ps ax

then to use kill and killall on Linux I use:

owen@linux-blog:~$ kill <processid>

owen@linux-blog:~$ killall <processname>

This is pretty straight forward but if you have say multiple FireFoxes open, you may want to just kill the process by using the kill <processid> command, otherwise all of your FireFox windows will probably close since killall kills all processes that match the name, regardless of if they actually are crashed or not.

If the process won’t die, you can use the following to kill it. Be aware that this is not the best thing to do but it will kill the process.

owen@linux-blog:~$ kill -9 <processid>

owen@linux-blog:~$killall -9 <processname>

Basically instead of killing gracefully you send a SIGKILL to the process which is basically tells it to commit suicide no matter what its currently doing. I’ve listed all of the signals you can send to kill a process at the end of this post.

Another method to kill a process is by using top. Top is an interface that shows you what processes are doing what. You can kill a process (once your in top) by pressing the k key. It then asks you what PID (Process ID) you want to kill. You can figure this out from the list. It then asks what type of signal you want to use. You can use the default first, and then if the process just wont die, you can use 9. Top is useful for killing a bunch of processes in a small amount of time.

List of all signals that you can send:

owen@linux-blog:~$ kill -l
1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL
5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE
9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2
13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGSTKFLT
17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP
21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU
25) SIGXFSZ 26) SIGVTALRM 27) SIGPROF 28) SIGWINCH
29) SIGIO 30) SIGPWR 31) SIGSYS 34) SIGRTMIN
35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3 38) SIGRTMIN+4
39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8
43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12
47) SIGRTMIN+13 48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14
51) SIGRTMAX-13 52) SIGRTMAX-12 53) SIGRTMAX-11 54) SIGRTMAX-10
55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7 58) SIGRTMAX-6
59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2

Recursive MD5 Sum Script

Filed under: Shell Script Sundays — TheLinuxBlog.com at 12:08 am on Sunday, December 9, 2007

This week I made this shell script to search one level deep and MD5 all of the files. I did this because I had multiple images and I wanted to see what images were the same so that I could merge them together. Its a pretty simple script & the output is the same as md5suming a file except there is more than one sum generated.

#MD5 Files in the directories
md5Dir () {
echo $directory;
for x in $(ls -1 $directory); do
md5sum $directory’/'$x;
done;
}
#Lists The Directories
for i in $(ls | grep active); do
directory=$i;
md5Dir;
done;

It only does one level deep but thats good enough for now. I am going to make it search recursively depending on the levels given by the user. I would also like to make it display files that are the same at the end.

It gets the job done for small directories, but if I wanted to run it on large multiple directories with lots of files in them I would definitely redirect the output to a file because it can be quite overwhelming. To run it just copy the code into a file and do the following:

sh <filename>

I hope this helps some one who is trying to MD5 multiple files in different directories!

Comment spam vent and an idea to combat it

Filed under: General Linux,The Linux Blog News — TheLinuxBlog.com at 10:48 am on Thursday, September 27, 2007

Ok I just have to vent a little. Spam Spam Spam. Spam bots and blog spammers in general are stupid. Why would you spam a site that doesn’t even have comments on blog posts? I don’t get it. I would love to get my hands on the code of these spam bots to see how they are working. It would be so simple for them to actually load the page and grep for the comment link. That way they could stop wasting mine and their time by spamming this site. All comments are marked as spam at the moment since that’s all I’m getting. If some one wants to send me a comment they could go trough the trouble of creating a <FORM> to send one but I think they would just send me mail at owen -at- <thelinuxblog.com>. This generally means that I can delete all of the comments I get as I doubt any one is really going to create a form just to send me a comment.

IP addresses are being recorded and I’ll continue to monitor the situation. Eventually I might do a security scan on repeat offenders and let the ISP of the host know what’s what. It would be nice to catch the spammer but I will settle for the knowing that the owners of the Zombie PC have been warned about the consequences and might send their computer in for repair.

After writing this post I have an idea for a Linux Blog project. The Linux Blog can be the first to submit data. Basically I propose a plugin for WordPress that would monitor comments and send the IP’s of those that were spam (either flagged manually or automatically) to a host. This host would log all of the IP’s submitted and when appropriate take action against them.
This reminds me of a project that I saw a presentation on at CarolinaCon 05. Except this project was targeted at bot nets and aimed to be malicious back or ban the host / subnet completely. This would not be necessary for this project; just a simple mail merge application written in Linux would work.
I suppose the real first step is to see if this application already exists. I’m sure I’m not the first person to think of this.
I guess I need to come up with a quirky name for the project and a plan. It goes without saying that the platform will be Linux and will use open source applications to achieve the final goal. Once research has been done I can either try to add it to my never ending list of projects to complete or find a team of developers willing to work on it.

For, While and Until Loops in Bash

Filed under: Shell Script Sundays — TheLinuxBlog.com at 3:45 pm on Sunday, August 12, 2007

Normally in a shell script it is desirable to do something repetitive.
I have touched on some loops in other posts but now would like to go over them in a bit more detail. Each of the examples in this post are intended to give an introduction to looping in bash.

For Loops
For loops allow you to repeat a section of code a number of times. Its very similar to other languages syntax but works a little differently. The for loop in bash only allows you to give a fixed list of values to loop over. A good way to remember how a for loop works is “For each of the dishes: clean and dry.”
For Syntax:

for i [in list]
do
statements [use $i]
done

For Example:

for x in 1 2 3
do
echo “Number: $x”
done

echo “Finished!”

This is a very simple script that just counts to 3 and then prints “Finished!”

While and until Loops
In essence while and until are the same in bash. The titles are pretty much self explanatory. A while loop would be explained in real life as “While the sink is still full: wash dishes” and a until loop would be “Until the sink is empty: Wash dishes.”
While and Until Syntax:

until/while [condition] do
statements
done

Example of a While loop:

count=1
while [ $count -lt 10 ]; do
echo $count
let count=$count+1
done
echo “Finished!”

Basically this loop will loop over the code while the count variable is less than 10. If we didn’t put the let statement in the script it would get stuck in the loop causing the user to press CTRL+C to end the script.

Doing the same thing can be done in a until loop except the condition has to be modified to get the same result.
Until example:

count=1
until [ $count -gt 9 ]; do
echo $count
let count=$count+1
done
echo “Finished!”

Now that you’ve figured out how to loop over something its probably a good idea to know how to stop the loop.
All that needs to be done to stop a loop is:

break

Break Example:

for x in 1 2 3 4 5
do
if [ $x = 3 ]; then
echo “Number is 3. Quitting”
break;
fi
echo “Number: $x”
done

This is a very easy to follow example. Its the same as the basic for loop except that if x is 3 it will stop the loop. This example has no real practical purpose. Since its a for loop the number 3 could just have been omitted.

Real World For Loop Example
Looping over all files in /etc and printing all of those that match “grep conf” and putting them in quotes.
The code to do this in a loop is:

for x in $(ls /etc -1|grep conf);
do
echo “$x”
done

The situation for many bash scripts is that there is normally a shorter way to do something. Take the Real World For Loop Example in this tutorial the same results can be achieved with:

x=$(ls /etc |grep conf); echo “$x”\n

This will get the job done but a loop may be better for esthetic purposes or for additional logic.

« Previous PageNext Page »