Linux Blog

An rsync primer

Filed under: General Linux,Linux Software — TheLinuxBlog.com at 12:55 am on Tuesday, January 13, 2009

An rsync Primer

rsync is a great tool used to copy or “sync” files locally or remotely. Having just lost a fair amount of important personal data its good to make the point that you can lose data if you use rsync, so make sure that you know what you are doing works before putting it into production.
“Whats so great about rsync?”
I hear you ask.

Well, there are many ways of copying data around, regular old cp, scp and rcp but what sets rsync apart is that it’s capable of syncing those files that have changed, or those that are missing. In other words, why do a full copy of all files, risking further corruption when only some have changed?
rsync also handles compression and verifies that the files were written correctly.
The syntax for rsync is fairly straightforward, those of you familiar with the rcp or scp should be easily able to relate to that of rsync.

The syntax for a local file copy is:

rsync [OPTION...] SRC… [DEST]

-avz is the option that I use most commonly.

-a for archive, -v for verbose and -z for compression. As lame as it may sound, I actually remember the syntax for this as “Alien Vs. Zeus.” Sounds stupid, but it actually works.

Since I do not normally sync directories locally, one of my hosts is usually remote. rsync is configured to use SSH by default, but rsh can be used if preferred. I stick with the default SSH for simplicity. The syntax is very similar to scp’s

To Pull: rsync [OPTION...] [USER@]HOST:SRC… [DEST]

To Push: rsync [OPTION...] SRC… [USER@]HOST:DEST

If you like to back up all files matching a pattern the same syntax applies but you specify a pattern. An example is you could backup all .conf files from /etc (provided you have read access) to another host by doing the following:

rsync /etc/*.conf [USER@]HOST:DEST

If you do the above, I am assuming that you would also want to backup other configuration files within the /etc/ directory. Instinct tells you if you have read the man page to just use -r however you may want to just do a full backup as rsync with a pattern by default does not traverse into directories.

Compression is really a nice feature for transferring large amounts of data. To demonstrate the speed increases that the -z option gives my full /etc/ directory which is 48M. It took 0m12.671s with compression and 0m35.657s without. This was over wireless from a dual core 2GHz laptop to a 1.8GHz wired desktop, so your results may vary. Either way compression or no compression rsync is a handy utility that should be able to make your life of copying files around a little easier.

Apache “Directory index forbidden by Options directive:”

Filed under: Quick Linux Tutorials — TheLinuxBlog.com at 9:29 pm on Friday, June 27, 2008

This is a common problem with the Apache web server. It is considered a security risk to show the directory contents by default. Especially those in the root directory. The way to fix this is pretty simple, you first need to make sure that there are no -Index directives in any of the configuration files. In your httpd configuration directory grep with a line number for Options like so:

grep -n Options *

If you don’t see any -Indexes then its possible there are included files. Check these directories if you know them, if you don’t grep your configuration file for the Includes:

grep -n Include *

If you’re still getting the test page or a permission denied error after removing the directive that disables Indexes then it may be a permissions issue. Apache needs +x access for all users to enable directory listings. Change the permissions and it should work.

On Fedora there is a file called welcome.conf in the conf.d directory. This can be removed and if you have Options All set, then you should be good to go. On other distributions like Slackware it is easier to accomplish. I wouldn’t recommend allowing any sort of directory listing in a production environment but in my development case, where I only allow access to my test server on a per host basis this is not a big problem.

Also, remember to restart the server after making changes.

Hope this helps, if it doesn’t be sure to drop a comment!