configuring RSYNC???

[geofoxer]

Hello all,

I have installed rsync from a .tar file, and I am wondering how I would go about finding the following files for further configuration of rsync:

1) rsyncd.conf
2) rsyncd.pid
3) rsyncd.secrets

Where are these files located? Does it make it difference if you compile a .tar compared to a .rpm file? I have used "whereis" to try to find these files, but unfortinitely without luck. My second question is whether or not these config files might be somewhere else, or perhaps was the installation bad?

[DimaYasny]

did you run make install ?

[geofoxer]

Yeah I ran that as well. I think that the config files were created with different names. I used locate to find the config files and it came up with the these results:

rsrsync.conf.5
rsync.conf.yo

It is not able to find the other files, is my install bad by chance? How do you fully uninstall a previously installed program?

[DimaYasny]

there was a great full rsync tutorial on linuxquestions.org
look it up there

[geofoxer]

I looked through the tutorials section and could not find the tutorial that you were talkin about. By chance do you have the link to the tutorial? But thanks for the suggestin on LinuxQuestions, its a great site.

[DimaYasny]

http://wiki.linuxquestions.org/wiki/Rsync

[geofoxer]

One last Question, is there a Forum for RSYNC? I have searched the web and have not found a forum that is dedicated to RSYNC. Thanks a lot for your help.

[DimaYasny]

no, not really.
what are you trying to do with rsync?

[geofoxer]

I want to back up about 4 to 5 remote servers at the end of everyweek to an external device on the linux machine that rsync is installed on. I have to do it by hand through putty and have been wanting to get something that does it automatically so I dont have to spend the extra time backing up each server one by one, and then running a final back up of everything to a computer on our local network with has a external 500GB HDD.

The way I back everything up right now works perfectly, but it takes a little time to do it, accessing each server over ftp to pull the backups. I want to set RSYNC to do that automatically.

[DimaYasny]

why not just script your actions?

[geofoxer]

You mean with RSYNC?

[geofoxer]

[Edit]

bad post... needs to be deleted.

[DimaYasny]

I mean with whatever tools you have successfully used up to now

[geofoxer]

Take a look at the following pic:

network diagram

Our servers make daily backups to individual backup servers. What we do once a week is make a final backup of all of the backup server onto the server outside of our network via ftp over putty. After we have all of the backups from the other backup servers, we make a final backup of that to a computer with a external HDD on our local network. THis is a pain in the butt, and I figured that I could configure RSYNC to pull the backups from the backup servers to a linux box on my network. I dont know if this would work. I have heard good things about RSYNC, so I thought that this would be a possible solution to make things a little easier and more automatic.

[DimaYasny]

what commands have you used up to now?

[geofoxer]

I am actually just getting started with rsync. I am very new with rsync, so right now I am just reading up on the program to get an understanding of how to use it and how it works. Do you have any suggestions on how I should start configuring rsync?

[DimaYasny]

as I said, before you start using rsync, maybe whatever you have been using up to now is good enough, or even better. so why not just script those actions, instead of changing the entire routine?

[geofoxer]

Thats the thing, I dont really understand how I can script that... :( I will google it and see what I can find!!!

[DimaYasny]

that is why I asked you to show what you have been doing up to now. that way we might try to script that, or see that scripting is impossible, and you need a different solution

[geofoxer]

I did find a forum for RSYNC, I will list it for anyone that may need it:

rsync forum

I actually talked to my boss and he said he wanted me to use rsync, so I will be playing around with it for the next two weeks. But if you are able to help me with scripting this stuff on the linux servers, that would be great. I have read up on rsync and I like how it runs its backups. I am sure that you already have read up on it yourself or are currently using it. What else can I do to give you a better idea in order to help me out? Are you using rsync yourself?

[EDIT: Here is a pic of what I want to see in my backup topology:

network diagram

Let me know what think!!! ]

[DimaYasny]

nope
I am currently working as an escalation point for system admins at datacentres all over the world, so I don't actually manage systems of my own, just help out when they can't manage on their own

[geofoxer]

Cool, would you mind if I asked what company you worked for, do you have offices located in Germany (Hamburg specifically)? Aside from getting off topic, how should I start scripting and what should I do for a script file in terms of my own case?

[DimaYasny]

the company probably has offices everywhere, and it is pretty famous :)

as for the script, can you post the commands you were using to do the manual backup? when I see that, we will be able to see if the action is scriptable.

Also, there are very good scripting gurus on this site, and I am far from being even close to their level of expertise

[geofoxer]

I first use " ftp <server name>" to log on from the backup server (the one where all backups from the other backup servers are copied too), enter the password to log on. Then I go to the directory where the daily backups are stored and use "mget *" to get the backups that I want to be copied.

I do that for all of the backup servers. Then once I have all of the backups copied from the other backup servers, I run a program on a computer thats on our local network with the user name and password of the server containing the backups and I click connect, drag and drop the file into the directory of the External HDD and let it do its work for about 2 hours!!!

[DimaYasny]

ok
1. ftp can be accessed directly by using the "ftp://username:[e-mail]", so no need to actually manually enter credentials
2. you can use wget to pull files off ftp in an unattended mode
3. mounting external drives automatically is easy with fstab entries
4. copying can be easily scripted, you can even use cp, but I'd go for tar

one question, why not pull all files directly from backup servers to the backup drives?
another question: why not use tar and run only incremental backups - that should really speed things up

[geofoxer]

Now that you mention it, you have a good point of backing up directly to the HDD on my linux machine. I am sure why we were not already doing that, but that will soon change (I was given the job to make the process faster and easier). And it makes sense to use incremental backups, it is obviously faster than backing up everything. But we only back up a weeks worth of file backups and that is it, we dont mess around with the older or previous backups.

For the scripting, I understand a little how I could make a script file with ftp to grab the files that I need. There is more than 1 server, so how I would want to set up the script file to backup one server, then when its finished, start the backing up the next one and so on until everthing is backed up? And one more question, how would I script the settings for an Incremental backup with mget?

[DimaYasny]

I don't think you can actually use mget to do incremental backups, but you can
1. http://www.google.ie/search?q=mount+ftp+linux
2. http://www.google.ie/search?q=tar+incremental+backup

the script would be to, one by one, mount a remote server's ftp share, tar the info over, umount and move on to the next server

[geofoxer]

Sweet, thanks for your help. Much appreciated!!!