Rsync on Windows disk to disk?

Wednesday 9 January 2008This is nearly 17 years old. Be careful.

I have a Western Digital 250Gb Passport drive for backing up my Windows laptop. I figured I would just use cygwin rsync to move data between the two drives. When I mount the Passport as drive Q, I use a command like this:

rsync -az --progress . /cygdrive/q/bak

This faithfully copies all the data, and avoids copying stuff that hasn’t changed, mostly. But some files are copied every time I run the command, even when they haven’t been changed, and in fact, a diff program shows that the source and the backup copy are identical. I figure it’s some Windows characteristic of the file that is throwing off rsync and making it think the file needs to be copied, but I can’t figure out what. Some of the files have capital letters in the filename, but not all of the afflicted files do. I never set permissions on files, so I don’t think that could be it (but I could be wrong).

Anyone have any experience with this?

Comments

[gravatar]
rsync isn't designed to understand Windows filesystems.

I think you might be better off with something like SyncToy.
[gravatar]
You may want to try the --modify-window option to rsync. I was having the same problem backing some data up to a FAT drive until I used --modify-window.
[gravatar]
The arguments I'm happy with for interaction with windows filesystems, is:

--size-only --delete --delete-excluded --exclude-from=exclude.txt -avz --no-group

although this grew over time, so some options might actually be redundant. I'd look at the man pages and search for no-group and --size-only particularly, to see if they are what you're looking for.

good luck,

eric
[gravatar]
I knew I'd get some good ideas! Of course, I neglected to bring the Passport to work with me so I could try them out right away. :(

The --modify-window suggestion led me to this page which has a number of suggestions that seem helpful.
[gravatar]
On Windows I have had better luck with Unison (http://www.cis.upenn.edu/~bcpierce/unison). The aforementioned SyncToy also works well.
[gravatar]
I don't know about rsync, but I do know about nsync. Also, I do know alot about the kitchensink, and also, that syncing feeling. Glad to help anytime, just aks.
[gravatar]
Try an MD5SUM on both to make sure the files had not changed. Windows (Especially MS Office) does not change the datetime stamp and or archive bit when it changes the user who last printed a file.
[gravatar]
I had the same exasperating experience with rsync. And with many of the always-copied files being rather large RAW photos and such, I was driven to abandon it in favor of a more hacky homebrew Python concoction that compared a manifest of files from both locations to decide what to copy. I will try SyncToy next.
[gravatar]
IIRC, the problem is that the granularity on file timestamps in Windows is too low -- something like 2s. So --modify-window works by telling rsync, "times differing by ____ are equivalent".
[gravatar]
I've had the same problem with rsync. I found the --size-only option fixed it. I also get strange windows permission problems from time to time on the backup drive. I fix them with a 'chmod -R 777 /cygdrive/f/'. It is a little extreme to set everything to 777, but for the purpose of my backup, it works OK.
[gravatar]
FAT32 has a 2 sec timestamp granularity, and --modify-window=1 solves that problem. That shouldn't be a issue if you're backing up FAT32 to FAT32 though.

The other problem is that FAT32 loses ownership permissons. The -a switch is equivalent to -rltpgoD, where the o and g tell rsync to check owners and groups. Try rsync -rltpD (that's what I use for backups on my FAT32 laptop).

Also, the z switch is most useful across the wire, not disk to disk, and is probably just wasting cycles if you're backing up a lot of compressed data and media (which is usually what's taking up the most space, anyway).
[gravatar]
You guys are the best! It turns out --modify-window=2 did the trick! Thanks for all the suggestions!
[gravatar]
Perhaps a bit intensive but worth it for anything you really care about....

-c

Adam
[gravatar]
Tks,
these tips were very useful to me.

My 2 cents below:

I was trying to sync my homedir with a pendrive (fat32), kubuntu 7.10.

To solve the problem I have to:

1) manually remount the pendrive with option shortname=mixed (default is lower).
2) --modify-window=2
3) remove the --perms option, actually I'm using -rltODvz
[gravatar]
Ward Christensen 8:44 AM on 9 Nov 2010
Hey great blog, great tips!!

If I have timezone trouble, I'd use --modify-window=3600 or even 7200 I presume?

I use rsync between windows XP and a wonderful NAS box called a Netgear NV+ (now I think replaced by a ?? Pro or something like that). If you can afford close to $1K for your small biz or family backups, it is really really nice! Even has telnet shell option.
[gravatar]
Reading through this old thread, im surprised no-one has mentioned robocopy, rsync does not make much sense in this instance http://en.wikipedia.org/wiki/Robocopy.

For windows rsync (over wan), i use yintersync http://www.yinter.net.
[gravatar]
Hi, great comments. I had problems with permissions and so on. Repalcing -a with -rltD solved this. Additional I added --modify-window=2 to prevent further problems ;)

Thanks a lot.
[gravatar]
What does backup.bat do?

Sync the entire profile for USERNAME to the /backup folder on the Pi. You can modify that source path to just about anything.
The --bwlimit flag is limiting the bandwidth used by rsync so you don't overwhelm your connection.
The exclusions on AppData are to prevent cyclical symlink transfers.
The timeout is specifying a delay in seconds before running the next command.
[gravatar]
Disk to disk can be done using simple win explorer too. but yeah its not good though. I recommend trying gs richcopy 360. It has many features and has helped me a lot till now. Great stuff, pretty dope software!

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.