Rsync – how effective is it?

There’s much excitement here as we’re about to release our Rsync Engine, which will enable data to be synchronized offsite in a bandwidth efficient manner.

I’ve written up some findings on Rsync in terms of how it performs for typical Windows data, including Exchange Information Store backups and SQL database backups. You can see the results here:

Rsync presentation

[Note – this is actually an excerpt from my full presentation on Server 2008 backup, available here: http://www.backupassist.com/SBS/sbspresentation.html]

To illustrate the point, I created a 100MB uncompressible file (actually a TrueCrypt container). I then Rsynced it across to a data host. I then changed one byte of the file, somewhere in the middle, and then reran the Rsync job. These were the results:

Data backed up: 100.00 MB

Bandwidth efficiency report:
Files transferred: 0
Size of changed files: 0.00 B
Total bytes sent: 2.13 kB
Total bytes received: 26.00 B
Speedup factor: 47468.36

Fantastic – if one byte in the file changes, Rsync will only transmit the relevant changed block.

I then decided to insert 5 bytes at the start of the file. This means that the entire file is shifted right by 5 bytes. Normally, a block level incremental backup algorithm (like those found in drive imaging programs) will actually need to backup the entire file again because of the shift in data (every single block is affected). However, Rsync is correctly able to identify the shift. These are the results:

Data backed up: 100.00 MB

Bandwidth efficiency report:
Files transferred: 1
Size of changed files: 100.00 MB
Total bytes sent: 12.20 kB
Total bytes received: 70.05 kB
Speedup factor: 1245.06

So we see here that 70kB was received, and 12kB was sent. Most of this data would have been checksums to try to detect what part of the file changed. Still an outstanding result! 12kB sent compared to 100MB.

So overall, Rsync is quite simply a sound choice for bandwidth efficient block-level delta incremental data transfers.

We’ll be releasing our Rsync engine for Beta testing in the coming days. Stay tuned!