n8blog
distraction in action

Like my work? Check out HexaLex, my game for iPhone & iPod Touch. It's a crossword game like Scrabble, but played with hexagonal tiles. http://www.hexalex.com

Note: Since everybody is linking to this page as the canonical Backup Bouncer document, I should point out that the project page with the latest version is located here, and the source code and issue tracker are located here. Also, the Backup Bouncer category of the blog has all the posts related to BB.

Do you back up your files? Of course you do! (Right? Right???) But do your backups work? Really? Are you sure? Have you checked?

Doing backups in OS X has always been a bit nerve-wracking. We’ve got resource forks, Finder flags, and various other odd metadata that don’t exist on other platforms. With the advent of Tiger, Apple told us that cp, tar, and rsync would finally be aware of our metadata and preserve it properly. And there was much rejoicing.

But then someone (or something?) named maurits actually tested these tools (and quite a few others), and the results weren’t pretty. This was a valuable wake-up call for many in the Mac community, but maurits never got around to releasing his test suite so we could do our own testing. Backup Bouncer is here to remedy this omission.

Backup Bouncer’s job is to keep the ugly backup tools out of the club. It’s here to show you exactly how bad (or, dare I say it, good?) your backup tools are. The way you use it is simple. First you create a couple of volumes:

bbouncer create-vol Src
bbouncer create-vol Dst

Next you create the test files on the source volume:

bbouncer create /Volumes/Src

Now you use any old method you want to copy the files from /Volumes/Src to /Volumes/Dst. Once that’s done, you verify the copy and see how well (or poorly) it went. You can start by checking the “critical” properties — properties that you’ll almost always want preserved:

bbouncer verify -d -T critical /Volumes/Src /Volumes/Dst

Next you can expand the scope to “important” properties — properties that power-users may find important or may be critical in the future:

bbouncer verify -d -T important /Volumes/Src /Volumes/Dst

Finally, you can see how the tool fares on the entire test suite:

bbouncer verify -d /Volumes/Src /Volumes/Dst

As an example of how the results look, here are the results for ditto on the entire test suite:

Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... FAIL
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... FAIL
   Sub-test:       on directories ... FAIL
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL
   Sub-test:              on dirs ... FAIL
ok
Verifying:                 fifo ... FAIL
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL
   Sub-test:     lots of metadata ... FAIL
FAIL

Pretty bad, eh? On a more optimistic note, here are the results from Shirt Pocket Software’s SuperDuper!, one of the only tools to even come close to passing all tests:

Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:         finder-locks ... ok
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... ok
ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok
   Sub-test:              on dirs ... ok
ok
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok
ok

Very nice!

Backup Bouncer also has a built-in set of “copiers”, to automate testing of command-line tools like cp, rsync, tar, asr, pax, ditto, and so on. You can run these tests using the “bbouncer copy” command:

bbouncer copy -d /Volumes/Src /Volumes/Dst

You’ll get several pages of output, so I won’t show it all here. Finally, I’ve included an autopilot script that you can use to create some volumes, initialize the source files, run all the copiers, and verify the results.

You can get Backup Bouncer here.

Update: My ACL tests were broken. I’ve updated the output shown here to reflect the corrected tests.

  • Share/Bookmark

  Comments:

1. maurits replies:

wow.

I haven’t looked at the code yet, but already from the description I can say: kudos! This is exactly what has been missing all the time. It’s too unfortunate I just haven’t had the time to do this (suffice to say – maurits is not something, but a grad student…). I remember we had a short conversation about a year ago, in which I vagely promised a test suite, but never followed suite. All the better that you’ve taken up the initiative yourself!

Not only have I had tons of (justified) requests to release a test suite over the past year, but also tons of (not less justified) requests to test about every single piece of backup software there is. An impossible task. But now, I hope people take your test suite, apply it to their pet software, and make the success (or failure?) of all the software out there more transparent. I have hope that the state of metatada conservation will improve.

Now, apart from all those more or less esoteric software packages out there, there are two things that would interest me, and probably others as well: does your test suite confirm my findings about the core command-line tools like cp, ditto, rsync, …? Also, how does the new rsync from 10.4.9 do?

Thanks again!

2. n8 replies:

Hey maurits! What a pleasure to have you as the first commenter on this topic. And you even posted at pi-time…

Being a grad student myself I can understand the time constraints. You may find that you have more free time if you simply abandon all hope of graduating. It’s done wonders for me! ;)

The obvious answer to your question about the core tools is “download Backup Bouncer and try it yourself!” :) But I’ll be a bit nicer and say that yes, my suite confirms your results for all the core utilities. I didn’t know there was a new rsync in 10.4.9, but if there is then it’s no better than the old one. It still barfs up a bunch of “File has vanished” errors for non-existent ._ files, then fails to copy almost all metadata. Here’s the test suite output, apologizing in advance for the inevitable formatting problems:

—————— rsync-apple ——————
This copier exited with error code 24
This copier produced log output in:
/Volumes/Dst/10-rsync-apple/log
Verifying: basic-permissions … ok
Verifying: timestamps …
Sub-test: modification time … ok
ok
Verifying: symlinks … ok
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok
Verifying: resource-forks … FAIL
Verifying: finder-flags … FAIL
Verifying: finder-locks … FAIL
Verifying: creation-date … FAIL
Verifying: bsd-flags … FAIL
Verifying: extended-attrs …
Sub-test: on files … FAIL
Sub-test: on directories … FAIL
Sub-test: on symlinks … FAIL
FAIL
Verifying: access-control-lists …
Sub-test: on files … ok
Sub-test: on dirs … ok
Sub-test: on symlinks … ok
ok
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … FAIL
Sub-test: lots of metadata … FAIL
FAIL

[...] However, the following could be a true breakthrough contribution. Testing for metadata conservation has, up to today, still been somewhat of a voodoo skill, and I’m not surprised that to the best of my knowledge only one person seems to have taken a serious shot at it after my posts. But now, this gaping hole has been filled by Nathaniel Gray with a fully automated test suite for metadata conservation. This could possibly make metadata conservation testing feasible for a wide audience. [...]

4. n8 replies:

BTW, Backup Bouncer writes a “meta” file for each test containing various version strings to help describe the test. Here’s the output for the version of rsync described above:

Darwin golux.localdomain 8.9.0 Darwin Kernel Version 8.9.0: Thu Feb 22 20:54:07 PST 2007; root:xnu-792.17.14~1/RELEASE_PPC Power Macintosh powerpc

copiers.d/10-rsync-apple.cp

rsync version 2.6.3 protocol version 28
Copyright (C) 1996-2004 by Andrew Tridgell and others

Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
inplace, IPv6, 32-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.

command = sudo /usr/bin/rsync -aH -E –rsync-path=/usr/bin/rsync src/ dst

I’m not sure how unique the rsync version string is. Perhaps I should include the md5 sum of the executable as well…

5. maurits replies:

it’s my pleasure – you made me write the first blog post in a year ;-)

re: rsync, that’s unfortunate. I did a diff of the sources, and the changes Apple did in 10.4.9 are really marginal. However, without spending too much time, I couldn’t find what exactly they have fixed. Not enough it seems… Problem is, the rsync situation is not completely black & white, the failure scenarios seem to be pretty subtle.

Regarding the state of rsync, I have been in contact with a senior staff member at Apple. I was asked me to file as many bugs on rsync as I could and bring them to his personal attention. In case you can afford to shift graduation a bit further out, I’m sure he’ll appreciate if you send a couple well-documented radars his way. I simply haven’t found the time yet to file metadata bugs beyond the handful I’ve already filed. Not graduating is no longer an option here ;-)

One more minor issue: when I did the testing, I sometimes observed some of the BSD flags to change seemingly spontaneously, without me doing much to the files. I am clueless what caused this, never did an in-depth analysis – after all, their importance is, um, limited. Forgot which flag exactly behaved erratically. Just watch out when you interpret your results that you don’t catch spurious changes that the copying tool is not to blame for.

6. maurits replies:

re: uniqueness of rsync: That’s tricky. In 10.4.9, they didn’t change a thing in the version number. Even the file size stayed the same. Only md5 changed; but I’m not completely sure if or if not prebinding destroys unique identifiability of binaries. To cross-check: on a PPC system, I get md5=e66b789a5e93e43863d575911e2ec862 for 10.4.8’s rsync and md5=b8c9bbff40b1620d7c81adeb13818b75 for 10.4.9.

7. n8 replies:

re: uniqueness of rsync: Oh blah, I forgot about prebinding. Yeah, my md5 doesn’t match your PPC one. I have a vague recollection of a tool that could hash executables while avoiding this problem but no idea what it was.

re: changing BSD flags: One thing that wasn’t 100% clear from your articles is that the Finder “locked” flag is, in fact, implemented using the BSD “uchg” flag. If you didn’t know these were the same you may have thought things were changing behind your back.

re: filing rdars: I’m not too keen on filing individual bug reports for failures. I’d rather just file one bug report, “get your tools to pass the ‘important’ tests in this suite.” Maybe that’s what I’ll do. If you can drop a line to your Apple contact and point him here that might be enough. I intend to post an announcement to darwin-dev as well.

8. maurits replies:

re: BSD flags. Yeah, I was probably not 100% precise in that respect. But I was of course well aware that Apple seems to have a 1:1 mapping between Finder locked flag and the uchg flag. But it wasn’t this one that changed; uchg was rock-stable IIRC. It was one of the more esoteric ones.

re: radars: My Apple contact was indeed very keen on getting a test suite. So I’m sure they’ll appreciate your work. I’ve already filed #5168240 with a pointer to your post. But what they like most, of course, is one bug for every single problem, with complete failure scenario analysis. That’s a pain to file, of course.

9. Chris replies:

One tool that will give you what it calls a “checksum” of a Mach-O file correctly ignoring prebinding info, is ctool. MacPorts has an entry (and a cached copy of the tarball), and references its dead website.

Presumably you could hack up the code to give you a more robust hash if you needed…

10. Chris replies:

The ctool sources say it uses md5 or sha1, not just a “checksum”.

11. Joe Simkins replies:

Thanks. I have wanted a suite to work on multi disk backups. Is there any way to increase the size of your files to where I could force the backup to occupy more than one media? Thanks again.

12. n8 replies:

chris: Thanks! ctool was what I was thinking of.

joe: BB is primarily a tool to test for preservation of metadata, so we don’t currently worry about file size. But you can easily add a test that creates one or more large files (using dd, for example).

13. ershler replies:

For anyone who is interested, here is how the backup program NetVault (running under OS X 10.4.9) from BakBone software holds up to the acid tests.

[admin@sandbox]:backup-bouncer-0.1.1> ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying: basic-permissions … ok
Verifying: timestamps …
Sub-test: modification time … ok
ok
Verifying: symlinks … ok
Verifying: symlink-ownership … FAIL
Verifying: hardlinks … ok
Verifying: resource-forks … ok
Verifying: finder-flags … ok
Verifying: finder-locks … FAIL
Verifying: creation-date … FAIL
Verifying: bsd-flags … FAIL
Verifying: extended-attrs …
Sub-test: on files … ok
Sub-test: on directories … ok
Sub-test: on symlinks … FAIL
FAIL
Verifying: access-control-lists …
Sub-test: on files … ok
Sub-test: on dirs … ok
ok
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok
ok

14. Jeffrey J. Hoover replies:

Lovely. A test to better define the problem.

Now to get solutions! :-)

Seriously, kudos to you and maurtis and others invovled in this. I’ve been a Mac Sysadmin for quite a while now and backups have *never* been fun.

15. Histrionic replies:

I don’t believe that rsync gets prebound — and I don’t even think it’s subject to prebinding at all. I get this output for otool, which according to this ADC article indicates that it is not (there’s no timestamp for a prebound executable).

$ otool -hv /usr/bin/rsync
/usr/bin/rsync:
Mach header
magic cputype cpusubtype filetype ncmds sizeofcmds flags
MH_MAGIC I386 ALL EXECUTE 13 1480 NOUNDEFS DYLDLINK TWOLEVEL SUBSECTIONS_VIA_SYMBOLS

Less gets prebound in Tiger than earlier releases, anyway: “In Mac OS X v10.4, dyld was improved in a way that eliminated the need for prebinding information in most situations. The system libraries are now the only executables that are still prebound, and they are prebound in a way that optimizes performance on the target system. Because of the improved prebinding of the system libraries, applications and third-party libraries no longer need to be prebound. A side benefit to this new behavior is that applications now have more usable address space than in previous versions of the operating system.”

16. Histrionic replies:

Also, looking at Radmind transcripts for PowerPC-based Mac systems running Tiger — which essentially allows me to see the history of file changes, including when they happened and what their sha1 checksums were at the time of the change — I have seen updates for /usr/bin/rsync in the following timeframes (minor updates or security/other updates that followed them but were released before the next minor rollup update):

10.4.9
10.4.8
10.4.6
10.4.5 (two updates)
10.4.0

17. mark_usha replies:

Hello everybody,

In need of some rsync-style solution (backup over network/ssh)
that can handle the creation date correctly I ‘ve discovered RsyncX. It’s old, but beside of extended attributes and ACL support it seems to be better than Tiger’s rsync:

banana:/Users/Shared/backup-bouncer-0.1.1 admin$ sudo /usr/local/bin/rsync -v -a -H --eahfs /Volumes/source/ /Volumes/target

banana:/Users/Shared/backup-bouncer-0.1.1 admin$ ./bbouncer verify /Volumes/source /Volumes/target

Verifying: basic-permissions ... ok
Verifying: timestamps ... ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... FAIL # symlink's ownership is set to the target's ownership
Verifying: hardlinks ... ok # great, option "-aH --eahfs" works !
Verifying: resource-forks ... ok
Verifying: finder-flags ... FAIL # the "busy"-flag (Z -> z) is lost.
Verifying: finder-locks ... ok
Verifying: creation-date ... ok # !!!!!! good !!! how did they do that ??
Verifying: bsd-flags ... FAIL # fails, but just a single flag (the opaque-flag) is lost
Verifying: extended-attrs ... FAIL # was to be expected ...
Verifying: access-control-lists ... FAIL #
Verifying: fifo ... ok
Verifying: devices ... FAIL #
Verifying: combo-tests ... FAIL # ... known limitations, RsyncX was designed for 10.3,

One more thing to consider: as stated by maurits, rsync of RsyncX doesn’t copy files of a folder whose bsd-flag “uappnd” is set. The folder itself gets copied, but than rsync doesn’t copy anything in that folder to the target volume.

18. blue replies:

I find it absolutely amazing that the world has shown callous disregard for integrity of user data for so long. Great work maurits and n8. Now I know for sure why I have never been happy with backup. Now all we need is a bit of media attention and the problem will be fixed….

19. blue replies:

It looks like asr could be the most reliable backup/restore code. Would you consider adding it as part of the standard suite?

20. blue replies:

Ooops – well previous version of asr was reliable. I wondr how apple managed to break it?

[...] After many hours and much fiddling trying to get backup happening. Rsync Lart 2.6.6 Darwinports version finally works for me on Tiger. rsync-lart version 2.6.6 for Mac OS X Not sure how good it is with resource forks but at least it can do a backup without the interminable broken pipe error. Well as good as it can….see Backup Bouncer for independent testing of capability: n8gray.org: Introducing Backup-Bouncer [...]

22. blue replies:

After many hours and much fiddling trying to get backup happening. Rsync Lart 2.6.6 Darwinports version finally works for me on Tiger – well at least without bugs.

http://rsync-lart.darwinports.com/

$ ./bbouncer verify -d /Volumes/Src /Volumes/Dst/13-rsync-lart
Verifying: basic-permissions … ok
Verifying: timestamps …
Sub-test: modification time … ok
ok
Verifying: symlinks … ok
Verifying: symlink-ownership … ok
Verifying: hardlinks … FAIL
Verifying: resource-forks … ok
Verifying: finder-flags … ok
Verifying: finder-locks … FAIL
Verifying: creation-date … FAIL
Verifying: bsd-flags … FAIL
Verifying: extended-attrs …
Sub-test: on files … ok
Sub-test: on directories … ok
Sub-test: on symlinks … FAIL
FAIL
Verifying: access-control-lists …
Sub-test: on files … ok
Sub-test: on dirs … ok
ok
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok
ok

23. blue replies:

rsync OK on hard links with -H switch.

24. blue replies:

If rsync keeps you happy, a gui front end is available in early version.

http://arrsync.sourceforge.net/

Presently needs some tweaks to the rsync argument defaults in the source code. Doesn’t have root permission – so fails fifo and devices, otherwise can do as well as rsync on local volumes.

25. Rik replies:

I’ve made some tests to DAR – “Disk Archive” (http://dar.linux.free.fr/). I’m sure you will find it quite interesting. It’s a pitty it passed everything except the last one. I can post the “dar” call and configuration files if you want to reproduce the test.

Well, the results:


Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... ok
Verifying: finder-flags ... ok
Verifying: finder-locks ... ok
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
ok
Verifying: fifo ... ok
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL
FAIL

26. Rik replies:

I’m sorry. The results above are wrong.

These are the good ones. I’ts a pitty though:


Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... ok
Verifying: finder-flags ... ok
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... FAIL
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... FAIL
Sub-test: on dirs ... FAIL
FAIL
Verifying: fifo ... FAIL
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL
FAIL

I think DAR isn’t good enough :-(

27. Kreisquadratur replies:

So which tool atm should I use to copy a time machine backup disk? My problem is that the destination drive is of course bigger but there are already data on it. So I cannot use the yet best disk utility build in.

And the next curious thing is, that man ditto says, from 10.5 on, it supports lots of meta data. But my backup copy did that 12 hours and I came from 140 gb to 280gb (but only the disk was full then).

28. Chris Ridd replies:

I used backup-bouncer-0.12 to test Prosoft’s DataBackup 3.0.4, as it is said to be Leopard compatible. Once I’d figured out that bbouncer wasn’t smart enough to find xattr-util, I got these results:


Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... FAIL
Verifying: resource-forks ... ok
Verifying: finder-flags ... FAIL
Verifying: finder-locks ... ok
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... FAIL
Sub-test: on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
ok
Verifying: fifo ... FAIL
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok
ok

29. Histrionic replies:

For reference, Leopard includes xar but comes with version 1.4. That’s not the version that is supposed to pass all the tests, and xar 1.5.1 (from June 10) was the most current version at the time Leopard actually shipped (October 26).

30. lst replies:

Anyone have a suggestion on what is the best tool for metadata backup in 10.4.11 or 10.5.1 so far?

31. johannes replies:

First of all: Thanks maurits for the initial idea and n8 for putting together this sweet little suite.

This morning I ran through the current version of CCC and SuperDuper! on my 10.5.2 with the following results:

CCC 3.0.1 – normal and block:

Verifying: basic-permissions … FAIL
Verifying: timestamps …
Sub-test: modification time … ok
ok
Verifying: symlinks … ok
Verifying: symlink-ownership … FAIL
Verifying: hardlinks … ok
Verifying: resource-forks … ok
Verifying: finder-flags … ok
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok
Sub-test: on directories … ok
Sub-test: on symlinks … ok
ok
Verifying: access-control-lists …
Sub-test: on files … ok
Sub-test: on dirs … ok
ok
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … FAIL
FAIL

SuperDuper 2.5:

Verifying: basic-permissions … ok
Verifying: timestamps …
Sub-test: modification time … ok
ok
Verifying: symlinks … ok
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok
Verifying: resource-forks … ok
Verifying: finder-flags … ok
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok
Sub-test: on directories … ok
Sub-test: on symlinks … ok
ok
Verifying: access-control-lists …
Sub-test: on files … ok
Sub-test: on dirs … ok
ok
Verifying: fifo … FAIL
Verifying: devices … FAIL
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok
ok

32. tms replies:

ACLs test does not include inherited ACLs which fail on many backup/restore operations.

34. Adrian replies:

How do you test a specific Backup Program like CarbonCopyCloner? Do you have to make a backup using CCC first?

I ran the ./autopilot and got the results from that. But I wasn’t sure how to run the test for a specific program.

Thanks for your help

35. n8 replies:

Just use your tool to copy from bbouncer’s /Volumes/Src to a fresh destination /Volumes/Dst and then run bbouncer verify -d /Volumes/Src /Volumes/Dst (the -d is for detailed results).

36. daisy replies:

Can I use your tool in a Windows environment?

37. n8 replies:

@daisy: Do you mean you want to verify OS X metadata on a backup, but from a windows environment? I’m assuming you understand that BB is entirely devoted to Mac OS file system metadata preservation, right?

38. daisy replies:

Thanks for your kind reply.

No, I don’t have any Apple products at all, but I thought maybe your tool could help me.

I (obviously) don’t understand metadata very well. So you are talking only about preserving metadata related to/added by the system itself and not metadata attached, say, to photos via the camera, or metadata added in Photoshop by the user? The latter is what I wanted to verify after a duplication with Retrospect from an NTSF disk to a FAT32 disk.

I’m assuming now this is not what your tool is for and it couldn’t be used on a Windows environment anyway. Please excuse my inexperience and apologies for taking your time. Thanks again.

39. n8 replies:

No problem. I’m not a windows user but I’d assume the metadata you’re talking about should be safe under any backup tool. Photo metadata generally lives inside the file itself, so as long as the file has been copied intact all the metadata should be safe. You *do* have to watch out for photo editors — they’ll sometimes drop metadata from files. (I’m big on photo metadata as well as filesystem metadata, by the way… :) )

40. daisy replies:

Thanks for the tips and reassurance!

I hope metadata preservation will soon be taken more seriously by all applications and backup solutions. The software should IMO clearly specify upfront what happens to the various metadata during any given operation.

We need more tools like yours! All the best in your endeavors.

41. Zemran replies:

Thanks a lot for this. I recently had to restore from a Time Machine backup and found that it was total rubbish. It took me a couple of days to update and fix the problems with the data that Time Machine missed. I still find it hard to accept how much TM failed to back up although I accept it did save my data, just not the system. It restores the system to the original install (from the install disk) state before updates and I, being a *nix fan, had also made a lot of changes to make my system more like *nix. I had to run all the system updates all over again and download and install all the *nix stuff. I want to be able to run a restore that actually restores my system to a running system. It even trashed Aperture which is an Apple programme… I had to reinstall that from the original disk and update it again, but as I said, all my data was safe.

I am now deciding on which ‘real’ backup solution to use in the future and this tool will be great in evaluating which is the best one to stick with.

Thanks a lot.

42. johannes replies:

Could anyone please perform a test of SuperDuper (latest version)?

I tested them all bbounces tells me, that ALL of them made NO mistake at all, which can’t be true, because my SuperDuper-version hasnt changed a bit since my last test. Something went wrong but I have no clou where to look.

43. toubabou replies:

FWIW I decided to try a little control group test.

bbouncer create-vol Src
bbouncer create-vol Dst
bbouncer create /Volumes/Src
bbouncer create /Volumes/Dst

Now, when I compare these two, I should get identical files, because they have been created the same way. Not so.

sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... ok
Verifying: finder-flags ... FAIL
Verifying: finder-locks ... ok
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
ok
Verifying: fifo ... ok
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL
FAIL

I also made a Finder copy of the Src disk image, mounted it as Dst and got :
sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying: basic-permissions ... FAIL
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... FAIL
Verifying: hardlinks ... ok
Verifying: resource-forks ... ok
Verifying: finder-flags ... ok
Verifying: finder-locks ... ok
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
ok
Verifying: fifo ... ok
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL
FAIL

G4 FW800 Mirrored drive doors, 1GHz, 1GB RAM Mac OS X Tiger 10.4.11

44. n8 replies:

@toubabou: Good to see you’re paying attention, but there’s nothing to worry about here. In the first case, the two volumes were not created in the same way for the simple reason that they weren’t created at the same time. I generally tried to ensure that the BB tests would not be time-dependent, but it turns out the finder flags test is (modification time is one of the finder flags). So the files you created had different finder flags and BB correctly reported that. (That’s the same reason for the “lots of metadata” failure, btw.) I’ll change the finder flags test to be time-independent, but this isn’t an issue that would cause problems verifying backups made with a backup tool.

In the second case, the problem is caused by the way you mounted the Dst volume. When mounting disk images, the Finder by default treats all files as being owned by you. This is generally a sensible thing to do, but not in this case. BB creates files that aren’t owned by you and checks to see that they stay that way. If you want to mount a volume preserving file ownership you need to do it from Terminal: hdiutil attach -owners on Foo.sparseimage Perhaps I should add a bbouncer mount-vol command for this…

Thanks for the feedback.

45. n8 replies:

@johannes: I’m sorry, I’m afraid I don’t understand the problem you’re having. Can you try to explain it once more?

46. johannes replies:

Hey n8,

the problem was, that I checked the before and after version using SuperDuper and bbouncer told me that EVERY metadata was kept. When I checked the programme two month ago, I had another result.

But somehow I managed to convince bbouncer to cast a more critical look.

The results of a little collection of backup-tools can be found here:

http://forum.macsofa.net/viewtopic.php?t=30508

The thread is in German, but I’m quite sure you’ll figure out the important points… :-)

47. blue replies:

Latest patched version of rsync is perfect – see:
http://www.bombich.com/mactips/rsync.html

Bombich reports a perfect score with BackupBouncer! Mission accomplished???

48. blue replies:

rsync3.0.2 test on OS 10.5.3:

$ sudo rsync -aNHAXx –protect-args –fileflags –force-change /Volumes/Src/* /Volumes/Dst/

$ ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying: basic-permissions … ok
Verifying: timestamps …
Sub-test: modification time … ok
ok
Verifying: symlinks … ok
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok
Verifying: resource-forks … ok
Verifying: finder-flags … ok
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok
Sub-test: on directories … ok
Sub-test: on symlinks … ok
ok
Verifying: access-control-lists …
Sub-test: on files … ok
Sub-test: on dirs … ok
ok
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok
ok

49. n8 replies:

You might want to try again with the latest version of BB. I added a new test that’s caused problems for Apple’s rsync. It should also be more clear which tests are really important and which are not.

50. blue replies:

Latest version test – still looks fine (rsync 3.02 patched on OS 10.5.3):
backup-bouncer-0.1.3 $ sudo rsync -aNHAXx –protect-args –fileflags –force-change /Volumes/Src/* /Volumes/Dst/14-rsync302
backup-bouncer-0.1.3 $ sudo ./bbouncer verify /Volumes/Src /Volumes/Dst/14-rsync302
Verifying: basic-permissions … ok (Critical)
Verifying: timestamps … ok (Critical)
Verifying: symlinks … ok (Critical)
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok (Important)
Verifying: resource-forks … ok (Critical)
Verifying: finder-flags … ok (Critical)
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs … ok (Important)
Verifying: access-control-lists … ok (Important)
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests … ok

51. blue replies:

(Sorry – forgot to run detailed test. Still a perfect score.)

backup-bouncer-0.1.3 $ sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst/14-rsync302
Password:
Verifying: basic-permissions … ok (Critical)
Verifying: timestamps … ok (Critical)
Verifying: symlinks … ok (Critical)
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok (Important)
Verifying: resource-forks …
Sub-test: on files … ok (Critical)
Sub-test: on hardlinked files … ok (Important)
Verifying: finder-flags … ok (Critical)
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok (Important)
Sub-test: on directories … ok (Important)
Sub-test: on symlinks … ok
Verifying: access-control-lists …
Sub-test: on files … ok (Important)
Sub-test: on dirs … ok (Important)
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok

52. blue replies:

Bombich http://www.bombich.com/mactips/rsync.html recommends:
rsync -aNHAX –fileflags –force-change

Without root permissions (using sudo), device copy fails. All other bb tests pass.

53. blue replies:

Lastest version of rsync 3.0.3 with patches for fileflags and crtimes works just the same. All bb tests pass.

(Follow instructions at http://www.bombich.com/mactips/rsync.html and change 3.0.2 to 3.0.3.)

54. RobK replies:

I don’t think your test suite tests for this common problem that exists in many Mac OS X backup software. It existed in CCC until the recent version 3.1.1 got released a few days ago.

See http://forums.bombich.com/viewtopic.php?t=12013&highlight=

If the file is locked and the backup programs copies the locked flag too early from the source file to the backup file, the backup programs will be unable to copy ANY other metadata such as extended attributes.

To get around this limitation in Mac OS X, the backup programs MUST copy ALL other metadata first and then copy the LOCKED flag LAST.

If the destination file is locked TOO early, other metadata such as extended attributes and possibly other BSD flags will NOT be copied to the destination file.

I do hope you test for this condition.

On another note, you should note that Mac OS X makes it impossible to make a perfect clone of a disk drive. One cannot clone of copy the CREATION dates of symlinks. Apple does not provide an API to do so.

RobK

55. MrWizard replies:

On my Leopard machine, the bsd-flags test is always succeeding. I think you need to use ls -d -l -O instead of ls -lo. At least then it’s conclusive.

Rsync 3.0.3 plus fileflags plus crtime is quite a good thing!

56. n8 replies:

MrWizard: Good catch! This was a really sneaky bug. Notice the difference in the output of ls -lo under sudo:

[n8gray@golux]% ls -lo README
-rw-r--r--  1 n8gray  staff  - 5020 Jun  9 12:49 README
[n8gray@golux]% sudo ls -lo README
-rw-r--r--  1 n8gray  5020 Jun  9 12:49 README

The “-” in the regular user output shows the BSD flags. So my test works as a regular user but not as root. Changing the command to ls -ldO fixes the problem. Thanks!

57. n8 replies:

RobK: You’ve got a very good point WRT locking. I changed the “lots of metadata” test so that the file is locked as well, and indeed, it does change at least one result. The version of xar I’ve got (1.6dev) fails in this case despite passing all other tests. I didn’t check the before and after results on the other copiers so I’m not sure if it changed them.

Unfortunately, the “lots of metadata” test can fail for lots of reasons — a copier can fail because it totally fails to preserve one or many metadata items, or it can also fail because it locks the destination file too early. I’m not sure how to communicate this result to the user in a useful way.

As for the inability to do a perfect duplication, it’s interesting, but not alarming IMHO. I wouldn’t expect the user or the system to care about the creation date of symlinks. I’ll consider adding a test for it just for the sake of completeness.

Thanks for the interesting feedback!

[...] haven’t tried it yet, but Backup-Bouncer looks like a very useful tool for verifying backup methods (it doesn’t actually verify [...]

[...] tools in order to find something more effective than Time Machine to use for my daily backup. Backup-Bouncer is an amazing test suit created by Nate Gray for the sole purpose of checking how well backup and [...]

[...] almost a year old, here’s a tool you can use to run your own tests on whatever you’d like, today – http://www.n8gray.org/blog/2007/04/2…ackup-bouncer/ Billy [...]

[...] Super Duper. It does complete images for free. And, here’s something else you may find useful – http://www.n8gray.org/blog/2007/04/2…ackup-bouncer/ Finally, there are times (not many) when Apple is a little too helpful. One is when the boot [...]

62. GuestReader replies:

Hi there.
Every time I run a test with “./bbouncer verify -d” I get some errors “xattr-util: command not found”.
Do you habe any hints for me? Many thanks!!!

63. n8 replies:

It sounds like you forgot to run “make” before running bbouncer. Does that fix it?

64. GuestReader replies:

*sigh* One should read the “ReadMe” (guess why it’s called so) and not only this website.
Thanks for the fast answer! And also thanks for this great tool!!!

[...] way I use a USB flash drive to keep my files synchronized between work and home laptops. You can get started from this post. Still, I hadn’t tested rsync because my process worked as far as I could tell. (Then I ran [...]

66. Philipp replies:

Hi n8, hi everybody,

is there already some knowledge if these test cases are sufficient for testing the reliability of backup tools for use under Snow Leopard? For example, there are compressed executables now [1]. Does this fact or any other potential changes need special attention?

Any feedback appreciated.

[1] http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/3

67. Simon replies:

Hi n8,

is there a version witch runs on 10.6 aka snow leopard?
I want to test cp, the new finder, rsync (built-in version), rsync (with patches)

When i run make i get the following errors:

computername:backup-bouncer-0.1.3 username$ make
cd util && make
cc -Wall xattr-util.c -o xattr-util
xattr-util.c: In function ‘main’:
xattr-util.c:107: warning: unused variable ‘value’
cc -Wall hardlink-util.c -o hardlink-util
(cd ctool-1.2.3; make)
cc -c safe_string.c
cc -c ctool.c
ctool.c: In function ‘process_file’:
ctool.c:412: warning: format ‘%d’ expects type ‘int’, but argument 3 has type ‘long unsigned int’
ctool.c: In function ‘checksum_section’:
ctool.c:524: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 5 has type ‘uint32_t’
ctool.c:524: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 6 has type ‘uint32_t’
ctool.c: In function ‘print_stats’:
ctool.c:692: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘dev_t’
ctool.c:693: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘__darwin_ino64_t’
ctool.c:698: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 4 has type ‘int’
ctool.c:699: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘int’
ctool.c:719: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘uid_t’
ctool.c:720: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘gid_t’
ctool.c:722: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘dev_t’
ctool.c:753: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘blksize_t’
ctool.c:754: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘__uint32_t’
ctool.c:755: warning: format ‘%lu’ expects type ‘long unsigned int’, but argument 3 has type ‘__uint32_t’
ctool.c:784:52: error: macro “curl_easy_setopt” requires 3 arguments, but only 2 given
make[2]: *** [ctool.o] Error 1
make[1]: *** [all] Error 2
make: *** [all] Error 2

68. n8 replies:

Hi guys,

I don’t have a Snow Leopard machine yet but I will within the next week or so. Once I have that set up I’ll update BB right away.

69. n8 replies:

I’ve released BB 0.2.0 that compiles on SL. Please read the README.SNOW_LEOPARD.txt file for details on a problem with Apple’s rsync in SL.

70. n8 replies:

@Philipp: I’m looking into tools for testing HFS compression right now.

[...] of you have probably noticed that I’ve been neglecting some of my other projects and complaining about being busy a lot lately. Well here’s what I’ve been working on. [...]

72. Jaime Gago replies:

Just tested rsync 3.0.7 (patched) on Os X Server 10.5.8 and 10.6.2 with Backup Bouncer 0.2.0 all good.
Great tool.

RTFM!


10.5.8 Server output

./bbouncer verify -d /Volumes/Src/ /Volumes/Dst/
Verifying: basic-permissions … ok (Critical)
Verifying: timestamps … ok (Critical)
Verifying: symlinks … ok (Critical)
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok (Important)
Verifying: resource-forks …
Sub-test: on files … ok (Critical)
Sub-test: on hardlinked files … ok (Important)
Verifying: finder-flags … ok (Critical)
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok (Important)
Sub-test: on directories … ok (Important)
Sub-test: on symlinks … ok
Verifying: access-control-lists …
Sub-test: on files … ok (Important)
Sub-test: on dirs … ok (Important)
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok

10.6.2 Output

./bbouncer verify -d /Volumes/src /Volumes/dst/
Verifying: basic-permissions … ok (Critical)
Verifying: timestamps … ok (Critical)
Verifying: symlinks … ok (Critical)
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok (Important)
Verifying: resource-forks …
Sub-test: on files … ok (Critical)
Sub-test: on hardlinked files … ok (Important)
Verifying: finder-flags … ok (Critical)
Verifying: finder-locks … ok
Verifying: creation-date … FAIL
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok (Important)
Sub-test: on directories … ok (Important)
Sub-test: on symlinks … ok
Verifying: access-control-lists …
Sub-test: on files … ok (Important)
Sub-test: on dirs … ok (Important)
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok

[...] test whether a backup product can restore Mac files with all their metadata is a test suite called Backup Bouncer. It’s a brilliant open source tool, available at [...]

75. Stefan Reitshamer replies:

Arq is an online backup product for the Mac that passes every Backup Bouncer test, I’m proud to say.

Thanks n8 for writing Backup Bouncer! It really helped to make Arq backup and restore things correctly.

[...] product can restore Mac files with all their metadata is an excellent open-source test suite called Backup Bouncer. It tests whether the backup product faithfully restored all the data and metadata of your [...]

77. Backup Review – Arq replies:

[...] The most important fact, bar none, is Arq’s ability to preserve backups in a 1:1 manner. Arq, along with Jungle Disk, passed all 20 tests put out by Backup-Bouncer: [...]

[...] Fortunately, Nathan Gray, an independent researcher at Caltech, has created a tool called Backup Bouncer that tests every type of Mac metadata. I’ve run the Backup Bouncer tests through Mozy, [...]

79. Ceriel Jacobs replies:

Thanks for this nice tool. Any plans in merging Carbon Copy Cloner’s improvements to Backup Bouncer?

80. Ceriel Jacobs replies:

Ahsay One Click Backup Manager 5.5.7.0 results:

Backup progress hangs (2+ minutes) on “Uploading new directory …: /Volumes/Src/90-fifo (100%)”.

Cancelling the operation by pressing the “Cancel” button, doesn’t change the “hang” for another 2 minutes. Can’t close the Backup window either. Stopping the application also doesn’t respond. So finally a “Force quit” is issued.

As a result: nothing is back-up.
Now excluding folder 90-fifo.

This backup results in two errors:
1. /95-devices/devvn0 Cannot open Data…
2. /95-devices/devzero Cannot open Dat…

Restoring onto the disk is not possible, need to first create a folder on the Dst disk and select that, before being able to restore with Ahsay. I place the mark in the box to restore file permissions.

Now the results:
$ sudo ./bbouncer verify -d -T critical /Volumes/Src /Volumes/Dst/Src/Src
Password:
Verifying: basic-permissions … FAIL
Verifying: timestamps … FAIL
Verifying: symlinks … stat: ./symlink1: stat: No such file or directory
FAIL
Verifying: resource-forks …
Sub-test: on files … ok
Sub-test: on hardlinked files … FAIL
Verifying: finder-flags … FAIL
Test dir ‘/Volumes/Dst/Src/Src/90-fifo’ does not exist

And the full results:
$ sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst/Src/Src
Verifying: basic-permissions … FAIL
Verifying: timestamps … FAIL
Verifying: symlinks … stat: ./symlink1: stat: No such file or directory
FAIL
Verifying: symlink-ownership … FAIL
Verifying: hardlinks … FAIL
Verifying: resource-forks …
Sub-test: on files … ok
Sub-test: on hardlinked files … FAIL
Verifying: finder-flags … FAIL
Verifying: finder-locks … FAIL
Verifying: creation-date … FAIL
Verifying: bsd-flags … FAIL
Verifying: extended-attrs …
Sub-test: [ on files ] … FAIL
Sub-test: creation time … ok
Sub-test: modification time … ok
Sub-test: [ on locked files ] … FAIL
Sub-test: creation time … ok
Sub-test: modification time … ok
Sub-test: [ on directories ] … FAIL
Sub-test: creation time … FAIL
Sub-test: modification time … FAIL
Sub-test: [ on symlinks ] … FAIL
GetFileInfo: could not refer to file (-43)
Sub-test: creation time … FAIL
Verifying: hfs-compression …
Sub-test: decmpfs xattr … not preserved
Sub-test: UF_COMPRESSED flag … not set
Sub-test: file contents … match
Sub-test: creation time … ok
Sub-test: modification time … ok
Sub-test: hard link inode … ok
Sub-test: hard link decmpfs xattr … not preserved
Sub-test: hard link UF_COMPRESSED flag … not set
Sub-test: hard link modification time … ok
ok
Verifying: hfs-compression_large …
Sub-test: decmpfs xattr … not preserved
Sub-test: UF_COMPRESSED flag … not set
Sub-test: file contents … match
Sub-test: creation time … ok
Sub-test: modification time … ok
Sub-test: hard link inode … ok
Sub-test: hard link decmpfs xattr … not preserved
Sub-test: hard link UF_COMPRESSED flag … not set
Sub-test: hard link modification time … ok
ok
Verifying: access-control-lists …
Sub-test: on files … FAIL
Sub-test: on dirs … FAIL
Sub-test: on locked files … FAIL
Sub-test: on non-inherited acls … ok
Sub-test: on inherited acls … FAIL
Test dir ‘/Volumes/Dst/Src/Src/90-fifo’ does not exist
Verifying: devices … FAIL
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … FAIL
Sub-test: lots of metadata … FAIL

81. n8 replies:

@Ceriel: I’ve seen those modifications to backup bouncer but unfortunately they’re a mix of actual improvements and pointless changes. For example, the modifications remove the printing of test importance information, which is useful for end users. Still, I would definitely like to pull in the extra tests, and I plan to do so when I have time to work on BB again.

[...] die Bedeutung dieser Metadaten und zeigt auf, wie «Arq»-Alternativen beim Vergleichen mittels Backup-Bouncer in dieser Hinsicht [...]

83. JS replies:

Running OS X v10.6.4, all up-to-date updates etc., I find *nothing* (no utility) manages to pass the ‘creation-date’ test, including stalwarts like cp, tar, rsync (v3.0.7 Apple or patched al. la. Bombich), ditto etc.
FWIW, here is an ./autopilot run result:

We're now going to create two disk images, create some files on one,
copy them to the other, and find out what metadata was preserved.

cd util && make
make[1]: Nothing to be done for `all'.
created: /Users/Shared/backup-bouncer-0.2.0/Src.sparseimage
/dev/disk2 GUID_partition_scheme
/dev/disk2s1 Apple_HFS /Volumes/Src
created: /Users/Shared/backup-bouncer-0.2.0/Dst.sparseimage
/dev/disk3 GUID_partition_scheme
/dev/disk3s1 Apple_HFS /Volumes/Dst
Cleaning: 00-basic-permissions
Cleaning: 05-timestamps
Cleaning: 10-symlinks
Cleaning: 15-symlink-ownership
Cleaning: 20-hardlinks
Cleaning: 30-resource-forks
Cleaning: 40-finder-flags
Cleaning: 45-finder-locks
Cleaning: 50-creation-date
Cleaning: 60-bsd-flags
Cleaning: 70-extended-attrs
Cleaning: 80-access-control-lists
Cleaning: 95-devices
Cleaning: 99-combo-tests
Creating: 00-basic-permissions ... ok
Creating: 05-timestamps ... ok
Creating: 10-symlinks ... ok
Creating: 15-symlink-ownership ... ok
Creating: 20-hardlinks ... ok
Creating: 30-resource-forks ... ok
Creating: 40-finder-flags ... ok
Creating: 45-finder-locks ... ok
Creating: 50-creation-date ... ok
Creating: 60-bsd-flags ... ok
Creating: 70-extended-attrs ... ok
Creating: 80-access-control-lists ... ok
Creating: 95-devices ... ok
Creating: 99-combo-tests ... ok
src = /Volumes/Src
dst = /Volumes/Dst
Enabling owners on src/dst disks
/dev/disk2s1 on /Volumes/Src (hfs, local, journaled)
/dev/disk3s1 on /Volumes/Dst (hfs, local, journaled)
Cleaning.........
Copying with: rsync-apple ... ok
Copying with: rsync-qdolan ... Skipping: can't find needed files
ok
Copying with: rsync-local ... ok
Copying with: cp-apple ... ok
Copying with: ditto ... ok
Copying with: tar ... ok
Copying with: pax ... ok
Copying with: xar-apple ... ok
Copying with: xar-svn ... Skipping: can't find needed files
ok

------------------ rsync-apple ------------------
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... FAIL (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... FAIL
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok

------------------ rsync-qdolan ------------------
This copier was skipped

------------------ rsync-local ------------------
This copier produced log output in:
/Volumes/Dst/25-rsync-local/log
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... ok (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... ok
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok

------------------ cp-apple ------------------
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... FAIL (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... FAIL (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... ok
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL

------------------ ditto ------------------
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... ok (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... ok
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok

------------------ tar ------------------
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... ok (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... ok
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL

------------------ pax ------------------
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... FAIL
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... ok (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... FAIL
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... FAIL
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok

------------------ xar-apple ------------------
Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... ok (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... ok
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... FAIL

------------------ xar-svn ------------------
This copier was skipped
Running BB v0.2.0 with a test added for rsync built to Mike Bombich’s instructions (al. la. http://www.bombich.com/mactips/rsync.html )

My only guess is – I’m in the UK, so times are conventionally written dd/mm/yy rather than the US-style mm/dd/yy as used by the SetFile and GetFileInfo developer tools – surely I’m not the only EC person to have noticed this problem? So I assume it’s something I’ve done to cause it – but what?

Any ideas/solutions gratefully accepted, as long as it doesn’t materially affect my real data :-)

84. n8 replies:

Hmm, I don’t know what could be causing that. You can look at tests.d/50-creation-date.test and see what it does. None of it should depend on your locale. I would recommend playing around with touch, SetFile, and GetFileInfo to see if you can isolate the problem.

85. blue replies:

Most likely, the copier from JS is not using the right rsync.

OS 10.6.4 and patched rsync 3.0.7 as per Bombich. Works fine – edited output of ./autopilot:

—————— rsync-3-patch ——————
Verifying: basic-permissions … ok (Critical)
Verifying: timestamps … ok (Critical)
Verifying: symlinks … ok (Critical)
Verifying: symlink-ownership … ok
Verifying: hardlinks … ok (Important)
Verifying: resource-forks …
Sub-test: on files … ok (Critical)
Sub-test: on hardlinked files … ok (Important)
Verifying: finder-flags … ok (Critical)
Verifying: finder-locks … ok
Verifying: creation-date … ok
Verifying: bsd-flags … ok
Verifying: extended-attrs …
Sub-test: on files … ok (Important)
Sub-test: on directories … ok (Important)
Sub-test: on symlinks … ok
Verifying: access-control-lists …
Sub-test: on files … ok (Important)
Sub-test: on dirs … ok (Important)
Verifying: fifo … ok
Verifying: devices … ok
Verifying: combo-tests …
Sub-test: xattrs + rsrc forks … ok
Sub-test: lots of metadata … ok

The following lines are needed to run the patched version of rsync and the right flags. See the program file residing in …/.copiers.d (I used an edited version of 10-rsync-apple and called it 11-rsync-3-patch:

rsync=/usr/local/bin/rsync

flags=”-aNHAX –fileflags –force-change –rsync-path=$rsync”

86. blue replies:

[Sorry the dashes got munched in formatting, forgot to use tags]
flags="-aNHAX --fileflags --force-change --rsync-path=$rsync"

87. JS replies:

@N8: had a quick look – but will now do so ‘properly’, thanks for commenting.

@Blue: Well, I don’t believe the wrong rsync is being used, because the invocation is via this:

rsync=/usr/local/bin/rsync
flags="-v -aNHAX --protect-args --fileflags --force-change --rsync-path=$rsync"
…8< SNIP
sudo $rsync $flags $1/ $2

Which should pick up the patched rsync, as that is where it lives.

In any case, why would ALL the tests fail the creation-date check, note ditto, cp, tar, xar, etc. all fail due to this creation-date test failing (that was why I included the entire run results :-)

Tanks for the comments – will do so checking – but if anyone has a bright idea in the meantime, I’ all ears (as this has me concerned).
cheers.

88. blue replies:

It’s definitely nothing to do with the date format, as my machine is in Australia with Australian localisation of date.

89. blue replies:

See if you can copy with Carbon Copy Cloner (which uses its own rsync) and see what results it gets on a manual bbouncer test.

90. JS replies:

@Blue: Good thinking (on running CCC) – that works – I’ll try and manually go through what the differences are between CCC and the patched rsync, as the patched one is supposed to work.
Will cover this checking when I do as Nate suggested, and try some manual manipulations with touch, SetFile, and GetFileInfo – but, rysnc isn’t my main concern!

So knowing CCC works, how do I now get cp, tar, xar, ditto, and (the other ‘built-in’ utilities to behave (ie pass the creation-date test) – any ideas?

91. JS replies:

rsync – think the problem is 32-bit versus 64-bit and ‘unclean 64-biit tools’ on SnowLeopard.

CCC’s rsync:

rsync version 3.0.6 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, no symtimes, file-flags
(Note the 32-bit inums, 32-bit timestamps)

my patched rsync:

rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, no iconv, symtimes, file-flags
(Note the 64-bit inums, 64-bit timestamps)

Note these are running on a 64-bit Mac (Snow Leopard) and here is the kicker: I *think* the Apple Developer tools are 32-bit (not 64-bit) hence why my tests fail, but CCC ‘works’ (for the wrong reason?)

Some evidence to support that assertion:

file /usr/bin/SetFile
/usr/bin/SetFile: Mach-O universal binary with 2 architectures
/usr/bin/SetFile (for architecture i386): Mach-O executable i386
/usr/bin/SetFile (for architecture ppc7400): Mach-O executable ppc

file /usr/bin/GetFileInfo
/usr/bin/GetFileInfo: Mach-O universal binary with 2 architectures
/usr/bin/GetFileInfo (for architecture i386): Mach-O executable i386
/usr/bin/GetFileInfo (for architecture ppc7400): Mach-O executable ppc
(is both are 32-bit/PPC bit neither are64-bit compatible)

As all of BackupBouncers creation-date tests are predicated on the Developer tools SetFile, and GetFileInfo, if they aren’t fully 64-bit compliant doesn’t that mean that Backup Bouncer is (currently) only suitable for 32-bit environments?

@N8: care to comment on this theory, please?

92. n8 replies:

@JS: It’s not a bad theory. I certainly haven’t tested BB on a 64-bit OSX kernel so it’s possible there are issues with that. But I still don’t understand why 32/64-bit mismatch would matter while we’re within the 32-bit unix epoch. We’re talking about this sequence of events:

touch some-file
SetFile -d 12/31/1999 some-file  # 32-bit operation
copy some-file some-file2  # ??-bit operation
x1="`GetFileInfo -d some-file`"  # 32-bit
x2="`GetFileInfo -d some-file2`"  # 32-bit
test "$x1" = "$x2"   # fails

Even if SetFile doesn’t set the date correctly due to some 32/64-bit mismatch and GetFileInfo doesn’t read it correctly, x1 should still be equal to x2 if the copier has worked properly.

93. JS replies:

Yes, like this:

cp some-file some-file2
orac:100907 paul$ x1="`GetFileInfo -d some-file`"
orac:100907 paul$ x2="`GetFileInfo -d some-file2`"
orac:100907 paul$ echo "$x1" = "$x2"
12/31/1999 12:31:00 = 09/07/2010 20:12:23
indeed it fails!

Clear as daylight now?

94. JS replies:

Re-post (a bit was swallowed during the ScreenSave copy-n-paste, extraneous bits snipped out):-

$touch some-file
$ SetFile -d 12/31/1999 some-file
l$ cp some-file some-file2
$ x1=”`GetFileInfo -d some-file`”
$ x2=”`GetFileInfo -d some-file2`”
l$ echo “$x1″ = “$x2″
12/31/1999 12:31:00 = 09/07/2010 20:12:23

95. blue replies:

Yep mine is 32-bit. The faults in the other copiers are a feature until apple fixes them or you find/write patched versions. Doing the manual tests requires running rsync-patched, since other copiers (cp, etc) don’t copy creation times, perhaps for valid reasons.

Yours is different is a few ways – 32vs64bit timestamps, iconv and symtimes. Presumably the timestamp is the problem.

My Leopard 10.5.8 machine:
rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, file-flags

My Snow Leopard 10.6.4 machine:
rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, file-flags

Presumably one can compile for 32-bit timestamps…

96. JS replies:

Hi Blue, thanks for taking the time and trouble to follow-up on this.
Note, I’m not ruling out that I’ve done something very, very silly here – it’s just that I’ll be the last to notice if so :-)

‘my’ patched rsync is built to Mike Bombich’s recipe here http://www.bombich.com/mactips/rsync.html

So, two additional patch files (patches/fileflags.diff and patches/crtimes.diff) are applied, and they seem to work – for Mike, and most others (which is why it may stll be a silly boo-boo on my part?)

Yes, you can build for 32-bit times, with everything else as-is and in my break tomorrow, I’ll try and do that – and report back the results :-)

cheers!

97. blue replies:

No JS hasn’t done anything very, very silly. Logic honing helps narrow the field of possibilities. You may have found a bug in 64bit rsync machine builds on Snow Leopard.

To date the Bombich recipe has been robust. Note Mike Bombich’s latest 3.0.7 rsync as on his website reports 64bit timestamps, but not bbouncer reports….

98. blue replies:

Mike Bombich has confirmed the 64-bit timestamp bug and and additional patch is available to fix it – see his page re: 64bit crtimes patch and bugzilla report:

http://www.bombich.com/mactips/rsync.html

99. JS replies:

Thanks for the tip-off Blue – my Mike was quick, as even late (my time) yesterday he hadn’t made that update – cool!

Oh, and yes, I have tested the new recipe, here are all the gory details:

cd util && make
make[1]: Nothing to be done for `all'.
created: /Users/test/100908/backup-bouncer-0.2.0/Src.sparseimage
/dev/disk3          	GUID_partition_scheme
/dev/disk3s1        	Apple_HFS                      	/Volumes/Src
created: /Users/test/100908/backup-bouncer-0.2.0/Dst.sparseimage
/dev/disk4          	GUID_partition_scheme
/dev/disk4s1        	Apple_HFS                      	/Volumes/Dst
Cleaning: 00-basic-permissions
Cleaning:        05-timestamps
Cleaning:          10-symlinks
Cleaning: 15-symlink-ownership
Cleaning:         20-hardlinks
Cleaning:    30-resource-forks
Cleaning:      40-finder-flags
Cleaning:      45-finder-locks
Cleaning:     50-creation-date
Cleaning:         60-bsd-flags
Cleaning:    70-extended-attrs
Cleaning: 80-access-control-lists
Cleaning:           95-devices
Cleaning:       99-combo-tests
Creating: 00-basic-permissions ... ok
Creating:        05-timestamps ... ok
Creating:          10-symlinks ... ok
Creating: 15-symlink-ownership ... ok
Creating:         20-hardlinks ... ok
Creating:    30-resource-forks ... ok
Creating:      40-finder-flags ... ok
Creating:      45-finder-locks ... ok
Creating:     50-creation-date ... ok
Creating:         60-bsd-flags ... ok
Creating:    70-extended-attrs ... ok
Creating: 80-access-control-lists ... ok
Creating:           95-devices ... ok
Creating:       99-combo-tests ... ok
sending incremental file list
./
bbouncer-vol
.Trashes/
.Trashes/507/
.fseventsd/
.fseventsd/fseventsd-uuid
00-basic-permissions/
00-basic-permissions/owned-by-me
00-basic-permissions/owned-by-root
00-basic-permissions/owned-by-www
00-basic-permissions/some-dir/
05-timestamps/
05-timestamps/some-hardlink
05-timestamps/some-dir/
10-symlinks/
10-symlinks/broken_symlink -> ./bogus_file
10-symlinks/link2broken_symlink -> ./broken_symlink
10-symlinks/some-file
10-symlinks/symlink1 -> ./some-file
10-symlinks/symlink2 -> ./some-file
10-symlinks/symlink3 -> ./symlink1
15-symlink-ownership/
15-symlink-ownership/some-file
15-symlink-ownership/symlink1 -> ./some-file
15-symlink-ownership/symlink2 -> ./some-file
15-symlink-ownership/symlink3 -> ./symlink1
20-hardlinks/
20-hardlinks/some-file
30-resource-forks/
30-resource-forks/hl-rfork2
30-resource-forks/some-file
40-finder-flags/
40-finder-flags/hidden-extension.txt
40-finder-flags/mucho-flags-dir
40-finder-flags/mucho-flags-file
40-finder-flags/system-file
40-finder-flags/type-and-creator
40-finder-flags/bundle-dir/
40-finder-flags/bundle-dir/stuff
40-finder-flags/invisible-dir/
45-finder-locks/
45-finder-locks/locked-file
50-creation-date/
50-creation-date/creation-date-test
60-bsd-flags/
60-bsd-flags/file-with-flags
60-bsd-flags/dir-with-flags/
70-extended-attrs/
70-extended-attrs/symlink-with-xattrs -> ./xattr-test
70-extended-attrs/xattr-test
70-extended-attrs/dir-with-xattrs/
80-access-control-lists/
80-access-control-lists/acl-test
80-access-control-lists/acl-test-dir/
95-devices/
95-devices/devvn0
95-devices/devzero
99-combo-tests/
99-combo-tests/many-metadata
99-combo-tests/xattr-with-rfork
05-timestamps/some-file => 05-timestamps/some-hardlink
20-hardlinks/link3 => 20-hardlinks/some-file
20-hardlinks/link2 => 20-hardlinks/some-file
20-hardlinks/link1 => 20-hardlinks/some-file
30-resource-forks/hl-rfork1 => 30-resource-forks/hl-rfork2

sent 4595 bytes  received 769 bytes  10728.00 bytes/sec
total size is 365  speedup is 0.07

Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... ok (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... ok
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok

Success!

With grateful thanks to all involved, here and Mike Bombich, appreciated.

100. JS replies:

For the sake of completeness, as this was a hard-one test success, here is the script used to produce the (above) results


#!/bin/bash
scriptdir="`dirname $0`"
cd "$scriptdir"
export PATH="`pwd`/util:/Developer/Tools:$PATH:."
set -x

make
./bbouncer create-vol Src
./bbouncer create-vol Dst
./bbouncer create /Volumes/Src
/usr/local/bin/rsync -v -aNHAX --protect-args --fileflags --force-change --rsync-path=/usr/local/bin/rsync  /Volumes/Src/ /Volumes/Dst
./bbouncer verify -d /Volumes/Src /Volumes/Dst

Let’s hope with that settled, we can continue doing ‘real work’ with ‘good backups’ again :-)

Cheers.

101. blue replies:

Mike does great work. I asked him the question and he looked into it and posted the answer very quickly. Saved a lot of bother for us. Thanks Mike.

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">

Please type this word with the letters reversed: live

Like my work? Check out HexaLex, my game for iPhone & iPod Touch. It's a crossword game like Scrabble, but played with hexagonal tiles. http://www.hexalex.com