n8blog
distraction in action

Like my work? Check out HexaLex, my game for iPhone & iPod Touch. It's a crossword game like Scrabble, but played with hexagonal tiles. http://www.hexalex.com

I just wrapped up Backup Bouncer 0.2.0, with the major change being that it now builds on Snow Leopard. Get it at the usual place. BB used to include a program called ctool that I had thought about using but never actually used. This ctool program was unmaintained and didn’t build on SnoLep, so I finally just wiped it out.

All is not quite well on the Snow Leopard front, however. The rsync that Apple ships (at least the one in 10.6.1) has a bug. When copying a FIFO it attempts to copy the contents of the FIFO. If you know what a FIFO is, you know why this is bad. So rsync hangs in the 90-fifo test of the test suite. When running from a terminal you can work around this by just hitting Control-C, which will cause the copy to stop and the other copiers to be tested. If you want to avoid the hang you can rename tests.d/90-fifo.test to tests.d/90-fifo.test.disabled or whatever floats your boat. I haven’t done that already because, well, a hang is arguably a valid test result!

Oh, and I also fixed a bug with the BSD flags test. If you care about that stuff you should re-test.

One other interesting tidbit — I changed the “lots of metadata” test to lock the file. This trips up some copiers that duplicate the “locked” status before setting up other metadata, causing the other metadata to be lost.

Much to my shame, I had an e-mail upheaval and I can’t find the names of the individuals who reported the BSD flags bug and the “early locking” phenomenon. If that was you, please drop me a message and I’ll give you the credit you deserve.

UPDATE: It was Rob Kennedy who reported the BSD flags bug and the “early locking” phenomenon. I hadn’t thought to search through my blog comments. Silly me! Thanks a lot for those tips, Rob!

  • Share/Bookmark

  Comments:

1. Joe replies:

Nice update.

Shame that Apple isn’t using this tool or an equivalent to at least track regressions.

2. Rob Kennedy replies:

I Just saw your post on the latest version of Backup Bouncer. I was the one the reported the BSD Flag Bug and the early locking phenomenon. See comment 54 on your blog.

I haven’t had a chance to try out the latest version to make sure it addresses this bug. Sorry. When I do I will let you know if there are any problems.

Thanks for all your hard work on Backup Bouncer.

3. HK replies:

Congratulation for your effort !!

Could you improve tests to manage compressed file on Snowleopard ?

Also you could improve the xattr management :
– huge resource-fork as internal API can read resource-fork in several times
– resource-forks on directory (add a picture with the Finder to change the default folder picture)
– big xattr name (127 char)

another question about the ctime. To my knowledge, there is no posix api to restore it, no equivalence of utime(). In a previous comment, somebody asks the same question without any answer. If anybody has the answer, I’m curious to know it.

Last thing : You have to port it now on Linux, Solaris, etc … :-)

4. n8 replies:

@HK:

I haven’t gotten into testing “huge” metadata items, since it’s not clear what “huge” would be for any given thing. One program might have a buffer overflow at 32 characters and another might have one at 16k. But if you know of a tool that works for items of some size but fails if they get too large I’m interested to hear about it.

- I’m planning on including compressed file tests.
- AFAIK you cannot have resource forks on directories. Changing the finder icon of a directory creates a hidden file within it called “Icon\015″ storing the info.

As for ctime, you are correct. There is no way I know of to restore it once it’s changed. I doubt that’s a big deal, since I’ve never heard of it being used for anything important.

The code is on github now so I’m just waiting for those Linux/Solaris/whatever forks to show up. :)

5. Rob Kennedy replies:

Yes, Apple does let you change the access and modification times via utime(). But Apple does NOT provide an API to change the creation time of a file. Apparently, this was done on purpose on philosophical grounds. The creation date is the date when the file was created. You should not be able to play around with that.

The unexpected consequence. It is impossible to make a perfect clone using common programs like SuperDuper and Carbon Copy Cloner etc. The creation dates will often be different on the backup vs the original drive.

6. Rob Kennedy replies:

Opps I forgot to add an important part.

To my knowledge, Apple will not let you change creation/modification dates on symlinks! Symlink files are the problem.

So your backups often will not have the right creation/modifications dates and times for the symlink files.

7. HK replies:

hello n8

“huge” means lots of differents xattr (1000, 10000, …) with long names (118 chars) and and maximum size (rsrc Fork -> 2GB (??), other 64 Kb)

I don’t know exactly the xattr limits but they seems to be bigger in Snwoleopard. Rsrc fork was limited to 64 K in Tiger and are now almost unlimited.

-rw-r–r–@ 1 root admin 90 Sep 11 14:49 xattr_size14m_ResourceFork.txt
com.apple.ResourceFork 14680064
com.apple.Finder.Info 2

compressed file : nice !!
resource forks on directories : you are right, it was what I was thinking about
ctime : perhaps remove this test until the availability of an API. It can be use to detect attribute change as xattr, usefull for an incremental backup.

other OS : I will see what can be done :-)

8. Harvey replies:

./bbouncer create /Volumes/Test

Creating: 99-combo-tests … chflags: many-metadata: Permission denied
FAIL

9. JIm replies:

Great tool. I’m testing my beta backup program (HashBackup) against it, and used the 0.2.0 version. It’s failing the flag test though, because the nodump flag is set on the files in 60-bsd-flags, so HB is not backing up the files. That’s what the nodump flag means: don’t backup these files, right?

A better test might be to remove the nodump flag on those 2 files and add 2 new files with nodump set, then make sure the 2 new files do not appear in the copy.

The beta site is: http://sites.google.com/site/hashbackup

The 0.9.9 version doesn’t handle ACL’s and flags on the Mac, but the next version will.

Thanks again for a good verification tool!
Jim

10. n8 replies:

@Jim: You’ve got an interesting point about nodump. I’ll think about it.

@Harvey: sudo?

11. HK replies:

hello n8

can I have a direct access to you ?

I have some modif to submit and a bbouncer version on linux

12. n8 replies:

@HK: I tried to email you but your address bounced. You can contact me at n8gray AT n8gray DOT org. Your best bet for contributing code is to start a fork of my code at:

http://github.com/n8gray/Backup-Bouncer

I’m crushed for time right now, and a fork will allow your work to reach like-minded people without waiting for me to merge it. Forking is really easy at github, but if you don’t want to do that you should open an issue on github and include your code. It’s not uncommon for me to drop patches accidentally if they’re sent to me by e-mail, and that doesn’t happen if there’s an issue open.

13. HK replies:

I will fork it this week after cleaning of my copy

and my email is now OK.

see you soon

14. HK replies:

help on github

to modify a file, it’s easy but how to rename or add a directory ? I can’t see any action on directories ?

I need it to make the switch between MacOS and Linux

I’m using uname -s (Darwin, Linux) to rename copiers.d and test.d

drwxr-xr-x 2 root root 4096 2009-11-23 15:23 copiers.Darwin.d/
drwxr-xr-x 2 hky vannes 4096 2009-11-23 11:21 copiers.Linux.d/
drwxr-xr-x 2 root root 4096 2009-11-23 15:23 tests.Darwin.d/
drwxr-xr-x 2 hky vannes 4096 2009-11-23 15:02 tests.Linux.d/

thanks

15. n8 replies:

With git you don’t need to do anything special for directories. You just set up all the files & directories the way you want them and then “git add –all .” in the root of the project. It’ll detect that things have moved and do the right thing. Just make sure it doesn’t add files that you don’t want added.

16. geekosaur replies:

ctime is not creation time; it is inode change time — and, ironically, its primary purpose is for backup programs to detect when the filesystem metadata for a file changes so they can back up the new metadata. As such, setting ctime is not a useful operation.

17. n8 replies:

@geekosaur: That’s correct, and that’s why BB doesn’t test ctime preservation. It’s a shame they called it ‘ctime’ instead of something like ‘itime’ — for a long time I just assumed it meant “creation time” until a colleague set me straight.

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">

Please type this word with the letters reversed: live

Like my work? Check out HexaLex, my game for iPhone & iPod Touch. It's a crossword game like Scrabble, but played with hexagonal tiles. http://www.hexalex.com