First, to access a shared printer on another machine, you have to turn on print sharing on your machine. This defies all logic, but there you go.

Second, for some utterly inscrutable reason Apple decided to *dis*able the CUPS browsing protocol, leaving only Bonjour browsing working. ???? Stupid. Stupid. Stupid. It’s as if they said “let’s make sure Mac users will have lots of trouble printing to Linux servers, that’ll improve our customer satisfaction!” To fix this lameness, do the following:

Combining these two tips, the shared printer finally appeared in the “Default” pane of the new printer dialog and printing to it just worked. Hopefully Apple makes this more sensible in Snow Leopard…

Here’s how you test Time Machine with Backup Bouncer:

$ mkdir ~/bb
# We need to trick BB into thinking this is a volume it created
$ touch ~/bb/bbouncer-vol
$ sudo bbouncer create ~/bb
# Now do a Time Machine backup and wait till it's done
$ mv ~/bb ~/bb-orig
# Now restore the ~/bb directory from Time Machine.  
# Try not to catch SPACE MADNESS!!!
$ bbouncer verify -d ~/bb-orig ~/bb

On my Leopard 10.5.3 machine here’s what I get with BB 0.1.3:

Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... FAIL 
Verifying:            hardlinks ... FAIL (Important)
Verifying:       resource-forks ... 
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... ok 
Verifying:        creation-date ... ok 
Verifying:            bsd-flags ... FAIL 
Verifying:       extended-attrs ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok 
Verifying: access-control-lists ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... FAIL 
Verifying:              devices ... FAIL 
Verifying:          combo-tests ... 
   Sub-test:  xattrs + rsrc forks ... ok 
   Sub-test:     lots of metadata ... ok 

The only annoyance is that hardlinks aren’t preserved, but that’s not surprising considering the way hardlinks are used internally by TM to prevent wasted space. (I should point out that the hardlink failure here is not the same as the Apple rsync one I mentioned earlier today, and it’s not nearly as bad.) These results are not perfect but I would have no problem recommending TM backups to any less-than-hardcore user.

------------------ rsync-apple ------------------
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok 
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ... 
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... FAIL 
Verifying:        creation-date ... FAIL 
Verifying:            bsd-flags ... ok 
Verifying:       extended-attrs ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... FAIL 
Verifying: access-control-lists ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... ok 
Verifying:              devices ... ok 
Verifying:          combo-tests ... 
   Sub-test:  xattrs + rsrc forks ... ok 
   Sub-test:     lots of metadata ... ok 

If you add a resource fork to a file that’s hardlinked to another file then only one of the files gets copied. This, no doubt, is due to the hackery involved to make HFS+ look like it has a real hardlinks when it actually doesn’t.

Some of the changes in BB 0.1.3 were a bit invasive and Bash is not the greatest of programming languages, so please keep your eyes open for bugs. If you see something in your results that doesn’t make sense just let me know.

So I wasn’t too surprised when a user forwarded me a somewhat exasperated-sounding note from Shirt Pocket software explaining why SD doesn’t pass BB’s FIFO and device file tests. I won’t paste the note in here, since I feel that would be poor form, but I will pass along their rationale for not copying these files. They feel that FIFOs and device files are normally system-created files that are not meant to be preserved across reboots or backups. For that reason, SD doesn’t back up these filesystem object types at all.

One can understand how Shirt Pocket arrived at this policy. Writing a good system-level backup tool is more subtle than it might appear. There are actually parts of the filesystem that you don’t want to back up, like caches, VM files, and so forth. You don’t want to back up the device files under /dev, since those get created dynamically depending on what devices are attached. In short, there are some parts of the filesystem that are tied to the state of the running system, and these should not normally be copied in a backup.

The problem with Shirt Pocket’s policy is that although it is certainly true that the system-created FIFOs and device files should not be copied in a system-level backup, there are legitimate reasons a user might create FIFOs or device files, and there’s every reason to preserve those user-created objects during a backup. Don’t get me wrong — this is obscure stuff that only advanced UNIX hackers would do, but hey, advanced UNIX hackers need reliable backups too! SD’s policy is, IMHO, overly broad. I would be much more comfortable if SD excluded filesystem locations rather than entire classes of filesystem objects.

The Shirt Pocket letter also accused BB of being “rather misleading to most” on this issue because it makes people worry about something that won’t cause them any problems. Although I vehemently disagree with the “misleading” characterization, I agree that not every BB test is equally important in practical terms. Most end-users will absolutely not care that tool X doesn’t backup FIFOs. This is why BB has always had a prioritization feature. Quoting from the usage string of bbouncer:

-T <set>     Use the given set of tests.  <set> can be either 
             "critical", for tests whose failure will cause problems
             for average users, or "important", for tests whose
             failure may not cause problems for many users, but which
             may be important to power-users or may qualify as 
             critical in the future.

The FIFO/device file tests are not included in either the “critical” or “important” set of tests. So I’m saying (and have always said) that most users, even most power users, won’t care about them. So why test for them at all? Consider the situation before Maurits’ blogged on backups and BB existed. Most people, myself included, didn’t even know what the full set of OS X filesystem object types and metadata was. Maurits filled us in on what existed, but knowing how to test for preservation was still a black art. BB was meant to democratize that testing, but also to act as an exhaustive test set. (I don’t claim to have achieved even close to 100% coverage, but that’s the goal.) So it’s important to me that BB include tests for any metadatum or filesystem object that can conceivably have a reason to be backed up.

Having said that, I’ll concede there’s a usability bug here. The -T option is unlikely to be discovered by casual users, and they’re the ones who need it most! It’s not used by autopilot and it’s not even documented in the README. So I think I’ll be making the following changes:

• I’ll document -T more pervasively
• I’ll have the priority of a test appear in BB’s output
• I’ll have BB default to running in “important” mode, changing the exhaustive mode to an opt-in option

Hopefully this will help prevent confusion for casual users and ease the support burden of tool developers while maintaining BB’s goals of providing easy backup tool validation and exhaustive test coverage.

I think the grownup meal is called a Sad Meal. It’s sad because it doesn’t have a toy.

Mind you, she was completely serious about this statement and delivered it with a totally straight face. Had my mouth been full of soda it would have come shooting out of my nose for sure…

Get it at the goog.

It doesn’t take long for you to start feeling the limits of your gear. Sure, you can get great results when the light’s good or the subject’s still, but when your kid’s running around in the living room and the light’s bad you’ve just gotta pop up the flash. After a few tries you realize that the built-in flash on your fancy dSLR isn’t a lot better than the one on your old point-n-shoot. Everything still ends up flattened. Blagh. The snowball starts rolling.

Once again you swipe the card — you pick up a nice, fast 50mm lens (a bargain at $120). Now you’ve got a better range, you’re taking shots where you couldn’t before, but guess what? At f/1.8 you’ve got a 2cm depth of field. You’d better hope you can nail the focus just right, or you’re outta luck. Plus, there are times when the light’s just not there, even at f/1.8. Oh dear, the snowball’s rolling faster now.

So you pick up a good flash ($190). But you make the mistake of reading strobist and start thinking about off-camera flash, so the snowball grows by another $60 in RF triggers and diffusers. (We’ll be generous and assume you didn’t get the stands, umbrellas, and gels that you wanted so badly but couldn’t justify to the spousal unit.)

I think you get the picture, but I’d just like to take a moment to enumerate a few more of the items that will contribute to that snowball’s mass. Mind you, I’m using “ghetto-gear” prices here. Buying actual reputable-brand gear means these prices go up, often significantly.

• You’ve gotta have a telephoto lens for birds and trips to the zoo. Something that at least gets out to 200mm. ($250 at least. Probably more like $450.)
• It would be nice to have some teleconverters for that lens to give you even more reach on the cheap. ($60-200)
• Ooh, macro’s fun. Close-up lenses are reasonably cheap ($40) and so are extension tubes ($85).
• Ah yes, the tripod! All the pros say it’s critical to have one, and a good one at that. Why, you’d be a fool to skimp here! ($250. Yes, you’re skimping. You’re poor. Boo hoo.)
• You really need at least a circular polarizing filter (multi-coated, of course) to get those magical cobalt-blue skies. ($75)
• Don’t forget about software! Photoshop is “only” $300 with the educational discount. You might be able to make do with Bibble Pro (great stuff, $120) and Krita ($0, thank God for open-source), but you know you’ll buy PS eventually.
• Another thing, if you care about getting good prints you have to have accurate color during your editing. That means you definitely need a calibrator for your monitors. ($80)
• Oh yeah, those RAW files? They take up space. You’ll be needing more hard drives at $100 each. (Yes, drives. You do make backups, right?)

But do you know what the funny thing is? You still truly believe that once you have all those things, once that snowball has reached some ideal size, it will just stop! Really!

Dear friend, the snowball never stops.

• User: root, Password: aardvark. (Repeat with various well-known usernames and every word in the English dictionary)
• User: abe, Password: abe. (Repeat with all common first/last names)

It’s not hard to think of simple methods to cut off these attacks without altering or otherwise restricting one’s ssh service (by, say, running sshd on a non-standard port):

• Throttle login attempts on a per-host basis. i.e. if a host tries and fails to log in N times in M seconds then start making it wait longer and longer between successive logins (or even blacklist the host entirely).
• Throttle login attempts on a per-account basis. i.e. if user bob gets N failed login attempts within M seconds then throttle or blacklist future login attempts on that account.
• Auto-blacklist hosts that attempt to log in to accounts that should never be logged in to. e.g. if somebody tries to ssh in as user “games” they’re toast.

Indeed, there are any number of programs out there that aim to provide exactly these capabilities.¹ What these techniques have in common is the need to measure the number of failed login attempts associated with a host and/or account over a period of time. So how do the existing programs do it? Sadly, a large number do it by scanning log files. In other words, a scanner will look for a certain pattern (typically using regular expressions) in the logging output for your system and assume that it indicates a failed login attempt. This is not a great idea.

What’s wrong with log file scanning, anyway? The first (and least important, IMHO) mark against it is performance. A typical system has dozens or hundreds of processes running at any time, each generating log output to some degree. Well-written programs generate a trickle, but it’s not unusual for a program to accidentally ship (or be misconfigured) with debug-level logging enabled and thus generate a torrent of debug output. By running a log file scanner you’ve now got a program that has to sift through every byte of that output looking for just one type of event! This is certainly not the most efficient way to find out about login attempts.

Another bad aspect of log file scanning is that it relies on the specific formatting of log messages from specific programs. If you replace OpenSSH with BogoSSH and BogoSSH uses a slightly different log output format then your scanner breaks. Similarly, if the OpenSSH people ever decide to change their logging format (not that this is likely) then your scanner breaks. If you install some new service that the scanner doesn’t know about it’s probably going to be totally unprotected unless it happens to format its log messages just like some other service the scanner understands.

But the worst thing about log file scanning is that sloppy implementations can actually open your system up to denial of service attacks! I recently came across this article by Daniel B. Cid that describes vulnerabilities in several popular log scanners that allow remote attackers to cause your machine to falsely blacklist any other machine of their choosing. Even worse, one scanner allowed attackers to cut a machine off from the network entirely! I refer you to the article for details, but the root causes of this problem are easy to describe:

• Regular expression matching is error-prone. People aren’t terribly good at understanding the full class of strings that a given R.E. will match. I’ve been doing complicated R.E. stuff for years now and I still have a terribly hard time writing bug-free regular expressions.
• Log files are chock full of strings that come from insecure sources, like, say, random users on your machine and script kiddies all across the internet who want to crack your machine. You do not want to be taking action automatically based on the contents of such an untrusted data source. The article doesn’t even begin to mention what an attacker with local access could do by writing and executing a program that mimics sshd’s log output.

So how should this problem be solved? By going straight to the horse’s mouth. If you want to know about authentication attempts, talk to the authentication system. Every modern *NIX system supports PAM, the Pluggable Authentication Module system. With a PAM module you can perform an action on any failed login attempt from any PAM-enabled service. This allows you to create a concise, trustworthy authentication log file with a consistent format, which can then be monitored by a very simple (and thus secure) scanner.

One project that takes the PAM approach is pam_abl (auto-blacklist). While it doesn’t use the specific strategy I described above (it builds a database rather than producing an authentication log file) it does get its info from the right source and is thus immune to these bogus log message attacks. Unfortunately pam_abl is a bit inflexible and limited in its capabilities. It doesn’t provide any way to blacklist a host if it tries to log in to a bogus account like ‘games’. It doesn’t provide a way to throttle rather than blacklist. And although it effectively ensures that an attacker will never have a chance to guess your account passwords it doesn’t provide a way of firewalling off a malicious host entirely. Even worse, the maintainer of pam_abl doesn’t seem to have time to work on it any longer. Hopefully somebody else will step up and turn this into the tool for stopping dictionary attacks dead in their tracks.

¹ Here are a handful:

• denyhosts
• fail2ban
• blockhosts
• blocsshd
• crackblock
• shellter
• sshdfilter
• sshit
• sshutout

Updated: Removed ssh-faker from the footnote since it effectively alters the sshd service.

The photo in the title is one I took while playing around with my new 50mm f/1.8 Nikkor lens. With the right adapter the lens can be mounted in reverse and used as a pretty decent macro lens. I didn’t have the adapter so I just held the lens up to the camera body instead and started goofing around. When I saw that shot I immediately thought about using it on the site, and it seems to fit perfectly IMHO. Since then I’ve bought the reverse adapter ring (they’re dirt cheap on ebay) so I’ll be sharing some more of those shots in the near future on my Flickr page.

The Flickr page is another “news” item. I’ve had a Flickr account for a long time now but never really used it because of various limitations in Flickr. But now I realize that I just don’t want to deal with maintaining my own photo hosting software. Flickr is reasonably priced, has unlimited storage/bandwidth, and gets my shots into an environment where real photographers might actually look at them and give feedback.

Finally, I thought I should mention that I upgraded across not 1, not 2, but 3 WordPress versions (from 2.0.something to 2.3.2) in a matter of minutes thanks to subversion. My installation is a subversion checkout of the WP developers’ repository, so all I needed to do was svn switch to the 2.3.2 tag to upgrade. I was able to use svn diff to find the minor tweaks I’d made to some of the WP files and svn revert them once I determined that they weren’t needed anymore. This is definitely the way to maintain a website. Happily, my theme only needed one minor adjustment to work in 2.3.

The change list is short and sweet. I’ve added UTIs for a few more languages (Tcl, JSP, Lua) and improved some highlighting modes. But the nicest change IMHO is the ability to configure the highlighting parameters with “defaults write …” commands. This means I can finally use my slateGreen theme day-to-day without worrying that I’m going to accidentally push it out as the default theme for the next release. It also means you can enable line numbers, wrapping, etc. without hacking on files in the QLCC bundle and worrying about your changes being overwritten on the next update.

BTW, QLCC has been downloaded over 5,000 times since I first released it. If only I had a dollar for each download…

Anyhow, get it at the project page.

As always, editing CSS has left me annoyed. I know that it’s important for CSS to be easy to parse and apply, but couldn’t we have simple variables or even functions? My CSS is liberally sprinkled with stuff like background-color: #f60; that says nothing about what the design intent of #f60 is. This from the organization that stresses separation of semantics from presentation? Imagine instead that I could do something like:

let border_highlight = #f60
let button_hover = #f60
...
.somebox { border: 1px solid border_highlight }
.somebutton:hover { background-color: button_hover }

Now if I want to change the border highlight but not the hover color it’s simple, and there’s no worries about accidental search/replace errors. The argument for functions is similar. If you want to define a bunch of related classes multiple times but with slightly different parameters, you end up defining one set, then overriding values to define the second, again for the third, and so on. It would be much cleaner IMHO to be able to define a function to generate these classes and then invoke the function several times. Possibly not a big win in terms of lines of code, but I think it would be easier to maintain.

It is, of course, possible to use the C pre-processor cpp with CSS, so that might be a way for me to get my wish, but I’m reluctant to add a compilation step into my already awkward web-development process. (Speaking of which, css3.info, MacFuse, MacFusion and SSHFS rock!)

So anyway, enjoy the bling, variables are good, functions are great.

Thumbnailing is supported now, and the syntax highlighting engine has changed from Pygments to Highlight, mainly because it’s about 10x faster but also because it covers more languages. It also has an ide-xcode theme that does a pretty decent job of imitating Xcode’s default colors!

Get it at the Goog.

Well, it turns out to be trivially easy to do a QL plugin that renders HTML. Mix in Pygments, the very capable syntax highlighting engine that can output HTML, and you’ve got a syntax highlighting QL plugin in under 100 lines of code!

Rather than host this project myself I’ve decided to try making a Google Code project. You can find it here:

http://code.google.com/p/qlcolorcode/

Enjoy!

Get them here. The standard disclaimers apply. They work for me and they validate in Font Book, but I don’t know squat about fonts and YMMV.

Here’s a screenshot from JEdit:

I just uploaded Backup Bouncer 0.1.2 for your backup-tool-testing pleasure. Get it at the usual place. Not a whole lot has changed, but there’s a fix to extended attribute handling that solves some problems in Leopard. Andreas Fuchs provided a patch for this and also mentioned that SuperDuper can’t handle dangling symlinks! I haven’t verified this, but if it’s true it’s a pretty huge failure for an otherwise excellent tool. In any case, I added a case to the symlink test to check for it.

I’ve also added a check for extended attributes on symlinks. This is pretty obscure, but a well-written tool shouldn’t care, since the extended attribute support would be orthogonal to the file type. If your favorite tool fails that test I wouldn’t worry about it too much, but you might want to file a bug report.

In related news, I read a while back that Apple was using xar for packaging in Leopard. After installing Leopard on my own machine I was a bit disappointed to find that Apple has installed xar 1.4, which does not do a good job of preserving metadata:

------------------ xar ------------------
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... FAIL
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... FAIL
   Sub-test:       on directories ... FAIL
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL
   Sub-test:              on dirs ... FAIL
FAIL
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL
   Sub-test:     lots of metadata ... FAIL
FAIL

This is a crying shame, since xar 1.5 and later do a great job. Hopefully people won’t pass judgment on xar based on the version shipped with Leopard.

On the other hand, rsync seems to have improved in Leopard:

------------------ rsync-apple ------------------
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... ok
   Sub-test:              on dirs ... ok
ok
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok
ok