Some of you have probably noticed that I’ve been neglecting some of my other projects and complaining about being busy a lot lately. Well here’s what I’ve been working on. I’m pleased to announce HexaLex, a new angle on crossword games! The first platform for HexaLex is iPhone and iPod Touch, but I’m planning to expand to Facebook in the near future.

HexaLex as it looks in-game

If you want to get a quick overview of the game and a list of features go ahead and visit the HexaLex.com website. If you want to read some great 5-star reviews or buy the game just pop on over to the App Store. Continue reading for a little more detail on the core concept of the game.

HexaLex is an idea I came up with a little over a year ago as a grad student. The core gameplay is a lot like Scrabble, Words With Friends, or Lexulous, so if you’ve played one of those games you’ll get the idea right away. You use tiles to build words on a game board. But HexaLex has a big difference — it’s played with hexagonal tiles. Looks interesting, right?

Playing a crossword game with hexagonal tiles sounds like a simple idea but it actually requires a bit of creative thinking to make it playable and fun. (This is probably why it hasn’t been done a million times already!) You see, when two words cross with square tiles they only interact at one tile — the tile of intersection. With hexagonal tiles, that’s not the case. Hex tiles cause interactions at the tile of intersection and its neighbors! This means that playing one word across another word also forms two new two-letter words. This is sort of hard to describe, but easy to see as a picture (like the one below).

Crossing START with CRAZY forms RT and RZ

Traditionally, in crossword games you have to make every word on the board valid. If you just take the standard rules and try to use them with hexagonal tiles you’ll quickly discover that the game is unplayably hard. It’s just frustrating to have to work out two valid two-letter words for each and every crossing play.

The way I solved this in HexaLex was to introduce the idea of junk words. A junk word is a two-letter word that’s not a valid word. So XR, TQ, and ZA are examples of junk words. (Oops, ZA is actually a valid word, at least in the strange world of crossword games.) In HexaLex you are allowed to play junk words subject to a few restrictions:

The main menu features a sliding word animation

With junk words, a simple play of one word crossing another is always possible provided the two words share a letter. The game is playable once again and balance is restored!

So now you know that when you look at a HexaLex game it’s OK that some of the two-letter words are bogus, but all of the longer words have to be legit.

Junk words aren’t the only break from Scrabble tradition in HexaLex but I’ll leave the rest for another post.

HexaLex is available now in the App Store. Give it a shot! Initial App Store reviews have been overwhelmingly positive, with fourteen unanimous 5-star ratings and reviews so far. If you enjoy HexaLex please let me know! You can also become a HexaLex fan on Facebook and/or follow HexaLexGame on Twitter. Also, please help spread the word by clicking the digg button!

To make matters worse, the system’s top-secret non-configurable algorithm for resolving conflicts appears to work backwards from what users might hope for. The Xcode plugin lives inside an Apple-supplied app bundle in a system directory. It’s installed automatically if you choose to install Xcode itself, so it can easily up on your system without you even knowing it. Compare this to QLCC. You, the user, have actively downloaded it and installed it for the express purpose of handling source code files in the Quick Look system. You can easily remove it if you decide you don’t like it anymore. Which plugin do you think should win if they conflict? Apple thinks different. The system always picks the Xcode plugin over QLCC.

But come on, if you’re going to have a system like this there ought to be a way of saying “I want plugin X to handle UTI Y.” This could be a nice shiny user interface somewhere or a down-n-dirty terminal command, but it needs to exist. I sent a bug report to Apple saying as much (rdar://7240036, mirrored here). I described my problem and made it clear that the solution ought to be a preference system for Quick Look. At the very least, I said, user-installed plugins should rule supreme.

Apple engineering’s response? You can register for more specific UTIs than public.source-code. Nothing about why they don’t want to have a preference system or let user-installed plugins override others. Just “register for more specific UTIs.” The idiocy of this suggestion is staggering. The whole purpose of the hierarchical nature of the UTI system is to allow folks like me to say “I handle source code” without having to list every type of source code. Highlight, the library that QLCC is based on, can colorize about 140 programming languages. Apple engineering, with a straight face, is telling me that I should be registering for 140 UTIs because Xcode decided to register for the one I actually need.

It gets worse. Even if I decided to take Apple’s advice, it’s impossible to say what those 140 UTIs should actually be, because there’s no central authority for registering UTIs. (This is another glaring flaw with the UTI system as implemented by Apple.) UTIs like org.ocaml.ocaml-source are generally defined and bound to file extensions like .ml by applications or other entities like Quick Look plugins. So what happens if you install one app that binds .ml to org.ocaml.ocaml-source and another that binds it to fr.inria.ocaml-source? Your guess is as good as mine. That’s another bit of non-configurable secret sauce. But I think it’s safe to say that only one will win.

So let’s go back to my hypothetical 140-UTI plugin. Imagine the confusion and headaches when a user’s .ml files stop being highlighted just because she happened to download some random app that bound .ml to fr.inria.ocaml-source and my plugin has registered for org.ocaml.ocaml-source. Can you imagine trying to solve these problems for random users of 140 programming languages, of which you actively use maybe 10?

But really, this is just an example of brush-off via bug report. Despite my slagging I do believe the engineers at Apple are generally quite talented and bright. They must know that the problem exists, but for whatever reason they’ve decided it’s not worth solving. Furthermore, they’ve decided it’s not worth telling some no-name hacker why they think it’s not worth solving. But the most insulting thing is that they didn’t even bother to mark my issue as a duplicate of some other issue requesting a prefs system for Quick Look. (You’d better believe that one exists.)

“Go register for more specific UTIs, kid, yer bothering me.”

¹ If you’re unfamiliar with the UTI system, John Siracusa’s encyclopedic review of Tiger has a detailed description.

For history’s sake, I’m observing this on Snow Leopard 10.6.1, Xcode 3.2, Instruments 2.0.

Continuing on my “update side projects for Snow Leopard” theme, I’ve just rolled a new release of QLColorCode. I added support for a few new languages, including Scala, which is a personal fave. I also fixed a bug or two and finally upgraded the bundled version of Highlight from 2.6.6 to 2.12, which was long overdue.

On the ‘meta’ side, I moved the project’s source code from google code’s subversion to github. I’ve been using git a lot lately so I expect I’ll do the same for my other projects as well. Once you’ve gotten used to firing off branches for every little thing that crosses your mind and seamlessly switching between them at a moment’s notice it becomes hard to work in the SVN mindset again.

Unfortunately, QLCC has hit a bit of a roadblock with Snow Leopard. More specifically, it’s not SnoLep that’s giving me a headache, it’s Xcode 3.2. With the new Xcode Apple decided to include their own QL plugin for source code. That’s great, I say, more choice is good! QLCC will render everything that Xcode renders plus OCaml files, Scala files, and all sorts of other interesting languages that Xcode can’t handle. I’m not worried about QLCC losing in a fair fight. But no, Apple doesn’t fight fair.

It turns out that Quick Look always picks the Xcode plugin when rendering source code files (on my machine, at least). I don’t think the Xcode plugin gets special treatment, it just happens to win in the algorithm QL uses to resolve plugin conflicts. What is that algorithm? Nobody outside of Apple knows. It could be a coin-flip for all we know. Heck, after a reboot it’s possible QLCC will always win! In any case, when there are two plugins that both want to render files of type public.source-code it seems like maybe letting the user pick one would be a good idea, right? Apple doesn’t think so. No, they don’t want users to worry their pretty little heads about such matters. Don’t worry about the algorithm, don’t worry about making choices, it’ll all be juuust fine. Have a lollipop.

Except it’s not all fine. Because let’s say a user decides that they prefer QLCC over the Xcode plugin. Their only option for getting QLCC consistently and predictably is to somehow disable Xcode’s plugin. They could remove it, rename it, or move it somewhere that quicklookd won’t find it. But who wants to do something like that to an Apple-supplied file? Nobody. Even worse, Xcode’s application bundle is cryptographically signed. I believe that disabling the plugin by any of those means will invalidate the signature, though I haven’t confirmed this as fact.

Update: After further research, it appears that disabling the plugin will not invalidate the signature on Xcode itself, so there shouldn’t be any terrible consequences if you choose to do so. The plugin is located at: /Developer/Applications/Xcode.app/Contents/Library/QuickLook/SourceCode.qlgenerator

So anyhow, QLColorCode 2.0 is out. It works with Snow Leopard. If you have Xcode 3.2 installed you’ll probably never get to see it do its thing, but maybe one or two of you will still get something out of it.

For what it’s worth, quick look still sends .plist files to QLCC. Xcode’s plugin won’t touch those things.

All is not quite well on the Snow Leopard front, however. The rsync that Apple ships (at least the one in 10.6.1) has a bug. When copying a FIFO it attempts to copy the contents of the FIFO. If you know what a FIFO is, you know why this is bad. So rsync hangs in the 90-fifo test of the test suite. When running from a terminal you can work around this by just hitting Control-C, which will cause the copy to stop and the other copiers to be tested. If you want to avoid the hang you can rename tests.d/90-fifo.test to tests.d/90-fifo.test.disabled or whatever floats your boat. I haven’t done that already because, well, a hang is arguably a valid test result!

Oh, and I also fixed a bug with the BSD flags test. If you care about that stuff you should re-test.

One other interesting tidbit — I changed the “lots of metadata” test to lock the file. This trips up some copiers that duplicate the “locked” status before setting up other metadata, causing the other metadata to be lost.

Much to my shame, I had an e-mail upheaval and I can’t find the names of the individuals who reported the BSD flags bug and the “early locking” phenomenon. If that was you, please drop me a message and I’ll give you the credit you deserve.

UPDATE: It was Rob Kennedy who reported the BSD flags bug and the “early locking” phenomenon. I hadn’t thought to search through my blog comments. Silly me! Thanks a lot for those tips, Rob!

Before you get your hopes up, please understand that this contact was made back in the early summer and nothing much has come from it. Hopefully a few “votes” for the issue will get the ball rolling again.

Last week I randomly decided to google around and see if anybody else had found a solution to the problem. Well, the results of the search were pretty interesting. The problem was ubiquitous — tons of folks with PCR controllers built around the same time as mine reported it on forums and review sites. (This was not the case when I first discovered the problem, by the way.) Some people reported that opening the keyboard and rubbing the key contacts with a stiff pencil eraser brought them back to life, which got me thinking. These things are just electrical contacts, right? Why not put conductive paint on the contacts? That should get them working again for good!

So I did a goog for “conductive paint” and one of the first links that came up was this one. And hey, can you believe it? They sell a “Rubber Keypad Repair Kit”! (MG Chemicals, cat. #8339.) Hallelujah, exactly what I wanted! It’s actually designed for repairing remote controls, but midi controllers work on the same principle. I decided to order one and gave it a shot. (BTW, my experience with Action Electronics was very good.)

How did it turn out? Well, it was good but not great. The kit did, in fact, manage to revive almost all of my keys. Why didn’t it work for all of them? Well, each key actually has two contacts underneath, and a bit of experimentation revealed that the time interval between closing the two contacts is quite important for the keyboard performance. (I assume this is how they sense the velocity with which you struck the key.) If they close in the wrong order, for example, no note is triggered. I think that the added thickness of the paint (which didn’t always go on particularly smoothly) caused timing problems with some of the keys, perhaps even causing the contacts to close in the wrong order. I tried sanding off the paint and repainting the bad keys, but it never quite worked for all the keys.

I tried to live with this for a while — it was, after all, a big improvement over the keyboard’s previous condition. But the fact is, there are times in any musician’s life when only D# will do, and using D or E as a workaround just doesn’t cut it. Finally, I decided that it couldn’t hurt to call Roland and try talking to a live person. After some bouncing around in their phone system (boy do they have awful muzak when you’re on hold!) they actually gave me approval to send in my keyboard! This despite it being many years out of warrantee and my admission that I’d tried to fix the problem myself!

I was a little bit afraid that their technician would investigate the problem, find the contacts covered with conductive paint, and stamp DENIED on my RMA ticket with an evil laugh, but no such thing happened. I suspect they just swapped out the whole keyboard assembly without ever uncovering my handiwork — why bother removing all the keys when this is a known defect? So after 3 days I had my PCR-30 back in fully working condition for no cost other than the time and gas required to drop the thing off at the nearby Roland factory and pick it up again. Kudos to Roland for making good after all these years!

So anyhow, the moral of the story is that talking to people by phone works better than e-mail, at least if those people are Roland employees. Or maybe it’s that conductive paint doesn’t work as well as you might hope.

First, to access a shared printer on another machine, you have to turn on print sharing on your machine. This defies all logic, but there you go.

Second, for some utterly inscrutable reason Apple decided to *dis*able the CUPS browsing protocol, leaving only Bonjour browsing working. ???? Stupid. Stupid. Stupid. It’s as if they said “let’s make sure Mac users will have lots of trouble printing to Linux servers, that’ll improve our customer satisfaction!” To fix this lameness, do the following:

Combining these two tips, the shared printer finally appeared in the “Default” pane of the new printer dialog and printing to it just worked. Hopefully Apple makes this more sensible in Snow Leopard…

Here’s how you test Time Machine with Backup Bouncer:

$ mkdir ~/bb
# We need to trick BB into thinking this is a volume it created
$ touch ~/bb/bbouncer-vol
$ sudo bbouncer create ~/bb
# Now do a Time Machine backup and wait till it's done
$ mv ~/bb ~/bb-orig
# Now restore the ~/bb directory from Time Machine.  
# Try not to catch SPACE MADNESS!!!
$ bbouncer verify -d ~/bb-orig ~/bb

On my Leopard 10.5.3 machine here’s what I get with BB 0.1.3:

Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... FAIL 
Verifying:            hardlinks ... FAIL (Important)
Verifying:       resource-forks ... 
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... ok 
Verifying:        creation-date ... ok 
Verifying:            bsd-flags ... FAIL 
Verifying:       extended-attrs ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok 
Verifying: access-control-lists ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... FAIL 
Verifying:              devices ... FAIL 
Verifying:          combo-tests ... 
   Sub-test:  xattrs + rsrc forks ... ok 
   Sub-test:     lots of metadata ... ok 

The only annoyance is that hardlinks aren’t preserved, but that’s not surprising considering the way hardlinks are used internally by TM to prevent wasted space. (I should point out that the hardlink failure here is not the same as the Apple rsync one I mentioned earlier today, and it’s not nearly as bad.) These results are not perfect but I would have no problem recommending TM backups to any less-than-hardcore user.

------------------ rsync-apple ------------------
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok 
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ... 
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... FAIL 
Verifying:        creation-date ... FAIL 
Verifying:            bsd-flags ... ok 
Verifying:       extended-attrs ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... FAIL 
Verifying: access-control-lists ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... ok 
Verifying:              devices ... ok 
Verifying:          combo-tests ... 
   Sub-test:  xattrs + rsrc forks ... ok 
   Sub-test:     lots of metadata ... ok 

If you add a resource fork to a file that’s hardlinked to another file then only one of the files gets copied. This, no doubt, is due to the hackery involved to make HFS+ look like it has a real hardlinks when it actually doesn’t.

Some of the changes in BB 0.1.3 were a bit invasive and Bash is not the greatest of programming languages, so please keep your eyes open for bugs. If you see something in your results that doesn’t make sense just let me know.

So I wasn’t too surprised when a user forwarded me a somewhat exasperated-sounding note from Shirt Pocket software explaining why SD doesn’t pass BB’s FIFO and device file tests. I won’t paste the note in here, since I feel that would be poor form, but I will pass along their rationale for not copying these files. They feel that FIFOs and device files are normally system-created files that are not meant to be preserved across reboots or backups. For that reason, SD doesn’t back up these filesystem object types at all.

One can understand how Shirt Pocket arrived at this policy. Writing a good system-level backup tool is more subtle than it might appear. There are actually parts of the filesystem that you don’t want to back up, like caches, VM files, and so forth. You don’t want to back up the device files under /dev, since those get created dynamically depending on what devices are attached. In short, there are some parts of the filesystem that are tied to the state of the running system, and these should not normally be copied in a backup.

The problem with Shirt Pocket’s policy is that although it is certainly true that the system-created FIFOs and device files should not be copied in a system-level backup, there are legitimate reasons a user might create FIFOs or device files, and there’s every reason to preserve those user-created objects during a backup. Don’t get me wrong — this is obscure stuff that only advanced UNIX hackers would do, but hey, advanced UNIX hackers need reliable backups too! SD’s policy is, IMHO, overly broad. I would be much more comfortable if SD excluded filesystem locations rather than entire classes of filesystem objects.

The Shirt Pocket letter also accused BB of being “rather misleading to most” on this issue because it makes people worry about something that won’t cause them any problems. Although I vehemently disagree with the “misleading” characterization, I agree that not every BB test is equally important in practical terms. Most end-users will absolutely not care that tool X doesn’t backup FIFOs. This is why BB has always had a prioritization feature. Quoting from the usage string of bbouncer:

-T <set>     Use the given set of tests.  <set> can be either 
             "critical", for tests whose failure will cause problems
             for average users, or "important", for tests whose
             failure may not cause problems for many users, but which
             may be important to power-users or may qualify as 
             critical in the future.

The FIFO/device file tests are not included in either the “critical” or “important” set of tests. So I’m saying (and have always said) that most users, even most power users, won’t care about them. So why test for them at all? Consider the situation before Maurits’ blogged on backups and BB existed. Most people, myself included, didn’t even know what the full set of OS X filesystem object types and metadata was. Maurits filled us in on what existed, but knowing how to test for preservation was still a black art. BB was meant to democratize that testing, but also to act as an exhaustive test set. (I don’t claim to have achieved even close to 100% coverage, but that’s the goal.) So it’s important to me that BB include tests for any metadatum or filesystem object that can conceivably have a reason to be backed up.

Having said that, I’ll concede there’s a usability bug here. The -T option is unlikely to be discovered by casual users, and they’re the ones who need it most! It’s not used by autopilot and it’s not even documented in the README. So I think I’ll be making the following changes:

• I’ll document -T more pervasively
• I’ll have the priority of a test appear in BB’s output
• I’ll have BB default to running in “important” mode, changing the exhaustive mode to an opt-in option

Hopefully this will help prevent confusion for casual users and ease the support burden of tool developers while maintaining BB’s goals of providing easy backup tool validation and exhaustive test coverage.

I think the grownup meal is called a Sad Meal. It’s sad because it doesn’t have a toy.

Mind you, she was completely serious about this statement and delivered it with a totally straight face. Had my mouth been full of soda it would have come shooting out of my nose for sure…

Get it at the goog.

It doesn’t take long for you to start feeling the limits of your gear. Sure, you can get great results when the light’s good or the subject’s still, but when your kid’s running around in the living room and the light’s bad you’ve just gotta pop up the flash. After a few tries you realize that the built-in flash on your fancy dSLR isn’t a lot better than the one on your old point-n-shoot. Everything still ends up flattened. Blagh. The snowball starts rolling.

Once again you swipe the card — you pick up a nice, fast 50mm lens (a bargain at $120). Now you’ve got a better range, you’re taking shots where you couldn’t before, but guess what? At f/1.8 you’ve got a 2cm depth of field. You’d better hope you can nail the focus just right, or you’re outta luck. Plus, there are times when the light’s just not there, even at f/1.8. Oh dear, the snowball’s rolling faster now.

So you pick up a good flash ($190). But you make the mistake of reading strobist and start thinking about off-camera flash, so the snowball grows by another $60 in RF triggers and diffusers. (We’ll be generous and assume you didn’t get the stands, umbrellas, and gels that you wanted so badly but couldn’t justify to the spousal unit.)

I think you get the picture, but I’d just like to take a moment to enumerate a few more of the items that will contribute to that snowball’s mass. Mind you, I’m using “ghetto-gear” prices here. Buying actual reputable-brand gear means these prices go up, often significantly.

• You’ve gotta have a telephoto lens for birds and trips to the zoo. Something that at least gets out to 200mm. ($250 at least. Probably more like $450.)
• It would be nice to have some teleconverters for that lens to give you even more reach on the cheap. ($60-200)
• Ooh, macro’s fun. Close-up lenses are reasonably cheap ($40) and so are extension tubes ($85).
• Ah yes, the tripod! All the pros say it’s critical to have one, and a good one at that. Why, you’d be a fool to skimp here! ($250. Yes, you’re skimping. You’re poor. Boo hoo.)
• You really need at least a circular polarizing filter (multi-coated, of course) to get those magical cobalt-blue skies. ($75)
• Don’t forget about software! Photoshop is “only” $300 with the educational discount. You might be able to make do with Bibble Pro (great stuff, $120) and Krita ($0, thank God for open-source), but you know you’ll buy PS eventually.
• Another thing, if you care about getting good prints you have to have accurate color during your editing. That means you definitely need a calibrator for your monitors. ($80)
• Oh yeah, those RAW files? They take up space. You’ll be needing more hard drives at $100 each. (Yes, drives. You do make backups, right?)

But do you know what the funny thing is? You still truly believe that once you have all those things, once that snowball has reached some ideal size, it will just stop! Really!

Dear friend, the snowball never stops.

• User: root, Password: aardvark. (Repeat with various well-known usernames and every word in the English dictionary)
• User: abe, Password: abe. (Repeat with all common first/last names)

It’s not hard to think of simple methods to cut off these attacks without altering or otherwise restricting one’s ssh service (by, say, running sshd on a non-standard port):

• Throttle login attempts on a per-host basis. i.e. if a host tries and fails to log in N times in M seconds then start making it wait longer and longer between successive logins (or even blacklist the host entirely).
• Throttle login attempts on a per-account basis. i.e. if user bob gets N failed login attempts within M seconds then throttle or blacklist future login attempts on that account.
• Auto-blacklist hosts that attempt to log in to accounts that should never be logged in to. e.g. if somebody tries to ssh in as user “games” they’re toast.

Indeed, there are any number of programs out there that aim to provide exactly these capabilities.¹ What these techniques have in common is the need to measure the number of failed login attempts associated with a host and/or account over a period of time. So how do the existing programs do it? Sadly, a large number do it by scanning log files. In other words, a scanner will look for a certain pattern (typically using regular expressions) in the logging output for your system and assume that it indicates a failed login attempt. This is not a great idea.

What’s wrong with log file scanning, anyway? The first (and least important, IMHO) mark against it is performance. A typical system has dozens or hundreds of processes running at any time, each generating log output to some degree. Well-written programs generate a trickle, but it’s not unusual for a program to accidentally ship (or be misconfigured) with debug-level logging enabled and thus generate a torrent of debug output. By running a log file scanner you’ve now got a program that has to sift through every byte of that output looking for just one type of event! This is certainly not the most efficient way to find out about login attempts.

Another bad aspect of log file scanning is that it relies on the specific formatting of log messages from specific programs. If you replace OpenSSH with BogoSSH and BogoSSH uses a slightly different log output format then your scanner breaks. Similarly, if the OpenSSH people ever decide to change their logging format (not that this is likely) then your scanner breaks. If you install some new service that the scanner doesn’t know about it’s probably going to be totally unprotected unless it happens to format its log messages just like some other service the scanner understands.

But the worst thing about log file scanning is that sloppy implementations can actually open your system up to denial of service attacks! I recently came across this article by Daniel B. Cid that describes vulnerabilities in several popular log scanners that allow remote attackers to cause your machine to falsely blacklist any other machine of their choosing. Even worse, one scanner allowed attackers to cut a machine off from the network entirely! I refer you to the article for details, but the root causes of this problem are easy to describe:

• Regular expression matching is error-prone. People aren’t terribly good at understanding the full class of strings that a given R.E. will match. I’ve been doing complicated R.E. stuff for years now and I still have a terribly hard time writing bug-free regular expressions.
• Log files are chock full of strings that come from insecure sources, like, say, random users on your machine and script kiddies all across the internet who want to crack your machine. You do not want to be taking action automatically based on the contents of such an untrusted data source. The article doesn’t even begin to mention what an attacker with local access could do by writing and executing a program that mimics sshd’s log output.

So how should this problem be solved? By going straight to the horse’s mouth. If you want to know about authentication attempts, talk to the authentication system. Every modern *NIX system supports PAM, the Pluggable Authentication Module system. With a PAM module you can perform an action on any failed login attempt from any PAM-enabled service. This allows you to create a concise, trustworthy authentication log file with a consistent format, which can then be monitored by a very simple (and thus secure) scanner.

One project that takes the PAM approach is pam_abl (auto-blacklist). While it doesn’t use the specific strategy I described above (it builds a database rather than producing an authentication log file) it does get its info from the right source and is thus immune to these bogus log message attacks. Unfortunately pam_abl is a bit inflexible and limited in its capabilities. It doesn’t provide any way to blacklist a host if it tries to log in to a bogus account like ‘games’. It doesn’t provide a way to throttle rather than blacklist. And although it effectively ensures that an attacker will never have a chance to guess your account passwords it doesn’t provide a way of firewalling off a malicious host entirely. Even worse, the maintainer of pam_abl doesn’t seem to have time to work on it any longer. Hopefully somebody else will step up and turn this into the tool for stopping dictionary attacks dead in their tracks.

¹ Here are a handful:

• denyhosts
• fail2ban
• blockhosts
• blocsshd
• crackblock
• shellter
• sshdfilter
• sshit
• sshutout

Updated: Removed ssh-faker from the footnote since it effectively alters the sshd service.