Digging into the latest Mac security flaw

Update: Apple has released a security update that addresses this issue.

Earlier this week a very serious Mac OS X security flaw was reported at heise online, and is now making the rounds at various tech outlets. Although the original report is accurate, some major news sites seem to missing the important details.

So I’ll reiterate here:

1. Yes, this is a very serious problem.
2. No, using Firefox instead of Safari will not completely mitigate the risks.
3. Apple needs to address this immediately (and by all accounts, it is doing just that).

I’m curious to see how Apple patches this hole. Because as I see it the actual flaw is deep within Mac OS X itself–in how the OS associates files with applications. Let me take a few minutes to explain.

Like Windows, in most cases the Mac consults a central file type registry when opening a file. So for example, double-clicking a .JPG opens the file in Preview, and double-clicking a .TXT opens the file in TextEdit.

However, the Mac is unique in how it also allows the default association to be overridden on a file-by-file basis. So one can say, “open .JPG files by default in Preview, but open this particular sunflower.jpg file in Photoshop”. One can even tell the Mac to associate sunflower.jpg with an application that doesn’t normally accept .JPG files. What’s more, this overriding association is not stored in a central registry. Rather, the association is stored within the sunflower.jpg itself, in a hidden resource fork.

Maybe you can see where this is going: what if sunflower.jpg is not really an image at all? What if it contains a malicious payload? And finally, what if this so-called .JPG is manually associated with an application that will execute its payload? I’ll leave out the exact recipe, but let’s just say that the Terminal application and shell scripts are involved, and the results are not pretty.

Essentially Mac OS X right now makes it very easy to package up the perfect trojan horse: the file is named as a .JPG, it even shows up with a .JPG icon in the Finder. But if someone double-clicks it, any script or application could be executed. Conceivably my home directory could get deleted before I knew what was happening.

In itself this is a pretty nasty vulnerability. But it gets much worse. This trojan file can be easily placed in a ZIP archive and emailed, IM’d or downloaded from one computer to another, and by virtue of the Mac’s enhanced zip/unzip tools, the resource fork remains intact: in other words, the deadly application association survives the transmission.

And now here’s the kicker: by default Safari automatically decompresses .ZIP files, and if the archive contains a single file with a “safe” extension, Safari automatically opens the file. In other words, a trojan file can be delivered and transparently executed with no user intervention: all it takes is a visit to a malicious web page.

Until Apple issues a patch, here’s what you can do:

1. If you use Safari, immediately disable the “Open safe files after downloading” option in its general preferences.
2. Never open the contents of a suspicious ZIP file, even if they seem like normal documents.
3. Use the Finder’s “Get Info” command to double-check that a file’s application association has not been tampered with.

You can also try this harmless proof-of-concept to see if your system is vulnerable.