According to CNET News, MS is getting ready to release a new file format for images. It promises to have fewer compression "artifacts" than JPEG (you know, that blocky, grainy effect you get when JPGs are highly compressed), but with much smaller file sizes. So, this could be good for serious digital photographers. Curious, I went to MS's site and grabbed the specs for this new format, to see what it was all about.

Turns out, it looks a lot like the venerable "TIFF" format. TIFF is an old and extremely accommodating format; it was designed to be flexible and can hold almost any kind of image data you can imagine - images with arbitrary amounts of color data per channel, arbitrary number of channels per pixel, arbitrary compression schemes, color representations, ways to represent the pixel data values, etc.. That's a blessing if you want to store something special, but a curse if you want to be able to read and display any TIFF your application comes across, since handling all the options and flavors and twists TIFF allows is simply impractical. So, the fact that MS went ahead and made a new format insteat of using TIFF is a little surprising to me; since TIFF is already capable of supporting different compression methods, and if MS just wanted to push their compression scheme they could probably have found a way to make it work inside of TIFF. But, this is MS, and they're never happy with existing standards. So, the fact that they went ahead and invented a new format instead of leveraging an existing one isn't really surprising. Disappointing, but not surprising.

One thing that was surprising to me was the way they chose to represent the "pixel format" identifier inside the file - that's the part that tells the reader how the pixels are encoded, organized, packed, etc.: monochrome like a fax, 24-bit RGB (like a JPG), 8-bit (like a GIF), 32-bit RGBA (like a transparent PNG), and so on. Most formats would represent that with a single number in the 0-256 or 0-32767 range (like TIFF does), since the odds of having more than a few dozen different formats is pretty slim. A number in the 0-32767 (a.k.a. 2¹⁶) range only takes two bytes, and more than adequately covers the range of possibilities. So, what did MS do? They used something called a "GUID" (aka, a Globally-Unique Identifier). That's a 16-byte value; or, a number in the range 0-2¹²⁸ (or 7 x 10³⁶). That's big. Really big. For comparison, it is estimated there are 10²³ grains of sand in the entire world. So a GUID is large enough to count the grains of sand on 70,000,000,000,000 Earths. Why did MS think they needed such an incomprehensibly large range of values?

Imagine picking one grain of sand off a beach somewhere on one of those seventy quadrllion worlds, then putting it back exactly where you found it. Imagine another person is randomly placed on one of those planets, on a random spot on a random beach. What is the probability that person will pick your exact grain of sand? Well, the answer is, of course, 1 / 2¹²⁸ - but for all practical purposes, it's zero. And likewise, as long as a new GUID is randomly chosen (and they always are), it is essentially guaranteed to be unique. That's why it's called a Globally Unique Identifier. They're very handy.

Now imagine you're a programmer and you've been asked to store some image data in this new file format, in a way that preserves some special characteristics of the data, and that no other pixel format you know fits the bill exactly, What do you do? You figure out how to store that data as you need it, then you grab a new GUID and proudly announce to the world "this is what pixel format blahblahblah does" (and nobody will care). But, you know how to read it, and you know that your GUID identifies it, and you know that nobody else will come along and try to read your file if they don't know what your pixel format is. So MS just made your job a little easier. You encode your image with your fancy new method, save it with a ".WMP" file extension, and move on to the next task. Handy. Clever.

Now, remember what I said way up above about TIFF and the complications of reading its many flavors and variations ? This new format has just superceeded that by a factor of potentially over 2¹⁰⁰. So, based on that feature alone, it sounds like a great format for preserving arbitrary data, but a horrible format for ensuring compatibilty.

---

Note: all of this is based on a quick read of the WMP (Windows Media Photo) spec, which promises (err. which the MS marketing dept promises) to be a useful format. Could be. And hopefully, things won't get as bad as my little mathematical musing says they could. One way this could be prevented is if MS tries to act as an official registrar of pixel formats - if you needed a new format, you would have to 'register' it with MS, and MS would publish a list of approved formats and how to deal with them. That's how TIFF handles many of its options - anyone is free to come up with any hare-brained scheme they want, but they run the risk of causing mass confusion and consternation if they don't register their format with the owner of the TIFF spec, Adobe.