The Essence of Images
What are images?
Much like arrays or strings in C, most image files are stored as long strings of numbers: so many colored squares across x so many colored squares high. These pixels together form an image.
“Convolution filters” in Photoshop show the underlying grid nature of images; entering numbers into the boxes transform an image by multiplying the pixel values and positions, creating something completely new. It’s matrix math.

The fact that images are strings of numbers is interesting because it evinces the duality of computer vision: text is number » number is image. There is no quantized scale to delineate seeing and knowing. The paths to Art and Science converge upon the same place.
What follows is An Anecdote and a short Treatise on Prior Art:
Alex recently created an interesting tool to decompose images into their component parts using ActionScript as the representation syntax. In doing so he has created a new graphics file format; one that stores as much image data as does a JPEG or GIF but in a format that Flash can easily operate on.
How does it work?
The img_analyzer interrogates an image to learn its height, width, and component pixels, and returns them as text/plain Actionscript code suitable for pasting into a Flash .fla movie file. His file format (we’ll call it .FPF for “formal play file”) looks something like this:
// .as doc : vegas_grdn_tiles3 //
ZOOM IN: Let’s deconstruct the deconstructing.
imgHeight and imgWidth are obvious — the height and width of the image.
incNum is less obvious — it’s the scaling factor or sampling rate at which pixels are extracted from the source image. An incNum of 2 would mean every other pixel is being sampled; 10 signifies that the image is sampled once every 10 pixels, horizontally and vertically.
The actual image data then follows the header block as a linear array of hexadecimal color values. Nb. that the array is not delimited along height/width parameters; this can be done manually by slicing the array into imgHeight arrays, each of which is imgWidth long.
The applications of a format like FPF are interesting: ActionScript has no built-in functions for operating on the individual pixels of an image, so converting images to FPF using img_analyzer yields a whole new way to manipulate images in Flash at the pixel level.
This format may appear familiar to the unix graphics heads amongst us: Who remembers XPM?
XPM is an old format. Sources show it dates from circa 1989. XPM is an older, C-language version of what Dr. Al_x Braidwood has created with his image decomposer.
XPM is relevant for two reasons: minor built redundancy compression using CLUTs and an available infrastructure that can deal with XPM files.
INFRASTRUCTURE
XPM would likely have been consigned to the trash heap of obsolete file formats by now, were it not for something called NetPBM, or it’s progenitor, PBMPlus. PBMPlus is the brainchild of a smart guy named Jef Poskanzer who has been programming more or less FOREVER.
The idea behind PBMPlus was both simple: create a universal graphic converter, one that would convert from any graphics format to any other. Think JPEG to Tiff, Pict to BMP, BMP to Pict, etc. Assuming there were only 10 graphics formats*, the naive approach would have been to write nine converters for each of the ten formats, to convert each format to any one of the other nine. That would have meant 10 * (10 - 1) or N2 converters. This would have been a waste of time because graphics formats aren’t that different from each other, and every time a new format was added the other format’s converters would need to be updated to support conversions to and from the new format. Bad idea.
* in reality there are dozens of graphics formats.
Instead, Jef did something strange: he created another graphics format (actually, a family of formats) called Portable Anymap files, or PNMs. PNMs are a simple binary format for storing images. But the odds are, unless you’re a hardcore unix geek, you’ve never seen a PNM file.
Why not? Because PNMs are really an intermediate format. They are a lingua franca of images, created to act as a go-between or interface when converting between graphics formats. Once you have PNMs, you only need two converters for each graphics format he wanted to convert between — one to PNM and one from PNM. That means the number of converters needed goes from ~100 to ~20.
BUT WAIT.
Now we can convert any file format to /from this new PNM format, but so what? PNM files aren’t that useful — what we want is a universal format converter.
Enter: PIPELINES
Jef’s approach was to achieve conversions by chaining operations together into a pipeline, like a plumber screwing adapters onto the ends of a pipe. Since all the converters could either read or write PNMs, any two converters could be “snapped” together by putting a PNM conversion in the middle. So to convert a Pict to a BMP file, you combined the Pict2PNM converter with a PNM2BMP converter: Picts go in one side, are converted to PNMs, and the PNMs get converted to BMPs, which falls out the other side.
As rands would say, holy shit.
The PBM toolkit means transformations between GIF, JPEG, etc. are simply shell script one-liners (which can then be called by perl, lisp, php, etc.).
COMPRESSION
Somewhat like a GIF, XPM has simple redundancy compression by storing a Color LookUp Table (or CLUT) in the file along with the pixel array. The CLUT is just a list of all the colors found in the image. each color gets an index. then, when the pixel array is stored, XPM just stores references to the CLUT index numbers instead of the longer hex values. Instead of storing ‘FFFFFF’ for every white pixel (for example), XPM images store ‘FFFFFF’ = ‘A’ once, and then store ‘A’ for each white pixel. Efficient!
This basically saves you space if your images have any expanses of the same color. Obvs.
Let’s see an example;
CONVERTING A JPEG TO XPM FORMAT IN ONE SHELL COMMAND:
show info on the jpeg (nothing up my sleeves):
jm3.net% jpeginfo seats.jpg
seats.jpg 305 x 216 24bit JFIF N 21324
convert it to text (XPM) format:
jm3.net% jpegtopnm seats.jpg | ppmtoxpm > seats.xpm
jpegtopnm: WRITING PPM FILE
ppmtoxpm: (Computing colormap…
ppmtoxpm: …Done. 21,708 colors found.)
voila! we now have a file called seats.xpm:
jm3.net% file seats.xpm
seats.xpm: X pixmap image text
…the first ten lines of which look like this:
jm3.net% head seats.xpm
/* XPM */
static char *noname[] = {
/* width height ncolors chars_per_pixel */
“305 216 21709 3″,
/* colors */
” c #320903″,
” . c #EACB9D”,
” X c #B6511B”,
” o c #672620″,
” O c #875C32″,
The XPM file, which is actually valid C source code, can now be read, run, or mangled by the thing of your choosing.
CLOSING THOUGHTS
I’m not sure how many lines of code the img_analyzer JPEG » ActionScript converter is; probably more than the one-line JPEG » C converter is, though. On the other hand, the C string syntax of XPM is a little more abstract than alex’s FPF syntax, and would take more coding to loop over and process, due to the need to de-reference pixel colors from the CLUT. I think the compromise I’d like to see is a new PPM to/from ActionScript module for PBMPlus.