For those who like to shoot RAW, the release of the Adobe Photoshop Camera Raw & JPEG 2000 plug-in bundle is probably one of the biggest recent news events. In addition to the much improved workflow and the tight integration to Photoshop, what interests people most would be possible improvements in image quality compared to vendor-supplied tools. Even though I do not have Photoshop myself, I was fortunate enough to receive an image processed by the the Adobe RAW plug-in from Dalibor Jelinek as part of the comparison against the Pixel Grouping algorithm, and found a few interesting things when I studied the processed image. Below is a report of my findings, and I hope you will find it an interesting read.
Nonetheless, keep in mind that this is only what I have found through studying one single image processed by the plug-in from a Minolta RAW file. This is by no means a complete review and should not be mistaken as such.
For starters, because of the current design of digital cameras, one thing any RAW converter (except that for the Sigma SD9) must do is perform color filter array demosaic operation that produces full RGB values at each pixel. Without undergoing the demosaic operation, each pixel would be purely red, green, or blue (or other colors depending on the color filter array used), and the image would not be very readable to the human eye.
However many RAW converters, including the Adobe plug-in, go far beyond that and offer many other image processing capabilities, including adjustments to brightness, contrast, white balance, and others. Even in their default settings, some adjustments are invariably performed so that the resulting image ends in a more usable condition, and this should not be surprising, because even if you shoot JPEG, the images also have went through a series of conversions in camera. Below is a comparison to show you what difference can these adjustments make: the image to the left is produced by the Adobe RAW plug-in in its default settings, and the one to the right is produced by ImageCooker with the Pixel Grouping algorithm and no other adjustments applied.
The original RAW file for both images comes from a Minolta DiMAGE 7i camera, and we will use this same pair of images throughout the rest of the report. To set them both on a more equal footing, I have applied curve adjustments to the 16-bit per channel Pixel Grouping TIFF image in CinePaint, so that it will possess comparable brightness and contrast to the Adobe sample from now on.
Since I do not have the source, I cannot really know for sure the internal workings of the Adobe RAW plug-in. Still we can make educated guesses by looking for certain characteristics in the produced image, and the results should not be too far off from the truth. First look at the following comparisons (we will customarily place Adobe samples at the top and Pixel Grouping samples at the bottom):
Judging from the patterns of image artifacts (in particular the zipper effect in horizontal and vertical edges), it is obvious that the algorithm used in the Adobe plug-in should be quite different from Pixel Grouping or Color Correction I, as they are both reasonably immune to that particular effect. The pattern also does not seem to exactly match that of the Variable Number of Gradients algorithm, so I think it is an existing algorithm that I am not familiar with or (more likely) a hybrid algorithm developed in Adobe that is used in the plug-in.
Another prominent characteristic exhibited by the Adobe plug-in generated image is the surprising uniformity of color in difficult areas (refer to the Localized Color Inconsistencies part of the Pixel Grouping algorithm description for details). The following comparisons illustrate this phenomenon (pay attention to the wrist watch band in the crops at the left):
I think the key to this extraordinary performance is the color space used for internal processing within the Adobe plug-in. Since most cameras use GRGB filter arrays, it is only natural for demosaic algorithms to work in the RGB color space. In fact, all of the algorithms listed in Ting Chen's survey works directly in RGB color space, so it is easy to forget that other possibilities may still exist. Instead of the RGB color space, I think the Adobe RAW plug-in works in the HSV (or perhaps LAB) color space. The reason is outlined in the following paragraphs (which assumes an GRGB type filter array is used).
A common trait of good demosaic algorithms (so nearest neighbor does not count) that work in the RGB color space is that the green channel of the image is always well reproduced, especially when compared to the red and blue channels of the same image. This is mostly due to the relatively high density of the green pixels in the filter array, so that accurate estimations to the missing values can be easily made. Below are the crops from the green channel of both images:
Pixel Grouping, as expected, produces smooth surfaces and transitions with little artifacts. The image produced by the Adobe RAW plug-in, however, contains strange dots and artifacts even in the middle of large blocks of solid color. This should not happen if the green color is in a separate channel and the missing green values are derived directly from that of the neighboring pixels. Now let's look at the hue channel in the HSV color space:
The image produced by the Adobe RAW plug-in obviously have smoother transitions in hue, even along edges (first crop from left), in areas with low luminance and saturation (second crop), and in places with great changes in luminance across small areas (third crop, compare with the wrist watch band crop above). From these evidence, I think it is safe to say that the Adobe RAW plug-in puts much more emphasis on maintaining the uniformity of color in the processed images, and more likely than not work in a color space different from RGB internally.
Of course. The paragraphs below describes some places where the Adobe RAW plug-in behaves differently from a straightforward implementation of a decent demosaic algorithm. Some are good and others are not, but in any case you should judge with your own eyes.
This is not very obvious at less than 200% magnification on screen, but the difference is there. It is possible that the demosaic routine can detect the flat regions in the image and switch to a softer processing mode, but no one can tell for sure.
We have discussed this before. It is possible that choosing the right color space to work with alone is enough to achieve this effect, however I suspect that the demosaic routine may actually seek out regions with the similar colors and actively make the hue values in the region close to each other. In any case its performance in this area is very remarkable and amazing.
Normally black objects of the image should have fairly low saturation, because it does not really have much color in it. Nonetheless, in practice there will always be noise in the image, so the actual saturation in dark areas would vary by probability. However there seems to be cases where saturation of supposedly black pixels would jump to extremely high values (sometimes to 255, which means fully saturated), and this may suggest design flaws in the demosaic algorithm. In fairness I must point out that the problem is invisible to our eyes, and you have to examine the saturation channel of the HSV color space to see it:
Come to think of it, this is quite similar to the Isolated Dots problem discussed in the Pixel Grouping algorithm description, just that it happens in the hue channel instead. So yet another evidence suggesting that the HSV color space is used in the demosaic routine.
There are some strange dots in large regions of solid color (red in this case) in the image generated by the Adobe plug-in. Somehow the flat-regions identification code failed to kick in (if it does exist), and even in that case, the situation should never have been this bad. After all, as far as demosaic algorithms are concerned, regions of any solid color are by far the easiest cases to handle, and there is no reason to get them wrong.
As digital photography started to go mainstream, methods for processing digital images also started to grow in delicacy and sophistication. Naive algorithms and implementations can no longer satisfy today's demanding users, and engineers will increasingly see the need to pack more intelligence into the image processing routines. Part of that intelligence comes from guessing: after all, no single method can be perfect in all cases. If you can guess which parts are flat areas and which areas are edges, then you can treat each in a more satisfactory manner, and improve the overall quality of the resulting image.
But somewhere there must be a limit. In addition to constraints in time and computational resources, most photographers also consider it important that the original scene be produced in a faithful manner. What if the software guessed wrong? What if the software guessed right, but decided to do things its way? Would there be a day when the software realizes that your Micky Mouse photo sucks and decides to put a better looking Micky Mouse in its place? If you think this is far-fetched, just consider how difficult it would be for software to recognize resolution charts and use special parameters (such as black-and-white only) to achieve optimal results: not very difficult at all.
I think we should all seriously think about the implications of intelligence in software systems.
Please direct all comments, suggestions, critiques, technical corrections and such, to the author. But please, no trolls and no flames. Thank you.