Photo Forum / Digital Photography / DSLR Cameras / April 2007
These Cnef files
|
|
Thread rating:  |
John Smith - 14 Apr 2007 02:37 GMT I know Cnef are not "true" raw files, but I shoot them exclusively, with the understanding that it's best for post work.
But I read an article the other day that *implied* there were serious issues with Cnef, but offered no *practical* examples.
Just a topic for discussion, or are there *practical* real life issues here?
DP
achilleaslazarides@yahoo.co.uk - 14 Apr 2007 03:22 GMT > But I read an article the other day that *implied* there were serious issues > withCnef, but offered no *practical* examples. A tad suspicious that there are no examples, no? Anyway, even theoretically, the tonal resolution of Nikon's compressed raw format (if that's what cnef is supposed to be) is enough to encode all useful information (after taking into account boson statistics).
So while one could still argue that there will be effects (eg something that exaggerates small tonal differences should, in principle, show a difference between compressed and not compressed; eg heavy sharpening), I have not managed to force anything to show. So I wouldn't worry about it (but if you do, switch to uncompressed if your camera has it for a while; if you see no difference...).
Which article is this?
achilleaslazarides@yahoo.co.uk - 14 Apr 2007 03:44 GMT On Apr 14, 6:22 am, achilleaslazari...@yahoo.co.uk wrote:
> boson statistics). argh... "boson statistics" should have been "photon counting statistics"... Sorry. Should not post when tired.
Floyd Davidson - 14 Apr 2007 08:38 GMT >I know Cnef are not "true" raw files, but I shoot them exclusively, with the >understanding that it's best for post work. Assuming you mean Nikon's compressed version of an NEF, I don't understand the idea that it is not a true raw file. It is.
>But I read an article the other day that *implied* there were serious issues >with Cnef, but offered no *practical* examples. Wellll, I don't know about any serious issues. But it is true that if you have some very delicate tonal gradients in the top two zones, there is in fact a difference between the compressed version and the non-compressed version.
Macro shots of flowers would be on place where it can be seen. A brides wedding dress would be another. But it does require an *extremely* critical eye.
>Just a topic for discussion, or are there *practical* real life issues >here? Depends. How critical are your eyes? :-)
 Signature Floyd L. Davidson http://www.apaflo.com/floyd_davidson Ukpeagvik (Barrow, Alaska) floyd@apaflo.com
John Smith - 14 Apr 2007 18:12 GMT >>I know Cnef are not "true" raw files, but I shoot them exclusively, with >>the [quoted text clipped - 20 lines] > > Depends. How critical are your eyes? :-) I wish I could find the article again. I found it by accident while at work with a search on the engine on my company's webpage searching for info on something else.
Wish I had printed it out now, but the issue *did* seem to be highlight performance.
As long as it has no "real world" limitations on my "expose to the right" preference and will still be supported in the future, it sounds like I don't have to be concerned about it.
I'm assuming that all the other advantages of raw, such as ability to refine the white balance are not affected?
DP
Floyd Davidson - 14 Apr 2007 23:37 GMT >> Wellll, I don't know about any serious issues. But it is true >> that if you have some very delicate tonal gradients in the top [quoted text clipped - 20 lines] >preference and will still be supported in the future, it sounds like I don't >have to be concerned about it. Here is one fairly reasonable discussion of exactly what it does:
http://www.majid.info/mylos/weblog/2004/05/02-1.html
An uncompressed NEF file uses a 12 bit linear encoder. The compressed version uses linear encoding for the lower zones, and at roughly what would amount to 18% grey it switches to a different curve with fewer bits. It works out to a 9.4 bit depth file, with 683 total values instead of 4096 values.
But all of the zones that have fewer values are in the brighter half of the range, where each zone from a linear encoder would have 512 or more levels, and where the eye is less sensitive to the changes. The brightest zone with a 12 bit linear encoder has 2048 levels, and has fewer than 200 with the compressed format.
To put that into context, it is generally said that if there are fewer than 8 levels in a zone the eye will be able to detect posterization. With 8 levels, the average error would be about 1/16th of an fstop. The indication is that we probably will not object to jumps in tonal gradation of 1/16th fstop.
Hence when Nikon says the compression is "visually lossless", they are referring to the fact that going from 2048 levels in the brightest zone to 160 levels still provides granularity that is *far* finer than we tend to notice.
All that said, when taking full frame high key images of delicate yellow flowers I discovered that Compressed NEF certainly did produce lower resolution than regular NEF files. I assume that white wedding dresses would be another way to find it significant. Essentially any high key image with a smooth tonal gradient in the brighter zones will be affected.
>I'm assuming that all the other advantages of raw, such as ability to refine >the white balance are not affected? Everything else remains the same.
 Signature Floyd L. Davidson http://www.apaflo.com/floyd_davidson Ukpeagvik (Barrow, Alaska) floyd@apaflo.com
achilleaslazarides@yahoo.co.uk - 15 Apr 2007 00:07 GMT > >> Wellll, I don't know about any serious issues. But it is true > >> that if you have some very delicate tonal gradients in the top [quoted text clipped - 43 lines] > 1/16th of an fstop. The indication is that we probably will not > object to jumps in tonal gradation of 1/16th fstop. If you actually check, at ISO 100 shot noise is larger than the accuracy of compressed raw data. So we're just quantizing noise. But this, by itself, doesn't really mean that it's useless.
> Hence when Nikon says the compression is "visually lossless", > they are referring to the fact that going from 2048 levels in [quoted text clipped - 7 lines] > it significant. Essentially any high key image with a smooth > tonal gradient in the brighter zones will be affected. Well, I once took some uncompressed raw shots and resaved them as compressed using nikon capture. I then loaded them into photoshop, and subtracted one from the other. The differences were invisible, I had to do some extreme level stretching to see anything (I really mean extreme). I tried this with images which had detail in all brightness levels. No visible difference between compressed and uncompressed (I really mean visible, I had to set the whitepoint of the result of differencing the two images to 1 or 2 to get the result to fill the range).
I'll try to dig up those shots and post them (but may not have time now).
Floyd Davidson - 15 Apr 2007 07:00 GMT >> But all of the zones that have fewer values are in the brighter >> half of the range, where each zone from a linear encoder would [quoted text clipped - 12 lines] >accuracy of compressed raw data. So we're just quantizing noise. But >this, by itself, doesn't really mean that it's useless. If in the brightest zone we are "just quantizing noise", then there is no visible image, at all. We'd have a frame with no detail at all. Clearly we are quantizing a great deal more than noise. (What you mean is that the LSB of the digitized value is noise.)
Because shot noise increases with the square root of the signal, even considering the increased SNR in the brighter zones it is indeed almost certain that a 12 bit linear encoder is over sampling, and hence wasting bandwidth by recording useless noise in the higher brightness zones.
The reason Nikon chose the particular curve they did was because it is a fairly close match to the eye's perception, which is why they say it is "visually lossless". And in general it is. It is only in certain circumstances that the difference can be detected. Nikon chose that particular compromise as opposed to greater reduction in file size by reducing the probability of over sampling even farther, or lesser reduction in file size by retaining some likelihood of over sampling (and thus keeping more of the useful data in instances where it is not over sampling).
Here is another article that discusses it,
http://www.nikonians.org/dcforum/DCForumID86/16055.html
with an interesting graph (explained in the above URL), at
http://home.comcast.net/~NikonD70/NikonInfo/NEF_Compression_files/image002.gif
which shows how how closely the Nikon 683 encoding matches CIE perceptual lightness.
Another discussion, using a 677 level format compared to a 12 bit linear format,
http://www.msss.com/http/near_cal/fs_algorithms/companding/companding.html
And I note your later article where you have discovered
http://regex.info/blog/photo-tech/nef-compression
which makes an (over dramatized, IMHO) attempt at a graphic visual representation of the differences between NEF and Compressed NEF formats. (Sometimes graphing noise distribution is interesting, and certainly mathematical analysis is too, but I just don't see where it generates a useful image to look at error signals.)
>> Hence when Nikon says the compression is "visually lossless", >> they are referring to the fact that going from 2048 levels in [quoted text clipped - 17 lines] >differencing the two images to 1 or 2 to get the result to fill the >range). If one format has, in zone 0, 2048 values, while another format has only 160 values, what is the result of a comparison between formats if it is done the way you describe?
First, what will be seen depends greatly on the images. Your description of "had detail in all brightness levels" does not necessarily mean it would show a significant difference. If, for example, the detail is high frequency and of sufficient amplitude, the difference between the two formats would be almost negligible. If it is lower frequency and of small amplitude, the errors might completely hide the detail.
For example, if a tonal gradient rapidly goes from one limit of that zone to the other limit, there are only two values of significance. Relative to the start of the zone, 0+ and 2048+ for one format and 0+ and 160+ for the other format. The quantization error on the first format is 1/4096 of an fstop and on the other is 1/320 of an fstop. Which is to say that the noise is small compared to the signal, and we cannot distinguish the noise with our eyes.
If the tonal gradient has half the amplitude, the SNR is reduced by half, because the possible error amplitude is the same. If there is a gradual gradient though, the SNR goes down even further because the possible error again remains the same, but the actual signal amplitude change from one pixel to the next gets smaller and smaller as the rate of change across the gradient is reduced. At some point all that is visible is, indeed, quantization noise.
 Signature Floyd L. Davidson http://www.apaflo.com/floyd_davidson Ukpeagvik (Barrow, Alaska) floyd@apaflo.com
achilleaslazarides@yahoo.co.uk - 15 Apr 2007 00:10 GMT Actually you can find a similar thing here http://regex.info/blog/photo-tech/nef-compression/ it's easy enough to reproduce using photoshop (or, I suppose, the gimp), although producing a compressed raw file from an uncompressed one may be a bit tricky without nikon capture.
|
|
|