Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
PhotoKB Home
Discussion Groups
Digital Photography
Digital PhotoDSLR CamerasZLR CamerasPoint & Shoot Cameras
Film Photography
35 mmLarge FormatMedium formatDarkroomFilm and LabsOther Equipment
Photo Technique
Nature PhotographyPeople PhotographyTechnique General
General Photo Topics
General TopicsAustralian PhotographyUK Photography
DirectoryPhoto Clubs

Photo Forum / Digital Photography / DSLR Cameras / April 2007

Tip: Looking for answers? Try searching our database.

These Cnef files

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
John Smith - 14 Apr 2007 02:37 GMT
I know Cnef are not "true" raw files, but I shoot them exclusively, with the
understanding that it's best for post work.

But I read an article the other day that *implied* there were serious issues
with Cnef, but offered no *practical* examples.

Just a topic for discussion, or are there *practical*  real life issues
here?

DP
achilleaslazarides@yahoo.co.uk - 14 Apr 2007 03:22 GMT
> But I read an article the other day that *implied* there were serious issues
> withCnef, but offered no *practical* examples.

A tad suspicious that there are no examples, no? Anyway, even
theoretically, the tonal resolution of Nikon's compressed raw format
(if that's what cnef is supposed to be) is enough to encode all useful
information (after taking into account boson statistics).

So while one could still argue that there will be effects (eg
something that exaggerates small tonal differences should, in
principle, show a difference between compressed and not compressed; eg
heavy sharpening), I have not managed to force anything to show. So I
wouldn't worry about it (but if you do, switch to uncompressed if your
camera has it for a while; if you see no difference...).

Which article is this?
achilleaslazarides@yahoo.co.uk - 14 Apr 2007 03:44 GMT
On Apr 14, 6:22 am, achilleaslazari...@yahoo.co.uk wrote:
> boson statistics).

argh... "boson statistics" should have been "photon counting
statistics"... Sorry. Should not post when tired.
Floyd Davidson - 14 Apr 2007 08:38 GMT
>I know Cnef are not "true" raw files, but I shoot them exclusively, with the
>understanding that it's best for post work.

Assuming you mean Nikon's compressed version of an NEF, I don't
understand the idea that it is not a true raw file.  It is.

>But I read an article the other day that *implied* there were serious issues
>with Cnef, but offered no *practical* examples.

Wellll, I don't know about any serious issues.  But it is true
that if you have some very delicate tonal gradients in the top
two zones, there is in fact a difference between the compressed
version and the non-compressed version.

Macro shots of flowers would be on place where it can be seen.
A brides wedding dress would be another.  But it does require an
*extremely* critical eye.

>Just a topic for discussion, or are there *practical*  real life issues
>here?

Depends.  How critical are your eyes? :-)

Signature

Floyd L. Davidson           http://www.apaflo.com/floyd_davidson
Ukpeagvik (Barrow, Alaska)                      floyd@apaflo.com

John Smith - 14 Apr 2007 18:12 GMT
>>I know Cnef are not "true" raw files, but I shoot them exclusively, with
>>the
[quoted text clipped - 20 lines]
>
> Depends.  How critical are your eyes? :-)

I wish I could find the article again. I found it by accident while at work
with a search on the engine on my company's webpage searching for info on
something else.

Wish I had printed it out now, but the issue *did* seem to be highlight
performance.

As long as it has no "real world" limitations on my "expose to the right"
preference and will still be supported in the future, it sounds like I don't
have to be concerned about it.

I'm assuming that all the other advantages of raw, such as ability to refine
the white balance are not affected?

DP
Floyd Davidson - 14 Apr 2007 23:37 GMT
>> Wellll, I don't know about any serious issues.  But it is true
>> that if you have some very delicate tonal gradients in the top
[quoted text clipped - 20 lines]
>preference and will still be supported in the future, it sounds like I don't
>have to be concerned about it.

Here is one fairly reasonable discussion of exactly what it does:

 http://www.majid.info/mylos/weblog/2004/05/02-1.html

An uncompressed NEF file uses a 12 bit linear encoder.  The
compressed version uses linear encoding for the lower zones, and
at roughly what would amount to 18% grey it switches to a
different curve with fewer bits.  It works out to a 9.4 bit
depth file, with 683 total values instead of 4096 values.

But all of the zones that have fewer values are in the brighter
half of the range, where each zone from a linear encoder would
have 512 or more levels, and where the eye is less sensitive to
the changes.  The brightest zone with a 12 bit linear encoder
has 2048 levels, and has fewer than 200 with the compressed
format.

To put that into context, it is generally said that if there are
fewer than 8 levels in a zone the eye will be able to detect
posterization.  With 8 levels, the average error would be about
1/16th of an fstop.  The indication is that we probably will not
object to jumps in tonal gradation of 1/16th fstop.

Hence when Nikon says the compression is "visually lossless",
they are referring to the fact that going from 2048 levels in
the brightest zone to 160 levels still provides granularity that
is *far* finer than we tend to notice.

All that said, when taking full frame high key images of
delicate yellow flowers I discovered that Compressed NEF
certainly did produce lower resolution than regular NEF files.
I assume that white wedding dresses would be another way to find
it significant.  Essentially any high key image with a smooth
tonal gradient in the brighter zones will be affected.

>I'm assuming that all the other advantages of raw, such as ability to refine
>the white balance are not affected?

Everything else remains the same.

Signature

Floyd L. Davidson           http://www.apaflo.com/floyd_davidson
Ukpeagvik (Barrow, Alaska)                      floyd@apaflo.com

achilleaslazarides@yahoo.co.uk - 15 Apr 2007 00:07 GMT
> >> Wellll, I don't know about any serious issues.  But it is true
> >> that if you have some very delicate tonal gradients in the top
[quoted text clipped - 43 lines]
> 1/16th of an fstop.  The indication is that we probably will not
> object to jumps in tonal gradation of 1/16th fstop.

If you actually check, at ISO 100 shot noise is larger than the
accuracy of compressed raw data. So we're just quantizing noise. But
this, by itself, doesn't really mean that it's useless.

> Hence when Nikon says the compression is "visually lossless",
> they are referring to the fact that going from 2048 levels in
[quoted text clipped - 7 lines]
> it significant.  Essentially any high key image with a smooth
> tonal gradient in the brighter zones will be affected.

Well, I once took some uncompressed raw shots and resaved them as
compressed using nikon capture. I then loaded them into photoshop, and
subtracted one from the other. The differences were invisible, I had
to do some extreme level stretching to see anything (I really mean
extreme). I tried this with images which had detail in all brightness
levels. No visible difference between compressed and uncompressed (I
really mean visible, I had to set the whitepoint of the result of
differencing the two images to 1 or 2 to get the result to fill the
range).

I'll try to dig up those shots and post them (but may not have time
now).
Floyd Davidson - 15 Apr 2007 07:00 GMT
>> But all of the zones that have fewer values are in the brighter
>> half of the range, where each zone from a linear encoder would
[quoted text clipped - 12 lines]
>accuracy of compressed raw data. So we're just quantizing noise. But
>this, by itself, doesn't really mean that it's useless.

If in the brightest zone we are "just quantizing noise", then
there is no visible image, at all.  We'd have a frame with no
detail at all.  Clearly we are quantizing a great deal more than
noise.  (What you mean is that the LSB of the digitized value is
noise.)

Because shot noise increases with the square root of the signal,
even considering the increased SNR in the brighter zones it is
indeed almost certain that a 12 bit linear encoder is over
sampling, and hence wasting bandwidth by recording useless
noise in the higher brightness zones.

The reason Nikon chose the particular curve they did was because
it is a fairly close match to the eye's perception, which is why
they say it is "visually lossless".  And in general it is.  It
is only in certain circumstances that the difference can be
detected. Nikon chose that particular compromise as opposed to
greater reduction in file size by reducing the probability of
over sampling even farther, or lesser reduction in file size by
retaining some likelihood of over sampling (and thus keeping
more of the useful data in instances where it is not over
sampling).

Here is another article that discusses it,

 http://www.nikonians.org/dcforum/DCForumID86/16055.html

with an interesting graph (explained in the above URL), at

 http://home.comcast.net/~NikonD70/NikonInfo/NEF_Compression_files/image002.gif

which shows how how closely the Nikon 683 encoding matches CIE
perceptual lightness.

Another discussion, using a 677 level format compared to a 12 bit linear
format,

 http://www.msss.com/http/near_cal/fs_algorithms/companding/companding.html

And I note your later article where you have discovered

 http://regex.info/blog/photo-tech/nef-compression

which makes an (over dramatized, IMHO) attempt at a graphic
visual representation of the differences between NEF and
Compressed NEF formats.  (Sometimes graphing noise distribution
is interesting, and certainly mathematical analysis is too, but
I just don't see where it generates a useful image to look at
error signals.)

>> Hence when Nikon says the compression is "visually lossless",
>> they are referring to the fact that going from 2048 levels in
[quoted text clipped - 17 lines]
>differencing the two images to 1 or 2 to get the result to fill the
>range).

If one format has, in zone 0, 2048 values, while another format
has only 160 values, what is the result of a comparison between
formats if it is done the way you describe?

First, what will be seen depends greatly on the images.  Your
description of "had detail in all brightness levels" does not
necessarily mean it would show a significant difference.  If,
for example, the detail is high frequency and of sufficient
amplitude, the difference between the two formats would be
almost negligible.  If it is lower frequency and of small
amplitude, the errors might completely hide the detail.

For example, if a tonal gradient rapidly goes from one limit of
that zone to the other limit, there are only two values of
significance.  Relative to the start of the zone, 0+ and 2048+
for one format and 0+ and 160+ for the other format.  The
quantization error on the first format is 1/4096 of an fstop and
on the other is 1/320 of an fstop.  Which is to say that the
noise is small compared to the signal, and we cannot distinguish
the noise with our eyes.

If the tonal gradient has half the amplitude, the SNR is reduced
by half, because the possible error amplitude is the same.  If
there is a gradual gradient though, the SNR goes down even
further because the possible error again remains the same, but
the actual signal amplitude change from one pixel to the next
gets smaller and smaller as the rate of change across the
gradient is reduced.  At some point all that is visible is,
indeed, quantization noise.

Signature

Floyd L. Davidson           http://www.apaflo.com/floyd_davidson
Ukpeagvik (Barrow, Alaska)                      floyd@apaflo.com

achilleaslazarides@yahoo.co.uk - 15 Apr 2007 00:10 GMT
Actually you can find a similar thing here
http://regex.info/blog/photo-tech/nef-compression/
it's easy enough to reproduce using photoshop (or, I suppose, the
gimp), although producing a compressed raw file from an uncompressed
one may be a bit tricky without nikon capture.
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.