RAW file sizes - Pentax vs Nikon

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Jeff R - 02 Apr 2006 06:17 GMT

My Pentax (*1stDS) RAW files (PEF) average out at a little more than 10Mb
(10-11.2).
My Nikon (D50) NEF files average out at around 5.5Mb (5.8-6.1).

So how come, given that they use the same sensor?
I thought RAW files were uncompressed. Obviously not.
While I'm at it, shouldn't they all be exactly the same, fixed, size?

<asbestos undies on>
Does this mean the Pentax files contain twice as much data?

--
Jeff R.
(discovering the joy of RAW)

Reply to this Message

w.beckley@gmail.com - 02 Apr 2006 06:53 GMT

Nikon's sub-$1500 bodies all use only compressed NEFs. Some claim the
compression is lossless, others think it crushes highlight detail.

No idea who is right, but that's the difference you're looking for.

Will

Reply to this Message

Ben Brugman - 02 Apr 2006 10:55 GMT

> Nikon's sub-$1500 bodies all use only compressed NEFs. Some claim the
> compression is lossless, others think it crushes highlight detail.

The compression algoritm used for the NEF's
(at least in a D70) is lossless.

BUT, before the compression starts data is thrown out. In the highlight
area information is discarded. Several values are grouped together as
one value.
Values 0-215 are stored exactly.
Values 216-4095 are stored in 653 groups.
This makes a total of 869 groups (or values).
(This would be just over 9.42 bits for each value).

After this throwing out information, this data is compressed but no
data is lost during the compression. Decompression of the data results
in exactly the same data.

So considering the complete process some detail is lost in the highlight's.
Actualy some detail is lost in values above the 216 value.

In total this results in a file of about 5.5 Mb.
(Without compression the data only would fit in about 7 Mb, so the
compression is not very effective).
(Without the throwing away of the data the data only would fit in about
9 Mb).

ben

> No idea who is right, but that's the difference you're looking for.
>
> Will

Reply to this Message

All Things Mopar - 02 Apr 2006 11:12 GMT

Today Ben Brugman commented courteously on the subject at
hand

>> Nikon's sub-$1500 bodies all use only compressed NEFs.
>> Some claim the compression is lossless, others think it
>> crushes highlight detail.
>
> The compression algoritm used for the NEF's
> (at least in a D70) is lossless.

> BUT, before the compression starts data is thrown out. In
> the highlight area information is discarded. Several values
[quoted text clipped - 17 lines]
> (Without the throwing away of the data the data only would
> fit in about 9 Mb).

I own a Rebel XT but I've been curious about D70 NEF sizes as
well. This explains why a D70 user gets files about half what
my calculator says they should be. Thanks for the explanation,
Will.

Signature

ATM, aka Jerry

"Whether You Think You CAN Or CAN'T, You're Right." – Henry
Ford

Reply to this Message

Ray Fischer - 02 Apr 2006 07:06 GMT

>My Pentax (*1stDS) RAW files (PEF) average out at a little more than 10Mb
>(10-11.2).
[quoted text clipped - 3 lines]
>I thought RAW files were uncompressed. Obviously not.
>While I'm at it, shouldn't they all be exactly the same, fixed, size?

Compression algorithms fall into two braod categories: lossy and
lossless. jpeg is a lossy algorithm because it throws out
information. mp3 is similar. Huffman, run-length encoding, LZW
are all lossless compression algorithms.

I'd guess that the D50 uses a lossless compression on RAW files and
the Pentax doesn't. Compression costs money and maybe the folks at
Pentax decided they wanted to skimp.

><asbestos undies on>
>Does this mean the Pentax files contain twice as much data?

Yes, but with twice as much redundancy.

Signature

Ray Fischer
rfischer@sonic.net

Reply to this Message

Richey - 04 Apr 2006 20:20 GMT

>>My Pentax (*1stDS) RAW files (PEF) average out at a little more than
>>10Mb (10-11.2).
[quoted text clipped - 17 lines]
>
> Yes, but with twice as much redundancy.

Have I mis-understood something here? I thought the idea was that a
'RAW' file was exactly that, RAW DATA, exactly as the camera or sensor
saw it. By definintion, if the D50 throws away data before compressing
(lossless or otherwise) the file is still a processed file, and
therefore cannot be 'RAW'

Reply to this Message

Ben Brugman - 04 Apr 2006 23:09 GMT

> Have I mis-understood something here? I thought the idea was that a
> 'RAW' file was exactly that, RAW DATA, exactly as the camera or sensor
> saw it. By definintion, if the D50 throws away data before compressing
> (lossless or otherwise) the file is still a processed file, and
> therefore cannot be 'RAW'

Yes and no, if you call the data which comes from the A/D convertor
raw this would be a 'fairly' correct term.

Nikon does some processing of this info, but no interpretation of
the bayer pattern. Then a lossless compression is done and they call
this raw. Why? you can't view it on normal viewers, you still HAVE to
process it to make it visible.

It's the same a a piece of uncooked meat on your plate, it is raw,
but not as raw at the time when the animal was still complete in
one piece.

Conclusion :
If some processing is still needed, it can be called RAW.

(A develloped negative can be called raw, and has undergone
some processing).

ben

Reply to this Message

Richey - 06 Apr 2006 18:25 GMT

"Ben Brugman" <Ben@niethier.nl> wrote in news:cbOdnWnzmfcPc6
_ZnZ2dnUVZ8qOdnZ2d@casema.nl:

>> Have I mis-understood something here? I thought the idea was that a
>> 'RAW' file was exactly that, RAW DATA, exactly as the camera or sensor
[quoted text clipped - 21 lines]
>
> ben

I understand your explanation about still needing some processing but can't agree
with you. The definition of 'RAW' was supposed to be 'Raw Data, exactly as the
camera or sensor saw it'. Ignoring the lossless compression, which shouldn't
actually change the picture, if the D50 throws away some of the highlight detail,
what it stores is NOT what it saw and therefore it cannot be true 'Raw'. If a
picture with less detail than the camera actually saw can be called raw, then why
can't a jpg be called raw. The point about needing further processing is not
really relevant as evan a jpg or bmp or tif need 'further processing' or the eye
wouldn't see it. They still have to be interpreted by software so that we ordinary
humans can see the picture.

A negative or slide is a different story really, since it can be push processed,
pull processed, cross processed (different chemicals) etc, there is no way you can
know exactly what has been done to it, so is it Raw?

All of which begs the question 'Does any camera generate a true Raw file, or does
the term Raw need to be redefined?'

Reply to this Message

Barry Pearson - 07 Apr 2006 08:05 GMT

[snip]

> I understand your explanation about still needing some processing but can't agree
> with you. The definition of 'RAW' was supposed to be 'Raw Data, exactly as the
> camera or sensor saw it'. Ignoring the lossless compression, which shouldn't
> actually change the picture, if the D50 throws away some of the highlight detail,
> what it stores is NOT what it saw and therefore it cannot be true 'Raw'.

[snip]

> All of which begs the question 'Does any camera generate a true Raw file, or does
> the term Raw need to be redefined?'

"... exactly as the camera or sensor saw it" is ambigous. Is there a
consensus that it is supposed to be the definition of raw?

The data stored in a raw file isn't a count of photons falling on the
sensor. It isn't a measure of electrons or electrical charge or some
other physical effect of the sensor in some internationally-recognised
units. It is always the result of some processing in the camera in
order to generate a number in the desired range.

It is raw because the camera manufacturer says it is raw!

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/

Reply to this Message

JPS@no.komm - 07 Apr 2006 21:30 GMT

>It is raw because the camera manufacturer says it is raw!

RAWer than JPEG or TIFF.

Signature

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS@no.komm>

><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><

Reply to this Message

Barry Pearson - 02 Apr 2006 09:03 GMT

> My Pentax (*1stDS) RAW files (PEF) average out at a little more than 10Mb
> (10-11.2).
[quoted text clipped - 3 lines]
> I thought RAW files were uncompressed. Obviously not.
> While I'm at it, shouldn't they all be exactly the same, fixed, size?

[snip]

Here is an article about D70 compression. Perhaps it applies to D50
too? This is potentially lossy compression.
http://www.majid.info/mylos/weblog/2004/05/02-1.html

The *istD PEFs average about 13MB! I guess the DL and DS PEFs are
slightly (lossless) compressed? Or perhaps the sensor data is simply
packed better, using 1.5 instead of 2 bytes per pixel?

DNGs from D50 and PEFs are about the same size: about 5-6MB. That
involves JPEG lossless compression.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/

Reply to this Message

John Bean - 02 Apr 2006 12:24 GMT

>The *istD PEFs average about 13MB! I guess the DL and DS PEFs are
>slightly (lossless) compressed? Or perhaps the sensor data is simply
>packed better, using 1.5 instead of 2 bytes per pixel?

The latter Barry, that's the only difference.

Signature

John Bean

Reply to this Message

John Bean - 02 Apr 2006 12:32 GMT

>While I'm at it, shouldn't they all be exactly the same, fixed, size?

No. They contain data other than the raw data - which is
fixed size in the DS files. The non-raw data includes both
full-sized and thumbnail JPEGs which vary in size with image
content.

Signature

John Bean

Reply to this Message

Jeff R - 03 Apr 2006 16:07 GMT

Thanks for all the replies, folks.

--
Jeff R.

Reply to this Message