 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
| PhotoKB HomeDiscussion Groups Digital Photography Digital PhotoDSLR CamerasZLR CamerasPoint & Shoot CamerasFilm Photography 35 mmLarge FormatMedium formatDarkroomFilm and LabsOther EquipmentPhoto Technique Nature PhotographyPeople PhotographyTechnique GeneralGeneral Photo Topics General TopicsAustralian PhotographyUK PhotographyDirectoryPhoto Clubs |
Photo Forum / Digital Photography / DSLR Cameras / January 2007Drawbacks of compressed NEF in D200
Hola, I'm still learning the many possibilities of my recently purchased D200. One of them is the possibility of using compressed NEF (RAW). The main advantage I see on using it is that file sizes are basically 50% of uncompressed sizes, but it is not clear to me yet what are the main drawbacks of compressed NEF. I googled a bit but most of what I read so far was mainly "theoretical", explaining how compression is done, etc., and not specifically referred to D200. What I would like to get are opinions from D200 users and their reasons for using or not using compressed NEF. TIA.  SignatureUn saludo, Miguel >Hola, > [quoted text clipped - 8 lines] >What I would like to get are opinions from D200 users and their reasons >for using or not using compressed NEF. Assuming the D200 uses essentially the same compression as the D2x... My experience has been that in typical photographs it makes no difference, except that you get to deal with a much larger sized file. However, in any image where fine detail in the highlights is significant, the compression will cause loss of detail. An example would be images of brightly colored flowers, where the flower itself is the brightest part of the image. If the exposure is set to push highlights right up to the maximum levels (by observing the "blink on over exposure" LCD display or a histogram), there will be a loss of detail. Where I noticed it the most was with bright yellow flowers. I shoot almost 100% using uncompressed NEF mode... SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com Hola Floyd, On 01/01/2007, at 16:00, you wrote: > Assuming the D200 uses essentially the same compression as the > D2x... My experience has been that in typical photographs it > makes no difference, except that you get to deal with a much > larger sized file. However, in any image where fine detail in > the highlights is significant, the compression will cause loss > of detail. > An example would be images of brightly colored flowers, where > the flower itself is the brightest part of the image. If the > exposure is set to push highlights right up to the maximum > levels (by observing the "blink on over exposure" LCD display or > a histogram), there will be a loss of detail. Where I noticed > it the most was with bright yellow flowers. > I shoot almost 100% using uncompressed NEF mode... Thanks for your reply. I'll make some test shots to see about highlights. And in Spring I'll check with some flowers. SignatureSaludos, Miguel > Hola, > [quoted text clipped - 14 lines] > Un saludo, > Miguel I always shoot compressed RAW on my D200. Though there can be a theoretical difference, haven't seen an example . I have seen several examples of how there is very little to no difference. Of course these are all reduced to web size, which has been the excuse of the "there has be a difference" group. It seems there is no compression of the image per se but an compression of the data stream through "binning", not enough of a computer expert to understand the difference. The one thing that is certain, the compression is not done as in jpeg or even LZW. Since most of the compression effects the upper reaches of the image if you shoot weddings you may want to use uncompressed RAW, but otherwise I'd save the card space. I'll have to shoot some snow pics this winter, see if there is a difference there. Again I'm a shoot who uses digital, not a computer guru, maybe someone who understands this better could chip in. Tom > Again I'm a shoot who uses digital, not a computer guru, maybe someone > who understands this better could chip in. > > Tom I do not now about the D200, but the D70 does binning as wel. Under this mail a table which gives the number of distinct values used for a specific zone. Binning mapping different values to one value. (Example the values 3001, 3002 and 3003 al get the same value). The lower zones have less distinct values than the higher zones. No binning is done up to zone 7. (See table) In Zone 8 a littlebit of binning is done, zone 12 is extremely binned from the 2048 possible values only 161 are used. But if this is a problem, why wouldn't binning of zone 9 which has only 77 values left be a problem. Everybody keeps complaining about the highlights, but zone 9 is not yet in the highlights, so I suspect that the complaining about the binning is done because they have heard of it and think this should not be done. But because I have not seen any complaints about a few stops under the highlights (zone 9 specific), I suspect that nobody has noticed the binning there. So my conclusion (carefull conclusion) would be IF 77 different levels in zone 9 do not present a problem, I would think that 161 levels in zone 12 would not be a problem either. Or the other way around if 161 levels in zone 12 is a problem, why is 77 levels in zone 9 not a problem. (A normal Jpg has only 128 values in the highest zone, and that is after a curve has been used, so the highest zone in a 'normal' Jpg probably covers 2 to 3 zones). Ben Brugman. What is binning. When using a 12 Bit A/D converter, there are 4096 distinct possible values. But they are not evenly spread over all the zone's. The highest zone (under zone 12) get's half of all values, where the lower zones get far less values. In General the number of values in the highest zone are not 'needed', so Nikon has decided to bin several values together to one 'distinct' value. This is a method of compression, and you lose information, but is reduces the amount of data makes further compression easier (because now there are less values to cope with) and can make a camera faster. (Depending where the binning is done, theoretically it is possible that the binning is done by the A/D converter, making it much faster, but it can also be implemented in hardware directly behind the convertor or in software). The reason for the choice of binning is that the higher zones have many more values than the lower zones, so taking some values away from those higher zones does not change the picture a lot. Binning does not alter the resolution at all. (Only for high lit zones with almost no contrast) Binning does not alter the dynamic range. Binning does alter the contrast, only when there is hardly any contrast this is changed a very little. Because 1 (rgb) pixel in the endresult is calculated from at least 5 hardware pixels for some and at least 7 pixels for some, the effects of binning are very limited in the endresult. The effect of binning is probably most noticable in zone 9 because there the number of remaining values is most limited for the higher zones. (Zone 1 to 7 are not binned at all, in zone 8 only 5 pairs of numbers are binned). The binning table for the D70. Zone From till number of values 1 0 1 2 2 2 3 2 3 4 7 4 4 8 15 8 5 16 31 16 6 32 63 32 7 64 127 64 8 128 255 123 9 256 511 77 10 512 1023 92 11 1024 2047 102 12 2048 4095 161 (I hope the table appears correctly after posting this message, there should be four columns in the table). Total number of values 683 (log2 this is 9.42 bits of information). After the bining compression without losing information is easier, because of the limited number of different values. > > Again I'm a shoot who uses digital, not a computer guru, maybe someone > > who understands this better could chip in. [quoted text clipped - 77 lines] > the bining compression without losing information is easier, because of the > limited number of different values. Why did Nikon use a term that flies in the face of standardized terminology? This is what binning is the the communities that use scientific CCDs: http://www.noao.edu/outreach/aop/glossary/binning.html > Why did Nikon use a term that flies in the face of standardized > terminology? > This is what binning is the the communities that use scientific CCDs: > > http://www.noao.edu/outreach/aop/glossary/binning.html I don't know if Nikon used specifically the term "binning" for what they do. But anyway, binning generically does refer to what Ben is talking about. See, for example, http://rkb.home.cern.ch/rkb/AN16pp/node18.html (or probably any statistics book, from vague memories of school statistics--but large grain of salt here as I spent most of the statistics school periods smoking rather than attending the lesson). But anyway, if Nikon did use such an inaccurate term, well... what do you expect from people who make plastic cameras, anyway? :) >> Why did Nikon use a term that flies in the face of standardized >> terminology? [quoted text clipped - 12 lines] >But anyway, if Nikon did use such an inaccurate term, well... what do >you expect from people who make plastic cameras, anyway? :) Can either of you explain what you perceive as different from what was described by Ben and what either of the two URLs provided describes? SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com > >> Why did Nikon use a term that flies in the face of standardized > >> terminology? [quoted text clipped - 16 lines] > what was described by Ben and what either of the two URLs > provided describes? Hi. Rich appears to think that "binning"=chunking in the spatial domain (ie what would be more specifically called pixel binning). I attempted to point out that binning=coarse-graining in general (a special case of which is what ben describes: coarse graining the signal; another is what rich says). I don't think i have to explain how what i say differs from what ben says, because I am in fact saying that it does not. >> >> Why did Nikon use a term that flies in the face of standardized >> >> terminology? [quoted text clipped - 25 lines] >I don't think i have to explain how what i say differs from what ben >says, because I am in fact saying that it does not. I misread your "generally does refer to what Ben is talking about". Probably still trying to recover from my mind being totally boggled by the previous article... ;-) SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com > I misread your "generally does refer to what Ben is talking about". Thought so :) Hola tomm42, On 01/01/2007, at 16:52, you wrote: > I always shoot compressed RAW on my D200. Though there can be a > theoretical difference, haven't seen an example . I have seen several > examples of how there is very little to no difference. This is my (little) experience so far, basically no difference. But I will check on highlights as pointed by others. Thanks. SignatureSaludos, Miguel > Hola tomm42, > [quoted text clipped - 5 lines] > This is my (little) experience so far, basically no difference. But I > will check on highlights as pointed by others. Thanks. Miguel, recently, this link was discussed here (don't remember the thread), but I found the information on the page very useful. Since reading that, I also shoot compressed NEF. http://regex.info/blog/photo-tech/nef-compression/ Cheers ink Hola ink, On 02/01/2007, at 14:03, you wrote: > recently, this link was discussed here (don't remember the thread), but I > found the information on the page very useful. Since reading that, I also > shoot compressed NEF. > > http://regex.info/blog/photo-tech/nef-compression/ Thank you for the pointer. Interesting and, for me, convincing article. I have just switch my D200 to 'RAW compression=ON' and I foresee it will stay that way most of the time. SignatureSaludos, Miguel >>Hola tomm42, >> >>On 01/01/2007, at 16:52, you wrote: > Miguel, > [quoted text clipped - 3 lines] > > http://regex.info/blog/photo-tech/nef-compression/ This seems clear that there is no useable benefit from using uncompressed RAW files. Thanks. >>>Hola tomm42, >>> [quoted text clipped - 8 lines] >This seems clear that there is no useable benefit from using >uncompressed RAW files. Thanks. That is true, unless you actually *use* the highlights. Wedding photography, flowers, snow scenes, ... SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com >>>>Hola tomm42, >>>> [quoted text clipped - 11 lines] > That is true, unless you actually *use* the highlights. Wedding > photography, flowers, snow scenes, ... Even so it's is negligible ugly grit from what I gather here. >>>>recently, this link was discussed here (don't remember the >>>>thread), but I found the information on the page very [quoted text clipped - 3 lines] >>>This seems clear that there is no useable benefit from using >>>uncompressed RAW files. Thanks. If you take a bit of effort to read that URL carefully, and check out some of the other links provided, it is very difficult to justify the "no useable benefit" claim. >> That is true, unless you actually *use* the highlights. >> Wedding >> photography, flowers, snow scenes, ... > >Even so it's is negligible ugly grit from what I gather here. As I've noted previously, that is not the experience that I've had. Others have said the same. I haven't got the slightest idea where you come up with the "ugly grit" concept. Negligible is a distinct possibility... On an average image, it there is probably no significant difference between compressed and uncompressed NEF images. On scenes that are high key, or in any way can use more detail in the highlights, there is benefit to using uncompressed NEF. This link, from the above URL, written by Fazel Majid, http://www.majid.info/mylos/weblog/2004/05/02-1.html quotes Thom Hogan, "Leaving off Uncompressed NEF is potentially significant--we've been limited in our ability to post process highlight detail, since some of it is destroyed in compression." And the author says, "In conclusion, Thom is right - there is some loss of data, mostly in the form of lowered resolution in the highlights." Majid then goes on to say, "Does it really matter? You could argue it does not, as most color spaces have gamma correction anyway, but highlights are precisely where digital sensors are weakest, and losing resolution there means less headroom for dynamic range compression in high-contrast scenes." Where do you see anyone saying ugly grit? SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com >>>>>recently, this link was discussed here (don't remember the >>>>>thread), but I found the information on the page very [quoted text clipped - 45 lines] > > Where do you see anyone saying ugly grit? On the right side, mouse over the word 'Exaggerated' I would call that ugly useless grit. He says the differences are "Wildly exaggerated. Spectacularly exaggerated." then 36 times more again! Perhaps this is useful for scientific analysis of some sort but I see no value for general photography. I'd love to get more out of highlights but this doesn't seem to help in any meaningful way at all. >>>>>>recently, this link was discussed here (don't remember the >>>>>>thread), but I found the information on the page very [quoted text clipped - 38 lines] >On the right side, mouse over the word 'Exaggerated' I would >call that ugly useless grit. What do you expect a set of pixel "differences" to look like. That "ugly useless grit" is *image detail*. >He says the differences are "Wildly >exaggerated. Spectacularly exaggerated." then 36 times more >again! Perhaps this is useful for scientific analysis of some >sort but I see no value for general photography. I'd love to get Nobody has said that it makes a difference for an average photograph. However, try telling a wedding photographer that high key detail is not significant. Try telling anyone who shoots images of brightly colored flowers that the "ugly grit" is useless. Try telling someone who photographs skiers, dogsled races, or polar bears that it isn't useful. >more out of highlights but this doesn't seem to help in any >meaningful way at all. Then you need to pay more attention to detail. (Irresistable pun intended.) SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com >>>>>>>recently, this link was discussed here (don't remember the >>>>>>>thread), but I found the information on the page very [quoted text clipped - 64 lines] > Then you need to pay more attention to detail. (Irresistable > pun intended.) Show me a meaningful improvement in normal photography that would for instance improve the apprearance of a white wedding dress and I'll switch to uncompressed raw. >>>>>>>>recently, this link was discussed here (don't remember the >>>>>>>>thread), but I found the information on the page very [quoted text clipped - 64 lines] >would for instance improve the apprearance of a white wedding >dress and I'll switch to uncompressed raw. Are you saying that wedding photography, or for that matter any photography with detail in the highlights, is abnormal? An interesting perspective you have, but not one that is valid. SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com >>>>>>>>>recently, this link was discussed here (don't remember the >>>>>>>>>thread), but I found the information on the page very >>>>>>>>>useful. Since reading that, I also shoot compressed NEF. >>>>>>>>>http://regex.info/blog/photo-tech/nef-compression/ >>>>He says the differences are "Wildly >>>>exaggerated. Spectacularly exaggerated." then 36 times more [quoted text clipped - 22 lines] > Are you saying that wedding photography, or for that matter any > photography with detail in the highlights, is abnormal? No, I often have trouble with blown highlights, I just don't see any benefit here. Perhaps for scientific purposes but not aesthetic improvement. >> Are you saying that wedding photography, or for that matter any >> photography with detail in the highlights, is abnormal? > > No, I often have trouble with blown highlights, I just don't see any > benefit here. Perhaps for scientific purposes but not aesthetic > improvement. Proper lighting is more important in this situation than whether or not the RAW file is compressed or not. It's all about compromise. Sometimes you can nail great lighting and sometimes you can't. The miniscule *perceived* gain between file compression is a non-issue. Rita >>>Show me a meaningful improvement in normal photography that >>>would for instance improve the apprearance of a white wedding [quoted text clipped - 5 lines] >any benefit here. Perhaps for scientific purposes but not >aesthetic improvement. You *still* aren't catching on. It is *not* a fix for "blown highlights", and with over exposed highlights it makes no difference if you use one NEF format or the other. It is when you have a very correct exposure of an object with important *detail* in the top two zones. That is *not* some sort of "abnormal" type of photography as you implied above. But it does not apply to the average snapshot either. If *you* don't happen to take pictures of wedding dresses, snowy scenes, bright flowers, jewelery, coins, or anything else the fits the description (or if you do and can't get the exposure correct to begin with), then *you* need not be concerned with whether compression is used or not. But many very "normal" photographers do. SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com >>>>Show me a meaningful improvement in normal photography that >>>>would for instance improve the apprearance of a white wedding [quoted text clipped - 22 lines] > > But many very "normal" photographers do. Show me an example where there is any aesthetic improvement. >>>>>Show me a meaningful improvement in normal photography that >>>>>would for instance improve the apprearance of a white wedding [quoted text clipped - 22 lines] > >Show me an example where there is any aesthetic improvement. Wedding pictures of women in white dresses. Brightly colored flowers. Textures that are white, such as a polar bear on an ice flow. Why do you keep saying these things to not exist? How many times do you need them listed? SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com > Wedding pictures of women in white dresses. > [quoted text clipped - 4 lines] > Why do you keep saying these things to not exist? How many times > do you need them listed? How about stop listing them and show just one good example, this should not be too hard should it? Scott > > Wedding pictures of women in white dresses. >> [quoted text clipped - 10 lines] > > Scott And upset the chance to remind all us "unwashed masses" how stupid we are for not agreeing, understanding and "needing them listed" a multiplicity of times so our feeble brains can grasp their theoretical intellectual superiority? Don't hold your hand over you-know-where waiting for that to happen. The predictability is enough to make one laugh! Bob > > Wedding pictures of women in white dresses. >> [quoted text clipped - 8 lines] >not >be too hard should it? Then it wouldn't be hard for you to show one good example that there is no difference. In fact, for something that subtle is indeed difficult. However, the URL cited previously did have everything needed to understand. Of course what was claimed to be "ugly grit" is the error that is introduced *into* the compressed data. It it is indeed "ugly grit", then that is indeed exactly why non-compressed is better. SignatureFloyd L. Davidson <http://www.apaflo.com/floyd_davidson> Ukpeagvik (Barrow, Alaska) floyd@apaflo.com > > > Wedding pictures of women in white dresses. > >> [quoted text clipped - 11 lines] > Then it wouldn't be hard for you to show one good example that > there is no difference. There already has been a link showing just that. > In fact, for something that subtle is indeed difficult. > However, the URL cited previously did have everything needed to > understand. Well there was talk, but is it true? > Of course what was claimed to be "ugly grit" is the error that > is introduced *into* the compressed data. It it is indeed "ugly > grit", then that is indeed exactly why non-compressed is better.The ugly grit was the difference between the compressed and uncompressed files, but it has to be processed a lot just to see it. If there really is value to using the uncompressed it should be possible to show this. Scott > > Wedding pictures of women in white dresses. >> [quoted text clipped - 8 lines] > not > be too hard should it? Hey stupid, you don't have a picture of a white wedding dress. Just freaking find one and you will see. Sheesh. > Scott > Hola, > [quoted text clipped - 8 lines] > What I would like to get are opinions from D200 users and their reasons > for using or not using compressed NEF. Consider converting your uncompressed Nikon RAW into lossless Adobe DNG. It saves anywhere from 20 - 40% of the filespace depending on image content. This won't help with "in camera" storage but will help with archiving space. Cheers, Alan Signature-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: Remove FreeLunch. Hola Alan, On 01/01/2007, at 19:15, you wrote: > Consider converting your uncompressed Nikon RAW into lossless Adobe DNG. > It saves anywhere from 20 - 40% of the filespace depending on image > content. > This won't help with "in camera" storage but will help with archiving space. Thanks for pointing this out, I wasn't aware of DNG's space saving. I still use my beloved RawShooter Premium for RAW processing and, although it can process DNGs, it can't save as DNG. I'll have to use PS CS2 for this. BTW, maybe an OT question for this group. When saving as DNG with ACR, why is there an option to "Embed Original RAW file"? SignatureSaludos, Miguel > BTW, maybe an OT question for this group. When saving as DNG with ACR, > why is there an option to "Embed Original RAW file"? For those who really want to be sure they have the original. In which case the DNG file contains the DNG + the camera RAW for a fairly large file (about 1.6 to 1.8 X the original in size). The only advantage to this is file management, eg: everything is kept together. I've begun using DNG for some projects and have every faith in it for the long term. Cheers, Alan Signature-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: Remove FreeLunch. Hola Alan, On 02/01/2007, at 15:51, you wrote: >> BTW, maybe an OT question for this group. When saving as DNG with ACR, >> why is there an option to "Embed Original RAW file"? > For those who really want to be sure they have the original. In which > case the DNG file contains the DNG + the camera RAW for a fairly large > file (about 1.6 to 1.8 X the original in size). > The only advantage to this is file management, eg: everything is kept > together. I think I'm missing something here. If DNG + the camera RAW are saved as a single file, is there somewhere an option to use one (DNG) or the other (original RAW) when opening it? Maybe I haven't looked well enough, but I haven't seen any such option yet. SignatureSaludos, Miguel > I think I'm missing something here. If DNG + the camera RAW are saved as > a single file, is there somewhere an option to use one (DNG) or the > other (original RAW) when opening it? Maybe I haven't looked well > enough, but I haven't seen any such option yet. Just tested this. When opening it I get the image in the RAW converter (PSE3). No indication as to whether it's the DNG or the original RAW data being used. (I assume it's the DNG). I have no idea how one would then get the original RAW. And frankly, I don't care as I won't be using the embed feature. Cheers, Alan Signature-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: Remove FreeLunch. Hola Alan, On 02/01/2007, at 18:17, you wrote: >> I think I'm missing something here. If DNG + the camera RAW are saved as >> a single file, is there somewhere an option to use one (DNG) or the [quoted text clipped - 6 lines] > > I have no idea how one would then get the original RAW. As Barry Pearson has indicated in his reply, with DNG Converter. > And frankly, I don't care as I won't be using the embed feature. And I would have said the same if I had replied to you before reading Barry Pearson's message and replying to him. But after playing a bit with DNG Converter I now have more questions than before. SignatureSaludos, Miguel [snip] > If DNG + the camera RAW are saved as > a single file, is there somewhere an option to use one (DNG) or the > other (original RAW) when opening it? Maybe I haven't looked well > enough, but I haven't seen any such option yet. You extract the embedded raw files using the DNG Converter, which has an "Extract ..." button. You identify the folder with the DNGs, (which are unchanged by this process), and a folder to hold the extracted raw files. I'm not aware of any products that, given a DNG file, will use the embedded raw file instead of the DNG data. (Perhaps there are such products?) SignatureBarry Pearson http://www.barry.pearson.name/photography/ Hola Barry, On 03/01/2007, at 0:13, you wrote: >> If DNG + the camera RAW are saved as >> a single file, is there somewhere an option to use one (DNG) or the >> other (original RAW) when opening it? Maybe I haven't looked well >> enough, but I haven't seen any such option yet. > You extract the embedded raw files using the DNG Converter, which has > an "Extract ..." button. You identify the folder with the DNGs, (which > are unchanged by this process), and a folder to hold the extracted raw > files. Thanks, I had completely forgot DNG Converter existed. And it works. > I'm not aware of any products that, given a DNG file, will use the > embedded raw file instead of the DNG data. (Perhaps there are such > products?) I'm not aware of any either. But, maybe because "I was born curious", the more I learn the more questions that come into my mind. For example, in the DNG Converter Preferences, if the 'Preserve Raw Image' option is selected for 'Image Conversion Method', an information text appears saying: ,----- [ ] | The image is stored in the original "mosaic" format, if possible, which | maximizes the amount of data preserved. Mosaic image data can be | converted to linear data, but the reverse is not possible. `----- And this raises the following questions (I'm not asking you or anyone, I'm just thinking out loud): 1.- What does "if possible" really mean? 2.- What does "which maximizes the amount of data preserved" really mean? Doesn't conversion from RAW to DNG preserve *all* the data? 3.- Is this the reason why original RAW can be embedded, because DNG perhaps "doesn't guarantee" that all original data is preserved? I guess that to satisfy my curiosity I need to do some more reading about DNG format. SignatureSaludos, Miguel [snip] > For example, > in the DNG Converter Preferences, if the 'Preserve Raw Image' option is [quoted text clipped - 10 lines] > I'm just thinking out loud): > 1.- What does "if possible" really mean? The DNG specification currently can't cater properly for the Sigma/Foveon X3F sensor data in its original state. So it does a simple raw conversion (using Foveon code, I believe) to a "linear" 3-channel form. (This is a non-raw version of DNG known as Linear DNG). http://www.barrypearson.co.uk/articles/dng/linear.htm > 2.- What does "which maximizes the amount of data preserved" really > mean? Doesn't conversion from RAW to DNG preserve *all* the data? It preserves what Adobe can understand and cater for by reverse engineering. Some formats, such as NEFs, CR2s, PEFs, etc, are well-behaved and reasonably-well understood, and the converter is claimed to preserve it all. Others, including ORFs, apparently have badly-laid-out metadata, and the converter loses some of it. The image data should be preserved. http://www.barrypearson.co.uk/articles/dng/safety.htm#completeness > 3.- Is this the reason why original RAW can be embedded, because DNG > perhaps "doesn't guarantee" that all original data is preserved? The main reason is that some products can read the original raw files but not DNGs. This is typical of camera manufacturer's products (although some handle DNG). http://www.barrypearson.co.uk/articles/dng/not_yet.htm I think that people should ask themselves how many copies of their raw image data, on different media, they want, in case of media failures? A disc may fail - are they willing to lose their original image data in that case? If not, they need at least 2 separate copies. (Embedding one in the other on the same disc won't do). In which case, don't embed, but keep a copy of the original raw files on one disc (could be CD or DVD) and use the DNGs on another disc. http://www.barrypearson.co.uk/articles/dng/safety.htm#workflow > I guess that to satisfy my curiosity I need to do some more reading > about DNG format. There are 15-20 pages here! http://www.barrypearson.co.uk/articles/dng/ SignatureBarry Pearson http://www.barrypearson.co.uk/photography/ Hola Barry, On 03/01/2007, at 11:33, you wrote: <snipped quite a bit> >> I guess that to satisfy my curiosity I need to do some more reading >> about DNG format. > > There are 15-20 pages here! > http://www.barrypearson.co.uk/articles/dng/ Thanks for this and your other replies in this same thread. You seem to be quite knowledgeable of this issue and I have learned a lot from your messages, But, the more I learn the more I feel I have to do some (much) more reading. Thanks for the pointers and links. SignatureSaludos, Miguel > [snip] >> For example, [quoted text clipped - 11 lines] >> I'm just thinking out loud): >> 1.- What does "if possible" really mean? > The DNG specification currently can't cater properly for the > Sigma/Foveon X3F sensor data in its original state. Why not? The LinearRaw PhotometricInterpretation value is supported by the current DNG specification; as it says in the spec, "The LinearRaw PhotometricInterpretation value is intended for use by cameras that do not use color filter arrays, but instead capture all color components at each pixel. It can also be used for CFA data that has already been de-mosaiced." > So it does a simple raw conversion (using Foveon code, I believe) to > a "linear" 3-channel form. (This is a non-raw version of DNG known > as Linear DNG). > http://www.barrypearson.co.uk/articles/dng/linear.htm What is so special about Foveon data that the current DNG spec can't support it? Andrew. On Jan 3, 5:39 pm, Andrew Haley <andre...@littlepinkcloud.invalid> wrote: [snip] > > The DNG specification currently can't cater properly for the > > Sigma/Foveon X3F sensor data in its original state. [quoted text clipped - 14 lines] > What is so special about Foveon data that the current DNG spec can't > support it? This is an area that that I don't understand well, partly because I know less about the handling of the X3F sensor than CFA sensors, and partly because I haven't got a response to questions about this I have posted in the Adobe forums. (Hence my qualifications above such as "I believe"). I believe the problem is that the algorithm for converting the X3F sensor values to more normal RGB values is not (currently) catered for in the DNG specification. The required parameters are not there. (The X3F values need work done to them before they conform to more normal linear RGB values). The key question is: can a DNG colour transformation matrix map between the X3F color space coordinates and CIE XYZ coordinates? I have been led to believe that the answer is "no" because the required mapping is more complicated. I believe what happens is that, instead of simply storing the original X3F values as PhotometricInterpretation = LinearRaw, it converts them before storing them, (as LinearRaw), so that the rest of the DNG parameters can work on the new values. The DPReview Sigma forum has several discussions on this, and they do not convince me that we know the full story. If anyone has evidence that the Original X3F values are stored, I would like to see it. SignatureBarry Pearson http://www.barrypearson.co.uk/photography/ > Consider converting your uncompressed Nikon RAW into lossless Adobe DNG. > It saves anywhere from 20 - 40% of the filespace depending on image > content. It's my understanding this just removes the embedded joeg in the raw file. >> Consider converting your uncompressed Nikon RAW into lossless Adobe >> DNG. It saves anywhere from 20 - 40% of the filespace depending on >> image content. > > It's my understanding this just removes the embedded joeg in the raw file. Not the way I understand it. My camera produces MRW's and JPG's (if so setup). The MRW's are about constant in size; the JPG's are variable. The DNG's are smaller, but not by the size of the JPG's. Following are MRW and in-camera JPG sizes from the Maxxum 7D. DNG's produced with Adobe DNG converter 3.1 from the MRWs'. -------- Sizes in KB ------- file MRW DNG Diff JPG 4616 8976 6128 2848 2087 4617 8968 5995 2973 1517 4618 8968 5702 3266 1755 4619 8968 5988 2980 1538 The Minolta scheme produces as separate THM file (thumbnail) file as well of about 4 KB. Cheers, Alan. Signature-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: Remove FreeLunch. > Hola, > [quoted text clipped - 10 lines] > > TIA. I have been using the Compressed NEF settings on and off for about 2 years in the D2 series, trying to find any differences in the finished product. So far none have been seen by me or others and until these theoretical differences in output become visible in my finished work, I will continue with my workflow as is. Of course the "Pixel Peepers" will point out the theoretical differences which I just "fail to see", but are "obvious to even the uneducated eye". So be it. I now mostly compress both in camera (compressed NEF) and in storage (compressed DNG with contained NEF). I also work from the compressed DNG (without the contained NEF) for output generation. Bob Hola Robert, On 01/01/2007, at 20:19, you wrote: > I have been using the Compressed NEF settings on and off for about 2 years > in the D2 series, trying to find any differences in the finished product. [quoted text clipped - 4 lines] > which I just "fail to see", but are "obvious to even the uneducated eye". > So be it. Thanks. From what you and others say I think that, for an amateur like myself, there should be no practical difference between compressed and uncompressed. > I now mostly compress both in camera (compressed NEF) and in storage > (compressed DNG with contained NEF). I also work from the compressed DNG > (without the contained NEF) for output generation. I have already asked this question but, if compressed DNG is lossless, why save the embedded NEF and what's the advantage? SignatureSaludos, Miguel According to Miguel <glruamREMOVETHIS@gmail.com>: > Hola Robert, > > On 01/01/2007, at 20:19, you wrote: [ ... ] > > I now mostly compress both in camera (compressed NEF) and in storage > > (compressed DNG with contained NEF). I also work from the compressed DNG > > (without the contained NEF) for output generation. > > I have already asked this question but, if compressed DNG is lossless, > why save the embedded NEF and what's the advantage? Since I don't see any information posted (yet) to your question, I'll have a try. 1) The sensor has in each sensel location filtering so the sensel produces information for green, red, or blue. (Out of four sensels forming a square, two are green, one red and one blue. 2) To produce a usable image, the values of these are combined in a weighted average system to produce pixels containing information from all three colors found in adjacent sensels. 3) There are various forms of weighting, and the color contributions come from multiple adjacent sites, so you have no way of reversing the process to recover exactly what was in each sensel before the weighted averaging. 4) As long as you are happy with the algorithm used to perform the weighted averaging, you can continue to use the output in the DNG image. 5) However, there may be a benefit to be obtained at a later date by using a different algorithm to perform the weighted averaging, and to do this, you must have the original individual sensel data. I hope that this helps, DoN. SignatureEmail: <dnichols@d-and-d.com> | Voice (all times): (703) 938-4564 (too) near Washington D.C. | http://www.d-and-d.com/dnichols/DoN.html --- Black Holes are where God is dividing by zero --- > According to Miguel <glruamREMOVET...@gmail.com>: [snip] > > I have already asked this question but, if compressed DNG is lossless, > > why save the embedded NEF and what's the advantage? > > Since I don't see any information posted (yet) to your question, > I'll have a try. [snip] > 4) As long as you are happy with the algorithm used to perform the > weighted averaging, you can continue to use the output in the [quoted text clipped - 4 lines] > and to do this, you must have the original individual sensel > data. [snip] The DNG contains the original image data - it doesn't demosaic. (Foveon sensors are different - I have just mentioned them in another post). Normal DNG is a true raw image file format. You CAN convert to Linear DNG, which demosaics, but that isn't the default, and isn't the reason for having an "embed" option. SignatureBarry Pearson http://www.barrypearson.co.uk/photography/ Hola DoN., On 03/01/2007, at 4:01, you wrote: >> I have already asked this question but, if compressed DNG is lossless, >> why save the embedded NEF and what's the advantage? [quoted text clipped - 25 lines] > > I hope that this helps, It certainly does help Don, thanks for taking the time. I now understand why there is an option to embed original RAW. I would assume so, but it would be nice to see if, given a RAW, the few products that can write DNGs would produce exactly the same DNG data from it. SignatureSaludos, Miguel [snip] > I would assume so, but it would be nice to see if, given a RAW, the few > products that can write DNGs would produce exactly the same DNG data > from it. Raw files, including DNG files, hold lots of types of information, not just the raw image data. For example, they can hold metadata describing the camera details, lots of information about the shot itself, perhaps a record of editing instructions, and things like JPEG previews, etc. http://www.barrypearson.co.uk/articles/dng/profiles.htm Some of these should be the same whatever converts to DNG. The raw image data itself should be preserved, although there are various ways it might be stored (for example lossless-compressed or not). This is what is most important, of course. Camera details can vary. I have seen different colour matrices in DNGs produced by Adobe and Pentax products, for example. These are partly derived from the particular instance of the camera used to generate the profiles - Pentax and Adobe presumably used different actual cameras even for the same model. I'm not convinced that Adobe's colour matrices are the best - they appear to over-emphasise reds. (I calibrated my camera to overide the default settings for it in ACR). I don't know why, and haven't had an answer from the Adobe forums. (In fact, sometimes Adobe have later recreated their colour matrices for particular camera models, and offer users the option of which to use). JPEG previews are allowed to vary in number, size and quality. http://www.barrypearson.co.uk/articles/dng/benefits.htm#previews Editing-metadata varies from one case to another. ACR stores the values of its own sliders in the DNG after editing, and at the moment Lightroom Beta stores different values from ACR! (It is intended that the released Lightroom 1.0 will use the same values as the released ACR 4.x in CS3). But another raw converter storing values in the DNG will typically store its own slider-values, which won't be the same as Adobe's. Note that much of what is stored in the DNGs is not stored at all by many products in other formats. Typically colour profiles, etc, are built into the raw converters. And they obviously vary from one products to another. So the fact that DNGs differ should not be a surprise - they are holding information that doesn't even exist in other raw files, some of it potentially product-dependent, especially after editing. Some information held in DNG files is optional, or has alternative permitted formats, for example manufacturer's proprietary data. It is important to ask the right questions about what is "the same"! Most important of all is the raw image data itself, and DNG allows that to be preserved exactly the same, except (I believe) for Foveon sensors. http://www.barrypearson.co.uk/articles/dng/safety.htm SignatureBarry Pearson http://www.barrypearson.co.uk/photography/ > What I would like to get are opinions from D200 users and their reasons > for using or not using compressed NEF. The compression throws away information in the highlights. How it's done was described in other replies. It doesn't appear to make much difference in the final picture, but it could do so in theory. Thus -- storage is cheap. I've never run out of storage in the field, or even come close, and if I do I'll buy more. I shoot uncompressed not because I've done tests and seen a difference, but because, well, why throw away information if you don't have to? DNG compression is lossless. I convert to DNG as soon as I download to the computer, and the file sizes are cut in half with no loss at all, which gives the benefit without the loss. SignatureJeremy | jeremy@exit109.com >> What I would like to get are opinions from D200 users and their reasons >> for using or not using compressed NEF. [quoted text clipped - 10 lines] > the computer, and the file sizes are cut in half with no loss at all, > which gives the benefit without the loss. There is a six page article (starting on page 174) on shooting "RAW" in the December issue of the magazine, "Practical Photography," The camera he is using looks to be a Nikon D200 or D2X. I use a D200 most of the time. His article has convinced me to start shooting mostly uncompressed "RAW." This same magazine reviews storage devices (page 122). Storage of large photos is not a problem if you use a device such as an "Epson Multimedia Storage Viewer." The Epson Storage Multimedia Storage Viewers" are sold in 20,40, 80, and 120 GB sizes. I stored about 1,000 photos on my last trip in a "Epson Multimedia Storage Viewer" and didn't use all the space. Robert Hola Robert, On 02/01/2007, at 6:49, you wrote: > There is a six page article (starting on page 174) on shooting "RAW" in the > December issue of the magazine, "Practical Photography," The camera he is > using looks to be a Nikon D200 or D2X. I use a D200 most of the time. His > article has convinced me to start shooting mostly uncompressed "RAW." Do you know if the article is available on-line? SignatureSaludos, Miguel Hola Jeremy, On 02/01/2007, at 5:37, you wrote: > The compression throws away information in the highlights. How it's done > was described in other replies. It doesn't appear to make much difference [quoted text clipped - 3 lines] > tests and seen a difference, but because, well, why throw away information > if you don't have to? > DNG compression is lossless. I convert to DNG as soon as I download to > the computer, and the file sizes are cut in half with no loss at all, > which gives the benefit without the loss. Thanks for your reply. As I have written in some of my other replies, I gather (as a summary) that there is no practical different (specially for an amateur like me) except perhaps for some fine detail in highlights in some shots. SignatureSaludos, Miguel |
Reply to this Message |
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2009 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.