 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
| PhotoKB HomeDiscussion Groups Digital Photography Digital PhotoDSLR CamerasZLR CamerasPoint & Shoot CamerasFilm Photography 35 mmLarge FormatMedium formatDarkroomFilm and LabsOther EquipmentPhoto Technique Nature PhotographyPeople PhotographyTechnique GeneralGeneral Photo Topics General TopicsAustralian PhotographyUK PhotographyDirectoryPhoto Clubs |
Photo Forum / Digital Photography / DSLR Cameras / February 2007Dynamic Range of RAW digital sensor data
I am looking for hard data on the dynamic range of different digital sensors, such as the sensors used in the Canon EOS 5D and Nikon D200. I've found a lot of articles that is some way or another shows how photon counts corresponds to RAW imager output values, such as the diagram on this page: http://www.dpreview.com/learn/?/key=dynamic+range However, the diagram only shows the relationship between the number of photons in a (hypothetical) image sensor and output DN /data numberss/ (presumably between 0 and 4096) on a log scale, which does not really reveal what the input luminosity was. I've also found some articles that reports on various dynamic range measurements /after/ gamma and tone curve adjustments has been applied to the data, which don't necessarily tell you what the dynamic range was before applying the adjustments. I am loking for is this kind of data (either numbers or diagrams) for real sensors, and data that shows the relationship between a spesific photon count and a specific real world luminance or EV (at the image sensor's "native" sensitivity). I've checked my library and searched the web, but to no avail. Do anyone know about a good source for this type of data?  Signature- gisle hannemyr [ gisle{at}hannemyr.no - http://hannemyr.com/photo/ ] ------------------------------------------------------------------------ Sigma SD10, Kodak DCS460, Canon Powershot G5, Olympus 2020Z ------------------------------------------------------------------------ > I've checked my library and searched the web, but to no avail. > Do anyone know about a good source for this type of data? Did you look at Roger Clarke's site? I seem to recall it is quite detailed in this regard, and may include the specifics you're looking for. Signature-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: Remove FreeLunch. >> I've checked my library and searched the web, but to no avail. >> Do anyone know about a good source for this type of data? > Did you look at Roger Clarke's site? I seem to recall it is quite > detailed in this regard, and may include the specifics you're > looking for. Yes, for instance "Dynamic Range and Transfer Functions of Digital Images and Comparison to Film": http://www.clarkvision.com/imagedetail/dynamicrange2/ - and a number of related pages linked to from that page, The title is of course promising, but unless I am missing something, the data he reports for both scene and output intensity are /data numbers/ (i.e. the numeric values of the pixels in his image files) before and after RAW conversion, which AFAIK don't reveal actual scene luminosity. Signature- gisle hannemyr [ gisle{at}hannemyr.no - http://hannemyr.com/photo/ ] ------------------------------------------------------------------------ Sigma SD10, Kodak DCS460, Canon Powershot G5, Olympus 2020Z ------------------------------------------------------------------------ > >> I've checked my library and searched the web, but to no avail. > >> Do anyone know about a good source for this type of data? [quoted text clipped - 13 lines] > image files) before and after RAW conversion, which AFAIK don't reveal > actual scene luminosity. Roger uses linear data, I don't know which converter he is using to do this but I do know he is looking at linear data for each channel before it has been de-mosaiced Scott >>> Did you look at Roger Clarke's site? I seem to recall it is quite >>> detailed in this regard, and may include the specifics you're >>> looking for. >> Yes, for instance "Dynamic Range and Transfer Functions of Digital >> Images and Comparison to Film": [quoted text clipped - 6 lines] >> image files) before and after RAW conversion, which AFAIK don't >> reveal actual scene luminosity. > Roger uses linear data, I don't know which converter he is using to > do this I states that he uses "Canon conversion software". > but I do know he is looking at linear data for each channel before > it has been de-mosaiced His input numbers (Scene intensity) are obviously linear data, but he don't pause to tell you what his SN numbers mean in terms of real world light (expressed in e.g. EV, foot-candles or LUX) or the gamma of the sensor data. For instance, if you look at figure 7 in his "dynamicrange2"-paper, you'll find than 100 DN "scene intensity" is converted into 1000 DN "output intensity" by the Canon converter. I can't see how this translates into real life dynamic range without knowing what 100 DN means in terms of light, or - at least - the native gamma of the sensor he pulls these data from. Signature- gisle hannemyr [ gisle{at}hannemyr.no - http://hannemyr.com/photo/ ] ------------------------------------------------------------------------ Sigma SD10, Kodak DCS460, Canon Powershot G5, Olympus 2020Z ------------------------------------------------------------------------ [] > I can't see how this translates into real life dynamic range without > knowing what 100 DN means in terms of light, or - at least - the > native gamma of the sensor he pulls these data from. Gisle, The sensor is linear - gamma = 1 There may be some black level offset, though, so the best-fit equation is likely to be y = Ax + B, where B is quite small. Of course, the lens aperture, transmission, light spectrum etc. would all come into working out light-levels from DN. Cheers, David "David J Taylor": >> I can't see how this translates into real life dynamic range >> without knowing what 100 DN means in terms of light, or - at least >> - the native gamma of the sensor he pulls these data from. > Gisle, > The sensor is linear - gamma = 1 Of course. Silly of me, gamma is a red herring. > There may be some black level offset, though, so the best-fit > equation is likely to be y = Ax + B, where B is quite small. Of > course, the lens aperture, transmission, light spectrum etc. would > all come into working out light-levels from DN. I just can't make sense of his DN numbers. If we look at fig. 8a, in "Dynamic Range and Transfer Functions of Digital Images and Comparison to Film": [ http://www.clarkvision.com/imagedetail/dynamicrange2/ ] he says that "the digital camera keeps going to the bottom end of data below 70 DN", and then refers to his "black hole in the scene measured at 19 DN, Well, the data in the black hole is obviously consisting of noise only (black current noise?) so the noise floor must lie above that. From the scatter plot in fig. 8a, it looks like the his data for the 1DII stretches from 70 DN to 70000 DN on the logarithmic scene scale. I.e. he have a scatter plot showing a 1000:1 linear ratio. This is about equal to 10 EV (or stops). But in the text, he claims that this data demonstrates a 11.7 stop dynamic range! He is obviously interpreting the data different than me, but how he interprets them is beyond me. Signature- gisle hannemyr [ gisle{at}hannemyr.no - http://hannemyr.com/photo/ ] ------------------------------------------------------------------------ Sigma SD10, Kodak DCS460, Canon Powershot G5, Olympus 2020Z ------------------------------------------------------------------------ [] > I just can't make sense of his DN numbers. > [quoted text clipped - 15 lines] > He is obviously interpreting the data different than me, but how > he interprets them is beyond me. I find the fig. 8a graph somewhat confusing, particularly (a) having "the human eye" response on there and (b) missing the major grid lines for scene intensity. I want to be able to compare the digital camera to a straight-line through the origin, and figure 8a doesn't easily allow this. It does show that my "linear" response was incorrect, because of some deliberate "smooth clipping" at the higher end to allow for a higher dynamic range. It's unclear where this is happening - it must be in the RAW to Image conversion. So it's a function of the software. Two points of detail: - He doesn't say the black-hole is at 19 DN, but that the noise measured 19 DN. - He doesn't say that the data he shows here demonstrated 11.7 stops range, but that "Other testing of the noise level versus intensity shows the Canon 1D Mark II has 11.7 stops of dynamic range." There is plenty of scope for explaining all this in a number of different ways, as some will approach if from film photography, some from the physics, some from digital signal processing, and others from a human vision characterisation! Cheers, David > But in the text, he claims that this data demonstrates a 11.7 stop dynamic range! http://www.clarkvision.com/imagedetail/evaluation-1d2/ has a table that shows 11.6 stops at ISO 100. The read-noise at this ISO is 13 electrons; the "minimum discernable signal" is subject to definition, and it appears Clark has defined it to 1 stddev over that distribution: 13 + sqrt(13) == 16 electrons. The 1D2, like all cameras worth owning, is virtually linear all the way until the pixel is full of electrons, some 53000 in this case. So the DR is then simple enough: log(compressed_signal/minimum_signal)/log(2) == log(53000/16)/log(2) == 11.7 stops. > "David J Taylor": > [quoted text clipped - 13 lines] > > I just can't make sense of his DN numbers. DN numbers are simply a linear scale. They could be scaled to electrons (photons). > If we look at fig. 8a, in "Dynamic Range and Transfer Functions of > Digital Images and Comparison to Film": [quoted text clipped - 10 lines] > is about equal to 10 EV (or stops). But in the text, he claims that > this data demonstrates a 11.7 stop dynamic range! The sensor noise goes below the 70 DN of the test chart, thus enabling the extrapolation beyond the test chart. > He is obviously interpreting the data different than me, but how > he interprets them is beyond me. Several points: I did not use the canon converter, I used ImagesPlus which allows extracting linear data from the raw file. The purpose the the http://www.clarkvision.com/imagedetail/dynamicrange2 web page is to compare to film. Like others, I have found measuring dynamic range with variable test targets to be quite difficult, so I have changed to the sensor electronics industry standards, which circumvent the problems, but at the cost of needing dozens of images and more data processing. My results are here: http://www.clarkvision.com/imagedetail/index.html#sensor_analysis Methodology is here: Procedures for Evaluating Digital Camera Sensor Noise, Dynamic Range, and Full Well Capacities; Canon 1D Mark II Analysis http://www.clarkvision.com/imagedetail/evaluation-1d2 Data for many sensors are here: http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary You asked about the Nikon D200 and Canon 5D. Detailed analysis of the D200 is here: http://www.clarkvision.com/imagedetail/evaluation-nikon-d200 I do not have the 5D detailed analysis, but the pixels are the same size and era as the 1D Mark II, so the performance is probably like the 1D Mark II, and analysis by others shows it is close. Roger > Yes, for instance "Dynamic Range and Transfer Functions of Digital > Images and Comparison to Film": > http://www.clarkvision.com/imagedetail/dynamicrange2/ Mr. Clark is using the same totally incorrect dynamic range definition that sensor manufactures use in their marketing. Ignoring the photon shot noise totally. He claims: --> "Further image analysis shows at least 10.6 stops are recorded --> by the canon 1D Mark II camera (the full range of of detail in --> this image, Other testing of the noise level versus intensity --> shows the Canon 1D Mark II has 11.7 stops of dynamic range. In order to have a photographically acquired image that truly holds 11.7 stops scene dynamic range one has to have full well capacity of 11 million electrons due to the photon shot noise alone. That in case of an ideal sensor, some electrons more in order to overcome the noises of a real sensor. 11.7 stops == 2^11.7 == 3327:1 in linear quantity. 3327^2 == 11068929 electrons or 11.07 million electrons full well. And 10.6 stops == 2^10.6 == 1552:1 in linear quantity, 1551^2 == 2408704 electrons or 2.4 million electrons full well. These "results" are _enormously_ incorrect, because of the incorrect definition of the dynamic range. Also, the definition of dynamic range goes down to S/N ratio of 1:1 or 1/sqr(1). The image information at signal level of 1 electron is not usable at all,, the S/N ratio of 1:1 just happens to be part of the definition of dynamic range. In case N electrons are required for acceptable S/N level then sqr(N) must be subtracted from the DR that is expressed in stops. Assuming ideal sensor, more in case of real sensor. BTW: For the material on the above linked page Mr. Clark was using Polaroid SprintScan 4000 scanner, that has way lesser dynamic range than any film. Timo Autiokari > > Yes, for instance "Dynamic Range and Transfer Functions of Digital > > Images and Comparison to Film": > > http://www.clarkvision.com/imagedetail/dynamicrange2/ > > Mr. Clark is using the same totally incorrect dynamic range definition > that sensor manufactures use in their marketing. Clark is correct as per standard engineering practice. Any "marketing" information derived from such is also correct. > Ignoring the photon shot noise totally. Yes. He also ignored the phase of the Moon as well. And why not? It is not relevant. The DR is a measure of how the system responds to its input, from the minimum discernable signal up until compression (however defined). > 11.7 stops == 2^11.7 == 3327:1 in linear quantity. 3327^2 == 11068929 > electrons or 11.07 million electrons full well. Ignoring the incorrect mathematics, your progression from a dimensionless number to a physical unit is most hilarious... > The DR is a measure of how the system responds to its > input, from the minimum discernable signal up until > compression The above is almost correct, should be "...up until clipping". The way the DR is most often expressed by the manufactures of imaging sensor (the same way that also Mr. Clark define it) is _not_ according to your own above statement. The full well capacity divided by the sensor noises is not at all the same as "how the system responds to its input". The input is light. This "DR" (the full well capacity divided by the sensor noises) has no direct relation with the f/stop range of the scene that the camera is able to capture. The only thing that can be derived from such "DR" value is that the true dynamic range of the system is always _much_ lesser. > He also ignored the phase of the Moon as well. And why not? > It is not relevant. It would be very beneficial for you to study the issue a little. Just google "photon shot noise" gets you started very well. Timo Autiokari > > The DR is a measure of how the system responds to its > > input, from the minimum discernable signal up until > > compression > > The above is almost correct, should be "...up until clipping". Serves me right to assume you are capable of abstraction. In most systems, there is a fair amount of gain compression prior to hitting the power supply rails ("clipping"). Optical sensors are something of an exception to this rule: they are staunchly linear until the pixel saturates. Not having designed any myself, I'll speculate that there probably isn't enough input signal even at pixel saturation to drive the following electronics into a significant amount of compression, so practically speaking the clip-point is the compression point of interest here. > The way the DR is most often expressed by the manufactures of imaging > sensor (the same way that also Mr. Clark define it) is _not_ according > to your own above statement. With exactly one exception -- specifically, you -- I haven't seen any use of the word "dynamic range" elsewhere that is substantially different. > This "DR" (the full well capacity divided by the sensor noises) has no > direct relation with the f/stop range of the scene that the camera is > able to capture. DR(camera) = DR(scene) - noise_figure Can't be more direct that that. The "noise figure" (go ahead, google that up too) of the Canon 1D2 at ISO 100 appears to be about 2 stops (assuming a quantum efficiency of ~1/4 and an MDS of 1 stddev over read-noise). > The only thing that can be derived from such "DR" value > is that the true dynamic range of the system is always _much_ lesser. Oh dear, ~14 stops to 11.7 stops. > > He also ignored the phase of the Moon as well. And why not? > > It is not relevant. > > It would be very beneficial for you to study the issue a little. Just > google "photon shot noise" gets you started very well. You desperately need a course in basic signal processing -- audio, RF, doesn't matter much: the concepts apply across the board. This stuff isn't even particularly hard. Certainly not has hard as explaining how you can start with a pure number and arrive at electrons... >> Yes, for instance "Dynamic Range and Transfer Functions of Digital >> Images and Comparison to Film": [quoted text clipped - 3 lines] > that sensor manufactures use in their marketing. Ignoring the photon > shot noise totally. He claims: Timo, You are way off base here. It is more than just marketing by sensor manufacturers, it is the standard used in engineering. (I would have responded sooner but I was in Africa the last couple of weeks.) I'll give the following link: http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary then go to the bottom of the page to the references and download the Kodak sensor data sheets (those marked KAF). You'll see the same methods and definitions used that I use. On the same above web page, I've plotted the Kodak data along with my data and those of others who have studies sensors. We have a collectively consistent picture. > --> "Further image analysis shows at least 10.6 stops are recorded > --> by the canon 1D Mark II camera (the full range of of detail in [quoted text clipped - 12 lines] > And 10.6 stops == 2^10.6 == 1552:1 in linear quantity, 1551^2 == > 2408704 electrons or 2.4 million electrons full well. You are confusing precision in intensity measurements with range of measurements. Let's try another analogy: estimating length with your eye. For a small length, like a mm, you might have an error of a fraction of a mm (let's say 0.25 mm error at length 0.5 mm). At 1 meter, you may have an error of a few cm. At 100 meters, you might have an error of a few meters. Over 100 meters, what dynamic range in length measurements do you have? It is on the order of: 100 meters / 0.5 mm = 100 /0.0005 = 20,000. The range of measurement is 20,000 even though the precision is not high. You don't need that 0.25 mm accuracy to know that 100 meters is big. Same with photons. The sensor can measure 50,000 photons (electrons) with an error of sqrt(50,000) =223, but you know it is still a big number regardless of the error bar. Then at small numbers of photons, e.g. 4, the noise is sqrt(4)=2. The range of such a measurement is 50,000/4 = 12,500, even though the signal-to-noise ratio never exceeds 223. So, don't confuse signal-to-noise ratio with dynamic range. > These "results" are _enormously_ incorrect, because of the incorrect > definition of the dynamic range. wrong. Your definition of dynamic range is incorrect. I use electronics industry standard definitions. > Also, the definition of dynamic range goes down to S/N ratio of 1:1 or > 1/sqr(1). The image information at signal level of 1 electron is not > usable at all,, the S/N ratio of 1:1 just happens to be part of the > definition of dynamic range. In case N electrons are required for > acceptable S/N level then sqr(N) must be subtracted from the DR that is > expressed in stops. Assuming ideal sensor, more in case of real sensor. We've been over this before, yet you have never shown any data that proves your point and you ignore data that proves you are wrong. See Figure 5 and the paragraph above Figure 5 at: http://www.clarkvision.com/photoinfo/night.and.low.light.photography where it is shown and discussed how less than a fraction of the noise still shows image detail. For example, Figure 5 set 5, patch A has 1.2 photons is clearly discernible (electrons)/pixel on average, with noise of 3.9 electrons, or over 3 times smaller than the noise. It is the same with high ISO film: the grain is quite large and a lot of image components are at a S/N less than 1. You have yet to respond with a definition of acceptable S/N ratio that will include film as a usable medium (and note it was acceptable for decades). > BTW: For the material on the above linked page Mr. Clark was using > Polaroid SprintScan 4000 scanner, that has way lesser dynamic range than > any film. Wrong. (Ironic, as your own definition of dynamic range says film effectively has no dynamic range!) To the contrary, the sprintscan has pulled data out of images that I nor professional labs could not print. It has more than adequate dynamic range for the task. Roger > I am looking for hard data on the dynamic > range of different digital sensors, Unfortunately such hard data is not available unless you find the manufacturer's data-sheet of the sensor. And if you do find such a data-sheet then you must calculate the effect of photon shot noise in to the specified properties since all the sensor manufacturers ignore the photon shot noise totally. Sensor manufacturers simply calculate the dynamic range as: The full well capacity in electrons in divided by the noise electrons that are induced by the sensor. This is the proper definition for many other instrumentation but not for any instrumentation that measures light (photons). Light has the property called photon shot noise (also called as the Poisson noise) and the quantity of this noise is the square of the electrons (electrons are those photons that gets detected). In photographic sense the sensor manufacturer's definition of the dynamic range is the same as a shooting situation where an object surface in the scene is captured by the camera in such way that the camera records the surface at the maximum output level (255 in 8-bit/c notation) but there is not a single photon reflecting from that surface (so it appears to be absolutely black). Obviously such definition and specification of the dynamic range is nonsense. For example, the true dynamic range for a full well that has the capacity of 50000 electron is sqrt(50000) or 223:1, due to the photon shot noise. Those noises that the sensor manufacturers regard as noises then decrease this further. In other words the true dynamic range of a light sensing sensor can never be equal to the square root of the full well capacity in electrons. It can be rather close to that in case the sensor induced noises are very small (this is the case with actively cooled sensors that are often used in scientific applications). Not all that 233:1 dynamic range is usable since we do not accept such image information as _useful image information_ that has signal to noise ratio of sqrt(1) or 1:1. E.g. at 16 electrons the signal to noise ratio is just 4:1 due to the photon shot noise only, such image information looks _very_ bad, very noisy. But if we do accept that then a sensor that has full well capacity of 50000 electrons has _useful_ dynamic range of 233/4 or 58.25:1 or 5.9 f/stops only. Now then, the task of measuring the dynamic range of a digital camera is incredibly a difficult one. One major error source are the internal reflections: Between the individual lenses of the camera lens, between the blur-filter and the surface of the exit lens of the camera lens, between the blur filter and the sensor, and inside the sensor compartment. These reflections create a more or less diffuse fog of light that adds to the measurement so in the dark end the measurement will be way incorrect. What happens is that when testing the dynamic range e.g. using a Stouffer step wedge even the 3.0D patch _seems_ to get recorded, the camera _seems_ to output some signal for the 3.0D patch but the reality is that the signal is mostly from the fog. But people happily go and announce that the dynamic range is more than 3.0D or more than 1000:1 or more than 9.966 f/stops. These reflections are one of the main reasons for the incorrect/unrealistic high DR test results that can be found on the Web. An other major error source is the noise reduction, some of it is performed already before the raw data is written. The noise reduction has the effect that even if a camera seem to detect some signal for a very dark, large, uniform patch of a step wedge, it can not deliver fine structured image detail that reside at equally low luminance levels, the noise reduction algorithms will clean such fine structured image detail away. So, such signal is not inside the useful dynamic range of the camera nor inside the true dynamic range of the camera. Unless the camera is only used for recording such large uniform surface areas like the patches of the step wedge. Timo Autiokari > > I am looking for hard data on the dynamic > > range of different digital sensors, [quoted text clipped - 4 lines] > the specified properties since all the sensor manufacturers ignore the > photon shot noise totally. Wrong. http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary several sensors analyzed at (and references to others): http://www.clarkvision.com/imagedetail/index.html#sensor_analysis > Sensor manufacturers simply calculate the dynamic range as: > The full well capacity in electrons in divided by the noise electrons > that are induced by the sensor. This is the proper definition for many > other instrumentation but not for any instrumentation that measures > light (photons). Wrong. It is the correct definition for light sensors and is the definition used in the electronics industry. > Light has the property called photon shot noise (also called as the > Poisson noise) and the quantity of this noise is the square of the > electrons (electrons are those photons that gets detected). Correct. > In photographic sense the sensor manufacturer's definition of the > dynamic range is the same as a shooting situation where an object [quoted text clipped - 3 lines] > (so it appears to be absolutely black). Obviously such definition and > specification of the dynamic range is nonsense. Wrong. You forget that 8-bit image data are gamma encoded. > For example, the true dynamic range for a full well that has the > capacity of 50000 electron is sqrt(50000) or 223:1, due to the photon [quoted text clipped - 4 lines] > sensor induced noises are very small (this is the case with actively > cooled sensors that are often used in scientific applications). Wrong. You confuse signal-to-noise ratio with dynamic range. > Not all that 233:1 dynamic range is usable since we do not accept such > image information as _useful image information_ that has signal to noise > ratio of sqrt(1) or 1:1. Wrong. > E.g. at 16 electrons the signal to noise ratio is just 4:1 due to the > photon shot noise only, such image information looks _very_ bad, very > noisy. But if we do accept that then a sensor that has full well > capacity of 50000 electrons has _useful_ dynamic range of 233/4 or > 58.25:1 or 5.9 f/stops only. Wrong. > Now then, the task of measuring the dynamic range of a digital camera is > incredibly a difficult one. No it is not if you have access to the raw data. > One major error source are the internal reflections: Between the > individual lenses of the camera lens, between the blur-filter and the > surface of the exit lens of the camera lens, between the blur filter and > the sensor, and inside the sensor compartment. If you use correct methods, none of the above are problems. Follow the procedures here, which is the industry standard method for measuring properties: Procedures for Evaluating Digital Camera Sensor Noise, Dynamic Range, and Full Well Capacities; Canon 1D Mark II Analysis http://www.clarkvision.com/imagedetail/evaluation-1d2 > These reflections create a more or less diffuse fog of light that adds > to the measurement so in the dark end the measurement will be way [quoted text clipped - 19 lines] > > Timo Autiokari I suggest more research before you post again, and then if you don't change, be prepared to tell why the entire electronics industry and scientists are wrong and you are right. There is a Nobel prize waiting. Roger |
Reply to this Message |
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2009 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.