 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
| PhotoKB HomeDiscussion Groups Digital Photography Digital PhotoDSLR CamerasZLR CamerasPoint & Shoot CamerasFilm Photography 35 mmLarge FormatMedium formatDarkroomFilm and LabsOther EquipmentPhoto Technique Nature PhotographyPeople PhotographyTechnique GeneralGeneral Photo Topics General TopicsAustralian PhotographyUK PhotographyDirectoryPhoto Clubs |
Photo Forum / Digital Photography / DSLR Cameras / January 200510D ISO 1600 is pushed one stop from 800
I just took blackframe and super-overexposed white wall images at ISOs 800, 1600, and 3200 on my 10D. I converted them all to uncompressed DNG files, and looked at the RAW data in a hex editor (set to look at the data as decimal numbers, assuming 16-bit unsigned data). The data patterns are the same for the 1600 and 3200, and both are a little strange. You get a long string of even numbers, then a long string of perfectly alternating odd and even numbers, then a long string of odd numbers, then a long string of alternating numbers again. I don't know if it's the camera or the DNG converter that is doing this to the data (adding or subtracting one to blocks and striped blocks), but it's quite clear that there are only 11 bits used for both ISO 3200 *AND* ISO 1600. Here is some sample data from the ISO 1600 blackframe: http://www.pbase.com/jps_photo/image/38034746 Note how only the vertical stripes marked have odd numbers.  Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< > I just took blackframe and super-overexposed white wall images at ISOs > 800, 1600, and 3200 on my 10D. I converted them all to uncompressed DNG [quoted text clipped - 12 lines] > > Note how only the vertical stripes marked have odd numbers. So what? >> Here is some sample data from the ISO 1600 blackframe: >> [quoted text clipped - 3 lines] > >So what? So, you aren't very bright if you replied to an on-topic thread that doesn't interest you. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< > >> Here is some sample data from the ISO 1600 blackframe: > >> [quoted text clipped - 6 lines] > So, you aren't very bright if you replied to an on-topic thread that > doesn't interest you. Well, I'm interested but as someone who is ignorant of what I'm looking at I have no idea what the significance of the odd numbers is even after reading your post 3 times. Greg >> >> Here is some sample data from the ISO 1600 blackframe: >> >> [quoted text clipped - 10 lines] >have no idea what the significance of the odd numbers is even after reading >your post 3 times. Note that I said, "that doesn't interest you"; not, "that isn't clear to you". The odd numbers are a peculiarity; I don't understand them myself. They occur in the raw data (which reads like english text in the image) not at all at the beginning of the image (where all levels are even numbers), then further into the image every other pixel has an odd level, and there are sections that are all odd as well. The DNG converter is not supposed to alter data at all, with the exception of filling in defective pixels with interpolated data (which would result in individual pixels breaking out of the pattern). One speculation that I have is that Canon is doing this to make the data look OK in a histogram (an equal number of odd and even values should be expected The fact, however, that the RAW data is all even or odd within patterns suggests that the data is not really 12-bit at its source, but rather, 11-bit. What does this mean for the user? It means that you get the same quality data by setting the camera to ISO 800 instead of 1600, if you are shooting RAW, with an EC of -1. It also means that you get an extra stop of highlights this way, as the camera would clip any value above 2023 if the camera were set to ISO 1600. For those of us who shoot in low light, this is actually very beneficial to know. I have suspected that the camera is cheating ISO 1600 for a long time, and consequently, I have been setting the camera to ISO 800 instead of ISO 1600 when shooting wildlife at dusk. That way, if there truly is enough light for ISO 800, I will get the better ISO 800 image, but if there is not enough light, it will shoot a shot that will "push" to ISO 1600 with the same quality and more dynamic headroom than if the camera were actually set to ISO 1600! I even set the EC to +1 at ISO 800 sometimes, if there aren't a lot of bright highlights. That will make the aperture stop down more, when there is sufficient light (effectively a better ISO 400 than if the camera were actually set to ISO 400, because more bits represent the subject's dynamic range), but will also work at ISO 800, or "1600" (as good or better than the camera does 1600 with 0 EC) when necessary. Also, "ISO 3200" or "H" is actually ISO 1600, under-exposed and pushed by a stop, the same way. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< > The DNG converter is not supposed to alter data at all, Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't, 3/4 of the data has to be interpolated. Think bayer patterns ... and then remember that (customarily) in each row there are green pixels, but red and blue pixels are usually on seperate rows. > The fact, however, that the RAW data is all even or odd within patterns > suggests that the data is not really 12-bit at its source, but rather, > 11-bit. This is but one of many possible causes. Programs have been known to have bugs, the bayer pattern interpolation may cause this, a non-perfect sensor (some cameras are even differently sensitive for UV light on their green pixels in alternating rows!) and many other causes may exist. > What does this mean for the user? It means that you get the > same quality data by setting the camera to ISO 800 instead of 1600, if > you are shooting RAW, with an EC of -1. It also means that you get an > extra stop of highlights this way, as the camera would clip any value > above 2023 if the camera were set to ISO 1600. Have you actually tested that, or is that just a guess? For all we know, the camera could also compress the highlights, e.g.: 0-1800 => 0-3600 (i.e. *= 2) 1801-4096 => 1801-2023 (compress) => 3602-4096 (*= 2) > I have suspected that the camera is cheating ISO 1600 for a > long time, How do you think the camera handles ISO 800, 400, 200, if not by 'cheating', if not by collecting many bits' depth at ISO 100 and --- in the easiest case --- simply bit-shifting them? > and consequently, I have been setting the camera to ISO 800 > instead of ISO 1600 when shooting wildlife at dusk. That way, if there > truly is enough light for ISO 800, I will get the better ISO 800 image, Assuming the image is _better_ in a photographic sense. Something you can _see_ in the finalized picture. > but if there is not enough light, it will shoot a shot that will "push" > to ISO 1600 with the same quality and more dynamic headroom than if the > camera were actually set to ISO 1600! Unless you overexposure your image (so you actually reach that "headroom"), said headroom is purely theoretical and does you not a bit of good. You could as well shoot at 1600 then. And if you use anything but RAW (and thus a much more laborious workflow) you loose dynamic range, since JPEG compresses said range ... > I even set the EC to +1 at ISO > 800 sometimes, if there aren't a lot of bright highlights. That will > make the aperture stop down more, when there is sufficient light > (effectively a better ISO 400 than if the camera were actually set to > ISO 400, because more bits represent the subject's dynamic range), So you basically 'expose to the right'? http://www.luminous-landscape.com/tutorials/expose-right.shtml > Also, "ISO 3200" or "H" is actually ISO 1600, under-exposed and pushed > by a stop, the same way. And there may be special routines helping the image look better being run as well. -Wolfgang >> The DNG converter is not supposed to alter data at all, > > Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't, > 3/4 of the data has to be interpolated. No, it doesn't. DNG doesn't require interpolation of the sensor data. It *allows* it, but by default the DNG converter does not do it. >> I have suspected that the camera is cheating ISO 1600 for a >> long time, > > How do you think the camera handles ISO 800, 400, 200, if not by > 'cheating', if not by collecting many bits' depth at ISO 100 and > --- in the easiest case --- simply bit-shifting them? Up to 800, analog gain is used. After that it's a mathematical process. This is not exactly news. SignatureJeremy | jeremy@exit109.com >Up to 800, analog gain is used. After that it's a mathematical process. >This is not exactly news. Well, the popular conception is that up to 1600 is analog, and 3200 is mathematical (based on 1600 analog). No one believed me in the past when I said that "ISO 1600" is actually ISO 800 (analog), pushed mathematically. This is sort of a proof, but it also contains a mystery (why there are patterns of odd numbers). The only explanation I can come up with is that Canon is trying to cheat the histograms. It's clearly not visual dithering, because it is inconsistent in the image. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< >> Up to 800, analog gain is used. After that it's a mathematical process. >> This is not exactly news. [quoted text clipped - 3 lines] > when I said that "ISO 1600" is actually ISO 800 (analog), pushed > mathematically. It seems to be that you get four stops of analog gain. Thus, the D70, for example, has ISO 200-1600 for "real", but not 100, while the announced D2x has 100-800 for "real" and 1600-3200 as digital boost. This seems common enough to be expected at this point. SignatureJeremy | jeremy@exit109.com Wolfgang Weisselberg <ozcvgtt02@sneakemail.com> wrote in news:9nsla2- seq.ln1@ID-52418.user.berlin.de: > Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't, > 3/4 of the data has to be interpolated. Think bayer patterns > ... and then remember that (customarily) in each row there are > green pixels, but red and blue pixels are usually on seperate rows. When you do not understand how Bayer interpolation works, it would be better if you did not post about it. Thank you. For those that are interested: The idea that ? of the data has to be interpolated is a falsehood created by a person on r.p.d under the name Geaorge Preddy. The fact is that each pixel location on Bayer grid sensors contains a sensel that captures information about one of the three colours (blue, red or green) and the other two colour values are calculated based on information from several neighbouring sensels as well as the measured data at that location. This means that with a Bayer pattern sensor the final output will consist of 2/3 of the data being interpolated. SignatureMark Heyes (New Zealand) See my pics at www.gigatech.co.nz (last updated 12-Nov-04) "There are 10 types of people, those that understand binary and those that don't" > Wolfgang Weisselberg <ozcvgtt02@sneakemail.com> wrote in news:9nsla2- >> Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't, >> 3/4 of the data has to be interpolated. Think bayer patterns > When you do not understand how Bayer interpolation works, it would be > better if you did not post about it. Thank you. You are right, I thinko'ed. > For those that are interested: > means that with a Bayer pattern sensor the final output will consist of 2/3 > of the data being interpolated. On the average, this is true, and for the green channel only 1/2 of the pixels need interpolation. -Wolfgang >> The DNG converter is not supposed to alter data at all, >Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't, >3/4 of the data has to be interpolated. All of the data in an RGB output is interpolated (if the method of interpolation is worth a dime), and the increase in data from the interpolation is equivalent to interpolating 2/3 of it; not 3/4. >Think bayer patterns >... and then remember that (customarily) in each row there are >green pixels, but red and blue pixels are usually on seperate rows. Which is totally irrelevant in a dark frame, as no light is passing through the Bayer CFA. All the data is from sensor and circuit noise, and a possible offset bias. >> The fact, however, that the RAW data is all even or odd within patterns >> suggests that the data is not really 12-bit at its source, but rather, >> 11-bit. > >This is but one of many possible causes. Programs have been known >to have bugs, the bayer pattern interpolation may cause this, No, there was no bayer interpolation in the data I was looking at. It was *RAW*. >a non-perfect sensor (some cameras are even differently sensitive >for UV light on their green pixels in alternating rows!) and many >other causes may exist. You didn't read my post very clearly; did you? The stripes I'm getting are vertical, not horizontal. Even if it were a difference between odd and even lines for green, there are red and blue on those lines, as well. You really wouldn't expect one line of green to be all odd values, and the next to be all even, would you? There are solid runs of odd pixels and solid runs of even pixels over multiple horizontal lines. >> What does this mean for the user? It means that you get the >> same quality data by setting the camera to ISO 800 instead of 1600, if [quoted text clipped - 4 lines] >Have you actually tested that, or is that just a guess? For all >we know, the camera could also compress the highlights, e.g.: >0-1800 => 0-3600 (i.e. *= 2) >1801-4096 => 1801-2023 (compress) => 3602-4096 (*= 2) It doesn't compress the highlights. RAW data is RAW and it is linear! >> I have suspected that the camera is cheating ISO 1600 for a >> long time, >How do you think the camera handles ISO 800, 400, 200, if not by >'cheating', if not by collecting many bits' depth at ISO 100 and >--- in the easiest case --- simply bit-shifting them? Nope. You are just full of error, aren't you? ISOs 100 through 800 are achived by amplifying the signal before the ADC stage. >> and consequently, I have been setting the camera to ISO 800 >> instead of ISO 1600 when shooting wildlife at dusk. That way, if there >> truly is enough light for ISO 800, I will get the better ISO 800 image, >Assuming the image is _better_ in a photographic sense. >Something you can _see_ in the finalized picture. There is no logical reason why this would not be so. The only difference is the ISO setting; same Tv mode shutter speed, and same wide-open aperture. >> but if there is not enough light, it will shoot a shot that will "push" >> to ISO 1600 with the same quality and more dynamic headroom than if the >> camera were actually set to ISO 1600! >Unless you overexposure your image (so you actually reach that >"headroom"), said headroom is purely theoretical and does you not >a bit of good. You could as well shoot at 1600 then. What for? >And if you >use anything but RAW (and thus a much more laborious workflow) >you loose dynamic range, since JPEG compresses said range ... That would require some testing. The JPEG compression would probably damage the shadows that you would want to boost. In any event, I was talking about RAW, which is what people who want maximum quality generally shoot. >> I even set the EC to +1 at ISO >> 800 sometimes, if there aren't a lot of bright highlights. That will [quoted text clipped - 4 lines] >So you basically 'expose to the right'? > http://www.luminous-landscape.com/tutorials/expose-right.shtml If I can. If things are uncertain, I might play it safe with default exposure. >> Also, "ISO 3200" or "H" is actually ISO 1600, under-exposed and pushed >> by a stop, the same way. > >And there may be special routines helping the image look better >being run as well. This is RAW data. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< >>> The DNG converter is not supposed to alter data at all, >>Oh yes, it is! Unless the 10D has a Foveon chip, which it hasn't, >>3/4 of the data has to be interpolated. > All of the data in an RGB output is interpolated (if the method of > interpolation is worth a dime), Naive me had assumed that the blue value on a blue pixel would have been taken for it's value (excluding edges in the image). >>Think bayer patterns >>... and then remember that (customarily) in each row there are >>green pixels, but red and blue pixels are usually on seperate rows. > Which is totally irrelevant in a dark frame, as no light is passing > through the Bayer CFA. All the data is from sensor and circuit noise, > and a possible offset bias. And it could not be that red, green and blue sensors get any different treatment inside the camera, like different gain, say, because one of these colors happens to need that to get the correct strength? >>> The fact, however, that the RAW data is all even or odd within patterns >>> suggests that the data is not really 12-bit at its source, but rather, >>> 11-bit. >>This is but one of many possible causes. Programs have been known >>to have bugs, the bayer pattern interpolation may cause this, > No, there was no bayer interpolation in the data I was looking at. It > was *RAW*. You used a converter. You have no knowledge what the converter does _inside_. >>a non-perfect sensor (some cameras are even differently sensitive >>for UV light on their green pixels in alternating rows!) and many >>other causes may exist. > You didn't read my post very clearly; did you? The stripes I'm getting > are vertical, not horizontal. The green pixels in different rows are also in different columns in any common pattern. > Even if it were a difference between odd > and even lines for green, there are red and blue on those lines, as > well. On _alternating_ rows, and on _alternating_ columns. > You really wouldn't expect one line of green to be all odd > values, and the next to be all even, would you? I would expect them all to be zero, in a dark frame, at least with dark frame substraction. How about you photograph some object at ISO 800, 1600 and H, while adjusting the shutter time to get equivalent exposure, run the RAWs through dcraw -d (of which you can see what it does) and examine the well-documented output format. See if you get the same even-odd rows in 1600 and in H. If you are right, you should see them there, too. If not, you have hit an artifact of sensor noise in dark frame situations. Actually, if you are right, you should see the 2 least significant bits in H being, ah, unusual, and nothing similar in 800. > There are solid runs of > odd pixels and solid runs of even pixels over multiple horizontal lines. That could well be because of sensor noise in dark frame situations. >>> What does this mean for the user? It means that you get the >>> same quality data by setting the camera to ISO 800 instead of 1600, if >>> you are shooting RAW, with an EC of -1. It also means that you get an >>> extra stop of highlights this way, as the camera would clip any value >>> above 2023 if the camera were set to ISO 1600. >>Have you actually tested that, or is that just a guess? For all >>we know, the camera could also compress the highlights, e.g.: >>0-1800 => 0-3600 (i.e. *= 2) >>1801-4096 => 1801-2023 (compress) => 3602-4096 (*= 2) > It doesn't compress the highlights. If you have pure bitshifting, true. > RAW data is RAW and it is linear! Not having the source code of the firmware I cannot tell you what the camera does. However, it _would_ be a pity to throw away all that headroom by bitshifting, and it's not as if it would hurt really, or would it? By arguing how things should be we don't get answers how they are, we get doctrines and should open a church. >>How do you think the camera handles ISO 800, 400, 200, if not by >>'cheating', if not by collecting many bits' depth at ISO 100 and >>--- in the easiest case --- simply bit-shifting them? > Nope. You are just full of error, aren't you? I also have prejudices, half-knowledge, insults, attacks and a very strong oppinion in me, so I scarcely can be *just* full of error. > ISOs 100 through 800 are > achived by amplifying the signal before the ADC stage. Which is --- basically --- the analogue analog of bit shifting (plus adding amplifier noise), isn't it? The only difference is that the precious 12-bit converter is brought into a better range (being exposed to the right, basically) and that one hopes that the amplifier noise is below the converter's threshold --- otherwise you get one (or more) bits of noise and 11 (or less) bits of signal. The only reason we bother with analog gain is that we don't have better converters and we'd prefer more dynamics at the cost of noise. Wait till we have 20-bit converters @ISO 100, then you can have 12 bits at ISO 25,600 without any extra analog gain --- and people will scream that the sensor 'cheats' and gets them less than 20 bits at ISO 6,400. >>> and consequently, I have been setting the camera to ISO 800 >>> instead of ISO 1600 when shooting wildlife at dusk. That way, if there >>> truly is enough light for ISO 800, I will get the better ISO 800 image, >>Assuming the image is _better_ in a photographic sense. >>Something you can _see_ in the finalized picture. > There is no logical reason why this would not be so. As they say, the proof is in the pudding ... Have you tested it? After all, according to your logical reason, JPEG will always be practically unusable, with most of the data being removed from the picture, lossy compression and all that. Same with Ogg Vorbis, mp3, DVDs (mpeg compressed), ... >>> but if there is not enough light, it will shoot a shot that will "push" >>> to ISO 1600 with the same quality and more dynamic headroom than if the >>> camera were actually set to ISO 1600! >>Unless you overexposure your image (so you actually reach that >>"headroom"), said headroom is purely theoretical and does you not >>a bit of good. You could as well shoot at 1600 then. > What for? If you don't use the headroom, what's it's use, except for more work to polish the image? >>And if you >>use anything but RAW (and thus a much more laborious workflow) >>you loose dynamic range, since JPEG compresses said range ... > That would require some testing. The JPEG compression would probably > damage the shadows that you would want to boost. JPEG compresses logarithmically. > In any event, I was talking about RAW, which is what people who want > maximum quality generally shoot. And here _I_ thought that would be slides of at least medium format, since 100+ megapixel cameras are still somewhat rare. >>> Also, "ISO 3200" or "H" is actually ISO 1600, under-exposed and pushed >>> by a stop, the same way. >>And there may be special routines helping the image look better >>being run as well. > This is RAW data. Does that mean no noise supression on the sensor chip itself? -Wolfgang > So, you aren't very bright if you replied to an on-topic thread that > doesn't interest you. Who gives a sh.t about the sequence of 1's and 0's? It's the end result that's important...it's about photography. >> So, you aren't very bright if you replied to an on-topic thread that >> doesn't interest you. > > Who gives a sh.t about the sequence of 1's and 0's? It's the end result > that's important...it's about photography. This group's about Photography not digits ? ;oO >>> So, you aren't very bright if you replied to an on-topic thread that >>> doesn't interest you. [quoted text clipped - 3 lines] > > This group's about Photography not digits ? ;oO Well, actually, it's about SLR systems. I would have thought that a better understanding of the internals of something might have enabled you to make the best use of it? I think John is trying to explain something, without shouting it too loudly. Happy New Year, David >> So, you aren't very bright if you replied to an on-topic thread that >> doesn't interest you. >Who gives a sh.t about the sequence of 1's and 0's? It's the end result >that's important...it's about photography. Digital photography completely depends on ones and zeros for the "end result". Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< >>> So, you aren't very bright if you replied to an on-topic thread that >>> doesn't interest you. [quoted text clipped - 4 lines] > Digital photography completely depends on ones and zeros for the "end > result". I think what he was saying is along the lines of, "It's not important to me how a refrgerator works, as long as it keeps the beer at 38.7? F". SignatureFrank ess >>>> So, you aren't very bright if you replied to an on-topic thread that >>>> doesn't interest you. [quoted text clipped - 7 lines] >I think what he was saying is along the lines of, "It's not important to >me how a refrgerator works, as long as it keeps the beer at 38.7º F". Yes, but if you know that filling the empty shelves with empty boxes allows less air to exchange when you open the door, and get rid of all the empty space, your beer will be colder, and your electric bills will be lower. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< >>I think what he was saying is along the lines of, "It's not important to >>me how a refrgerator works, as long as it keeps the beer at 38.7º F". > Yes, but if you know that filling the empty shelves with empty boxes > allows less air to exchange when you open the door, Please look up how little enery it needs to cool down air compared to water, beer and even the plastic and metal of the fridge itself. > and get rid of all the empty space, your beer will be colder, > and your electric bills will be lower. Buying a smaller fridge in the first place will handsomely remove all that empty space. This also reduces the surface of the refrigerator unit, thus reducing the amount of heat that leaks through from the outside, thus lowering your electric bill Your beer will not be cooler in any way, if you cool it long enough and don't have the door open a lot of the time. F'up set since OT. -Wolfgang > Digital photography completely depends on ones and zeros for the "end > result". You guys get too bogged down in the electronics/computer end of things. >> Digital photography completely depends on ones and zeros for the "end >> result". > >You guys get too bogged down in the electronics/computer end of things. It only sounds "bogged down" to you because you're simple. This is natural, ABC, 123 stuff to me. I'm smart enough to know that the more you know how things really work, the simpler it is for you to deal with them. Less magic, and less black boxes with no controls. Your less-bogged-down approach will result in images with more noise, and less shadow detail. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< > Your less-bogged-down approach will result in images with more noise, > and less shadow detail. I doubt it. >> Your less-bogged-down approach will result in images with more noise, >> and less shadow detail. > >I doubt it. ------------------------ If someone told you your 8MP camera really only had 6MP, you'd be upset, right? Or suppose your 48-bit scanner was really only using 16-bits. Well think of not using all the bits in those terms, maybe then you'll appreciate what is being described here. Or look up the word > Who gives a sh.t about the sequence of 1's and 0's? It's the end result > that's important...it's about photography. Spoken like a poor carpenter who doesn't understand tools. Signature-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: there's no such thing as a FreeLunch. > I just took blackframe and super-overexposed white wall images at ISOs > 800, 1600, and 3200 on my 10D. I converted them all to uncompressed DNG [quoted text clipped - 17 lines] > John P Sheehy <JPS@no.komm> > ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< Well, I'm quite familiar with bits, bytes, hex and all that, having been in programming for about 20 years, but I am at a loss to understand what you are driving at here, John. 12 bits converts to decimal 4095, or 4096 separate steps, but the data in your graphic, since it is for blackframe, is averaging around 8 or 9 bits (decimal 240 - 268 approx). The odd numbers imply that bit 0 is a 1, that's all, but I can't see from this that the highest binary value is represented by 11 bits, or 2048 decimal. Could you elucidate further, please? Colin >Well, I'm quite familiar with bits, bytes, hex and all that, having been >in programming for about 20 years, but I am at a loss to understand what [quoted text clipped - 4 lines] >from this that the highest binary value is represented by 11 bits, or >2048 decimal. >Could you elucidate further, please? If the winning lottery numbers were all odd one year, odd every other week and even on the others the next year, and all even the next year, what would you think? Do all numbers have an equal chance at any given time? Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< > If the winning lottery numbers were all odd one year, odd every other > week and even on the others the next year, and all even the next year, > what would you think? Do all numbers have an equal chance at any given > time? In the lottery I'd assume (nearly) equal chances. With sensors I don't. -Wolfgang >> If the winning lottery numbers were all odd one year, odd every other >> week and even on the others the next year, and all even the next year, [quoted text clipped - 3 lines] >In the lottery I'd assume (nearly) equal chances. >With sensors I don't. Of course there will be *ranges* of RAW data that are inlikely. Any value less than 120 is unlikely with Canon DSLRs. However, every *range* has both odd and even numbers, in equal distribution. When the same pixels are always odd and others always even, neither has the chance of being the other, and there is one bit of precision missing. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< >>In the lottery I'd assume (nearly) equal chances. >>With sensors I don't. > Of course there will be *ranges* of RAW data that are inlikely. Any > value less than 120 is unlikely with Canon DSLRs. However, every > *range* has both odd and even numbers, in equal distribution. You assume that there is an equal distribution in the range, concerning the physical sensor. > When the > same pixels are always odd and others always even, neither has the > chance of being the other, and there is one bit of precision missing. To show to a sufficienrt degree that these self-same pixels are _always_ odd or even, you'd need far more than one picture. Even if in the "dark frame" range that observation is true --- that does not make it necessarily true in brighter ranges at the same ISO setting. -Wolfgang >>>In the lottery I'd assume (nearly) equal chances. >>>With sensors I don't. >> Of course there will be *ranges* of RAW data that are inlikely. Any >> value less than 120 is unlikely with Canon DSLRs. However, every >> *range* has both odd and even numbers, in equal distribution. >You assume that there is an equal distribution in the range, >concerning the physical sensor. That is a very safe assumption to make; about as safe as assuming that the floor is there when I get out of my bed in the morning. >> When the >> same pixels are always odd and others always even, neither has the >> chance of being the other, and there is one bit of precision missing. >To show to a sufficienrt degree that these self-same pixels are >_always_ odd or even, you'd need far more than one picture. I should have said in "the same pattern". I have looked at other images, and the pattern is the same, except when the data is clipped at 4095. >Even if in the "dark frame" range that observation is true --- >that does not make it necessarily true in brighter ranges at the >same ISO setting. I looked at image RAW data also; the same thing. I really don't understand why you are so stubborn about accepting this pattern. It's as plain as the grid on a graph paper, and only exists at ISO 1600 and 3200. At 800 and below, any pixel can have an odd or even value. Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< >>>>In the lottery I'd assume (nearly) equal chances. >>>>With sensors I don't. >>> Of course there will be *ranges* of RAW data that are inlikely. Any >>> value less than 120 is unlikely with Canon DSLRs. However, every >>> *range* has both odd and even numbers, in equal distribution. >>You assume that there is an equal distribution in the range, >>concerning the physical sensor. > That is a very safe assumption to make; about as safe as assuming that > the floor is there when I get out of my bed in the morning. Only assuming a sensor without non-random (coloured) noise and a perfect digitizer. >>> When the >>> same pixels are always odd and others always even, neither has the >>> chance of being the other, and there is one bit of precision missing. >>To show to a sufficienrt degree that these self-same pixels are >>_always_ odd or even, you'd need far more than one picture. > I should have said in "the same pattern". I have looked at other > images, and the pattern is the same, except when the data is clipped at > 4095. Do I understand you correctly: - you have looked at at least a dozen pictures with ISO 1600, of several opjects - at least half a dozen of these pictures have been exposed correctly for ISO1600 - all of these pictures are in RAW - when converting the image file from camera RAW to readable RAW again, the identical image comes out (why should any 'cheat at histograms' algorithm not be in the converter?) - all these pictures show a pattern of stripes - however, it's not always the same pixels that are even and odd, this varies from picture to picture - when you turn up the contrast way way high, you can actually see the stripes or am I missing something? > I really don't understand why you are so stubborn about accepting this > pattern. I do accept that you have seen some pattern in an HEX editor (and up to now I only remembered you writing of one single dark frame). However, the DNG format is not very trivial --- see for yourself[1]. For example, what is the orientation --- it is set as a single byte and is not optional. Could your stripes be horizontal (page 13)? What is the BlackLevel, and the BlackLevel pattern (page 19) and BlackLevelDeltaH and BlackLevelDeltaV (page 20) --- and could that pattern, as substracted (see page 37), unstripe your observed 'stripes'? > It's as plain as the grid on a graph paper, But whatever it is that you see --- is it what you think it is? Without the raw data[2] (and not having a 10D, producing them is kind of hard) I cannot see what you see, only what you show me, interpreted through your eyes. Of course I am sceptic, I owe it to myself. > and only exists at > ISO 1600 and 3200. At 800 and below, any pixel can have an odd or even > value. But then you'd get only ..0 ..4 ..8 .12 .16 with 3200 (i.e. 2 unused bits), _unless_ 3200 has more analogue gain than 1600 ... do you get that? -Wolfgang [1] http://www.adobe.com/products/dng/pdfs/dng_spec.pdf [2] .cr2, not the already processed .dng >Do I understand you correctly: > - you have looked at at least a dozen pictures with ISO [quoted text clipped - 8 lines] > - however, it's not always the same pixels that are even > and odd, this varies from picture to picture I lost track of this thread, but now I have a better demo. No, the stripes are different in every image; however, they are being applied very evenly (odd and even number constitute 49.8 to 50.2% of the images). This is an ISO 1600 image from the 10D, but showing only the LSB (0=black; 1=white) http://www.pbase.com/jps_photo/image/38841732 As far as it affecting the image is concerned, it is certainly too much of an offset to be useful dithering, if in fact some are +1 and some are -1. I believe they are offset all the same way, though, as a blurred blackframe tends to have similarly-striped sections that look lighter and darker than each other, after the blur is histogram-equalized (i.e., significant low-frequency content). Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< > >Well, I'm quite familiar with bits, bytes, hex and all that, having been > >in programming for about 20 years, but I am at a loss to understand what [quoted text clipped - 12 lines] > time? > -- With respect, John, lottery winning numbers aren't binary, and I cannot see your analogy here. I would think that if only 11 bits are being used as you suggest, then it will be the most significant bit that will be dropped, not the least significant, i.e. bit 0. But that seems to be what you are implying, since whether or not a decimal conversion is odd or even depends on the state of bit 0. At least that is what I think you are saying, but I may have misinterpreted your earlier post. A further point that has me confused is that, if the blackframe values are represented by an 8- or 9-bit binary number, then even with 12 bits available, the entire brightness range of the image has to be represented with 3 or 4 bits, i.e. a range of 16:1 max. Yet the images appear to have much greater range than that. I wll have to read up some more on this, it appears. Colin. >With respect, John, lottery winning numbers aren't binary, and I cannot >see your analogy here. [quoted text clipped - 4 lines] >state of bit 0. At least that is what I think you are saying, but I may >have misinterpreted your earlier post. I believe that I said or implied that all the numbers should be even, if only 11 bits are used. The RAW data is in 12-bit format, and the numbers 0 through 2047 doubled are 0 through 4094, all even, and all with 0 for the least significant bit, in binary. >A further point that has me confused is that, if the blackframe values >are represented by an 8- or 9-bit binary number, then even with 12 bits >available, the entire brightness range of the image has to be >represented with 3 or 4 bits, i.e. a range of 16:1 max. Yet the images >appear to have much greater range than that. Not at all. The camera has pixels which are not exposed to light (masked). This "black" data is used to generate a "blackpoint" value, probably from some position near the beginning of the expected bell-curve histogram for the black pixels. This number is *subtracted* from all RAW image values, before conversion to a full RGB bitmap. In this case, the value might be 260. That means that the actual image uses a range of 260 to 4095, or 0 to 3835 after the blackpoint subtraction. That is only slightly less dynamic range than 0 to 4095 (which will never happen, as the blackpoint is lowest at ISO 100 and with a 1/4000 shutter speed is about 126 at room temperature, on the 10D). >I wll have to read up some more on this, it appears. I don't know if much is written on this! Signature<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<> John P Sheehy <JPS@no.komm> ><<> <>>< <>>< ><<> <>>< ><<> ><<> <>>< |
Reply to this Message |
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2009 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.