Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
10-19-2010, 10:16 AM   #16
Veteran Member
paperbag846's Avatar

Join Date: Jul 2010
Photos: Gallery
Posts: 2,396
The only really glaring problem with the OP is the assertion that the K-5 has a max analogue ISO of 1600, which is not true according to anything else I have read. I believe that the max analogue ISO is 12800, and as stated before, 25600 and 51200 are underexposed (by 1 or 2 ev) and then pushed. This would not suprise me, as this is within the realm of "software expanded ISO" and software expanded iso looks like pushing an underexposed RAW anyways. All this means is that the K-5 only looks stellar until 12800 (oh boo hoo!)

If he was talking about "native" (not the right word but whatever) up to 1600, I think he would be referring a few generations back, to the days of the K10d..!

10-19-2010, 10:30 AM   #17
Veteran Member




Join Date: Oct 2009
Location: United States
Photos: Gallery | Albums
Posts: 392
OP may be right

I am no technical expert on the image processing, but I remember that someone else commented on this forum about the same thing.
He/she pointed out that k-x can do only upto 800 ISO. Anything above that is basically underexposed images at 800. K-x goes upto +5 stops from 800, ending up with a maximum of 25600.

This is a similar argument - although for a k-5 where the threshold is 1600 in place of 800.
10-19-2010, 10:43 AM   #18
Pentaxian




Join Date: Apr 2007
Location: Sweden
Posts: 1,883
The most chocking part is that the OP acts as if this is something new and only Pentax is "cheating" with high ISO and applying NR to raw files.
It's not, it's common practice.
10-19-2010, 10:58 AM   #19
Senior Moderator
Loyal Site Supporter
Parallax's Avatar

Join Date: Apr 2008
Location: South Dakota
Photos: Gallery
Posts: 15,104
Okay, if it's all done by the processing engine, then why did they bother with a new sensor?

10-19-2010, 11:02 AM   #20
Pentaxian
bobmaxja's Avatar

Join Date: Mar 2010
Location: Laval, Quebec Canada
Posts: 1,700
WEll if this is true, this is true for any brand not only Pentax
10-19-2010, 11:11 AM   #21
Senior Member
eigelb's Avatar

Join Date: Mar 2009
Location: Italy
Posts: 174
in bulb mode iso 1600 is the max selectable iso valu because during long exposer 1 sec and up noise, banding and hot pixels increase. They increase with longer exposer and higher iso setting.
you get only shit with a 10 min. exposer at iso 12.000. So it isnt selectable
10-19-2010, 11:12 AM   #22
Senior Moderator
Loyal Site Supporter
Parallax's Avatar

Join Date: Apr 2008
Location: South Dakota
Photos: Gallery
Posts: 15,104
Interesting. I'm not trying to bust your chops here, but only a few hours before starting this thread asserting that the highest native ISO is 1600, you asked in this post, in another thread if what you had read about 6400 being native was true. Again, if the processor is doing the work, why the new sensor?

In a second post in this thread you said:
QuoteQuote:
I have collected this information from various sources, including various web articles, observations from real-world users in the PentaxForums, and my own knowledge of electronics. I quickly banged this out before going to bed (now i can't sleep!!!), so i didn't bother citing all the sources. Some of it is speculation.
Perhaps if the last sentence of that post had been the first sentence of the first post, this wouldn't have stirred so much controversy.

Last edited by Parallax; 10-19-2010 at 11:36 AM.
10-19-2010, 11:19 AM   #23
Pentaxian
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,862
QuoteOriginally posted by rawr Quote
Falconeye and others with more expertise would be best to discuss this.


QuoteOriginally posted by secateurs Quote
This is my understanding of how the K5 handles high ISO. If I am mistaken, please correct me!!!

Ok, so first of all, your K-5’s highest “native” ISO is 1600. That’s why ISO 1600 is the highest selectable ISO in bulb mode. All higher sensitivities are done with smoke and mirrors, I’m afraid!

The interesting thing about this is your RAW files aren’t really RAW!

Here’s how it works (in slightly simplified language):

At ISO 1600 and lower, when you press the shutter button the Prime II processor sends a request to the sensor for a read-out of all the pixel locations. On previous generations of sensors, this information was sent to the processor in analogue form to be processed into digital. However, on this new sensor things are done a bit differently. The sensor gathers analogue data from the photosites. This information is immediately converted to digital data (1’s and 0’s) on the sensor. Please note that no new information is added to the image from this point on. The digital data is then sent to the Prime II for processing into Jpegs or demosaic’d (I think?) and sent straight to the card in the case of RAW.

To explain what happens above ISO 1600 I will use the example of shooting at ISO 3200. Because the sensor is incapable of capturing images any faster, the camera does some tricks to get this “higher sensitivity”. In this case, when you press the shutter, the Prime II asks the sensor to take a readout at -1EV. In other words, the image is underexposed. This under-exposed analogue image is immediately converted to digital on the sensor and sent to the Prime II for processing. This is where it gets interesting when shooting in RAW.

Theoretically, RAW images are digital conversions of the analogue signal received by the sensor, with as little processing as possible, so that the user can do their own custom processing on their computer. However, when the K-5 is set to ISO3200 the digitized image from the sensor is actually underexposed by 1 stop! The Prime II then adds 1 stop (+1EV) to the image (,adds NR?) then sends the image to the card as a RAW file. This is a fair bit of processing for a “RAW” image! What we’re getting here is what you might call a “pseudo-RAW” file! So much for a RAW option which gives the user the power to process the image as they prefer!

The situation gets worse when you get to ISO 52100. By this stage, the sensor is supplying an image to the PRIME II that is 5 stops underexposed (-5EV)!!! This is then brightened by +5EV and (most likely) NR’d by the PRIME II before sending to the card as a RAW file. All this processing explains the wide variation of file sizes for RAW images. Generally, the higher the sensitivity, the larger the file size! Which basically means more noise information is being recorded.

In film terms, what the K-5 is essentially doing at ISO 52100 is push-processing ISO 1600 "film" by 5 stops in-camera!


The point being, you can get a more “true” RAW file at ISO 52100 by shooting at ISO 1600 at -5EV! And if you did, you wouldn’t have the reduced-buffer issue seen at higher in-camera ISO settings! The problem is, of course, reviewing the shots in-camera would be nearly impossible to do (especially accurately) because the images you’ve just captured are so under-exposed! I would expect the LCD would look almost black. Histograms would be useless of course.

In my view this calls for a feature request from Pentax in an upcoming firmware release: Can we please have “true” RAW image capture at high sensitivities? This would mean brightening the image with the PRIME II, but only for in-camera review (maybe include a jpg “thumbnail” that has been processed to correct EV only?). The RAW file actually stored on the card must be the under-exposed image, with as little processing as possible done to it. We will take it from there, thanks!

Now, if all the above is correct, we should be able to get results just as good from a RAW image shot at -5EV at ISO 1600 as we can get from a normally-exposed image shot at ISO 52100. Can someone with a K-5 please post one of each in RAW so that an NR whiz can do their thing? Thanks!


And thanks for reading. I'll be interested to see the responses I get.
secateurs, sorry that I left the entire quote.

But I had to keep it to clarify how much words you used to say a very simple thing:

QuoteQuote:
The max. analog gain in the K-5 sensor is 16.
Let me stick to my more compact language because otherwise, I will never succeed to formulate my thoughts (I am just unable to say it in many words ).
  1. The "max. analog gain in the K-5 sensor is 16" claim is based on an observation made by GordonBGood. Namely that beyound ISO 1600, the histogram isn't equally populated anymore. Theoretically, if you were right, it should only be populated by 1/N where N are the stops above 1600. But this doesn't seem to be the case.

    Therefore, GordonBGood (and I follow him in this respect) only guesses that 16 is the max. analog gain. We both don't know and don't make this claim. But it is likely.

  2. Analog gain isn't necessarily better than digital gain (digital gain is bit-shifting). To the photographer, they are almost the same. Digital gain is more technically challenging because read-out noise is amplified the same as the signal. But the Sony sensor seems to live up to the challenge.

  3. If a vendor implements digital gain with low read-out noise then this is the preferred solution. Because the firmware could lazy-decide the digital gain factor after determining the brightest pixel. This feature isn't yet in the K-5 or D7000 firmware. But if I were them, I would hurry to be the first... (BTW, do I get consulting money? -- make a note to accounting )

  4. Pentax obviously doesn't do simple bit shifting. Rather, they apply some sort of NR to the raw data after applying the digital gain. Probably, a mild form of pixel binning, possibly restricted to some color channels or to dark colors only. And no, demosaicing does never happen at that stage. BTW, raw never is real raw. The sensor already subtracts reference dark feeds, both on the analog and digital end and e.g., hot pixels are mapped out. Also, AFAIK, non linear and location-variant sensitivities are corrected (fixed pattern noise removal) and probably, cos^4-alpha vignetting is removed as well. That's all cheap operations.

    The NR above 1600 could happen below 1600 as well, just restricted to very dark colors then and just not detected prior to pushing levels. DxO wouldn't see it and I didn't study this.

    Bit shifting alone is still keeping raw data. At least if no clipping occurrs.

  5. RAW files at higher ISO levels are bigger. But not at all because of digital gain. It's all because of noise. A noisier image compresses less and therefore, is bigger. The effect is more pronounced for JPG but is still there for the lossless DNG/PEF compression. An ISO 6400, the noise is already so large that lossless compression becomes impossible. An uncompressable K-5 raw is 4928 x 3264 x 14Bit or 26.84 MB. It is actually a bit bigger because of the preview image and black border pixels, about 30 MB.

    At lower ISO, noise is less and neighboring piexls start to be correlated, allowing lossless compression, resulting in DNGs to be about 20 MB.

    To think that digital gain increases the size or raw files is strange. At best, it should decrease the size at max ISO (51k) to (14-5)/14 or 64% what we don't see.

  6. A low analog gain (large digital gain) is a prerequisite for excelent dynamic range. Because dynamic range depends on the digital gain quality.

    Therefore, the best possible sensor is one with zero read-out noise and a max. analog gain of 1. User-selectable ISO is so 2010-ish...



Last edited by falconeye; 10-19-2010 at 11:25 AM.
10-19-2010, 11:20 AM   #24
Pentaxian




Join Date: Apr 2007
Location: Sweden
Posts: 1,883
QuoteOriginally posted by Parallax Quote
Again, if the processor is doing the work, why the new sensor?

Because you need a really clean picture to begin with if want to push it 5 stops digitally.
10-20-2010, 09:19 AM   #25
Veteran Member
Eruditass's Avatar

Join Date: Jul 2009
Location: Pittsburgh, PA
Photos: Gallery | Albums
Posts: 2,206
QuoteOriginally posted by falconeye Quote
To think that digital gain increases the size or raw files is strange. At best, it should decrease the size at max ISO (51k) to (14-5)/14 or 64% what we don't see.
That's the main thing that bugs me about the theory.
10-20-2010, 09:41 AM   #26
Site Supporter
jpzk's Avatar

Join Date: Jan 2009
Location: Québec
Photos: Gallery | Albums
Posts: 6,974
QuoteOriginally posted by falconeye Quote
RAW files at higher ISO levels are bigger. But not at all because of digital gain. It's all because of noise. A noisier image compresses less and therefore, is bigger. The effect is more pronounced for JPG but is still there for the lossless DNG/PEF compression. An ISO 6400, the noise is already so large that lossless compression becomes impossible. An uncompressable K-5 raw is 4928 x 3264 x 14Bit or 26.84 MB. It is actually a bit bigger because of the preview image and black border pixels, about 30 MB.
Is this the reason why once we PP a large high ISO RAW file using NR software that the resulting file will have decreased in size?
And yes, I noticed that as one gets higher ISO images, the larger they get.

JP
10-20-2010, 05:08 PM   #27
Pentaxian
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,862
QuoteOriginally posted by Eruditass Quote
That's the main thing that bugs me about the theory.
I don't understand what bugs you here ...

Let me repeat one thing to make it crystal clear ...

ISO is always a marketing term if you like so. No ISO setting will convince a single photon to enter your lens which wouldn't have done otherwise.

Analog signal amplification prior to A/D conversion serves a single purpose: to minimize the addition of extra noise along the transport of the analog signal to the A/D converter and in the A/D converter itself. If no noise is added along these stages, there is no reason to do an analog amplification of the signal. After all, all what happens up to the A/D converter exit is to count the number of electrons in the well. You better don't amplify if you want accurate counts... And a digital amplification (a bit shift) is always more accurate then. BTW, the variable gain amplifiers which Samsung has put on the K20D/K-7 sensor are believed to actually have added more noise than they helped to avoid.

The Sony Exmor HD sensor have a short signal path to the A/D converter which is on-chip. And it may be good enough to do the signal quantization with neglegible error. Then there is no need for a large analog amplification at all. I predict that soon CMOS sensors will do no analog amplification at anymore. After all, it's just a dubious hack to mask suboptimal A/D conversion of the well potential. It's a hack because it damages dynamic range.

There is NO real difference between EV compensation and ISO setting (except an advice by how much to push the image in raw conversion). AFAIK, Hasselblad never did any analog amplification in their cameras.

My personal opinion is that all sensors with excellent dynamic range have a small analog gain and a large digital gain. Otherwise, they wouldn't have their excellent DR.

QuoteQuote:
Are the compressed RAW filesizes the same for ISO 1600 EV-1 and ISO 3200? ISO 1600 EV-2 and ISO 6400 when averaged over multiple shots of the same scene?
That's a good question in theory. And they probably would be if not Pentax.

But Pentax recomputes the RAW after the digital shift which will alter the file size. Moreover, the file size reaches its max. at 30MB, so it's not a sensitive test at higher ISO.

What is possible though is to look at the population density of 14 Bit raw histogram positions modulo 1,2,4,8 etc.

QuoteOriginally posted by jpzk Quote
Is this the reason why once we PP a large high ISO RAW file using NR software that the resulting file will have decreased in size?
And yes, I noticed that as one gets higher ISO images, the larger they get
Yes. Same if you would artificially add "grain" to an ISO 100 file for artistic reasons: the file size would grow. Holds true for JPG in particular.

It's the reason too why video at low light uses more of the 4GB limit than video at good light. Some people wonder why the K-7/5 stops recording video before the 4GB limit is reached. It does because it precomputes max. recording time with a lot of noise in mind.

The 30MB/Raw on the K-5 is kind of a maximum though, reached after lossless compression became impossible. I am not sure, but I think it is possible that the JPG quality depends on the ISO setting too (ie., it tries to compress more aggressively at higher ISO settings. Maybe, there's a difference between 3 and 4 star quality here).
10-20-2010, 05:54 PM   #28
Veteran Member
Eruditass's Avatar

Join Date: Jul 2009
Location: Pittsburgh, PA
Photos: Gallery | Albums
Posts: 2,206
QuoteOriginally posted by falconeye Quote
I don't understand what bugs you here ...
I was agreeing with you and re-iterating my initial criticism of the OP
QuoteQuote:
Let me repeat one thing to make it crystal clear ...

ISO is always a marketing term if you like so. No ISO setting will convince a single photon to enter your lens which wouldn't have done otherwise.

Analog signal amplification prior to A/D conversion serves a single purpose: to minimize the addition of extra noise along the transport of the analog signal to the A/D converter and in the A/D converter itself. If no noise is added along these stages, there is no reason to do an analog amplification of the signal. After all, all what happens up to the A/D converter exit is to count the number of electrons in the well. You better don't amplify if you want accurate counts... And a digital amplification (a bit shift) is always more accurate then.
If the voltage range matches the ADC. If we are at a lower exposure and only using 1/2^8 th of the voltage range the ADC can see, a LNA will undoubtedly help significantly.. An ADC will add a fixed amount of noise with the digitization (you know, effective bits), so if you amplify the signal before the ADC adds its fixed amount of noise, it will be more accurate. To see when this tradeoff would be, we'd have to do some more calculations of effective bits and noise figures.
QuoteQuote:
BTW, the variable gain amplifiers which Samsung has put on the K20D/K-7 sensor are believed to actually have added more noise than they helped to avoid.
I'd imagine they would suck. If they had the real-estate, multiple signal paths and LNA's would be so sweet.
QuoteQuote:

The Sony Exmor HD sensor have a short signal path to the A/D converter which is on-chip. And it may be good enough to do the signal quantization with neglegible error.
Maybe, maybe not. Depends on the ADC. In what I've worked with, an LNA up front is invaluable for reducing the system noise figure. Yes, if you get a good enough ADC and a very short path from the photosites to the ADC, it might be neglegible, but I'm sure price for price, an LNA will easily best a ADC in noise figure.
QuoteQuote:
Then there is no need for a large analog amplification at all. I predict that soon CMOS sensors will do no analog amplification at anymore. After all, it's just a dubious hack to mask suboptimal A/D conversion of the well potential. It's a hack because it damages dynamic range.
How does it damage dynamic range more than bit-shifting at higher than base ISO?

Of course at base ISO, I agree that bypassing an amplifier will probably add noise compared to not having anything in the signal path
QuoteQuote:

There is NO real difference between EV compensation and ISO setting (except an advice by how much to push the image in raw conversion). AFAIK, Hasselblad never did any analog amplification in their cameras.

My personal opinion is that all sensors with excellent dynamic range have a small analog gain and a large digital gain. Otherwise, they wouldn't have their excellent DR.

That's a good question in theory. And they probably would be if not Pentax.

But Pentax recomputes the RAW after the digital shift which will alter the file size. Moreover, the file size reaches its max. at 30MB, so it's not a sensitive test at higher ISO.
Do you have any more information as to what they do? I mean, with empty bins, how can you not compress that? Do they throw random things into the empty bins or something?
QuoteQuote:

What is possible though is to look at the population density of 14 Bit raw histogram positions modulo 1,2,4,8 etc.

Last edited by Eruditass; 10-20-2010 at 06:14 PM.
10-20-2010, 06:48 PM   #29
Pentaxian
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,862
I'll respond w/o quotes...

Voltage range ADC can see... Well, at 14Bits and e.g., 40000 full well capavity, it should see all electrons after a preamplification of only 2.4

There are reasons to believe that read out noise may be as low as 4e-. Of course, all of this is speculative. But the LNA's gain may be kept small without adding noise to the system.

You mention the quantization noise introduced by the ADC. But w/o any LNA, all the ADC does is counting electrons. No quantization error. An LNA could actually add a Moire effect to extreme dark pixels. I think however that the readout noise prevents that.

I trust the Sony engineers that they gave the LNA quite the right minimum and maximum gain. They seem to be doing a great job. Anybody here calling that job a marketing gimmick would have to come up with the exact figures you mentioned indeed

Also, I tend to agree that they use a LNA at all ISO stages. The point is just that as soon as you make the ADC 16 Bit, there is no need to make the gain variable.

The analog gain hack damages DR because it allows you to live with a noisy ADC. And a noisy ADC prevents good DR.

Pentax NR: I don't know what they do. It seems they make neighbor pixels correlate (DxO) and they partially fill what you call empty bins. So, my bet is they make a weighted average of a sensel with its neighbor sensels of equal color. Where the weight may depend on the color channel and luminosity value in original e-.
10-20-2010, 09:32 PM   #30
Veteran Member
jolepp's Avatar

Join Date: Oct 2010
Location: Finland
Photos: Albums
Posts: 3,196
QuoteOriginally posted by falconeye Quote
I'll respond w/o quotes...
...
You mention the quantization noise introduced by the ADC. But w/o any LNA, all the ADC does is counting electrons. No quantization error.
...
The point is just that as soon as you make the ADC 16 Bit, there is no need to make the gain variable.
...
I think I get your point: an ideal ADC would simply count the passing electrons as they are hoovered from each photosite by a voltage gradient and assuming it had a long enough counter any analog amplification would be only be a potential source of noise (?). The trouble is that an ADC can never be quite that accurate as electrons are elusive fellows due to the strangeness - from our point of view - of the quantum world they natively live in. Also, the practical ADC designs I know of are based on multiple analog voltage comparisons spread in in either in time (slow, simple, cheap) or space (fast, complex, expensive) so it seems they'd absolutely need the analog amplification for sample and hold at least.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
camera, card, dslr, file, images, information, k-5, k-5 ii, k-5 iis, k5, pentax k-5, sensor
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Black & White This is what ISO 3200 used to look like! brkl Post Your Photos! 5 01-10-2010 03:35 PM
iso 3200 jolee1990 Troubleshooting and Beginner Help 10 12-22-2009 08:26 AM
K-x vs K7 at iso 3200 artemapei Pentax DSLR Discussion 21 11-18-2009 08:05 PM
K 7 iso 3200 snooked Pentax DSLR Discussion 38 07-21-2009 02:53 PM
Tests iso 200 to iso 3200 with k100d Deni Post Your Photos! 0 06-20-2007 05:17 AM



All times are GMT -7. The time now is 08:53 AM. | See also: NikonForums.com, part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top