Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
10-11-2019, 03:26 PM - 9 Likes   #1
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2007
Location: Flagstaff, Arizona
Posts: 747
How Linear Are Some Pentax Camera Sensors?

Pretty darn good - at least if you have a Pentax K-1, K-3, or Q7.

Have you ever wondered how your camera was responding to the light incident on the sensor?

Optical astronomers worry about this a lot: most telescope cameras these days have CCD detectors very similar to those in our cameras. Most research with these cameras requires an accurate measure of “how bright/how much light” is being detected. Generally, the CCDs are quite linear - more light gives more signal (more charge stored in a CCD pixel) in a 1-to-1 relation (after nuisance corrections such as bias frames, dark frames, flat fielding, etc.).

So, I was curious about my Pentax cameras in this regard.

How do you measure it? The easiest way is to take a series of exposures of, say, a gray card with constant illumination using different shutter speeds and determine the pixel values that result. (One could also vary f-stop or even ISO, but shutter speed is easy to change over a wide range, and, according to my measurements, exceedingly accurate in most cases.)

This is not quite as simple as it sounds, though. The problem is getting the raw pixel data out of the camera. Pretty much all programs which display images for us on our PC monitors interpret the data in distinctly non-linear ways. For example, here is what Photoshop Elements (PSE) displays as a function of shutter speed over a range of almost 10 stops (the different colors lie nearly on top of each other, since a custom white balance was set):




If PSE was linear, the results should look like this:





After some web searching and a question here, the solution to getting the raw data out (short of writing my own program to take advantage of dcraw software (dcraw - Wikipedia)) was to use the RawDigger (RD) software (RawDigger | Everything You Always Wanted to Know About Raw(But Were Afraid to Ask)). RawDigger reads in camera RAW files, and will report the actual R, G (two values, since most cameras with Bayer grids actually have twice as many green-sensitive photosites as red or blue), and B values measured by the camera sensor, along with various statistics.

I used a Gretag-Macbeth 24-square Color Checker card and set RD to measure 3 or more of the white-gray-black squares at the bottom of the card. I set a custom white balance for my lights (see next paragraph), and then made exposures over a wide range of shutter speeds, typically from around 1/500 to several seconds, in half-stop intervals (for the Q7 and a Canon G15, I used 1/3 stop intervals, since these are the only option). ISO was set at 100, and a typical f-stop was 4. Some diffusing plastic ensured uniform lighting of the target. The results shown below have been corrected for the “incorrect” shutter speeds displayed by a camera ( True Shutter Speeds compared to Nominal Shutter Speeds - Actual Measurements - PentaxForums.com ). Thus, all shutter speeds have the correct square root of 2 (cube root for the Q7) values.

My first attempt came to a halt when I realized that the results were obviously inconsistent at short shutter speeds (1/60 second and shorter) just from looking at the images- the brightness varied all over the place. I quickly realized I was seeing effects due to the 60 Hz power line frequency. I was using line-powered LED lights initially and they respond quickly to the fluctuating voltage. I cured the light fluctuation by using a DC power supply to run other LED lights, some 12 V mini-floods (for the K-1 exposures) and a pair of 6V work lights for the K-3 and Q7 results.

I put all the RD data into excel spreadsheets for analysis and display.


Here are the results for my K-1, in all three colors (the green values have been averaged here):




The results show phenomenal linearity, for at least 10.5 stops in the blue! I have been taking scientific data for more than 50 years (in the lab and actual astronomical data), and I have NEVER seen anything so good. For perfectly linear data, the power of x in the power law fits should be exactly one. The values here deviate by less than a percent from that. The R-square value indicates the goodness of fit in excel. 1 indicates a perfect fit, and these are that good to 4 digits or so. At the very low end (pixel values below 10) seen on the darkest squares (not shown here), the values are a bit above the fit line, almost certainly due to noise at such low values. The largest pixel data values for the K-1 (and K-3) are 16316 - just slightly below the absolute maximum we might expect for 14 bits: 2^14 = 16384 . Looks like Pentax left themselves a bit of breathing room at the top of their analog-to-digital converter. Values approaching 16000 appear to be real (i.e. actually represent true variations in light intensity).


Here are the K-3 results:




Again, phenomenal fits and superb linearity, over at least 11 stops (in the blue). The powers here are a bit bigger then one, but only by a percent or two, with goodness of fit coefficients again close to one. This time, too, the maximum pixel values reported are 16316, consistent with the expected 14 bits.

The astute observer will note that the relative strength of the blue and red data have switched between the plots. Some of that may be due to the relative color sensitivity of the two sensors, but I think it could also be an effect of the relative color content of the LED lights that were used. The K-3 lights were definitely bluer.


And, finally, the Q7 results:



And, once more, the camera sensor is exceedingly linear over more than 10 stops, with powers very close to unity and goodnesses of fit very close to one as well. The exposures here are mostly for full stop steps, with just a few at intermediate 1/3 stop values. The maximum pixel values now are around 4060 - again, slightly lower than the maximum for 12 bits: 2^12 = 4096. This time, the red and green data lie nearly on top of each other. The lighting here is the same as used for the K-3. Evidently, the Q7 relative sensitivity to red and green is different than for the K-1 and K-3.


I made some similar measurements with my Canon G15. The results, while still quite linear, are not nearly as good as for these Pentaxes. There are kinks in the curves around shutter speeds of 1/20 second. And for exposure times of more than a second, the camera insisted on changing the ISO from 100 to 80. Correcting for the ISO sensitivity change did not quite restore linearity. Just realized: I need to try properly accounting for the 1/3 stop step size in ISO - hmmm, that’s a less than 1 per cent effect for that step - not big enough to explain the effects I see.


Another question of interest, alluded to at the top of this discussion, is how the actual pixel values are displayed by Photoshop as RGB values. I will address this in a second write-up real-soon-now.

10-11-2019, 04:48 PM   #2
Loyal Site Supporter
Loyal Site Supporter
Mikesul's Avatar

Join Date: Sep 2009
Location: Phoenix, AZ
Posts: 6,472
Thanks for putting this together. It is very interesting.
10-11-2019, 05:26 PM   #3
Pentaxian
photoptimist's Avatar

Join Date: Jul 2016
Photos: Albums
Posts: 4,457
Wow! Great work! And reassuring results!

There are a few of potential sources of the tiny amount of non-linearity in the data:

1) Unless you used the electronic shutter, some of the nonlinearity might be in the nonuniformity or inaccuracy of the mechanical shutter.

2) And even if you use the electronic shutter, then some of the some of the nonlinearity might be due to imperfections in the electronic first curtain which probably don't perfectly empty the well during a reset. These will bias the shutter time results in a way that makes the slope a bit shallower.

3) Dark current (including dark current between when the shutter closes and the read out gets the pixels) would slightly lift the pixel values. You might find the slope of the line differs between the upper edge and the lower edge of the sensor based on the lower or greater delay between curtain and read-out.
10-11-2019, 05:31 PM   #4
Senior Member




Join Date: Oct 2018
Posts: 241
QuoteOriginally posted by AstroDave Quote
Another question of interest, alluded to at the top of this discussion, is how the actual pixel values are displayed by Photoshop as RGB values. I will address this in a second write-up real-soon-now.
De-mosaicing is a big topic I really like how pixel shift resolution compensates for the "short-comings" of the sensor.

10-11-2019, 05:45 PM   #5
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,002
QuoteOriginally posted by AstroDave Quote
The results show phenomenal linearity
Amazing, IMHO. This speaks loads for the shutter as well. Again...amazing...


Steve
10-11-2019, 05:50 PM   #6
Senior Member




Join Date: Jan 2012
Location: Utica,Il
Posts: 118
Very interesting data set! Thanks for the hard work.
10-11-2019, 05:55 PM   #7
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,002
QuoteOriginally posted by AstroDave Quote
Another question of interest, alluded to at the top of this discussion, is how the actual pixel values are displayed by Photoshop as RGB values. I will address this in a second write-up real-soon-now.
I am looking forward to your write-up. I have a back-burner project that asks similar questions using synthetic capture data. What I found was mildly alarming with ACR taking huge liberties with the data for low values on import and even bigger liberties when doing a pull of exposure or shadows. I suspect that if shadow detail is being clobbered, as claimed by some on this site, it is being done through capture settings (data paucity in the shadows) and creative remapping by ACR rather than by the post-capture NR activities.


Steve
10-11-2019, 06:03 PM   #8
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,002
QuoteOriginally posted by AstroDave Quote
After some web searching and a question here, the solution to getting the raw data out (short of writing my own program to take advantage of dcraw software (dcraw - Wikipedia)) was to use the RawDigger (RD) software (RawDigger | Everything You Always Wanted to Know About Raw(But Were Afraid to Ask)).
Out of curiosity, which version did you license? I got the "research" edition. BTW...the articles on their site are pretty informative too.


Steve

10-11-2019, 06:13 PM   #9
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2007
Location: Flagstaff, Arizona
Posts: 747
Original Poster
QuoteOriginally posted by stevebrot Quote
Out of curiosity, which version did you license? I got the "research" edition. BTW...the articles on their site are pretty informative too.
I got the profile edition. Courtesy (perhaps) of one of my email addresses (referencing my Ph.D. school), I got an academic discount (40%). I told them this was mostly out of curiosity, and the reply was sort of, "hmmm, maybe we should consider a curiosity discount"! So, if you want to upgrade, you might inquire along such line.

---------- Post added 10-11-19 at 06:19 PM ----------

QuoteOriginally posted by photoptimist Quote
1) Unless you used the electronic shutter, some of the nonlinearity might be in the nonuniformity or inaccuracy of the mechanical shutter.
I've measured my shutter times for all three cameras, and as long as you use half stop intervals, they are good to better than 1% (I've upgraded my multi-LED timing blinker to 256 "pixels"). If, instead using 1/3 stop intervals, both the K-1 and K-3 show a very systematic 3% error (dead on for each stop, but then 3% high and 3% low for intervening values). Somebody at Pentax can't do arithmetic?!

A report on all the shutter timings is in my to-do list.

Last edited by AstroDave; 10-11-2019 at 06:14 PM. Reason: fix a typo
10-11-2019, 06:40 PM   #10
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,002
QuoteOriginally posted by AstroDave Quote
I got the profile edition. Courtesy (perhaps) of one of my email addresses (referencing my Ph.D. school), I got an academic discount (40%). I told them this was mostly out of curiosity, and the reply was sort of, "hmmm, maybe we should consider a curiosity discount"! So, if you want to upgrade, you might inquire along such line.
I would have a hard time pulling that off, my degrees are in life sciences and in the distant past.


Steve
10-11-2019, 07:25 PM   #11
Pentaxian
swanlefitte's Avatar

Join Date: May 2015
Location: New Orleans
Photos: Gallery
Posts: 3,281
Very interesting. Thanks for both the work and the write up.
10-12-2019, 12:28 AM   #12
Pentaxian




Join Date: Feb 2015
Photos: Gallery | Albums
Posts: 7,914
QuoteOriginally posted by AstroDave Quote
The largest pixel data values for the K-1 (and K-3) are 16316 - just slightly below the absolute maximum we might expect for 14 bits: 2^14 = 16384
IMO, the result of digital linearization done by the image processor at conversion of the sensor ADC data into PEF or DNG data. Likely one of the jobs handed over to the accelerator chip in the KP and K1 Mk II to free up some computing capacity of the main CPU for operations that apply post RAW..

---------- Post added 12-10-19 at 09:37 ----------

Unlike what a number of people assume, the RAW data isn't the data straigth from the sensor, the data of RAW files is the result of sensor data processed. For example, on the K1 (not the K1 II), the high ISO data are noise filtered even with NR disabled, a comparison of ISO 1600 single exposure to a 100 ISO addition stack of underexposed images show that the 1600 ISO RAW data has been through a low pass filter, while the ISO 100 data has not. I would say, sensor data is pre-processed (at the rate of the frame rate...) so that photographers already get optimal files to work with, something which we should be thankful about.

Last edited by biz-engineer; 10-12-2019 at 12:43 AM.
10-12-2019, 01:06 AM   #13
Site Supporter
Site Supporter
microlight's Avatar

Join Date: Sep 2011
Location: Hampshire, UK
Posts: 1,699
Fascinating thread, thanks for posting.
10-12-2019, 05:34 AM   #14
Loyal Site Supporter
Loyal Site Supporter
Madaboutpix's Avatar

Join Date: Dec 2011
Location: North Rhine-Westphalia
Posts: 926
Awesome research and write-up, AstroDave. Really appreciated. It would seem, once again, that our Pentax cameras are technically very sound and precise photography tools, with linearity being no exception.
10-12-2019, 06:33 AM - 1 Like   #15
Pentaxian
photoptimist's Avatar

Join Date: Jul 2016
Photos: Albums
Posts: 4,457
QuoteOriginally posted by AstroDave Quote
I've measured my shutter times for all three cameras, and as long as you use half stop intervals, they are good to better than 1% (I've upgraded my multi-LED timing blinker to 256 "pixels"). If, instead using 1/3 stop intervals, both the K-1 and K-3 show a very systematic 3% error (dead on for each stop, but then 3% high and 3% low for intervening values). Somebody at Pentax can't do arithmetic?!

A report on all the shutter timings is in my to-do list.
LOL!

Pentax probably used 1.3 instead of 1.25992104989 for the cube root of 2. That approximation would create about a 3.2% error if each full-stop value was divided by 1.3 and multiplied by 1.3 to get the next shorter and next longer 1/3 stop value respectively.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
camera, curiosity, data, edition, ii, intervals, iso, k-3, k1, lights, photography, photoshop, pixel, post, q7, result, results, sensor, shutter, speeds, values
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Quick Question re: Linear Polarizers BruceBanner Pentax Camera and Field Accessories 27 04-19-2018 03:11 PM
K-3 EV top LCD display from numeric to back to linear bar values lukulele Troubleshooting and Beginner Help 4 10-27-2017 05:57 AM
Linear or circular polariser Pentax 645? Alpinus Pentax Medium Format 17 02-24-2017 07:24 PM
Which sensors in new Pentax APS-C camera (speculation)? ogl Photographic Industry and Professionals 22 02-18-2012 08:00 AM
For Sale - Sold: Pentax DA 50-200, Nikon 3T & 4T Closeup lenses, Tiffen Linear Polarizer infosyn Sold Items 3 05-13-2008 08:14 PM



All times are GMT -7. The time now is 12:46 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top