Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
02-23-2021, 01:41 AM - 2 Likes   #1
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,225
Test charts not agreeing with what I see

Pentax K-1 vs Sony A7RIV: Comparing test charts (curves) from photonstophotos.net, dxomark and side-side comparison of raw image noise at ISO800 and ISO1600.
- According to photonstophotos PDR curve, above ISO 320 camera setting, the Sony A7RIV should be superior to Pentax K1, thanks to the dual gain sensor of the Sony A7RIV that the Pentax K1 doesn't have...
- If I look at DXO curves ("print" option selected) at same measured ISOs, dynamic range curves are basically on top of each other. Where I see a bump of "performance" on the photonstophotos PDR curve for the A7RIV, I don't see it in DXOmark (measured ISO).
- Now, if I go to DPR studio comparison tool, select Pentax K1 and Sony A7RIV RAW ISO800 or RAW ISO1600, comparison mode. I check camera settings in both cases (ISO, lens aperture and shutter speeds are the same for both K1 and A7RIV). A visual side-side comparison, same image size for both cameras, RAW format, show that the Pentax K1 has slightly less noise than the Sony.

So why photonstophotos PDR curve shows that the Sony A7RIV is "superior" to K1 above ISO320 setting (presumably thanks to dual gain technology)?

02-23-2021, 02:49 AM - 4 Likes   #2
Site Supporter
Site Supporter




Join Date: Feb 2009
Location: Cork
Photos: Albums
Posts: 1,882
QuoteOriginally posted by biz-engineer Quote
Pentax K-1 vs Sony A7RIV: Comparing test charts (curves) from photonstophotos.net, dxomark and side-side comparison of raw image noise at ISO800 and ISO1600.
- According to photonstophotos PDR curve, above ISO 320 camera setting, the Sony A7RIV should be superior to Pentax K1, thanks to the dual gain sensor of the Sony A7RIV that the Pentax K1 doesn't have...
- If I look at DXO curves ("print" option selected) at same measured ISOs, dynamic range curves are basically on top of each other. Where I see a bump of "performance" on the photonstophotos PDR curve for the A7RIV, I don't see it in DXOmark (measured ISO).
- Now, if I go to DPR studio comparison tool, select Pentax K1 and Sony A7RIV RAW ISO800 or RAW ISO1600, comparison mode. I check camera settings in both cases (ISO, lens aperture and shutter speeds are the same for both K1 and A7RIV). A visual side-side comparison, same image size for both cameras, RAW format, show that the Pentax K1 has slightly less noise than the Sony.

So why photonstophotos PDR curve shows that the Sony A7RIV is "superior" to K1 above ISO320 setting (presumably thanks to dual gain technology)?
You can't be banding about logic based questions with proof just like that. The whole industry would shut down. Journalists write what they are told. All new tech is better.
02-23-2021, 04:01 AM   #3
Site Supporter
Site Supporter
Michail_P's Avatar

Join Date: Nov 2019
Location: Kalymnos
Photos: Gallery
Posts: 3,006
QuoteOriginally posted by robbiec Quote
You can't be banding about logic based questions with proof just like that. The whole industry would shut down. Journalists write what they are told. All new tech is better.
Most amusing opinion, and an one I share. It’s sad and true that the new Sony will always be “better” in any possible way. But what can you do anyway? Every page, blog, channel, praises what they are told or paid to. Marketing is a devil. You don’t win the market fingers crossed.
There’s even a small piece of photo “experts” that believe they can push the industry forward by suggesting technical improvements and stuff like that. Unfortunately I don’t think the line works like that. If you make it better you can always sell , if you make it the best , it’s an overkill.

Last edited by Michail_P; 02-23-2021 at 04:08 AM.
02-23-2021, 06:02 AM   #4
Loyal Site Supporter
Loyal Site Supporter
macman24054's Avatar

Join Date: Aug 2013
Location: Axton, VA
Photos: Gallery | Albums
Posts: 461
I would have to agree. The “story” is that mirror less and the latest tech is king. Who are we to question the “conventional wisdom” being pushed on us.

02-23-2021, 06:29 AM - 3 Likes   #5
Loyal Site Supporter
Loyal Site Supporter




Join Date: Mar 2012
Location: Rochester, NY
Photos: Gallery
Posts: 5,325
Mirrorless fans remind me of Toyota Tacoma fans. Even when the model years with the frames that rusted through in two years are still better than any other mid size pickup truck out there. Evey product has it's fanatics.


For the record, Ansel Adams used a mirrorless camera for his best shots. They are great not because he waited (sometime months), for the right lighting or had great composition and exposure. They were not great because he was a master of the craft of B&W photography. They were great becssue he use a mirrorless camera.
02-23-2021, 07:11 AM - 2 Likes   #6
Loyal Site Supporter
Loyal Site Supporter
UncleVanya's Avatar

Join Date: Jul 2014
Photos: Gallery | Albums
Posts: 28,384
I think those of you blaming marketing aren’t even close to the mark. Photons to photos isn’t that kind of site. It also clearly shows the excellent performance of the KP over many rivals.

I would read the tests data carefully. PDR <> Noise. While a system with a higher PDR is expected to be somewhat less noisy, there are other test charts like shadow recovery that may be more relevant. PDR more closely approximates the DXOMARK dynamic range data. The sports measurement in DXOMARK is the one that gives you a measure of low light performance.

Also, photonstophotos data is largely community sourced. You can directly contribute. Essentially you take controlled shots of some test images on your lcd.

I have some thoughts on why the data obtained this way might be limited. This also applies to raw files compared on an lcd. Most lcd panels fail to represent the dynamic range adequately. Most have limited dynamic range. True hdr with true sRGB color space is very expensive. This is why I think that the data at photonstophotos tends to get a little bunched up at the top since the best sensors dynamic range may be compressed a bit by the lcd panels.

I don’t know how many samples are used for a given camera, but I do know the driving force behind that site is quite math centric and his data is rigorously examined.

If you’re still reading, I’d suggest looking deeper into the explanations of the data on the site. Picking the right chart is key to interpreting what you see.

DXOmark on the other hand is less clear in their methods. They also have a history of not publishing data - withholding 645z tests for example -that suggests some motives that aren’t pure.

Lastly, the differences in the top sensors these days is very small - particularly if lcd display is how you plan to view the images. Most of the top sensors can out perform our common display devices at low iso. This is a really amazing situation. I personally rarely think these tiny performance gains play a role in most people’s photography. Only at the very edge of performance do these really become viable decision points. At that level I would rent the systems and directly compare them.

Last edited by UncleVanya; 02-23-2021 at 07:18 AM.
02-23-2021, 07:52 AM   #7
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,225
Original Poster
QuoteOriginally posted by UncleVanya Quote
While a system with a higher PDR is expected to be somewhat less noisy, there are other test charts like shadow recovery that may be more relevant. PDR more closely approximates the DXOMARK dynamic range data.
I think there are a lot of assumptions in photon2photo PDR, starting with output image as input data and going back all the way back to electrons based on a lot of assumptions that aren't verified. There is a lot of efforts into it, but are resulting figures and curves representative of reality. DXO use calibrated light input (somehow controlled light level) and estimate the real ISO. If I compare images at same ISO with shadows pushed by +4 or +5ev, I can see in the images that the K1 doesn't have more noise than say another camera with dual gain at the same ISO, which is what DXO dynamic range curves based on measured ISO agree with. If I look at the PDR of Photo2photos, normally when pushing shadows at ISO400 (to have dual gain triggered), according to the curve I should see less noise with the dual gain, I don't see it happening. I think, behind the very complicated explanations that I read on the p2p site, there is a major flaw:
- first, there is no need to get back to a virtual FWC , electrons, to measure dynamic range, if the only data input is the image output, why not make things a lot more simple and real by just looking at image output instead of going through a lot of assumptions to double guess what the input of the sensor may have been.
- second, if the goal is to measure the input based dynamic range (the range of light levels that can be coded by the image sensor), there is not proof that the theoretical models used to figure this out from output images actually works
So, the PDR curve can be nice, but disconnected with the actual image quality seen in real images.
I'd like to see evidence in real images, with exact same shutter speed and lens aperture, that dual gain sensors deliver significantly better dynamic range.


Last edited by biz-engineer; 02-23-2021 at 08:02 AM.
02-23-2021, 07:56 AM   #8
Loyal Site Supporter
Loyal Site Supporter
UncleVanya's Avatar

Join Date: Jul 2014
Photos: Gallery | Albums
Posts: 28,384
QuoteOriginally posted by biz-engineer Quote
I think there are a lot of assumptions in photon2photo PDR, starting with output image as input data and going back all the way back to electrons based on a lot of assumptions that aren't verified. There is a lot of efforts into it, but are resulting figures and curves representative of reality. DXO use calibrated light input (somehow controlled light level) and estimate the real ISO. If I compare images at same ISO with shadows pushed by +4 or +5ev, I can see in the images that the K1 doesn't have more noise than say another camera with dual gain at the same ISO, which is what DXO dynamic range curves based on measured ISO agree with. If I look at the PDR of Photo2photos, normally when pushing shadows at ISO400 (to have dual gain triggered), according to the curve I should see less noise with the dual gain, I don't see it happening.
I think that’s a fair way to discuss this. But blaming marketing and hype isn’t. Photonstophotos data isn’t as clean as I’d like but they are the only source for some like the KP. DXOmark history is murky. Also of note is that there a strong correlation of data between the sites albeit at different absolute numbers.
02-23-2021, 08:06 AM - 1 Like   #9
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,225
Original Poster
QuoteOriginally posted by UncleVanya Quote
I think that’s a fair way to discuss this. But blaming marketing and hype isn’t. Photonstophotos data isn’t as clean as I’d like but they are the only source for some like the KP. DXOmark history is murky. Also of note is that there a strong correlation of data between the sites albeit at different absolute numbers.
Sure. I suppose I should spend more time looking at actual images and rely much less on test charts. At the end of the day, with images, what you get is what you see, or what you see is what you get. KP images look fantastic.
02-23-2021, 08:31 AM - 1 Like   #10
Loyal Site Supporter
Loyal Site Supporter
UncleVanya's Avatar

Join Date: Jul 2014
Photos: Gallery | Albums
Posts: 28,384
QuoteOriginally posted by biz-engineer Quote
Sure. I suppose I should spend more time looking at actual images and rely much less on test charts. At the end of the day, with images, what you get is what you see, or what you see is what you get. KP images look fantastic.
Totally agree. And I’m not saying you shouldn’t ever look at Test data, it is a way to compare before buying - but the integrity and applicability factor is hard to judge sometimes.

My main objection was the earlier posts claiming intentional bias. DXOmark scores might have shady history, but I don’t think photonstophotos has an agenda.
02-23-2021, 09:12 AM - 1 Like   #11
Pentaxian




Join Date: Dec 2011
Posts: 3,112
QuoteOriginally posted by biz-engineer Quote
So why photonstophotos PDR curve shows that the Sony A7RIV is "superior" to K1 above ISO320 setting (presumably thanks to dual gain technology)?
I will give you the advice not only to not consider any "curves" a reliable source of information.

With regards to your question the much more relevant advice is to look at more image material than you did. Why?
A K-1 user has not need to look any further than ISO 100 in any case, because the camera is so wonderfully ISO invariant.
No dualgain, triple gain or whatever will ever trump proper ISO invariance.

Reason: Higher ISO will still always lead to at least 1 stop less dynamic range per ISO step.

So at ISO 6400 the Sony has around 9,5 Stop of DR according to dxomark (assume this is right for a second).

On the more/fully ISO invariant K-1 (see Image comparison: Digital Photography Review) you have no need to ever go away from ISO 100 as you simply pull up the shadows in post.
You do get an equally noisy image with ISO 100 pulled 6 stops on the K-1 as with The a7r4 at ISO 6400.
BUT: The K-1 image retains the full glorious 14,6 stops of DR. It will be so much better with highlights than any Sony at ISO 6400.

K-1 users simply can laugh at anyone caring for DR at higher ISOs. Our ISO-invariance for Raw shooters means, we have 14,6 stops of DR accessible at ISO100-6400 (at least).

And you can verify this with your own eyes with the available raws.

For the curve junkies: Shadow Improvement of Photographic Dynamic Range versus ISO Setting shows the ISO invariance. Ideally the whole curve stays on the 0-line (for a perfectly ISO-invariant sensor).
02-23-2021, 09:15 AM - 1 Like   #12
Site Supporter
Site Supporter
JensE's Avatar

Join Date: Dec 2015
Location: Leipzig
Photos: Gallery | Albums
Posts: 1,964
QuoteOriginally posted by biz-engineer Quote
Pentax K-1 vs Sony A7RIV: Comparing test charts (curves) from photonstophotos.net, dxomark and side-side comparison of raw image noise at ISO800 and ISO1600.
- According to photonstophotos PDR curve, above ISO 320 camera setting, the Sony A7RIV should be superior to Pentax K1, thanks to the dual gain sensor of the Sony A7RIV that the Pentax K1 doesn't have...
- If I look at DXO curves ("print" option selected) at same measured ISOs, dynamic range curves are basically on top of each other. Where I see a bump of "performance" on the photonstophotos PDR curve for the A7RIV, I don't see it in DXOmark (measured ISO).
- Now, if I go to DPR studio comparison tool, select Pentax K1 and Sony A7RIV RAW ISO800 or RAW ISO1600, comparison mode. I check camera settings in both cases (ISO, lens aperture and shutter speeds are the same for both K1 and A7RIV). A visual side-side comparison, same image size for both cameras, RAW format, show that the Pentax K1 has slightly less noise than the Sony.

So why photonstophotos PDR curve shows that the Sony A7RIV is "superior" to K1 above ISO320 setting (presumably thanks to dual gain technology)?
You have to read the charts a bit more carefully. First, a difference of 1/3EV is really hard to see and it doesn't all have to come from the bottom end (noise) but can also come from the top-end (highlight clipping). This depends on the choice of designated ISO, at least for the sites that don't measure the ISO according to their own definition, but use the camera's designated ISO. It's a useful diagram from photographer's point of view for each individual camera (you control nominal, not 'real' ISO), but of course opens the door form 'beautifying' test results by overstating ISO, as Fuji was reported to do.


What I can see in the photonstophotos PDR curve is that the A7RIV's PDR declines more rapidly that the K-1's in the 'low gain' range. The gain switch stops that, so that the curves of both cameras become basically parallel over a large range, with a mere 0.4EV difference. Now compare the DXO curve: Note that the first two A7RIV points are taken at nominal ISO50 and ISO200 - first point in the 'non-native' low ISO range! If you connect the ISO 50, 200, 400 points only on the photonstophotos PDR curve, you could not see a jump either. Instead, you see a very similar shape as on the DXO curve, especially if you move the ISO50 up to the measured 72. No contradiction here from a 'shape' perspective. Why the K-1 data is lower at base gain, conveniently comparable in DxOMark Derived Photographic Dynamic Range versus ISO Setting, I don't know. Note that this comparison is at nominal ISO. Taking the measured ISO of DXO into account, we find that at, say, measured ISO200 we have less than 0.1EV difference at the interpolated data in the DXO diagrams - which already benefit a bit from the 'lift' at ISO400 in the A7RIV case.

The visual appearance can be quite different even at similarly measured noise amplitude (or mean square averaged - RMS ... or whatever they define PDR as). I noticed that when I compared pixel-shift K-1 and 645Z test pictures on dpreview's test shots: Head to head at medium ISO, the 645Z data looks quite a lot cleaner at low ISO, even when giving the K-1 a one stop ISO advantage. The photographic dynamic range curves don't tell that story, both cameras should equally improve. Noise in the 645Z pictures is higher in amplitude, but less correlated that in the K-1 (at one stop lower ISO), leaving a much nicer rendition of fine detail on the 645Z. This isn't taken into account in the PDR definition, but can be observed visually.

On dpreview's test shots, I also notice that the dark areas render with a bit more contrast on the A7RIV compared to the K-1. Lens differences may be at play. This again make a difference in visual impression of shadow detail.
02-23-2021, 09:32 AM   #13
Site Supporter
Site Supporter
JensE's Avatar

Join Date: Dec 2015
Location: Leipzig
Photos: Gallery | Albums
Posts: 1,964
QuoteOriginally posted by beholder3 Quote
BUT: The K-1 image retains the full glorious 14,6 stops of DR. It will be so much better with highlights than any Sony at ISO 6400.K-1 users simply can laugh at anyone caring for DR at higher ISOs. Our ISO-invariance for Raw shooters means, we have 14,6 stops of DR accessible at ISO100-6400 (at least).
That is unfortunately not helping to capture extra detail in the shadows with a given aperture + exposure time. The dual-gain design actually works and will provide extra detail. If you have to accept slightly more noise at base ISO (as in the A7RIV) and the gain switch lowers that ISO to below your ISO100 level (and below the K1's very good one, too), you do get a benefit that may outweigh your loss at the highlights in terms of actually available dynamic range. It's a bit more pronounced with the A7RIII, btw. - which is nearly perfectly ISO-invariant in both gain sections and does provide a photgraphic dynamic range advantage this way over the otherwise identical K-1.
02-23-2021, 09:50 AM - 1 Like   #14
Loyal Site Supporter
Loyal Site Supporter




Join Date: Mar 2009
Location: Gladys, Virginia
Photos: Gallery
Posts: 27,650
I guess I think that numbers aren't images. Things like banding can confuse the situation (see the Z series of cameras in particular) and make some shadow detail unusable. The accelerator makes the numbers look better, but it doesn't actually give better shadow detail.

I too, don't see images that look better with the A7r IV, but maybe it is just me. But I don't really think the accelerator is the most wonderful thing ever. I own a K-1 and K-1 II and while the K-1 II images look a bit better in high iso situations, I can usually get K-1 images to look similar with a little bit of effort.
02-23-2021, 09:51 AM   #15
Pentaxian




Join Date: Dec 2011
Posts: 3,112
QuoteOriginally posted by JensE Quote
That is unfortunately not helping to capture extra detail in the shadows with a given aperture + exposure time. The dual-gain design actually works and will provide extra detail. If you have to accept slightly more noise at base ISO (as in the A7RIV) and the gain switch lowers that ISO to below your ISO100 level (and below the K1's very good one, too), you do get a benefit that may outweigh your loss at the highlights in terms of actually available dynamic range. It's a bit more pronounced with the A7RIII, btw. - which is nearly perfectly ISO-invariant in both gain sections and does provide a photgraphic dynamic range advantage this way over the otherwise identical K-1.
That sadly is marketing b/s.

The Sony dual gain sensors deliver no more detail, they lag behind actually at given lighting situations.


The a7r3 is actually the worst performer with regards to ISO invariance of those three cameras. The ten year old K-5 (within its apsc limits) outshines an a7r3 on ISO invariance until ISO 1600, same as a Nikon D7000.

Looking at the actual images easily reveals this.


However you try to twist it, the Sonys always lag on noise and dynamic range.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
charts, comparison, comparisons, conversion, curve, curves, dslr, full frame, full-frame, iso, k-1, k1, noise, pentax, pentax k-1, pixel, readout, sony, stage
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
An interpretation of MTF charts biz-engineer Pentax SLR Lens Discussion 5 02-12-2021 03:23 AM
Yvon Bourque's Focus Charts - Where Are They Now? Dale H. Cook Pentax SLR Lens Discussion 5 05-25-2019 01:56 PM
Test charts for Pentax DA 18-55mm 3.5-5.6 vs Pentax 16-50mm 2.8 at 18mm F8 - shocking Foma2 Pentax SLR Lens Discussion 10 06-15-2014 03:52 AM
Focusing Charts jbuck92 Pentax SLR Lens Discussion 8 11-21-2012 04:02 PM
Agreeing w/ Rand Paul.. wonders never cease!on jeffkrol General Talk 1 10-10-2012 12:22 PM



All times are GMT -7. The time now is 07:20 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top