[wildman]

From DXO:
"Sports & action photography: Low-Light ISO

Unlike the two previous scenarios in which light is either generous (studio) or stability is
assured (landscape), photojournalists and action photographers often struggle with low
available light and high motion. Achieving usable image quality is often difficult when pushing ISO.

When shooting a moving scene such as a sports event, action photographers’ primary
objective is to freeze the motion, giving priority to short exposure time. To compensate for t
he lack of exposure, they have to increase the ISO setting, which means the SNR will decrease.
How far can they go while keeping decent quality? Our low-light ISO metric will tell them.

The SNR indicates how much noise is present in an image compared to the
actual information (signal). The higher the SNR value, the better the
image looks, because details aren't drowned by noise. SNR strength is
given in dB, which is a logarithmic scale: an increase of 6 dB
corresponds to doubling the SNR, which equates to half the noise for
the same signal.

An SNR value of 30dB means excellent image quality. Thus low-light ISO is the highest ISO
setting for a camera that allows it to achieve an SNR of 30dB while keeping a good dynamic
range of 9 EVs and a color depth of 18bits.

A difference in low-light ISO of 25% represents 1/3 EV and is only slightly noticeable."

----------------------------------------------------------------------------------------------

Just trying to wrap my head around what all this means in practical real world terms.

Put it this way -

given two cameras with an ISO rating one of 1000 and one of 2000.
All else being equal does this mean a gain of 1 stop in ISO performance for instance?

So at an ISO rating of 1000 you have a certain level of noise, say X, at an ISO of 800 and at that
same level of noise on the camera with a ISO rating of 2000, all else being equal,
you have the same level of noise (X) at ISO 1600?

In other words:
DXO ISO rating of 1000 = X@ISO 800
DXO ISO rating of 2000 = X@ISO 1600

And DXO defines X as a SNR of at least 30db at a DR of 9EVs at 18bits.

Just trying to translate this rating into something more useful and intuitive.

---

Last edited by wildman; 08-02-2014 at 05:28 AM.

[Adam]

All it means is the highest ISO at which the camera achieves the image quality standards you quoted above. So the better camera would be at least as good at ISO 2000 as the other camera at ISO 1000, and yes, that would be a 1-stop advantage. But because the DxO test conditions don't apply to every practical scenario, you have to take the results with a grain of salt and factor in the primary use of your camera.

[Rondec]

Basically, the DXO Mark folks have identified the point for each sensor where noise increases too much. I find that the DXO Mark score tends to be overly conservative since it does look at SNR/noise more than dynamic range. I find that noise is "cleanable," while absence of dynamic range gives you photos that are pretty much OK for black and white. The K5 has a sport iso score of 1162, but captures dynamic range of 8.32 EVs at iso 6400. My experience with that sensor is that I can easily shoot to iso 3200 without much noise reduction at all and up to 6400 with some gentle noise reduction applied, but excellent quality photos.

I guess the other thing that this score doesn't mention is banding. Some sensors are prone to banding and this is much more difficult to deal with. The K7 had a sensor that did this and some of Canon's sensors do as well. This can limit high iso as much as noise and lack of dynamic range.

[wildman]

OK...

... so given all that, comparing the k5 to the Sony A7s, 3702/1162 = approximately a 3 stop advantage of the Sony over the K5.

Thus K5@ISO 800 = A7s@ISO 6400 so far as noise is concerned.
Does that sound about right in the real world?

I'm using an ISO of 800 as my standard reference because, based on my experience on the K5, unless I'm doing some extreme cropping there is usually no significant practical difference from IS0 80-800 when using the full frame. So for normal shooting I'm fine with an ISO of 800 or below when using the K5.

---

Last edited by wildman; 08-02-2014 at 12:56 PM.

[Rondec]

OK...

... so given all that, comparing the k5 to the Sony A7s, 3702/1162 = approximately a 3 stop advantage of the Sony over the K5.

Thus K5@ISO 800 = A7s@ISO 6400 so far as noise is concerned.

I'm using an ISO of 800 as my standard reference because, based on my experience on the K5, unless I'm doing some extreme cropping there is usally no significant practical difference from IS0 80-800 using the full frame. So for normal shooting I'm fine with an ISO of 800 or below.

Does that sound about right?

It doesn't look like that much of a difference to me. A7s at iso 800 has dynamic range of 12 EV, while the K5 has 11.1 EV. SNR is 37 dB at iso 800 for A7s and 32 dB for the K5. Both of those equate to slightly less than a stop of observable difference at iso 800 between the two cameras.

Where the A7s shines is in dynamic range at high iso, maintaining 9EVs of dynamic range all the way out to is 20,000. So there is a big separation in the dynamic range curves between the A7s and K5 (and even the A7r) above iso 6400, but the SNR curves continue to track about a stop apart.

[normhead]

Interesting....

Looking at the images at IR the Sony ATS at 6400 appears to be well ahead of the K-5 at 800, in both Dynamic Range and noise. Everytime I do a test like this, it turns out there is more to reality than what DxO knows about.

[Rondec]

Interesting....

Looking at the images at IR the Sony ATS at 6400 appears to be well ahead of the K-5 at 800, in both Dynamic Range and noise. Everytime I do a test like this, it turns out there is more to reality than what DxO knows about.

But, Norm, this isn't a scene to test dynamic range. Take an A7s out and shoot a high dynamic range at iso 6400 and see what you get and how much you can push the shadows. The K5 looks pretty good at iso 6400 too, in such a test.

I know you don't like DXO Mark, but their testing jives pretty closely what I have seen with the various cameras I have owned and how much noise they have and how much I can push the shadows on an image.

This is iso 1600 on a K3 -- above the "sports iso" score for the K3, but still fine at web sizes.



And iso 3200 on the K3.

---

Last edited by Rondec; 08-02-2014 at 07:09 AM.

[normhead]

But, Norm, this isn't a scene to test dynamic range.

But it is a scene that uses constant lighting and focus for both cameras, so you can use it for whatever you want. I find the concept of setting up a scene to test for Dynamic range as opposed to using everyday results somewhat repugnant. If the DxO scores can't be used to predict results in various uses they are worthless. Based on the quoted scores, results in the images i showed would not be anticipated. that has nothing to do with me liking or disliking DxO. I looked at what you posted. Went to a source of images shot under the same conditions, and came to a different conclusion than DxO did. That has nothing to do with liking or disliking DxO. That has to do with how reliable their data is when comparing cameras. And the way to prove that they are worth something , would be to come up with a couple images that support their data, which is what I was trying to do. You need to ask yourself, if their data is worth something, why is that so difficult? Why is it so hard for me to find a set of independently taken test images, where the cameras perform the way DxO says they do.

I guess it didn't occur to you that DxO makes money by getting people to trust their expertise, whether it's worth anything or not. That's how they get their pay check. Con men or performing a valuable service. The only way I can check is to see if their assertions can be independently verified. So far, they are striking out woefully.

ANd for the record, I used to love DxO. They gave me the impression of being people who knew something. However a number of people suggested I take a closer look, and when I did, the whole DxO magic thing fell apart. I know they're good a selling themselves , they had me fooled. Only because I was too lazy to read the fine print.

Their data is good, only of those shooting in living room with only a few dim artificial light sources., and no natural light. If you shoot in those conditions, well then, use them. Open a curtain and let some light into the room or shoot outdoors... they don't test in those conditions, it says so right on their website. I seriously cannot believe how many people who quote their data, don't know that.

So what's the problem with that you say?

They don't test much blue spectrum, and you can get much higher resolution from blue spectrum, than from red, especially on APS_c where the red is diffraction limited after 16 Mp. Their test lighting is not the same as daylight or even studio lighting. If you can't understand how that skews their results, I have a sense of how it works, but not nearly a clear enough understanding to explain it. Suffice to say, that's why their results don't make sense in the real world, or any world with a daylight balance or an abundance of blue light.

[Rondec]

But it is a scene that uses constant lighting and focus for both cameras, so you can use it for whatever you want. I find the concept of setting up a scene to test for Dynamic range as opposed to using everyday results somewhat repugnant. If the DxO scores can't be used to predict results in various uses they are worthless. Based on the quoted scores, results in the images i showed would not be anticipated. that has nothing to do with me liking or disliking DxO. I looked at what you posted. Went to a source of images shot under the same conditions, and came to a different conclusion than DxO did. That has nothing to do with liking or disliking DxO. That has to do with how reliable their data is when comparing cameras. And the way to prove that they are worth something , would be to come up with a couple images that support their data, which is what I was trying to do. You need to ask yourself, if their data is worth something, why is that so difficult? Why is it so hard for me to find a set of independently taken test images, where the cameras perform the way DxO says they do.

I guess it didn't occur to you that DxO makes money by getting people to trust their expertise, whether it's worth anything or not. That's how they get their pay check. Con men or performing a valuable service. The only way I can check is to see if their assertions can be independently verified. So far, they are striking out woefully.

ANd for the record, I used to love DxO. They gave me the impression of being people who knew something. However a number of people suggested I take a closer look, and when I did, the whole DxO magic thing fell apart. I know they're good a selling themselves , they had me fooled. Only because I was too lazy to read the fine print.

Their data is good, only of those shooting in living room with only a few dim artificial light sources., and no natural light. If you shoot in those conditions, well then, use them. Open a curtain and let some light into the room or shoot outdoors... they don't test in those conditions, it says so right on their website. I seriously cannot believe how many people who quote their data, don't know that.

So what's the problem with that you say?

They don't test much blue spectrum, and you can get much higher resolution from blue spectrum, than from red, especially on APS_c where the red is diffraction limited after 16 Mp. Their test lighting is not the same as daylight or even studio lighting. If you can't understand how that skews their results, I have a sense of how it works, but not nearly a clear enough understanding to explain it. Suffice to say, that's why their results don't make sense in the real world, or any world with a daylight balance or an abundance of blue light.

Well, I am not buying what DXO Mark is selling -- their products that is. I do shoot real world images and the question I have is can I lift the shadows on an image or, not. Cameras prior to the K5 were lousy at doing that. Take a K10 or K7 image at iso 400 and try to pull up shadows and you get noise all over the place. That is not the case with the K5 or K5 II. But when you get up to iso 3200 or 6400, you can't lift shadows any more (or at least not much) -- your exposure is what you've got.

There just is always hype around cameras. When the K3 was released there were people claiming all sorts of things about its high iso ability. Better than full frame, etc. The reality is that it is the same. The A7s is minimally better than the A7r at isos below 6400, but certainly it just isn't that different (assuming same printing size/viewing size).

Iso 6400 on K3.

[normhead]

To show different (or not that different) you have to have two images. So would you say the images I posted above are different, or not much different. Maybe we just differ over what's different, or not very different.

[Rondec]

To show different (or not that different) you have to have two images. So would you say the images I posted above are different, or not much different. Maybe we just differ over what's different, or not very different.

I struggle with you saying that you want real life images and then you post well lit static scenes from IR that seem quite the opposite of real world.

[normhead]

I struggle with you saying that you want real life images and then you post well lit static scenes from IR that seem quite the opposite of real world.

You didn't answer my question...

Listen, I read the stats, I did a quick check with a known resource. The results didn't jive with the DxO results. End of story. if you want to nit pick, fine... I'm just saying, DxO results only apply in their lab, and checking images on other sites confirms that. That's the simply stated , state of affairs. Theoretical postulations must be confirmed to be of use. DxO will neither provide original data nor has their process been confirmed by independent review or replication. They are not scientific, they are hype who's only purpose is to sell product. Is that so hard to comprehend?

It may be true that their data are a great resource of low light photographers... my problem is, they present their findings as if they were relevant for all shooting conditions, and they demonstrably aren't, not even for normal shooting conditions. I'm not saying there is something wrong with their research. I'm saying they, and many other people present it as something it's not. Many scientists make the same mistake... a scientist is often some one who knows a great deal about a very narrow sliver of the pie, and who elevates the importance of that sliver above the whole rest of the pie. My guess is DxO falls into that category.

I'm not saying they aren't great at what they do, only that what they do isn't relevant to photography as a whole.

[wildman]

If I may be permitted to play moderator for a moment...


... My original question was, given DXO's methodology for determining ISO performance how am I to make a meaningful practical comparison between different cameras?
In other words how does DXO expect the ordinary photographer to understand and use their data for such a comparison?

My question was not about calling into question the efficacy of their methodology in the first place - that's a topic for another thread.

Frankly at this point I don't care if their methodology is any good or not I'm just trying to better understand what DXO thinks it's saying with their ISO ratings and how to best use these rankings.

[normhead]

Interpret their data to be accurate as sampled. If you want info on camera performance in low indoor light like you'd find at night in someone's living room, as described on their website, take it as gospel. it's not accurate for images taken with studio lighting, or outdoor light... just my opinion of course.

Do I have to explain everything?

[Rondec]

If I may be permitted to play moderator for a moment...


... My original question was, given DXO's methodology for determining ISO performance how am I to make a meaningful practical comparison between different cameras?
In other words how does DXO expect the ordinary photographer to understand and use their data for such a comparison?

My question was not about calling into question the efficacy of their methodology in the first place - that's a topic for another thread.

Frankly at this point I don't care if their methodology is any good or not I'm just trying to better understand what DXO thinks it's saying with their ISO ratings and how to best use these rankings.

Well, I think you need to remember as you compare sports iso scores, that iso is a logarithmic scale -- there would be one stop difference between the K5 at 1162 and a camera that had a sports iso of 2324 and two stops with a camera that had a sports iso of 4648. So I guess that puts a stop and two thirds difference between the two cameras, just using the sports iso as your comparison point.

As I said above, I think the big thing about the A7s is how it holds onto dynamic range at high iso. This to me, should make the camera quite a bit more usable at high iso ranges than any other camera out there.