[Andrews]

Not sure how accurate these dxomark number are.

I have downloaded RAW image for D300 and D90 for iso 800 and 1600 from imaging-resource for same scene.
Converted without any noise reduction and can't see much difference in 100% crop.
50D images looks noiser to me.

[rawr]

What does "Score" mean? There are no units - is it S/N, grain size distribution, max grain size . . . what? Are these "data" for RAW images or processed ones?...
Jer

For a full outlineof their methodology, visit their site:

A description of their low-light ISO score is:

Unlike the two previous scenarios in which light is either generous (studio) or stability is assured (landscape), photojournalists and action photographers often struggle with low available light and high motion, so that achieving usable image quality is often difficult when pushing ISO.

When shooting a moving scene such as a sports event, action photographers’ primary objective is to freeze the motion, giving priority to short exposure time. To compensate for the lack of exposure, they have to increase the ISO setting, which means the SNR will decrease. How far can they go while keeping decent quality? Our metric, Low-Light ISO, will tell them.

The SNR indicates how much noise is present in an image compared to the actual information (signal). The higher the SNR value, the better the image looks, because details aren't drowned by noise. SNR strength is given in dB, which is a logarithmic scale: an increase of 6 dB corresponds to doubling the SNR, which equates to half the noise for the same signal.

An SNR value of 30dB reflects an excellent image quality. Low-Light ISO is then the highest ISO setting for the camera such that the SNR reaches this 30dB value while keeping a good dynamic range of 9EVs and a color depth of 18bits. As cameras improve, the Low-Light ISO will continuously increase, making this scale open.

Photojournalism & action photography: Low-Light ISO

And all of their stuff reflects working from RAW alone.

For more info on their process, methodology etc:

Questions & Answers

I think their work is useful. It has limitations etc - as they make clear - but it is useful to have this sort of structured data around.

[rawr]

Are we looking at a CPU benchmark here??

No, a camera sensor sensitivity test, essentially.

[defiant]

From those graphs the K20D scored horribly! But i wonder if these scores are looked upon accurately? If i were to look at actual high-ISO photos comparing the cameras, it always looks like the K20D maintains the most amount of detail (not including full-frame cameras). The K20D also shows more noise usually, but noise can always be reduced in post-processing and so that's not as big a deal to me when compared to having lost significant detail in the first place, which cannot be recovered from post-processing.

For example:
Pentax K20D review Cameralabs noise results
If i were to compare the detail at 1600ISO, particularly in sensitive areas like the roof "texture" or the leaves or the layers of the brick walls, the K20D shows significantly more detail than the 450D, even though the 450D scored higher in the graph. The K20D also shows more noise than the 450D, but for my personal taste i much prefer more detail in the original shot at the cost of noise, as long as the noise is nice and random instead of patterny, and as long as the noise doesn't obscure detail. I don't know if i'm just in the minority in wanting more detail instead of lower noise in the pre-processed shot though?

[falconeye]

About DxO mark ...

They managed to produce a bag of numbers they don't properly understand themselves (e.g., it took them ages to start to realize that number of pixels does play a rôle ...). Let me say just this much: those DxO physicists wouldn't be able to publish their stuff in a renowed physical publication.

Until they get their stuff fixed, I decided to completely ignore their work (their DxO converter is great, though). It creates more confusion than anything else.

[rawr]

I think the key to understanding the way they construct the simple Low-Light ISO score is this paragraph:

An SNR value of 30dB reflects an excellent image quality. Low-Light ISO is then the highest ISO setting for the camera such that the SNR reaches this 30dB value while keeping a good dynamic range of 9EVs and a color depth of 18bits.

So (and I may be wrong...) the raw number means, for example in the RAW output from the camera:

- for the D700, at an ISO setting of 2303, the signal-to-noise ratio still reaches 30db, whilst still holding onto a dynamic range of 9EV and a color depth of 18bits;

- for the K20D, only at a lower ISO setting of 639 can the same parameters for SNR, dynamic range and color depth apply.

- ditto for the D90 at 977, the EOS450D at 692 etc.

It seems quite a tough test, actually, because those ISO's are quite low. It is even saying that the D90 can only deliver those results at a comparatively low ISO.

For the K20D the real comparisons should be within the APS-C space, and here the results for the K20D aren't much different from any of the newer Canon's. Only the D90/D5000 break away from the rest of the APS-C pack, so far.

APS-C size sensors are fundamentally quite boxed in when it comes to achieving good SNR whilst holding onto dynamic range and color depth, I think. Certainly when compared to full-frame.

And DXO do make it clear that their benchmark has nothing to do with all of the matters that may impact optical or camera quality:

General questions

I'm not too sure how the concept of 'detail retention' works its way into their calculations though. One would need to define what detail retention means, in terms of what a camera sensor does. I guess if anything, detail retention must also be a function of SNR, dynamic range and color depth?

Anyway, don't shoot the messenger. I think there stuff is useful because it's a methodically assembled dataset. But I'm no expert on sensors, SNR etc etc.

Hopefully this sort of stuff will encourage Pentax/Samsung to redouble their efforts to tweak their k7 firmware etc to do better against the Canikons.

Maybe even push them a bit harder along the way to cooking up a full frame sensor too, to match Canikon/Sony.

[Sailor]

For a full outlineof their methodology, visit their site:

A description of their low-light ISO score is:


Photojournalism & action photography: Low-Light ISO

And all of their stuff reflects working from RAW alone.

For more info on their process, methodology etc:

Questions & Answers

I think their work is useful. It has limitations etc - as they make clear - but it is useful to have this sort of structured data around.

Thanks much. I'll take a look.

Jer

[designinme_1976]

I have compared some high ISO images over from imaging resource, and unless I am missing something, I think the K20D is BETTER than the D300??

The detail is completely smeared with the D300? And now I see this list of actual ISO ratings, and I am starting to wonder why this is so difficult to see?

Maybe These images are not real world examples, or maybe my recent copy of the k20D is above average, but in my limited few days of shooting, I feel that the K20D achieves in my opinion better high noise handling while keeping detail than any of the previous cameras I have owned including.. the Canon 10D, Canon 40D, Nikon D200, and Pentax 10D.

Now, I completely understand that the K20D is a newer model than those listed above, yet I am convinced Canon-Nikon are smearing their high ISO images and though at first glance the images look cleaner, they lack substantial detail compared to the K20D, including examples I have seen online from the D300...

Am I alone here?

[Pentaxor]

I have compared some high ISO images over from imaging resource, and unless I am missing something, I think the K20D is BETTER than the D300??

The detail is completely smeared with the D300? And now I see this list of actual ISO ratings, and I am starting to wonder why this is so difficult to see?

Maybe These images are not real world examples, or maybe my recent copy of the k20D is above average, but in my limited few days of shooting, I feel that the K20D achieves in my opinion better high noise handling while keeping detail than any of the previous cameras I have owned including.. the Canon 10D, Canon 40D, Nikon D200, and Pentax 10D.

Now, I completely understand that the K20D is a newer model than those listed above, yet I am convinced Canon-Nikon are smearing their high ISO images and though at first glance the images look cleaner, they lack substantial detail compared to the K20D, including examples I have seen online from the D300...

Am I alone here?

you are not alone. I had seen samples before and they do lack detail when their ISO is boosted above 1600.

[asdf]

I have compared some high ISO images over from imaging resource, and unless I am missing something, I think the K20D is BETTER than the D300??

The detail is completely smeared with the D300? And now I see this list of actual ISO ratings, and I am starting to wonder why this is so difficult to see?

Maybe These images are not real world examples, or maybe my recent copy of the k20D is above average, but in my limited few days of shooting, I feel that the K20D achieves in my opinion better high noise handling while keeping detail than any of the previous cameras I have owned including.. the Canon 10D, Canon 40D, Nikon D200, and Pentax 10D.

Now, I completely understand that the K20D is a newer model than those listed above, yet I am convinced Canon-Nikon are smearing their high ISO images and though at first glance the images look cleaner, they lack substantial detail compared to the K20D, including examples I have seen online from the D300...

Am I alone here?

Canikon shots tend to be smeared from ISO100 up.

[SpartanWarrior]

I think that the K20D does really great at high ISO, you know i always hear how canon ISO is more clean, now i like canon too and have shot with them for 6 yrs but i believe i will stay with pentax now, here are just a few comparison shots with the K20D DA 17-70, and 40D Tamron 17-50 and to my eyes Pentax ISO is just as good or should i say even

k20D 1600ISO


40D 1600ISO


K20D 3200ISO


40D 3200ISO


They are right out of camera.

Also here is another from the K20D at 6400ISO i still think it looks great especially that lady on TV

[Lowell Goudge]

I think there are a lot of things than need to be considered in the graph shown.

As some have mentioned, part of this needs to consider ballancing the ISO setting and exposure. if the ISO is actually understated, it is not a fair test or comparison.

So, take the same shots used to assess each score and modify the exposure so they are all identical. Then measure noise again, Ill bet that changes things.

Also, measure or sort the quality based upon pixel pitch. i.e. a 14 mP camera on ASP-C has an equal pitch to a 31mP camera Full frame. No one is there yet, so compare noise against pixel pitch at high ISO.

Lastly look at noise vs age, newer sensors are better than older ones, show that improvement, because that is what it is all about.


Lastly, for the person who finds the noise of any present camera a limiting factor to their photography, I have absolutely no sympathy. Try shooting film at ISO 3200. Until you have done that, you will complain about DSLR high ISO noise, once you have done it, you will stop all thebitching because digital is so far superior to film in low light/high ISO situations that there is no comparison at all

[rawr]

I am pretty sure the DXOLabs guys have attempted to put all the camera sensors on a level playing field - as best anyone can, since over time things do move along and there are lots of different sensor technologies in their sample.

They've tested a LOT of cameras - from Hasselblad, Phase One etc as well as dozens more than those I've put in the chart - so I think they are pretty much on top of the complexities of benchmarking sensor RAW output... I find their efforts useful.

I look forward to seeing their results for the K7 - without too much delay after the device gets commercially released, hopefully.

[Ratmagiclady]

At least as comparing the K20d or any high-ISO setting, I find myself once again in the position of seeing something like the industry all mad for 'Fine grain,' years ago: ....back then I'd be like, 'Who cares how fine the grain is if it looks like crap?'



I definitely think I fall in the 'I'd rather have detail and tonal range than try to make it not look like a photograph.'

I also like there to be texture. As long as it's not *ugly,* And I think the question with this is ...with any given thing, what *can* you get.

[Mister Guy]

I am pretty sure the DXOLabs guys have attempted to put all the camera sensors on a level playing field - as best anyone can, since over time things do move along and there are lots of different sensor technologies in their sample.

They've tested a LOT of cameras - from Hasselblad, Phase One etc as well as dozens more than those I've put in the chart - so I think they are pretty much on top of the complexities of benchmarking sensor RAW output... I find their efforts useful.

I look forward to seeing their results for the K7 - without too much delay after the device gets commercially released, hopefully.

I've been reading through the links you've posted and trying to make sense of them. Professionally, I'm a computer programmer in the defense industry. I'm not going to go into much detail about what I do, but these guys are setting off all the alarm bells I've developed over the years in dealing with problematic customers. They have a set of data they meticulously collected and they are attempting to interpret it in a fairly meaningless way. The problem I'm discovering is they have developed a formula based around what they already believe. These are not objective measured numbers coming out of their formulas; they are a series of heavily weighted criterion for which they don't seem to be very open about the weighting process.

In other words, what their score says is that if you have the same expectations as they do, this is the sensor that most "correctly" meets it, but they won't tell you what the expectation is, beyond something nebulous about studio lighting and landscapes.

If anyone can actually dig up some real detail on how they do their weighting, I'd like to see it. Otherwise, I'm with Falconeye, I'm not sure they even know what their own numbers mean -- they seem to have just rigged the curves to look how they expect them to look.