[PentaxPoke]

Several people have already made the point that static numbers and synthetic benchmarks are only moderately good as predictors of real-world behavior. They're so bad, in fact, that the only reason we use them is that nobody has come up with another way. The fractal color/resolution/bandwidth map sounds like an excellent solution to me, but if cameras follow audio and computers, it wouldn't be long before manufacturers were building sensors specifically to perform well on those particular charts - and they would cease to correlate to real-world performance.

After hours and hours and hours of poring over images from pixel-peeper.com and various review sites, I've come to a somewhat heretical conclusion: There is far more difference from photographer to photographer than from camera to camera or lens to lens.

Additionally, people make trivial differences sound as though they're penultimate, conclusive analyses. The difference between some of the sensors that people are so religious about is less than the differences between some of the films of the 70s and 80s that people were so religious about.

If your photography is a laboratory activity, then all these numbers might be critical. But if it's an artistic endeavor, then all of these numbers are naught, as art is subjective. Well, not completely irrelevant, but they should be only suggestive rather than decisive. Pick the platform that meets your artistic requirements - whether that's noiseless low-light performance or stunningly sharp macro images or grainy B&W images. Then take some pictures!

Amen. Go to Imaging Resource "Comparometer" and compare even the k2000 (a $400 camera) to the D3x (an $8000 camera). Even at ISO 3200 you have to download the images and open to a relatively large size before you see the difference. The concept of diminishing return comes to mind. (sorry to provide bait for the FF fans. )

---

Last edited by PentaxPoke; 07-17-2009 at 10:35 PM.

[rawr]

It is easy to agree with you guys (jstevenwhite, Pentaxpoke) in principle. But I bet neither of you guys for example use 3MP point and shoots to do your photography, even though similar arguments could be applied to them too.

Even though I actually do have several thousand images from an old Ricoh Caplio 3MP point-and-shoot that look great when printed for example, the camera was hopeless after dark or indoors and I recognised long ago the need to move on if I wanted better images.

It's the same story here in this thread. The tech is evolving. As it evolves it opens up new opportunties for photographers.

The numbers tracked by stuff like the DXOMark scores aren't tablets from God about the art of photography, they are just data about digicam sensor evolution.

[tarsus]

PentaxPoke is correct. One of these pictures cost $8000 and the other $400. How big are people printing. I am at a loss to understand the need for "100 percent straight out of the camera."

[TomE]

Love that second shot! I expect !3200 was a typo for ISO using the D3, if not that was not a fair test using 3200 for the D200.

Tom

[rawr]

One of these pictures cost about $400 and the other about $200. How big are people printing? And why are people bothering to buy complicated DSLR's too.

[tarsus]

Rawr. Low light and fps.

[rawr]

tarsus. Then get the F100fd - it does equal or better fps than the K2000 and is actually excellent in low light....

But real low-light performance isn't highlighted at all by these sort of Imaging Resource test shots, which is why I don't like them at all.

Drop the lighting down to 'walking around the city streets at 9pm' levels, make the sensor really do some work, and then the differences between camera low-light ISO capabilties (meaning ability to get a good image under low light whilst still maintaining a certain minimum standard of IQ) really jump out at you, whether pixel peeping or printing. These studio shots don't do any of that.

[tarsus]

Rawr. I happen to completely agree with you on the validity of the test shots under those controlled conditions. They are both, however, at ISO 3200 and serve to show that ISO differences in cameras, K2000 and D3x in the above case, are greatly influenced by print size.

[rawr]

I am in complete agreement with you then tarsus. The image quality by print size issue can be a real leveller between cameras. Sometimes even a David vs Goliath leveller, as you have shown.

[pcarfan]

It is easy to agree with you guys (jstevenwhite, Pentaxpoke) in principle. But I bet neither of you guys for example use 3MP point and shoots to do your photography, even though similar arguments could be applied to them too.

...................

+1

Also, think about cropping.

[Haakan]

I did do a test with RAW files from Imaging Resource that I posted in another thread
https://www.pentaxforums.com/forums/668540-post29.html

Since they are relevant also in this discussion I have copied them here. These are 100% crops, so they take into account cropping. As I state in my previous post, I think that the differences between "best in class" APS format cameras are almost undistinguishable.

The RAW files from Imaging Resource are for K20D (K7 not yet available), D90 and EOS50D. I converted them all in Raw Therapee, using "zero" settings for all values, except for exposure which was adjusted to give all images a similar white level. I used was the "multi targets" from Imaging Resource from which I cropped out the "Kodak Target" as a good representative image.

I have attached JPEGs of the resulting images below, scaled to the same sensor resolution (12 Mpix), 100% crops. When looking over the images, there seems to be some inconsistencies in the focusing, but since I only look at the noise structure, that should not be an issue.

When I looked at the resulting images, this was my take on them (note that this were subjective and others might come to other conclusions).

1. All 3 cameras are extremely capable and as indicated by the DxO SNR curves, they are all very close in overall "image noisiness", with a slight advantage to K20D at lowest ISO, in having 1 dB lower noise at its lowest ISO vs. the D90. I.e. at its best setting the K20D should produce a slightly cleaner overall images than the D90. But at higher ISO, there is a slight advantage to the D90.

2. Looking at the higher ISO images (ISO 1600 and 6400) for all tonal ranges, except for the darkest tones, there seems to be no big difference in the noise levels. To me they all give a similar "noise impression". The D90 looks cleaner in the darkest tones whereas the K20D looks cleaner in the lighter tones. Overall the EOS50D looks slightly noisier but the difference is quite small.

3. Looking at the character of the noise, I personally felt that the noise for K20D is somewhat smoother looking than for the D90. The D90 seems to have more "posterized" noise. When I cleaned up the images in "Neat Image" I felt I liked the result from K20D better than for D90. But with the differences being quite subtle, I think that Nikon and Canon owners would be equally right in claiming a preference for their respective brands.

Best regards,
Haakan

K20D ISO 100


D90 ISO 200


K20D ISO 1600


D90 ISO 1600


EOS50D ISO 1600


K20D ISO 6400


D90 ISO 6400


EOS50D ISO 6400

[rawr]

Thanks Haakan. That is a useful sequence of images. Not much seems to seperate the tested sensors, does it, under those tests.

The Imaging-Resource test was conducted under bright studio lighting presumably - I wonder how the results would go if the lights were dimmed - eg to replicate about EV 3-6

Exposure value - Wikipedia, the free encyclopedia

Hmmm. I may be exposing my ignorance here, but logically, I know that there shouldn't any difference between exposing in bright lights or exposing in dim light in order to test high ISO. The camera will adjust shutter/ aperture etc in order to deliver the light needed for a viable exposure regardless of the level of light falling on the subject.

But I can't help but think there would be a difference in the results because of the variable of luminance.

I may be in error here, but if the subject is dimly lit, there is less light (luminance) at source for the sensor/lens to work with, no matter what adjustments the camera or photographer may make to shutter speed or aperture in order to achieve an accurate exposure. The sensor simply can't capture light that wasn't there to begin with.

While the imaging resource Ektacolor test chart images are useful, I think tests that work across a wider luminance range are probably more revealing, I think, of sensor capabilities under low-light. FWIW, I believe DXOMark test that way.

[daleroy]

Amen. Go to Imaging Resource "Comparometer" and compare even the k2000 (a $400 camera) to the D3x (an $8000 camera). Even at ISO 3200 you have to download the images and open to a relatively large size before you see the difference. The concept of diminishing return comes to mind. (sorry to provide bait for the FF fans. )

The problem with those images is they're taken in a well lit environment. Not many of us here would be shooting ISO 3200 with that much light available.

Take candlelit scene at ISO 3200 with say a D700 and a K2000 and at say 8x10 print size, you will definitely see a huge difference.

Also, the D3x isn't known for fantastic high ISO performance, it's not what it's really good at (which is resolving an insane amount of detail).

There is a law of diminishing returns, like all technology. If you want the best, you have to pay for it. Some of us don't mind spending a few extra thousand dollars to get a camera that we really like. And lets face it, if people didn't buy the latest and greatest, the camera companies wouldn't be getting return on their R&D and we wouldn't see better sensors etc.

[Haakan]

I may be in error here, but if the subject is dimly lit, there is less light (luminance) at source for the sensor/lens to work with, no matter what adjustments the camera or photographer may make to shutter speed or aperture in order to achieve an accurate exposure. The sensor simply can't capture light that wasn't there to begin with.

I will not claim to be a sensor expert, but I was thinking that the the signal output is based on the number of photons hitting the sensor (pixel), multiplied by the gain (varying by ISO). So if that number of photons was reached by a better illuminated object and small aperture/high shutter speed, or a less illuminated object and larger aperture/slower shutter, should it not give the same result?

The only thing that I can think of (again claiming no expertize), is that noise might increase over time, thus a longer exposure time might have a negative effect on noise.

If that is the case, that could certainly explain the results of the images posted in the thread K7 vs D90 https://www.pentaxforums.com/forums/pentax-dslr-discussion/66028-k7-vs-d90.html, where the settings where quite different for the two cameras, (e.g. for the ISO 3200 shots the K7 had 1/10s and f/22 whereas the D90 was set to 1/160s and f/6.3

I.e. for the K7 the aperture was closed down to give much longer exposure times for the K7 than for the D90. So if your theory is correct then the comparison in that thread was severely biased to make D90 look better.

How DxOMark obtains there higher ISO reading, I do not know, but looking at their noise readings for the different cameras at different ISO levels, I think it is fair to say that they are very close to each other with differences so small that it would be hard to see in images. (I have attached the graph for the normalized/printed results with the 3 cameras that I compared by the RAW files from Imaging Resource)

Best regards,
Haakan

[rawr]

The higher DXOMark low-light reading for the D90 comes about because for DXOMark

Low-Light ISO is then the highest ISO setting for the camera such that the SNR reaches this 30dB value while keeping a good dynamic range of 9EVs and a color depth of 18bits.

While the results between the D90, K20D and 50D are line-ball on SNR, the D90 sensor does very well in terms of dynamic range, as you can see in the chart below, which helps give the D90 low-light ISO score a boost over the Pentax and Canon.