Originally posted by Dan Rentea Yes, it's quite visible the difference in the Imaging resources test compared to the one from DPreview website. And bare in mind that this is still a lab test where you can set a custom white balance and you have controled lights. In real life shooting conditions the difference is even more visible. But, people trust DXO and I don't intend to make them change their minds. I just think that there are other/better tests that can be done if I want to see which camera will give me the best results in various shooting conditions.
Those are good points and the show that as detailed as all these tests may seem, they don't always fully-characterize sensor performance. That's especially true with respect to color and performance under less controlled lighting.
Do any of the test quantify the RGB channel gains of the sensor (which are required to correct for differences in the chip's sensitivity to different wavelengths and the specific design of the color filter array). There's some tricky design trade-offs in the color filter array that trade-off color saturation and ISO which then affects the required gain on each of the RGB channel and the "color" of the noise in the resulting images.
Less-perfect light also has a impact. Tungsten light, for example, has about 1 stop less green light and 2 stops less blue light than does daylight (I say "about" because it's a more complicated function of the sensor's color filter array, too). Thus, an ISO "1600" image taken in tungsten light, the green channel is probably being boosted to ISO 3200 and the blue channel is probably being boosted to ISO 6400. Two cameras might have identical performance at ISO 1600 but if they differ in performance at higher ISOs, then a tungsten-lit scene will look different for the two cameras.