[MJSfoto1956]

So a year ago, DxOmark reviewed the K-1 and gave it an excellent score of 96, putting it in the top 5 best full-frame cameras ever.
However a year later, they have yet to publish the data on the K-1 + Pentax lenses. Note that they have the data as the K-1 is fully supported with DxO Optics Pro v11. Wish they would get off their backsides...

Attached Images

 

---

Last edited by BigMackCam; 01-25-2017 at 12:48 PM.

[VoiceOfReason]

They have also been hiding the 645z results for some time. They made a mistake and published them once on accident.

[MarkJerling]

Indeed! The leaked 645Z results. 101! See https://www.pentaxforums.com/articles/photo-industry-news/pentax-645z-ranked...y-dxomark.html https://www.dxomark.com/Cameras/Pentax/645Z and Pentax 645z scored 101 at DxOMark? | Pentax Rumors

[lithedreamer]

They've gotta be doing this on purpose, but... why?

[luftfluss]

It's rigged.

[MarkJerling]

They've gotta be doing this on purpose, but... why?

There's been lots of theories why, but no response has ever been forthcoming from DXOmark.

[bm75]

Hmmm... there's not a single Fujifilm camera test on dxomark....hmmmm... Why so???!!!! Aren't the Fuji lenses good enough to be tested?

[surfar]

Aren't the Fuji lenses good enough to be tested?

They are too good!

[bm75]

Exactly !! why the hell a APSC lens would resolve so much lpmm on a 16 mpx sensor when Dxo states that the more the resolution, the more the sharpness and consequently the biggest dxo scores (thus FF being the best)?

that said, they have a scientific basis in their tests, so I don't want to be unfair towards them.

[tvdtvdtvd]

I think the bigger mystery is why anyone still pays attention to DxO. Their bias is obvious, so why even bother.

[Na Horuk]

that said, they have a scientific basis in their tests, so I don't want to be unfair towards them.

Many faults, though.
a) They don't publish exactly how they do the tests. Tests could be faulty, are unverified, have not been reproduced, and we don't know how many lenses they used for each test (because of sample variation, this could be a significant problem).
b) They test lenses on various cameras, which means you cannot compare the lenses between different cameras (not even from the same brand or with same sensor). Even if you look at the same lens tested on K-5 and K-3II, there are differences - and not just the ones you would expect. One lens gets much better, another only a little better, some get worse.. its weird. They should test lenses on an independent sensor (or extremely high resolution film - these do exist) so we could then compare lenses across brands. This should be fairly simple and everyone could reproduce the test
c) Same thing for camera tests - they get tested with some lenses, but I am not sure they use the same exact ultra high resolution lens for all camera tests, which would be required for objective camera comparisons. Some cameras get tested in special hires modes, but others don't. Some cameras have NR and sharpening built-in into the "raw", which also skews results.
d) Finally, not reviewing lenses on K-1, not publishing 645Z results (despite doing the test!), not using Pixel shift mode correctly with the K-1 review (but they do use advanced features like PS when testing their own camera!),... either these are big blunders, or purposeful bias. I don't know why, but I assume its because Pentax doesn't have a big budget to pay for adverts and positive reviews (unlike Nikon, Sony, and Canon)

When looking for lens info, I look at other websites, as well. LensTip, Photozone,.. and user reviews on this forum. You can find sample photos on 500px, Flickr, in threads here.. relying purely on some numbers from DXo is a bad choice

[bm75]

There are some repetitive patterns, though. I suppose their lens evalueting system is strongly affected by f/stop number: this thing itself means that , from the biginning they consider a f/4 lens just worse than a f/2.8 one (not exactly what describes the character of a lens, in my opinion...). Then the mpx count is obviously of critical importance for them: the same lens performs from 12 to 27 depending on the camera used (typically 12 on 12 mpx APSC , 27 on 36mpx FF - Nikon system ) . Those results are repetitive: 16 mpx k5 and d7000, sharing the same sensor, will affect lenses test in the same way - typically scores not exceeding 18, no matter the brand. Same lens, different sensor, no matter the brand, performs better on FF. You'll recognize some lenses they consider crappy because the difference between tests at 24 mpx and 36 mpx are of minus difference, thus involving the lens not being able to perform correctly at 36 mpx.
I think that there could be some scientific method basis.

---------- Post added 01-26-17 at 12:14 AM ----------

When looking for lens info, I look at other websites, as well. LensTip, Photozone,.. and user reviews on this forum. You can find sample photos on 500px, Flickr, in threads here.. relying purely on some numbers from DXo is a bad choice

You're right. I do the same but I find Dxom Iso tests quite credible. For sure they give just numbers and their results have to be taken "cum grano salis".

[Na Horuk]

I think I really lost faith in DXo some time ago when I was looking at DA 40mm reviews. What really confused me is the SMC DA 40mm, DA 40mm XS, and HD DA 40mm. The order of best to worst of these three is different when tested on K-5 and K-3. Even though they have really similar, if not identical, optics (the HD has newer coatings, plus it has rounded blades)

[bm75]

We must remember that those kind of tests are not conducted "in the field", with real subjects ,but in a laboratory (we hope ) under controlled conditions. Real world testing is something different and surprises are just round the corner. in my opinion lenses perform always better in the field than in the laboratory.

[Digitalis]

they have a scientific basis in their tests

They don't. They have only partially published their testing methodology and it is rather questionable. They should allow users to have access to their raw data, even if it only to paying members.