[asw66]

The difference between the K-5 and the D7000 in the above numbers is quite dramatic.

As it happens, a friend of mine has had a lot of trouble getting accurate focusing with his D7000. If I understand him correctly, the D7000 has a bad habit of focusing on a point of higher contrast than the actual selected focal point.

On a more positive note, he's found that the D7000 is a pleasure to focus manually -- it has little indicators that indicate whether you're focusing in front of or behind the selected point. So he's refined his kit to a rather Pentax-like pair of 24mm and 105mm primes instead of the big VR zooms that he used to use.

[falconeye]

Thanks to froeschle, I now had a chance to have a deeper look into the test.

First, the test is more complete and consistent than I would have thought. It certainly is the best test published in the print media about AF accuracy that I am aware of. But it isn't without its own flaws.

First, what the test does right:

1. First, they work with experts.
   Image Engineering - test charts . camera testequipment . image quality analysis is who did the test and I guess, it wasn't cheap. Image Engineering is on the better side of testers. I know about one who did his thesis there and I've seen his diploma thesis about the efficiency of shake reduction. Good work overall.
   It measures sharpness in an objective way. The test method is described here:
   Der neue Kameratest - Testversion 1.6 - colorfoto - Magnus.de (section "Auflösung").
   
   Basically, they measure the spatial frequency where MTF drops to 10% aka MTF10. That's a more intelligent measure than MTF50 and they use a star chart rather than edge blur to do so. Their method may therefore be superior to Imatest's (and photozone's or mine). For an AF test, it is a bit overkill though. More cumbersome, it is a resolution test not allowing an easy mapping into focus accuracy (what it more easily possible in my tests based on edge blur).
   .
2. They use a focus series to determine best focus
   .
3. They use a multitude of lenses and focal ranges (however, they seem to use each lens wide open only).
   .
4. They make 1300 shots overall, 10 shots at each setting. That's good enough statistics.
   .
5. They defocus the lens prior to every test shot.

Second, the results in a bit more detail:

Based on this setup, they find that Pentax performs best in the phase AF camp. As I said, they declare Panasonic to be the winner (overall). But they also declare Pentax to be position 1 (Platz 1) in the phase AF camp. It isn't a headline in the text though.

But the opening double page shows six cameras with bar charts and it clearly shows Pentax to be ahead of everybody except Panasonic who is the clear winner.

Pentax wins #2 because most of the lenses performed strong.
- Strong in particular: DA21, DA14, Sigma70-300, DA50-200
- Average (which is good): DA15, Sigma10-20, Sigma18-250, DA12-24
- Weak: DA*55, Sigma50/1.4
For the zooms, the average lenses were strong at some focal lengths. They tested 10 lenses, 20 focal lenghts and 200 shots for Pentax.

It was 410 shots for Canon and 270 for Nikon. Canon only had two strong lenses. But the 16-35 (average at the wide end) was very strong at standard and tele end. Similiar for Nikon: no strong lens, but e.g., the Sigma70-200 at 135mm was strong. So, YMMV.

They made 30 shots with a NX11 50-200 and 10 shots with a NEX-5 16mm but did not rank them. Both were very good.

Olympus ranked behind Pentax (despite contrast AF) because of a few weak lens/focal length combinations, such as 14-42 at 42mm.

Sony SLT was good overall, but the 70-200 showed catastrophic results, with obviously some totally blurred images.

BTW, Pentax was second fastest in the phase AF camp too, with 390ms for 1000 Lux (Sony SLT was fastest with 330 ms). Actually, the Panasonic GH2 won the speed comparison too (280ms!, despite contrast AF, 450ms for the GH1). That's pretty amazing and another headline, maybe. Almost twice as fast as a D7000 (460ms). But at such good lighting conditions, static AF speed isn't the real problem anymore anyway.

the quote

This is the table Class A posted, complemented by the two contrast AF entries:

My rank	Type	Camera	% sharp	% acceptable	% not sharp	speed 1000lx
1	contrast	Panasonic GH2/1	85.3%	14.0%	0.7%	280/450ms
2	phase	Pentax K5	62.4%	28.9%	8.6%	390ms
3	contrast	Olympus EP1	57.0%	29.0%	14.0%	1040ms
3	phase	Sony Alpha 55	62.1%	16.4%	21.4%	330ms
5	phase	Nikon D7000	35.2%	40.0%	24.8%	460ms
5	phase	Canon 7D	40.2%	29.7%	30.0%	440ms

Third, the flaws in the test:

IMHO, there are 4 major flaws in the test:

1. The sample variation of bodies is large. They did not write that they did AF fine tuning for each lens tested, prior to the test. For the K-5, I found this to have a large impact on results for such a test. A 5% drop in MTF10 is a rather fragile measure.
   
   This means that the tested K-5 had good calibration out of factory. Which is a combination of luck and better than competition quality control. Quite some surprise
   .
2. They make an error in their scale:
   
   They measure sharp to be 95% spatial frequency resolution of best focus. IMHO, that's nonsense and a result of the difficulties to map MTF10 to focus accuracy. I.e., a soft lens to start with, which has a small aperture, will have an advantage because the criterion allows for much less accurate to be still considered sharp.
   
   This explains why many mediocre lenses (like DA50-200) performed nicely in the test. Worse even, it introduces an arbitrary factor into the test: test enough bad lenses (or closed down lenses) and you'll win the AF competition. I don't see that they did this. But it is a flaw of the test methodology.
   
   Moreover, a body with high resolution is at disadvantage too, because then the 100% measure is higher. That certainly was an extra burden for the 7D.
   
   They should have recomputed focal plane accuracy (in µm/sensor size) from image quality. This is doable as I've shown in my K-5 low light focus test.
   .
3. They don't select from even lens categories, like one fast portrait lens, two standard zooms, a wide and a tele prime etc. With equal aperture settings in the test.
   
   And they don't predetermine aperture across similiar lens tests.
   .
4. They only tested at 1 luminosity: 1000 Lux.
   .

One may argue that a higher resolving lens or camera requires better focus accuracy and that the test automatically accounts for it. However, I dismiss this argument. If I want to know what to do to produce a sharp photo when hanging on a wall, I want to know what camera and lens to choose. And not the other way round.


Overall

Despite the methodological flaws, I think that the result holds true. They print bar charts for all 1300 shots! And their entirety speaks a clear language:

The tested K-5 outperforms the tested 7D, D7000 and A55 in terms of phase AF accuracy (and speed actually).

If this only holds true for the tested samples or is a general result on the market remains to be seen. But Pentax always had a good reputation for focus accuracy and maybe, for a very good reason. I think overall, the test result is credible.

[Class A]

The intersection of the lens sets used for the K-5 and D7000 is
Sigma 10-20/3.5
Tokina/Pentax 12-24/4
Sigma 18-250/3.5-6.3

Would be interesting to see how the duel between the cameras ends up when looking at the results for these lenses only,

I'm a bit puzzled by the mixed lens bags approach.
FWIW, the average focal length and aperture of the D7000 lenses is 94.4mm & f/3.1 respectively.
The average focal length and aperture for the K-5 lenses is 63.2mm and f/3.5 respectively.
(For zooms I took the middle of the zoom range and the middle of the aperture range).
The average "blur factor" (focal length / f-ratio) for the D7000 lenses is 35.1 and for the K-5 19.1 (calculated per lens and then averaged). I believe this could explain an advantage for the K-5.

Not complaining about the good press for Pentax, just wondering how well deserved it is. Certainly it seems that quite a bit of luck was involved for the tests to use a K-5 that was well calibrated out of the factory.

[Kenn100D]

Frankly, I don't know.
Theoretically, a lens shouldn't have to be calibrated as much as the body. Because the AF module anyway sees any defocus.
But I didn't systematically check it.
From two K-5 bodies and two lenses (FA31 and DA70, i.e., four combinations) I'd say that there is significant sample variation between bodies and some bodies need less AF correction than others. When needing less AF correction, K-5 phase AF often beats contrast AF accuracy. With the lens being the smaller factor. But not always.

The best focus I achieved at 31mm at f/5.0 was 1.35px with contrast AF while the two best phase AF measurements I got were 1.24px (which is a fabulous value) and 1.45px. That's all in one ball park.

I don't have enough data to isolate factors which make phase AF the option to go for. I am sorry.

But let me say this much (which is the safe variant of statement I can make):

With the K-5, phase AF can be more accurate than contrast AF. But the performance is more fragile and in other cases, contrast AF may win because it delivers a more stable accuracy (more stable across bodies, lenses and apertures). OTOH, contrast AF can fail miserably and fortunately by so much that it is visible to the naked eye.

Note that the above is a non trivial statement. Many testers assume that Contrast AF or Manual LV focus is the way to go for tests. Not necessarily so though with the K-5.


Note:
I said several times that I consider mechanical focus precision, in particular a play (or better its absence) between the forward and backward focus direction, to be crucial for accurate focus. At least for focus algorithms which go forth and back. Most contrast AF algorithms do it while it can be avoided in many cases for phase AF. But better contrast AF algorithms (better than Pentax uses) can avoid this too, I worked on them. So, if a lens lacks mechanical precision engineering, it may face some focus accuracy problems for reasons other than optical. So, the build quality may actually pay off here. After all, top notch focus accuracy is much better than the thickness of a hair.

Thank you for the answer Falk. Now I begin to wonder why Pentax include the AF adjustment to K-10, K-20 & K-5. Just in case i bought a lens which it's build quality is not up to standards, i can still use it using this function. I believe they always know some lenses that come out from the factory have some defects.

[jbinpg]

And there is the assumption that the one K-5 body tested is representative of the whole. A comprehensive test would have tested more than one camera body. I realize this is an expensive option for the tester, but it would lead to a more solid conclusion.

Jack

[Edvinas]

Unfortunately this didn't force my two back focusing K-5s to start focus precisely

[goubejp]

Not a scientific test but I really enjoyed a couple of week ago the performance of the K5 with the DA*300; all shots in AF-C mode, these big birds flying quickly all around me; more than 80 % of the shots are perfectly in focus

[falconeye]

The intersection of the lens sets used for the K-5 and D7000 is
Sigma 10-20/3.5
Tokina/Pentax 12-24/4
Sigma 18-250/3.5-6.3

Would be interesting to see how the duel between the cameras ends up when looking at the results for these lenses only,

The good thing is they publish a large table of individual results. So, here we go:

Lens	K-5 %	+/-%	D7000 %	+/- %	7D %	+/- %
10-20/3.5 10mm	97.7	2.2	91.3	13.5	99.5	0.4
10-20/3.5 15mm	93.3	6.7	72.8	5.7	86.5	15.4
10-20/3.5 20mm	85.4	10.6	78.3	9.3	98.4	0.6
12-24/4 12mm	98.1	1.4	93.4	7.2	37.2	27.0
12-24/4 18mm	90.2	7.9	88.2	5.2	52.0	10.8
12-24/4 24mm	98.0	1.6	90.6	8.1	56.2	4.2
18-250/3.5-6.3 18mm	80.3	21.6	91.8	9.0	61.8	46.6
18-250/3.5-6.3 135mm	97.5	2.6	95.4	13.1	67.3	25.7
18-250/3.5-6.3 250mm	94.2	2.0	96.8	2.7	91.7	6.5
Sigma 50/1.4	67.3	25.1			97.7	1.6
DA* 55/1.4	83.4	13.0
Sigma 85/1.4			77.1	10.5	73.7	24.3
Nikkor 85/1.4			70.1	35.9
Average over f/1.4 glass	75		74		86

I marked a result in red (worst) or green (best) if the corresponding result is statistically significant to be different enough.

The %-figure is the MTF10 spatial resolution achieved on average, compared to the best value from a manual focus bracketing which was defined to be 100%. The +/-% figure is the standard deviation between the 10 test shots. A large standard deviation means that the best shot (within 10) was well above average; but the worst was well below it. Therefore, a small standard deviation is a better result.

Moreover, I don't see Sigma lenses to perform worse than genuine lenses on Canikon bodies as far as test results go.

Overall, this detailed look as suggested by Class A confirms a slight but significant lead by the K-5 over the big two.

However, this doesn't seem to apply to fast glass wide open which seems to be a difficult task for all vendors. Within significant error margins, they all deliver only 70-80% resolution except for a single "perfect match" on the 7D.


Side comment ...
look at the 7D results for 10mm/3.4 vs. 12mm/4. One case is wonderful while the other is a big miss. No wonder some call end of life for the phase AF method. Technology should work more reliably IMHO.

[falconeye]

Not a scientific test but I really enjoyed a couple of week ago the performance of the K5 with the DA*300

Nice photos.

But if no crops, really not possible to judge focus from web-sized samples.

A web-sized photo like yours (1280px wide) only has 25% of max. resolution which is less than even the worst results published in the test this thread is all about.

You may be proud of your photos because it is from Pentax AF.C and birds in flight is a real challenge with any camera.
And you did well!
25% resolution actually already is a good result for this situation. But it is a topic somewhat unrelated to this thread.

[RonHendriks1966]

I see a lot of red with the 7D so that wasn't the best camera for the test and maybe an error to use.

Good for us, and good to see that the DA*55mm scores well at f1.4 because mine does to.

[RonHendriks1966]

They swapped the results of DA*55 and Sigma 50 between the table (which I reproduced) and the bar chart graphics. So, it remains an open question if the Sigma50 or DA*55 numbers apply to the DA*55.

O my .... Let's hope they didn't switch the 7D with the K-5

[goubejp]

Nice photos.

But if no crops, really not possible to judge focus from web-sized samples.

A web-sized photo like yours (1280px wide) only has 25% of max. resolution which is less than even the worst results published in the test this thread is all about.

You may be proud of your photos because it is from Pentax AF.C and birds in flight is a real challenge with any camera.
And you did well!
25% resolution actually already is a good result for this situation. But it is a topic somewhat unrelated to this thread.

Sorry; being an engineer, I understand your point; I wanted to compliment your post with real life examples; here are the 100 % posts (original pictures are already recomposed because, you know, not possible to adjust image composition with these fast moving birds); as the camera had to deal with a severe contra light situation there is noise added by adding additional lightning in post processing; there is also some blur induced by haze (hot air) and of course bird mouvement; but the focus is exactly where it should be; can you see the poor little mouse in the first picture ?

[philbaum]

As it happens, a friend of mine has had a lot of trouble getting accurate focusing with his D7000. If I understand him correctly, the D7000 has a bad habit of focusing on a point of higher contrast than the actual selected focal point.

On a more positive note, he's found that the D7000 is a pleasure to focus manually -- it has little indicators that indicate whether you're focusing in front of or behind the selected point. So he's refined his kit to a rather Pentax-like pair of 24mm and 105mm primes instead of the big VR zooms that he used to use.

Regarding the manual focusing, a friend with a Sony A900, FF, mentioned that he manually focused a picture we were both talking about. Further comment seemed to indicate that Sony had some sort of assist programming for manual focus. I had forgotten this comment till yours came along. I'm wondering if Pentax has fallen slightly behind in this sort of visual aid???

This is rare and good news for Pentax:

Pentax wins (sort of) a comparative AF test executed by German magazine ColorFoto, as reported in issue 09/2011 in hands now by subscribers to the magazine.

The cameras participating in the test have been:

• Phase AF (in order of respective accuracy, best is first):
  
  Pentax K-5
  Sony Alpha A55
  Canon 7D
  Nikon D7000
  .
• Contrast AF (in order of respective accuracy, best is first):
  
  Panasonic GH1
  Olympus

The headline reads:

The phase AF fraction surprises with an outsider being their number one: Pentax focusses most accurately, ahead of Sony and even Olympus (contrast AF).

.

Falk, much thanks for giving us an analysis of the test results - a real learning experience and not a small effort on your part!

Pentax takes it on the chin once in a while in tests such as these, so it doesn't bother my conscience in the least to see them win one big comparison test occasionally.

Its ironic that when we see Pentax come up with a good result, we see comments like: Pentax was lucky that they had a good body in the test. Well, of course that is fortunate, But when Nikon or Canon has a good result, we don't say anything comparable about them being lucky These kind of statements imply that Nikon and Canon never issue deficient bodies - and we know that isn't true. Before i bought my K5, i used to read the Nikon D7000 forum(dpreview). There were complaints by some D7000 owners of oil stains, low light focus, and customer service that were interchangeable between Nikon and Pentax owners.

The takeaway that i get out of this test is that within its market niche, Pentax can compete with the best of Canikon in the same niche.

Sony, for me, is one of the most interesting camera companies to watch. They seem to have the hot hand in sensor design with their exmor designs, their product designs are highly innovative, Nex line, like them or not, and they have the strategic sense to sell their sensors to all body manufacturers. Smart! No surprise they came in 2nd in the phase detect competition.

[falconeye]

the focus is exactly where it should be; can you see the poor little mouse in the first picture ?

Poor mouse indeed.
The 2nd shot has perfect focus indeed. Well done!!

[rawr]

Someone hide this thread from garyk

He's recently spent lots of money getting a 7D and very expensive Canon mount super-telephotos due to what he felt were the inadequacies of the K-5's AF.