Originally posted by falconeye As I said, to be taken serious outside the Pentax community, any test would have to be done with every vendor's best powerhouse AF lens. I.e. 70-200/2.8G for Nikon. Otherwise, the test will have no impact whatsoever on undecided buyers.
Imagine a Nikon forum doing a test D7100 vs. K-3 where K-3 is paired to a Tamron 55-200 lens. Nobody would care about the reasoning, but just dismiss the test because the Pentax was ill-paired.
Pair K-3 with the Sigma 70-200 if that's the fastest AF lens (is it true btw?). Maybe include the result of D7100/Sigma. But do not skip the crucial Nikon/Nikon reference test.
Wrt #keepers and K-3 vs. K-5.
As I said, we don't know nothing from Adam's test here. Because of K-3's higher burst, all the extra keepers could be from the slow changing end of the test which does not challenge the AF.C. Maybe, the K-3 just hesitated more at the fast end. The kind of statistics based on #keepers is meaningless. I tried to explain why higher up in the thread.
Our best guide would still be Adam's subjective impression if K-3 tracks better or not than K-5. And this is what he shared in the review.
The test you described would be of interest to compare auto focus of Camera A equipped with Lens B versus versus Camera X equipped with Lens Y; using this technique you could even compare auto focus speed of Camera A equipped with Len B versus Camera A equipped with Lens Y. Seeing how total AF performance changes with lenses may be of keen interest.
I am interested in how good the K-3 auto focus system has improved. The only way to do this is to eliminate the variables, such as different lenses, so that you can conclude how the auto focus of one camera compares to another.
I agree that if I were looking to evaluate cameras equipped with a lens, lets say 500mm for action photography then I would be interested in various cameras being tested with 500 mm lenses so I can make a comparison of that combination.
Many Pentax users who read the forum have interest in knowing how the new K-3 auto focus performs in comparison to other Pentax cameras and to competitive cameras. To make those assessments the lens has to be the same. If you also want to compare the Pentax K-3 auto focus performance to the auto focus performance of say a Nikon or Canon camera, then the lens would also need to be the same.
For people looking to buy a new system to shoot 300mm, 400mm or 500mm with the premium on auto focus performance for a particular lens, then the test you propose makes sense. It may be that for 500mm a Canon solution provides better auto focus performance than a Nikon solution while at 400mm the roles are reversed. Your test would be useful to make that evaluation.
However with that same test, where cameras use different lenses of the same focal length, the person doing the test or interpreting the results would not be able to say the Canon is better than the Nikon or vica versa because ... a common lens was not used.
My personal interest is knowing how the auto focus of the K-3 improved relative to other Pentax cameras, which the Forum review addresses with keeper rate versus total keepers. A secondary interest is to see how the Pentax K-3 measures up to competitive cameras from Nikon and Canon and perhaps how the K-3 stacks up against higher performance and higher priced solutions, such as the Nikon D610. To compare the camera performance you need to eliminate the lens variably or you are back to comparing Camera A with Lens B versus Camera C with Lens D versus Camera X with Lens Y.
So perhaps there are just people with different interests. My interest is how the K-3 camera auto focus performs. I acknowledge that the current Pentax lens line up from Pentax and third parties has room for improvement in pure speed of focus and other areas.