Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
07-08-2016, 10:35 AM - 7 Likes   #1
Pentaxian




Join Date: Dec 2011
Posts: 1,340
Results from a really repeatable AF.C test (no Pentax inlcuded) as reference

The German photo magazine "C't Digitale Fotografie" in issue 02/2016 had the only autofocus test procedure I have ever (!) seen which is repeatable, well documented and to be taken seriously.
Quite a contrast to the clown show at dpreview.

I just provide the results here to provide some facts versus the fanboy claims of "brand x nails every shot" level.

The magazine spent two full pages to document their test setup and methodology and measuring, which I am far to lazy to repeat here.
The shot about 3,900 images in total!

Setup in short:
  • They set up a white beamer canvas against which a beamer projected images of a small flash program which displayed either a swallow or a female face and these were moving at a predefined speed across the image.
  • The camera was sideways to the canvas and shot the image from a 30 degree angle (the subject images were electronically distorted to provide a normal look from 30 degrees).
3 Test scenarios for all cameras:
  1. Tracking
    The camera on a tripod was zoomed to cover the whole image area with AF points. A bird (black and white with extra contrast marker) and also a face (much softer contrasts) image flew across the canvas from edge to edge (remember the edges are in different distances). The lens was manually set to middle distance at start.
  2. Follow focus center AF point
    Same bird, but now the camera was set on a swivel panoramic head and a user turned the camera to keep the subject under the AF point. So the camera just had to adjust z-axis
  3. Focus with obstacle
    A little real life plant was set in the middle between the canvas and the camera. A face moved from left to right over the screen (thus was hidden briefly by the plant), while the camera was panning, following the face. Just the type of scenario you set your Pentax AF hold parameter to "high" for.
"Sharpness" evaluation criteria:
They did not even rely on human arbitrary judgementto call an image "sharp", they used an automated analysis of the structured canvas and beamer signal creating moire where the focus area was sufficiently sharp (they looked at contrast curves. I can not really explain how they look, but it is extremely obvious where the focus was) . Their criteria were possible due to the fact that by using the angled canvas approach their target subject always showed some depth and it was not just a flat object which appeared sharp or not.

  • "sharp" = target subject fully in focus
  • "mediocre" at least half of the target subject in focus area (like the back wheel of a bicycle)
  • "unacceptable" = target out of focus


Results:







  1. Not one camera even came close to "nailed every shot". Let alone one brand.
    The authors declared Canon cameras to be in the lead by a small margin. They were completely disappointed from the Sony.
  2. Hugely lens dependant
  3. Practically nothing there where one can generalize from. Hugely dependent on exact camera/lens/scenario combination.
Anyone could redo the exact same test as it is well documented and the subjects are computer generated. You just need a flash program or a video of the target. I do consider the setup to be rather smart. You can redo that in your large garage.
Obviously the range difference obtainable is limited by the size of the canvas, so you can not simulate that fighter plane directly coming at you from infinity. But given the soso results this setup is interesting enough.

This is a test that comes close to scientific requirements and standards.


Last edited by beholder3; 07-08-2016 at 01:19 PM.
07-08-2016, 11:15 AM   #2
Loyal Site Supporter




Join Date: Aug 2011
Photos: Albums
Posts: 1,861
Thanks for sharing!
I recently helped a Nikon shooter fine adjusting her D800 and her lenses. She had her camera for quite a while and never thought of AF error, but many of her photos were out of focus.
Long story short, it turns out many of her lenses need adjustment, just like my lenses on K5. And during the testing and adjustment, I also noticed AF inconsistency that bothers me a lot on K5 and K5II. some lenses could focus spot on 3 times in 5 tests, but other 2 shoots were off. I guess many Pentaxian have experienced such thing. And now I can tell you, at least on this D800, things are not much better.
I think many pentaxians on this forum are sensitive or knowledgeable enough to notice the problem, and many Canikon users are not, like the lady I helped. She started with a super zoom on her D800 and never found serious AF problem (of course). But after she bought in more faster lenses, she complained better lenses gave her worse result, but never thought it was camera/lens issue - I guess partially because of the reputation of "nailed every shot".
07-08-2016, 11:34 AM - 1 Like   #3
Site Supporter
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 27,415
Thanks for the very interesting post. The test methodology appears to be sound, but it is the results that blew me away. What I see in the tables reflects field experience and discussions I have had with various photographers over the years. Nobody is particularly happy with their AF and some (mostly Sony folk) simply shrug and say, "why bother".

I suspect that much of the emphasis on frame rate and buffer size is due to the need to "spray 'n pray" as remedy for poor AF performance.


Steve
07-08-2016, 11:59 AM   #4
Site Supporter




Join Date: Nov 2013
Photos: Gallery
Posts: 156
What this tells me: AF is a vastly complicated area with a combination of Camera (hardware and software), Lens (hardware and software) decisions, tradeoffs, and bugs that all go in to the melting pot of perceived AF accuracy. There are a near infinite number of permutations of scenarios. The DPReview people picked on a particular scenario that they *knew* would foil Pentax AF and based their whole eval on that one scenario. Interestingly their IQ testing is far more rigorous and hence much more acceptable. Why they employ such rigor in one aspect and such half-assedness in another is beyond me. They also pretty heavily weighted their final conclusion on the one disappointing aspect.

Frankly, in the end I don't care. Haters gonna hate. I'm very happy with my K-1. I shoot both landscape and sports (horse shows). In landscape, the K-1 with pixel shift is phenomenal. My D800 toting friends are actually jealous (though the 800/810 is still a very fine camera in its own even though its going on 4 years old). In sports, the K-1 AF + 150-450 lens is giving me higher hit rates than ever before (I would guesstimate in the 80%-90% range) in some difficult situations. I can actually shoot with AF.C now (9 spot center focus, not single spot), where I never could before. Is it perfect? No. Neither are the 810, 7rii, 6D, etc.

I do feel bad for Pentax that DPReview gave them a slightly tarnished silver for what is beyond doubt the best value in full frame on the market. I sure hope it doesn't significantly affect their sales. I find much to commend in the review but the AF portion is completely amateurish. I don't care if they want to bitch about menus being too deep or other qualitative nitpicks. they are entitled to voice opinion on that. But AF is quantifiable and if they're going to be judging it then they should come up with a better way to *actually* quantify it. Thanks for the link to the AF study above. Too bad it doesn't include Pentax. Hopefully they do an update, this is the kind of more measured testing that I can appreciate.

07-08-2016, 12:38 PM   #5
Loyal Site Supporter
monochrome's Avatar

Join Date: Mar 2008
Location: Kirkwood (St. Louis) MO
Photos: Gallery | Albums
Posts: 20,423
The only good news is Pentax sales in the USA are so abysmally low that this kerfluffle won't even register a blip on global success of K-1 for Ricoh.
07-08-2016, 12:43 PM   #6
Pentaxian




Join Date: Dec 2011
Posts: 1,340
Original Poster
I just added some sentences on how sharpness was assessed.
07-08-2016, 12:52 PM   #7
Pentaxian




Join Date: Jul 2013
Posts: 1,457
Wow ! This chart Blew Me Away ! What a AF Novice like myself sees is ALL cameras pretty much Suck in autofocus quite a bit....thanks for sharing.
07-08-2016, 02:00 PM   #8
Loyal Site Supporter




Join Date: Mar 2012
Location: Rochester, NY
Photos: Gallery
Posts: 2,178
QuoteOriginally posted by Dlanor Sekao Quote
Wow ! This chart Blew Me Away ! What a AF Novice like myself sees is ALL cameras pretty much Suck in autofocus quite a bit....thanks for sharing.
That is why an old fart like me still wants optical focusing aids on the focus screen.

07-08-2016, 02:21 PM   #9
Loyal Site Supporter




Join Date: Apr 2015
Photos: Albums
Posts: 514
Crikey. That's almost as bad as me with my old film cameras. Doh.

The lens dependence is huge, though. While the K-1 itself is a good value, add in the full frame lenses and maybe not so much, especially as we see here for something like sports that wouldn't exactly help in AF performance. But as others have noted, the new mount and new lenses may make a big difference.

Thanks for the info; interesting.
07-08-2016, 11:56 PM   #10
Pentaxian




Join Date: Dec 2011
Posts: 1,340
Original Poster
This well done test does serve well as a reliable credibility measure when you read some posters claims.

Probably little competence and credibility in other posts as well if you see an author who once claimed fanboy things like "nails every shot" or "not a single frame out of focus ever".
07-09-2016, 06:31 AM   #11
Pentaxian
wizofoz's Avatar

Join Date: Apr 2009
Location: Melbourne, Outer east.
Photos: Gallery | Albums
Posts: 1,655
Thats an impressive test, unlike the DPR random biker.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
af.c autofocus test, camera, canvas, dslr, focus, full frame, full-frame, image, k-1, k1, pentax k-1, results, setup, test
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Another K-3 AF-C test crewl1 Pentax K-3 17 12-16-2014 05:12 PM
Shots taken my Pentax 645 and advice needed as results are not as expected spotreunion Pentax Medium Format 14 10-15-2013 04:20 PM
K-5 user AF (FF/BF) test results jolepp Pentax K-5 80 03-14-2011 04:10 AM
Pentax K-x: focus confirmation in AF-S, AF-C, not AF-A manteiv Pentax DSLR Discussion 24 12-10-2010 07:25 PM
Streets Pentax 6x7 as a street camera - some first results Rense Post Your Photos! 13 06-10-2010 01:19 AM



All times are GMT -7. The time now is 01:03 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top