Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
11-12-2010, 07:50 PM   #16
Senior Moderator
Loyal Site Supporter
Parallax's Avatar

Join Date: Apr 2008
Location: South Dakota
Photos: Gallery
Posts: 19,324
QuoteOriginally posted by falconeye Quote
The new forum layout has no perfect place for such threads but I want it here...

.......................
Well, this seems like as good of a place as any, so we'll leave it here for now.


Last edited by Parallax; 11-13-2010 at 11:28 AM.
11-14-2010, 07:28 AM   #17
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
Original Poster
All cameras AF.C

I couldn't resist.
I know it isn't very reasonable. But by collecting that many data, I could nonetheless describe a reliable trend of AF.C performance of various cameras.

I took the Labo FNAC data from 2008, 2009, 2010 and 2011.

Then I decided that the nearest distance where a sharp photo was still reported is the best single number measure of performance.

Then I noted a trend to favour low Megapixel. As it turned out, this trend was by pixel pitch rather Megapixel or Sqrt(MP), i.e., full frame cameras were not in disadvantage by the higher MP.

So, I defined an AF.C performance figure as P1 = 100 / nearest distance [m] / pixel pitch [µm].

I plot P vs. MP so everybody can apply his own visual compensation for the effect. Not that at the same MP number, dividing by pitch disadvantages full frame and you may not like that.

So, I attached a 2ns chart w/o the pitch compensation:

P2 = 20 m / nearest distance

Moreover, I spotted a few errors where FNAC reversed the scale (nearest and farest distance) when transfering results to 2009. In one case, they spotted the error 2010. Eventually, I dropped the 50D and D300 results as they have been unplausibly good. The next one to drop would be the 1000D result (which BTW is the reversed and later corrected result).

The resulting chart is as shown in the appendic.


Discussion:

There is a certain cluster formation which I believe is more significant than any single result.

Except 7D and 550D, all "good" cameras sit in the 2-3 band, more or less.

The D3s / D300s are reported to outperform a 7D in the field, so the Canons may be particularly well adapt to this synthetic uniform motion benchmark which is trivial to predict. Assuming a very strong predictor, the red Canon squares should all be lowered a bit for field relevance.

Otherwise, the K-5 is the first Pentax entry in the "band of interest" (it sits on one spot with D7000 and 1DmkIV). That's great news! And the other green spots at the bottom of the chart are no fun

Moreover, the test is a strong indicator that K-x and K-r (same spot in the chart) share the same AF module, despite Pentax calling it Safox IX now.

Also quite astonishing is that µFT starts to be able to compete; look at GH and EPL1.

And now the chart, enjoy
Attached Images
   

Last edited by falconeye; 11-14-2010 at 07:53 AM.
11-14-2010, 08:02 AM   #18
Site Supporter
Site Supporter
ManuH's Avatar

Join Date: Dec 2007
Location: Montreal
Photos: Gallery | Albums
Posts: 1,249
Interesting but having used the K20D and the K-7 I cannot see how the K20D could be better. There's no contest as the K-7 having the fastest AF (-S or -C).
11-14-2010, 08:31 AM   #19
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
Original Poster
QuoteOriginally posted by ManuH Quote
Interesting but having used the K20D and the K-7 I cannot see how the K20D could be better. There's no contest as the K-7 having the fastest AF (-S or -C).
This is why I made the chart. I gives an overall impression and allows to interpret individual results.

They don't run enough tests and don't control parameters well enough. So, there is at least +/-0.5 variation in the individual results when plotted the way I did.

But by being able to average over several similiar models, an overall image still emerges which I believe is pretty accurate. So, from this context information, we can probably correct the K-7 value to be 1.3. And the K-7 would have scored 1.3 indeed if the last two images wouldn't have been classified fuzzy where they have been sharp for the K-x at the same distances (the K20D actually had no images at these distances but the last sharp one was nearer). Bad luck and FNAC makes the big mistake not to run multiple tests and not to compute sharpness numerically. So, individual scores may vary but I still think the overall picture is a good indication.

For instance, you see 3 clusters in Nikon land: D60/D3000; D90/D3100/D5000/D7000; D300/D300s/D700/D3/D3s


Last edited by falconeye; 11-14-2010 at 08:50 AM.
11-15-2010, 02:31 AM   #20
Senior Member




Join Date: Feb 2008
Location: Italy -> Canada -> Belgium -> Switzerland
Photos: Gallery
Posts: 233
QuoteOriginally posted by ManuH Quote
Interesting but having used the K20D and the K-7 I cannot see how the K20D could be better. There's no contest as the K-7 having the fastest AF (-S or -C).
Using the nearest distance as the sole performance indicator is questionable. FPS should be accounted as it may skew the result of the nearest distance.

Following FNAC approach, a camera that shoots 5 frames in the 50m range with the last not blurred picture being at 10m *to me* is worse than a camera that shoots 20 frames with the last not blurred one being at 12m.
11-15-2010, 03:44 AM   #21
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
Original Poster
QuoteOriginally posted by soalle Quote
Using the nearest distance as the sole performance indicator is questionable. FPS should be accounted as it may skew the result of the nearest distance.

Following FNAC approach, a camera that shoots 5 frames in the 50m range with the last not blurred picture being at 10m *to me* is worse than a camera that shoots 20 frames with the last not blurred one being at 12m.
We are entering difficult terrain here.

What you say represents a majority point of view. However, I think I found out that the usual good vs. bad metrics is rather useless. As is the total #good metrics. I ran into the same trap when doing my own AF test for the K-7.

The #shots metrics is already covered by the fps figure (the tested rather the spec sheet figure). What I was talking is a metrics which measures the AF performance.

Btw, faster fps is favoured in my metrics too: Because the slower system may simply skip the nearest good point and score worse than necessary.

In your example, the 5 frames camera would have the better AF module indeed, except that statistical errors could still reverse the result. The problem of FNAC numbers. Not exact enough.

I see how you could prefer the 20 shot camera. But because of its faster burst, not better AF.



One last point.
In the back of my mind, I have a model of how AF behaves. There, AF works flawlessly if a subject moves slow enough. For the camera, a slow movement is a slow variation of the subject distance/focal distance which is nothing but a slow variation of the magnification. And the AF struggles if the magnification variation becomes large enough. Therefore, the maximum magnification variation speed where the AF can still maintain focus characterizes its overall perfomance. More than anything else. Moreover, I assume that the breakdown point is rather abrupt and well defined for a given set of testing parameters which includes the lens used and contrast.
Now, it turns out that the inverse nearest sharp distance is the same as this max. possible magnification variation speed, provided the 35mm equivalent focal length is constant. And it is. That's why my chart is the way it is. For a comparison of AF between various lenses but same body, one would have to normalize speed and/or distance though. Max. magnification change speed is the measure and it is in a unit like %/s.

BTW, I believe that Canon characterizes their AF module perfomance internally by some nearest distance measure too.

Last edited by falconeye; 11-15-2010 at 03:49 AM.
11-15-2010, 04:21 AM   #22
Veteran Member




Join Date: Jan 2009
Location: Atlanta, GA
Photos: Gallery
Posts: 317
Well, because the sensor is sooooo good in low light and has a crazy DR, Pentax probably didn't prioritize an improvement in SR since most users will now have the option to just turn the ISO up.

11-15-2010, 06:11 AM   #23
Site Supporter
Site Supporter




Join Date: Jan 2008
Location: SE Michigan USA
Photos: Gallery
Posts: 1,300
QuoteOriginally posted by falconeye Quote

As posted on DPR, FNAC in France has published test results for the K-5 including an AF.C test...


Do firmware updates 'move the cheese'?

Seems kinda silly to wade so deeply into interpretation, commentary and analysis until the firmware dust settles, don't you think?

Cheers...
11-15-2010, 08:07 AM   #24
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
Original Poster
QuoteOriginally posted by Michaelina2 Quote

Seems kinda silly to wade so deeply into interpretation, commentary and analysis until the firmware dust settles, don't you think?
Hi Michaelina,

it may be silly to look that deeply into any detail, you're right

However, FNAC seems to test all cameras when they are brand new so it's fair enough to compare them at v1.0 firmware level. Moreover, I don't think that the firmware can change much for the distance where the AF.C mechanism starts to fail. It could have helped the K-7 though. The K-5 result looks pretty regular though. And the announced update is for buffer efficiency and stability. No mention of AF.
11-15-2010, 12:19 PM   #25
Site Supporter
Site Supporter
jpzk's Avatar

Join Date: Jan 2009
Location: Québec
Photos: Gallery | Albums
Posts: 8,251
Falk,

This is all good (to you and a very few others) but would you care to give a short explanation, in English terms (()) for those like me not so well versed in mathematical and statistical analyses?

What does this mean in terms of the K5/K7 AF.C capabilities ?

A short explanation would be nice.

Thanks.

JP
11-15-2010, 05:40 PM   #26
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
Original Poster
QuoteOriginally posted by jpzk Quote
What does this mean in terms of the K5/K7 AF.C capabilities ?
JP, I'll give it a try

I assume that AF.C can be characterized by its ability to cope with a change of subject distance. I think that's pretty evident. With a static subject, any AF.C will perform great

Now, assume a 50 km/h vehicle at 1 mile distance: almost static. Next, assume the same vehicle at 10 m distance: dramatic action!

So, counting the number of good images means nothing. As they may all be from the static phase. The moment where the AF.C stops being able to follow the action means everything. And this moment is defined by the nearest sharp image, given that speed, contrast, field of view are all constant.

If you want a measure of the speed which the AF.C can capture, then just take the inverse, like 20m/nearest distance, and you get a relative capturable speed.

What does it mean in practical term?

Well, if I read the graph correctly, in the past Pentax could deal with a speed of "1" where the best managed to deal with "3", three times as fast. Now, it seems the K-5 closed the gap and is in a good average position, like "2". Twice as fast an action as with previous Pentax and in one league with D7000. The best are still 50% faster (at least).

In practise, an AF.C either follows the action or not. So, in one event the difference may be insignificant where in another, the difference is between win and fail. It really depends on what you shoot. The K-5 can just cope with more situations. And there are still situations no camera of 2010 can deal with.

Moreover, for fast action and long tele, the ability to track the subject (follow a subject with its active focus point) is important too. An ability Pentax still lacks. FNAC did not look into 3D tracking though.
11-16-2010, 03:55 AM   #27
Senior Member




Join Date: Feb 2008
Location: Italy -> Canada -> Belgium -> Switzerland
Photos: Gallery
Posts: 233
QuoteOriginally posted by falconeye Quote
What you say represents a majority point of view. However, I think I found out that the usual good vs. bad metrics is rather useless. As is the total #good metrics. I ran into the same trap when doing my own AF test for the K-7.

The #shots metrics is already covered by the fps figure (the tested rather the spec sheet figure). What I was talking is a metrics which measures the AF performance.
Probably I did not explain myself correctly.

What I meant is that FPS may skew (in a positive or negative way) the results of the nearest in-focus distance.

Suppose an oracle tells you that two cameras have a nearest in-focus distance of 10m - *same AF module*

Now camera A does a shot every 5m whereas camera B does a shot every 3m.

Camera A will be able to shoot at 50,45,40,35,30,25,20,15,10 (and 5m out-of-focus). Result: nearest in-focus distance of 10m.

Camera B will be able to shoot at 50,...,20,17,14,11 (and 8m out-of-focus). Result: nearest in-focus distance of 11m.

So, even if we focus solely on the AF performances, different FPS may give different results in the nearest in-focus distance.

QuoteOriginally posted by falconeye Quote
Btw, faster fps is favoured in my metrics too: Because the slower system may simply skip the nearest good point and score worse than necessary.
Not true as explained above. It *may* be favored, it *may not* be favored
11-16-2010, 04:43 AM   #28
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
Original Poster
soalle, I see what you mean now.

My reply is still applicable though. What you describe is an alias effect on testing results. Assume, you would run many tests and start the trigger with some random delay device and then take the average over results. Then and I think you agree, one would get a result free of any bias.

Unfortunately, FNAC doesn't care to run many tests, introduce a random delay or measure actual blur widths rather than green/orange/red. So, my chart averaging over many similiar cameras must substitute this.

Back to a single test result: you get a number with an alias error. The alias error is zero for the lucky case where the last sharp photo happens at the very last moment. And it is maximum if it happens just a tiny bit later. Because then, the 2nd last photo counts. The error is always positive, i.e., the performance is always underestimated. And the average error is half the distance travelled between two shots at the given fps. So, faster fps results in less underestimation of true performance. This is what I meant by "faster fps is favoured" in my previous reply.

Nonetheless, I don't think it renders my chart invalid. One must just be careful when interpreting any single data point. It may be offset significantly. But the overall pattern should still apply.

The chart should be viewed by colored clusters, not single points. And then, it may be the best if not only such chart availabe on the internet. OTOH, there still is the effect of lens selection which I think is more serious. Using the FA* 80-200/2.8 is an interesting choice. Is it the lens giving the best possible performance for Pentax? Or just some odd accident?

Last edited by falconeye; 11-16-2010 at 04:55 AM.
11-16-2010, 04:58 AM   #29
Senior Member




Join Date: Feb 2008
Location: Italy -> Canada -> Belgium -> Switzerland
Photos: Gallery
Posts: 233
Agreed.

Since we are stucked with single tests, maybe a performance indicator considering nearest in-focus distance and 2nd nearest in-focus distance would decrease further the bias.
11-16-2010, 05:01 AM   #30
Senior Member




Join Date: Feb 2008
Location: Italy -> Canada -> Belgium -> Switzerland
Photos: Gallery
Posts: 233
Btw, what struck me the most is the same performance for K-r and K-x... very strange since they should have 2 different AF modules...
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
af.c, bodies, body, camera, dslr, k-5, k-5 ii, k-5 iis, k5, pentax, pentax k-5, test
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Tamron (esp. adaptall) 90mm owners? (CCD flare) kxr4trids Pentax SLR Lens Discussion 28 03-29-2011 02:09 PM
Pentax video samples? (Esp. ~ 300mm and SR) emr Video Recording and Processing 5 02-28-2010 05:21 AM
K-7: tethered shooting, esp. video? virgilr Pentax DSLR Discussion 0 06-12-2009 03:03 PM
Kodak ESP-5 Printer Ed in GA Digital Processing, Software, and Printing 6 01-04-2009 04:15 PM
Photo sharing/selling options (esp. Canada friendly) David Whiteley General Talk 5 10-02-2008 02:56 PM



All times are GMT -7. The time now is 09:28 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top