Precision: the degree of deviation over multiple samples/attempts. Focus can be precise without being accurate. Precision may be evaluated using statistical tools.
Accuracy: whether results overall conform to a standard or known value. Focus can be accurate without being precise. That being said, determining accuracy when precision is poor is an exercise in futility.
Acceptable: whether results overall meet expectations within the limits of capture medium, lens performance, and user skill. Experience fielding questions on this site has taught me that acceptable focus performance is unattainable by some users, even with the best hardware.
Real world performance is strongly influenced by intended point of focus, the nature of the subject, and the quality/quantity of light. This is true regardless of focus system. Within the full range of available options (both manual and AF) here are a few points to consider:
- All systems are severely challenged by receding, curved, or complex (high relief) surfaces
- All focus systems are hampered by low contrast, low light, and lack of target detail. Any of the those factors may result in inability to consistently detect the out-of-focus condition resulting in poor precision, hunting, missed focus, and/or inability to attain focus.
- Best all-round performance (speed, precision, and accuracy) likely goes to manual focus wide-baseline optical rangefinder systems (e.g. Leica M-series bodies) (I said "likely" because I am unaware of any comparison studies.)
- Calibration of an imprecise system is a waste of time (scatter is greater than the range available for calibration)*
- CDAF offers potential for absolute accuracy with precision limited by implementation, subject, light quality/quantity, and hardware. The size of the focus area being evaluated as well as the algorithms used may limit the ability to do critical or selective focus and increase the potential for missed focus with some subjects. The linked article in the original post illustrates these points nicely.
- CDAF is relatively slow depending on implementation, lens match to system, and what it is being compared to
- PDAF accuracy is potentially high, but may be hampered by implementation, system optics, and hardware
- PDAF precision has the potential to be abysmal** resulting in inconsistent accuracy
- PDAF can be very fast
- PDAF precision may be limited by lens maximum aperture with no system able to detect the out-of-focus condition at better than if an f/2.8 lens is mounted
- Manual focus using magnified output from the image sensor (either Live View or EVF) is the gold standard for digital camera accuracy/precision. Accuracy is absolute with precision for a given subject and lighting limited only by sensor resolution and mechanical precision of the focus mechanism.
- Manual focus using focus confirm or catch-in-focus is no more accurate or precise than the PDAF it uses
- Manual focus using the stock focus screens on most (all?) current model dSLRs has accuracy and precision only slightly little better than PDAF on consumer-level bodies.
- Manual focus using screens with split-image focus aides and/or optimized matte field rival the precision/accuracy of optical rangefinder systems
Rant over...returning to the original post...
I just read the linked article in detail and suggest that any extrapolation beyond the author's description and discussion would be risky. The setup was intended to characterize observations from an earlier session and is a lead-in to additional work to follow. It appears he found a fairly specific issue with the Fuji spot-mode CDAF algorithm. The Fuji engineers will be thrilled, I am sure
BTW...he was not measuring CDAF precision
per se.
Steve
* Most of the questions on this site regarding PDAF calibration problems (e.g. "No calibration works") are related to poor AF precision for the lens and focus setup. The user makes an adjustment based on a single test image and experiences a rash of missed focus the next time out shooting. A series of shots of the slant scale often shows scattered points-of-focus on both sides of some center, often with "0" as the center. Even an accurate AF fine calibration value will not improve ratio of missed focus shots. The fault is usually traceable to the lens and may include optical issues, mechanical/structural deficiency, and/or a failure in the control loop. Some focus calibration tools assess precision as part of the process. The LensAlign FocusTune software does just that. A series of photos is required and if precision is poor the series is rejected.
** I know that is a strong statement, but some combinations of subject, light, and lens present an image to the detector with no unambiguous points for comparison (the PDAF sensor is essentially a split-image rangefinder). The result is that the detector will either fail to assign focus or will do so with low precision between attempts.