Guide: DSLR Lens Resolution Testing at Home

Using freeware programs and charts

By beholder3 in Articles and Tips on Sep 21, 2017
Guide: DSLR Lens Resolution Testing at Home

Many enthusiasts spend a lot of money on camera gear and then want to know how good it is. While every experienced photographer can tell the optical qualities of a lens from fitting sample images there are ways beyond subjective quality evaluations. These are about testing lens sharpness and resolution using either an optical bench (which costs more than the average new car, so not really a good fit for "homegrown" testing), or shooting test charts and analyzing the results mathematically.

The article is about the latter: resolution testing at home using freely available test charts and software to analyze it.

First Software Option: LPH

The first option you have is to download the free, lightweight Windows software LPH.

Since downloading .exe files from the internet can pose a risk, feel free to run it through online virus scanners such as VirusTotal.

Test Chart

You also need to download the test chart which is a Siemens star and print it on standard printer paper (Letter or A4) and place it on some flat, robust surface.

How To

  1. Shoot the test target in even, bright light with the sensor plane fully parallel with the test chart. Shoot outside on an overcast day or use at least 2 flashes at 45 degrees with diffusers.
    1. Distance: ensure the Siemens star's height is less than 1/5 of the frame height. This is so that measuring the center of the star is not limited by printer resolution, but by camera resolution.
    2. Camera: use a tripod, mirror up, cable release or remote, and electronic first curtain if available
    3. Focus: manually using live view magnification and focus peaking
    4. Exposure: Shoot in M mode at ISO 100
    5. Image format: select JPEG with settings you typically use. You can only compare results to others when you used the exact same in camera JPEG settings.
      1. Why not RAW? Because there are too many converters out there with varying results and settings. We all here might have the same camera, so we can discuss out-of-camera results fairly.
      2. My own setup: JPEG L ***; natural; contrast +1; sharpening +2;
  2. Start LPH.exe and push the "Open" button to the left to load your JPG file.
  3. Use the mouse scroll wheel to magnify the image and single click into the center of the Siemens star
  4. Now you'll get both a MTF50 and a MTF10 value in line pairs per picture height (LPH)

What to Do Now

  1. You can shoot the target at varying positions in the frame to check how your lens resolves in the center versus toward the edges.
  2. You can vary the light level, aperture, ISO, or JPEG settings to see how this affects the results

Some Observations

  1. Differing light levels have major impact, even at the same ISO.
  2. Bright sunlight is clearly worse than diffuse light
  3. JPG compression settings (*,**,***) have nearly no impact
  4. Sharpening does have some impact but it is not huge. Sharpening at 0 versus +4 changed MTF10 by less than +10%.
  5. The three types of in-camera sharpening (normal, fine, extra-fine) produced similar results
  6. Contrast settings have minimal impact, below 5% between 0 to +4
  7. Using a RAW file developed in Lightroom showed some small impact of the camera profile (remember it does not only contain color information, but also contrast adjustments)
  8. Downsampling to fewer megapixels has an impact but not as much as you'd think
  9. Using a D FA 100 macro my best JPG result at F5.6 had an MTF50 of 1416 LP/PH

LPH Paid Version

The author offers a "pro" version which includes graph outputs and more convenient batch processing. It's nice to have but not required (€25).

If you want to use it, be sure to download the full installer version of GNUplot, which is not included in the LPH files. I had some issues with integration when using the zip version.

Second Software Option: MTF Mapper

You can download the software via SourceForge for free.  You have to run it as an administrator to avoid errors.

Test Charts

MTF mapper includes a command line tool to generate vector graphics test charts, which is very nice. The test charts will be created as "chart.svg" in the same directory that the command line .exe resides in, so be sure to open the shell window as an administrator.

For general testing of a lens the "grid" chart is recommended.

For testing focus position, the default perspective chart type is recommended. This is to be shot at 45 degrees angle and best for autofocus adjustment testing. You autofocus on the big black edge in the center and the MTF mapper output will show you if you focused correctly.

Here is a screenshot of the shell window that shows how you can create the grid test chart using "mtf-generate_test-chart -t grid":

How To

  1. Shoot the "grid" test target in good diffuse light using the exact same suggestions provided above for shooting the LPH test target.
    1. The chart should fill the frame this time around
    2. MTF mapper can work with both JPGs and RAWs
  2. Start mtf_mapper_gui.exe, which will give you the following window:
  3. Now click on File/Open in the menu and open your test shot file. Here take a look at the check boxes at the bottom, which define the output analysis you will get:
  4. The three default ones (annotated image, profile, grid) plus "Lens profile" are my suggestions to start with. In some cases, one or more of these check boxes can cause issues, so if you get an empty output, try unchecking the boxes one at a time.  For example, when I checked "focus position" I never got to any results. Checking "chart orientation" gives results but it tells me that I have the wrong chart.
  5. Upon loading the image the program automatically starts analysis with can take 10-30 seconds to complete. Once the calculations have completed you can see sub-entries below the filename on the right side of the window. You need to expand the list clicking on the grey triangle and then click on each entry to show it:
  6. By clicking on the "save all results" button on the lower right the software saves all result graphs as .png files in the same directory where the photo is.
  7. Comments:
    1. As MTF Mapper can use raw files it will base its analysis on the unsharpened raw data. So expect raw file results to be significantly worse than JPG/TIFF results if they had any soft of sharpening applied (not easy to avoid).
    2. The best test target for slanted edge is not any printed chart but an actual physical razor blade shot against bright light (sample 1, sample 2)

Result Chart Types

Please note the sample charts  shown here are based on an extremely sloppy handheld test shot of the grid test chart shown above.

Profile

The "profile" simply shows all edge measurements of the image as cy/px on the y-axis and with the pixel distance from the chart center towards the extreme corners as x-axis. For a Pentax K-1 with 7360x4912 pixels, the diagonal pixel count of any image is 8848 pixels (the square root of the sum of both dimensions squared). So the distance from center to corner is 8848/2 = 4424 pixels.

For me, this is the core graph to look at.

In the below sample you see a maximum sharpness of about 0.32 cy/px in the inner region (0.32 * 4912 = 1571 LP/PH.

Grid2D

The Grid2D takes the measurements and shows them as a colored distribution. Normally you would expect the center to be sharper than the corners. It shows two separate charts for meridional and sagittal edges.

Grid3D

This is the same basic concept as the Grid2D, just with a nicer 3D look.

Lens Profile

This here turns the data into a graph close to what manufacturer lens MTF charts look like. It also shows the spread of actual data as clouds.

Observations

  1. Using a JPG test shot which I have exposed more (EV compensation +2) to brighten the whites actually produced worse results / lower MTF (I expected the increased contrast to deliver better results.

Understanding MTF Values

The MTF10 value is best at representing a lenses resolving power for fine detail, while the MTF50 is a commonly used value for representing human perceived sharpness.

You can easily convert LPH results which are given as LP/PH  (line pairs per picture height) into C/P (cycles per pixel) by dividing the LP/PH by the pixel count of the sensor height (for the Pentax K-1 it is 4912 pixels).

Test Target Types

Between the two software you see they use different chart types. Both have advantages according to Imatest.

  1. Siemens star: Relatively insensitive to noise, moderate sensitivity to sharpening and noise reduction.
  2. Slanted edge: May give optimistic results in systems with strong sharpening and noise reduction (i.e. it can be fooled by signal processing, especially with high contrast edges. Gives inconsistent results in systems with extreme aliasing (strong energy above the Nyquist frequency), especially with small regions. Most sensitive to sharpening, especially for high contrast edges; less sensitive for low contrast edges. Least sensitive to software noise reduction.

Final Thoughts

The article can only be a teaser for all the stuff you can try to do with these two software options. And there are certainly millions of questions you could ask as well as many, many things which are not "truth" but rather opinion and will lead to good long discussions.

For most users all this measuring is not worth anything, they will want to shoot real photos. But some want to have a closer look and understand technical relationships. I assume they will find these options useful.

For the more casual user I think there are two benefits especially when looking at the more trivial LPH:

  1. You can quickly compare your lenses against each other as long as the test setup and conditions are the same
  2. Using out-of-camera JPGs with well documented settings you can compare your lens + camera combo results more objectively with other users around the world

The Big Caveat

  1. You can not compare your absolute figures to any test site's figures. They use different software and different setups.
  2. You can not compare results for a lens across different cameras as the measurement is always for the camera + lens combo, never the lens alone.

References

  1. Sample lens testing using MTF mapper
  2. Tips for shooting test targets
  3. More explanations on MTF
  4. How to get performance curves
  5. MTF mapper blog home

I invite further discussion on this topic in the lens forum.

Comments

 

 


facebook.com/PentaxForums PentaxForums @PentaxForums News | Reviews | Forum

Support Pentax Forums Donate to Pentax Forums Support Pentax Forums