Roger Cicala: why I don't use an MTF bench to test my own lenses

swanlefitte · 10-26-2020, 01:24 PM - **2 Likes**

I hate to say Roger is at DPReview. I heard about this a few days ago but could'nt figure out why I couldn't find it at Lensrental's blog. I figured it was an old article from 2010 or so. No he has 2 articles over there and Roger is worth listening to.
Roger Cicala: why I don't use an MTF bench to test my own lenses: Digital Photography Review

pschlute · 10-26-2020, 02:21 PM - **9 Likes**

Good for him. I have been taking photographs for over 40 years.

I am firmly of the opinion that in the hands of a good photographer who knows about : light;shapes;patterns; and knows what image he wants to present.........

a broken bottom of a milk bottle will be sufficient glued to the front of a camera.

The lens test junkies are the photographic equivalent of the rich people that know the price of everything and the value of nothing.

**jatrax** · 10-26-2020, 02:54 PM - **4 Likes**

Originally posted by pschlute

a broken bottom of a milk bottle will be sufficient glued to the front of a camera.

I tried that. It didn't work. Otherwise I generally agree.

photoptimist · 10-26-2020, 03:18 PM - **3 Likes**

I'm of two minds on this issue.

On the one hand, most lenses really can successfully take most pictures. Given a new lens, a few ordinary test shots suffice to prove if the lens is decent or awful. Precision lab testing really isn't needed to reveal ugly decentering, poor contrast, nasty flare, or strong chromatic aberrations. Lab test perfection simply isn't needed for most photography.

On the other hand, few lenses can successfully take some select categories of pictures (astro, high-detail landscape, archival photography, intricate macro, low-light photography, etc.) that demand low field curvature and corner-to-corner sharpness often while wide open. I find that informal tests seldom provide a good picture (;-)) of just how good a picture a lens can make when shot wide-open on a high-detail subject. For example, I'm shocked by all the lenses I have that are great day-to-day but suck for astrophotography. For those seeking the quixotic pleasures of pixel-peeping, the lab tests ensure all lenses get tested fairly and to their limits of performance.

Thus, Roger Cicala's lab tests have their place in the photographic firmament even if they aren't needed in most cases.

stevebrot · 10-26-2020, 04:18 PM - **1 Like**

Originally posted by swanlefitte

I hate to say Roger is at DPReview.

He has been a sometime contributor there for some time, but now apparently as an unpaid* opinion contributor on a more regular basis. I still trust his analysis. FWIW, he has covered the MTF and utility of testing matter in the past on the lensrentals blog, but not before they got their new state of the art optical bench.

RCicala: Member profile: Digital Photography Review

Steve

* Sort of like some of us that sometimes write for the PF home page.

automorphism · 10-26-2020, 05:50 PM

Note that Mr. Cicala here isn't saying people shouldn't rely on MTFs to make lens purchasing decisions. He is saying that he doesn't test his personal lenses using Imatest and other systems, to check to see if his copy is good or not.

**BigMackCam** · 10-26-2020, 11:54 PM - **2 Likes**

I don't mind who Roger chooses to write for, so long as his opinions remain his own and uninfluenced by his hosts.

Interestingly, for some time I've used photos of my lawn as a casual test for the lenses I buy. It is indeed a quick and reliable way to get some idea of sharpness, de-centering and field curvature (at short distances), just as Roger points out. I particularly like his use of the "find edges" feature in determining field curvature. I use GIMP, and I assume the "edge detect" feature should work in much the same way. I'll have to give that a try!

**BarryE** · 10-27-2020, 01:46 AM

Pragmatic sense. However, this pragmatism does have to be learnt, especially in these days. Newcomers, are probably unlikely to appreciate what he's saying, picking their first/second lenses based on reviews and charts. I know I did when I got back to digital. In Zenith/Canon AE1 days, I just bought what I could afford and got on with it.

I suppose chart tests can be useful as they can suggest how a lens might behave in the field. The DA* 16-50 f2.8, my first Pentax lens, is a complicated performer. The charts helped me understand it a little better.

**BigMackCam** · 10-27-2020, 02:10 AM

Originally posted by BarryE

Pragmatic sense. However, this pragmatism does have to be learnt, especially in these days. Newcomers, are probably unlikely to appreciate what he's saying, picking their first/second lenses based on reviews and charts. I know I did when I got back to digital. In Zenith/Canon AE1 days, I just bought what I could afford and got on with it.

There's a lot to be said for that approach... Buy the best lenses you can afford and get on with it. Test-bench results aside, there really aren't (m)any bad lenses these days. Yes, some perform better than others or have more pleasing rendering depending on individual preferences and use cases, but in most cases you have to view images up close at 100% reproduction to find fault (unless, of course, there's something seriously wrong with the lens). They're almost all capable of capturing excellent images if used proficiently...

**Rondec** · 10-27-2020, 02:56 AM

I don't know that you need to shoot test charts, but I do think that when you get a new lens you should probably shoot a brick wall or two just to be sure you didn't get a decentered copy. I'm not in search of perfection, but sometimes (particularly with wide angles) one side of the lens is just way softer than it should be. You might not notice it right away in routine testing and when deciding whether to keep a lens or not, that does seem important.

house · 10-27-2020, 05:02 AM

Originally posted by BarryE

Pragmatic sense. However, this pragmatism does have to be learnt, especially in these days. Newcomers, are probably unlikely to appreciate what he's saying, picking their first/second lenses based on reviews and charts. I know I did when I got back to digital. In Zenith/Canon AE1 days, I just bought what I could afford and got on with it.

I suppose chart tests can be useful as they can suggest how a lens might behave in the field. The DA* 16-50 f2.8, my first Pentax lens, is a complicated performer. The charts helped me understand it a little better.

Buying the most expensive lens you can afford is an expensive way of going about it. A cheap lens can have a better performance profile for what you shoot than a pricey one.

Generally speaking you want the bench test *before* you purchase. Afterwards its too late but for catching manufacturing issues. Reviews with numbers are a useful data point and helps you with what to look for in sample images. I've leanred about lens flaws uncomfortably late across the globe with no way of repeating the shot. Rogers testing regime catches a fair few issues and is quick enough.

**BigMackCam** · 10-27-2020, 05:39 AM - **1 Like**

Originally posted by house

Buying the most expensive lens you can afford is an expensive way of going about it. A cheap lens can have a better performance profile for what you shoot than a pricey one.

This is why I referred to buying the best (rather than the most expensive) lenses you can afford. As you rightly infer, the most expensive lens doesn't necessarily translate to better performance depending on what you shoot and how you shoot it. For example, there's no point buying an expensive constant aperture f/2.8 zoom if you're shooting primarily landscapes at f/8 - f/16 and can get similar performance from a good quality variable aperture lens covering similar fields of view...

house · 10-27-2020, 06:17 AM

The drift in this discussion is that Rogers method is for testing faulty lenses and learning details about the lens performance after purchase.

Reviews including ones with mtf charts are useful to decide which lens to buy when you have no opportunity to try them out.

**BigMackCam** · 10-27-2020, 06:43 AM

Originally posted by house

Reviews including ones with mtf charts are useful to decide which lens to buy when you have no opportunity to try them out.

I agree

... and so, by inference, does Roger:

Quote:

"It’s not because the lab stuff doesn’t give useful information. The lab gives a LOT of useful information. Most people don’t have time to learn how to interpret it, or learn its value and limitations, but it’s useful information nonetheless. And the lab is fast; I can test a lens about 32 different ways in a couple of hours. My ‘test a lens with photography’ time is a half a day or more. So the lab is faster, gives tons of information, and makes cool graphs. But I still don't use it to test my personal lenses."

photoptimist · 10-27-2020, 08:33 AM - **2 Likes**

Originally posted by BigMackCam

This is why I referred to buying the best (rather than the most expensive) lenses you can afford. As you rightly infer, the most expensive lens doesn't necessarily translate to better performance depending on what you shoot and how you shoot it. For example, there's no point buying an expensive constant aperture f/2.8 zoom if you're shooting primarily landscapes at f/8 - f/16 and can get similar performance from a good quality variable aperture lens covering similar fields of view...

Exactly!

And sometimes the definition of "best" takes one in directions far removed from optical test bench definitions.

For example, superzooms look like horror shows on the optical bench with all manner of distortions, aberrations, vignetting, and flare. But a superzoom may be the "best" lens for whirlwind tour-group travel that does not allow any time to switch lenses.

Sometimes the best lens has to be the one on camera.

10-26-2020, 01:24 PM - 2 Likes	#1
swanlefitte Pentaxian Join Date: May 2015 Location: Minneapolis Photos: Gallery Posts: 4,068	Roger Cicala: why I don't use an MTF bench to test my own lenses I hate to say Roger is at DPReview. I heard about this a few days ago but could'nt figure out why I couldn't find it at Lensrental's blog. I figured it was an old article from 2010 or so. No he has 2 articles over there and Roger is worth listening to. Roger Cicala: why I don't use an MTF bench to test my own lenses: Digital Photography Review

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Buck Roger Flash Blast Attack Ship in Space	SteveinSLC	Monthly Photo Contests	2	05-08-2020 09:41 AM
LensRentals: Painting Zoom Lenses with a Broad Brush – Roger’s Law of Wide Zoom Relat	interested_observer	Pentax SLR Lens Discussion	6	03-19-2017 12:37 AM
LensRentals' Roger Cicala shoots Pentax K-1	hyyz	Pentax K-1 & K-1 II	3	12-30-2016 03:08 PM
Great mirrorless camera posts by Roger Cicala	Raffwal	Canon, Nikon, Sony, and Other Camera Brands	5	08-26-2012 07:30 AM
Katzeye Optibrite - Why don't camera makers install their own versions as default?	Unsinkable II	Pentax DSLR Discussion	20	11-01-2010 09:17 PM

10-26-2020, 02:21 PM - 9 Likes	#2
pschlute Moderator Loyal Site Supporter Join Date: Mar 2007 Location: Surrey, UK Photos: Gallery Posts: 8,202	Good for him. I have been taking photographs for over 40 years. I am firmly of the opinion that in the hands of a good photographer who knows about : light;shapes;patterns; and knows what image he wants to present......... a broken bottom of a milk bottle will be sufficient glued to the front of a camera. The lens test junkies are the photographic equivalent of the rich people that know the price of everything and the value of nothing.

10-26-2020, 02:54 PM - 4 Likes	#3
jatrax Site Supporter Join Date: May 2010 Location: Washington Cascades Photos: Gallery \| Albums Posts: 12,991	Originally posted by pschlute a broken bottom of a milk bottle will be sufficient glued to the front of a camera. I tried that. It didn't work. Otherwise I generally agree.

10-26-2020, 03:18 PM - 3 Likes	#4
photoptimist Pentaxian Join Date: Jul 2016 Photos: Albums Posts: 5,123	I'm of two minds on this issue. On the one hand, most lenses really can successfully take most pictures. Given a new lens, a few ordinary test shots suffice to prove if the lens is decent or awful. Precision lab testing really isn't needed to reveal ugly decentering, poor contrast, nasty flare, or strong chromatic aberrations. Lab test perfection simply isn't needed for most photography. On the other hand, few lenses can successfully take some select categories of pictures (astro, high-detail landscape, archival photography, intricate macro, low-light photography, etc.) that demand low field curvature and corner-to-corner sharpness often while wide open. I find that informal tests seldom provide a good picture (;-)) of just how good a picture a lens can make when shot wide-open on a high-detail subject. For example, I'm shocked by all the lenses I have that are great day-to-day but suck for astrophotography. For those seeking the quixotic pleasures of pixel-peeping, the lab tests ensure all lenses get tested fairly and to their limits of performance. Thus, Roger Cicala's lab tests have their place in the photographic firmament even if they aren't needed in most cases.

10-26-2020, 04:18 PM - 1 Like	#5
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by swanlefitte I hate to say Roger is at DPReview. He has been a sometime contributor there for some time, but now apparently as an unpaid* opinion contributor on a more regular basis. I still trust his analysis. FWIW, he has covered the MTF and utility of testing matter in the past on the lensrentals blog, but not before they got their new state of the art optical bench. RCicala: Member profile: Digital Photography Review Steve * Sort of like some of us that sometimes write for the PF home page.

10-26-2020, 05:50 PM	#6
automorphism Veteran Member Join Date: Jun 2015 Location: Ontario, Canada Photos: Albums Posts: 791	Note that Mr. Cicala here isn't saying people shouldn't rely on MTFs to make lens purchasing decisions. He is saying that he doesn't test his personal lenses using Imatest and other systems, to check to see if his copy is good or not.

10-26-2020, 11:54 PM - 2 Likes	#7
BigMackCam Digitiser of Film Join Date: Mar 2010 Location: North East of England Posts: 20,673	I don't mind who Roger chooses to write for, so long as his opinions remain his own and uninfluenced by his hosts. Interestingly, for some time I've used photos of my lawn as a casual test for the lenses I buy. It is indeed a quick and reliable way to get some idea of sharpness, de-centering and field curvature (at short distances), just as Roger points out. I particularly like his use of the "find edges" feature in determining field curvature. I use GIMP, and I assume the "edge detect" feature should work in much the same way. I'll have to give that a try!

10-27-2020, 01:46 AM	#8
BarryE Site Supporter Join Date: Jan 2013 Location: Hampshire, UK Posts: 1,653	Pragmatic sense. However, this pragmatism does have to be learnt, especially in these days. Newcomers, are probably unlikely to appreciate what he's saying, picking their first/second lenses based on reviews and charts. I know I did when I got back to digital. In Zenith/Canon AE1 days, I just bought what I could afford and got on with it. I suppose chart tests can be useful as they can suggest how a lens might behave in the field. The DA* 16-50 f2.8, my first Pentax lens, is a complicated performer. The charts helped me understand it a little better.

10-27-2020, 02:10 AM	#9
BigMackCam Digitiser of Film Join Date: Mar 2010 Location: North East of England Posts: 20,673	Originally posted by BarryE Pragmatic sense. However, this pragmatism does have to be learnt, especially in these days. Newcomers, are probably unlikely to appreciate what he's saying, picking their first/second lenses based on reviews and charts. I know I did when I got back to digital. In Zenith/Canon AE1 days, I just bought what I could afford and got on with it. There's a lot to be said for that approach... Buy the best lenses you can afford and get on with it. Test-bench results aside, there really aren't (m)any bad lenses these days. Yes, some perform better than others or have more pleasing rendering depending on individual preferences and use cases, but in most cases you have to view images up close at 100% reproduction to find fault (unless, of course, there's something seriously wrong with the lens). They're almost all capable of capturing excellent images if used proficiently...

10-27-2020, 02:56 AM	#10
Rondec Loyal Site Supporter Join Date: Mar 2009 Location: Gladys, Virginia Photos: Gallery Posts: 27,653	I don't know that you need to shoot test charts, but I do think that when you get a new lens you should probably shoot a brick wall or two just to be sure you didn't get a decentered copy. I'm not in search of perfection, but sometimes (particularly with wide angles) one side of the lens is just way softer than it should be. You might not notice it right away in routine testing and when deciding whether to keep a lens or not, that does seem important.

10-27-2020, 05:02 AM	#11
house Pentaxian Join Date: May 2015 Photos: Albums Posts: 3,306	Originally posted by BarryE Pragmatic sense. However, this pragmatism does have to be learnt, especially in these days. Newcomers, are probably unlikely to appreciate what he's saying, picking their first/second lenses based on reviews and charts. I know I did when I got back to digital. In Zenith/Canon AE1 days, I just bought what I could afford and got on with it. I suppose chart tests can be useful as they can suggest how a lens might behave in the field. The DA* 16-50 f2.8, my first Pentax lens, is a complicated performer. The charts helped me understand it a little better. Buying the most expensive lens you can afford is an expensive way of going about it. A cheap lens can have a better performance profile for what you shoot than a pricey one. Generally speaking you want the bench test before you purchase. Afterwards its too late but for catching manufacturing issues. Reviews with numbers are a useful data point and helps you with what to look for in sample images. I've leanred about lens flaws uncomfortably late across the globe with no way of repeating the shot. Rogers testing regime catches a fair few issues and is quick enough.

10-27-2020, 05:39 AM - 1 Like	#12
BigMackCam Digitiser of Film Join Date: Mar 2010 Location: North East of England Posts: 20,673	Originally posted by house Buying the most expensive lens you can afford is an expensive way of going about it. A cheap lens can have a better performance profile for what you shoot than a pricey one. This is why I referred to buying the best (rather than the most expensive) lenses you can afford. As you rightly infer, the most expensive lens doesn't necessarily translate to better performance depending on what you shoot and how you shoot it. For example, there's no point buying an expensive constant aperture f/2.8 zoom if you're shooting primarily landscapes at f/8 - f/16 and can get similar performance from a good quality variable aperture lens covering similar fields of view...

10-27-2020, 06:17 AM	#13
house Pentaxian Join Date: May 2015 Photos: Albums Posts: 3,306	The drift in this discussion is that Rogers method is for testing faulty lenses and learning details about the lens performance after purchase. Reviews including ones with mtf charts are useful to decide which lens to buy when you have no opportunity to try them out.

10-27-2020, 06:43 AM	#14
BigMackCam Digitiser of Film Join Date: Mar 2010 Location: North East of England Posts: 20,673	Originally posted by house Reviews including ones with mtf charts are useful to decide which lens to buy when you have no opportunity to try them out. I agree ... and so, by inference, does Roger: Quote: "It’s not because the lab stuff doesn’t give useful information. The lab gives a LOT of useful information. Most people don’t have time to learn how to interpret it, or learn its value and limitations, but it’s useful information nonetheless. And the lab is fast; I can test a lens about 32 different ways in a couple of hours. My ‘test a lens with photography’ time is a half a day or more. So the lab is faster, gives tons of information, and makes cool graphs. But I still don't use it to test my personal lenses."

10-27-2020, 08:33 AM - 2 Likes	#15
photoptimist Pentaxian Join Date: Jul 2016 Photos: Albums Posts: 5,123	Originally posted by BigMackCam This is why I referred to buying the best (rather than the most expensive) lenses you can afford. As you rightly infer, the most expensive lens doesn't necessarily translate to better performance depending on what you shoot and how you shoot it. For example, there's no point buying an expensive constant aperture f/2.8 zoom if you're shooting primarily landscapes at f/8 - f/16 and can get similar performance from a good quality variable aperture lens covering similar fields of view... Exactly! And sometimes the definition of "best" takes one in directions far removed from optical test bench definitions. For example, superzooms look like horror shows on the optical bench with all manner of distortions, aberrations, vignetting, and flare. But a superzoom may be the "best" lens for whirlwind tour-group travel that does not allow any time to switch lenses. Sometimes the best lens has to be the one on camera.