[house]

You can't beat the MTF curve, it's the calculated maximium and is independent of the camera or the photographer or the subject. An optical testing bench will use lasers to verify problems with a real specimen. How to Measure MTF - Optikos

Sorry I meant to refer to the imatest "mtf" which tests the whole system rather than just the lens as a normal mtf would. Even so I believe Pentax release calculated not tested mtf right?

[tibbitts]

If a reviewer finds they can't replicate the published MTF results, especially in symmetry, they have to understand they've got a dud copy. Real reviewers will have Imatest software or similar, if they can't be bothered paying for and using it, they're not real testers, they're being amateurs. That's most 'testers' on the web, by the way, the standard of science, journalism and even their photography skills can be appalling. I saw that Angry Photographer guy in a video taking portraits of a chick in daytime with a Fuji MF, and for some reason the dumb so-and-so was letting his ISO automatically go up to 12,800 or whatever, he was real Uncle Bob level during the shoot, that was being filmed by Jason Lanier's team.

Your implication is that any lens that cannot meet the published MTF results is defective. I'm okay with that, but wouldn't that make almost all production lenses defective? Of course the ideal situation would be for manufacturers to test MTF as part of the production process and remanufacture any lenses not in compliance. I suspect part of the process might be some tweaking of that baseline MTF.

[normhead]

Your implication is that any lens that cannot meet the published MTF results is defective. I'm okay with that, but wouldn't that make almost all production lenses defective? Of course the ideal sitauation would be for manufactures to test MTF as part of the production process and remanufacture any lenses not in compliance. I suspect part of the process might be some tweaking of that baseline MTF.

The Pentax published MTF results, are based on theoretical performance, not measured results. It is unlikely any lenses actually meet those specs. The specs used by the repair people to determine a lens is in spec have been shown to be far below published MTF results. Poorly performing lenses have been returned to people as "in spec." when they were as bad as they were when they were sent in.

I would suspect the reason for doing it this way is "everybody does it."

My guess is running an MTF test to every lens leaving the factory would be at least $100 to the cost of every lens. And for what? Most of the time, a few hundred lw/ph makes no difference to the final image.

I suspect my DA*60-250 is a pretty average copy. But the original design spec was so high, it's still a great lens. That's a possibility that explains the value of manufacturer theoretical lens charts. You know what the potential is, and you also know, you have more chance of getting a useful lens even if everything isn't up to the published spec. When you sart with a substandard design spec, the bottom 1/3 of the sample could be atrocious.

My FAJ 18-35 and FA 28-200 (Tamron design) have such low design specs, even if you get one that's right at the top of the sample variation scale, you probably won't be happy with them on a K-1. Maybe on a 6 MP sensor, but I no longer have one of those to test my theory.

[tibbitts]

The Pentax published MTF results, are based on theoretical performance, not measured results. It is unlikely any lenses actually meet those specs. The specs used by the repair people to determine a lens is in spec have been shown to be far below published MTF results. Poorly performing lenses have been returned to people as "in spec." when they were as bad as they were when they were sent in.

I would suspect the reason for doing it this way is "everybody does it."

My guess is running an MTF test to every lens leaving the factory would be at least $100 to the cost of every lens. And for what? Most of the time, a few hundred lw/ph makes no difference to the final image.

I suspect my DA*60-250 is a pretty average copy. But the original design spec was so high, it's still a great lens. That's a possibility that explains the value of manufacturer theoretical lens charts. You know what the potential is, and you also know, you have more chance of getting a useful lens even if everything isn't up to the published spec. When you sart with a substandard design spec, the bottom 1/3 of the sample could be atrocious.

My FAJ 18-35 and FA 28-200 (Tamron design) have such low design specs, even if you get one that's right at the top of the sample variation scale, you probably won't be happy with them on a K-1. Maybe on a 6 MP sensor, but I no longer have one of those to test my theory.

Even on my K5 with its AA filter I can see differences across the frame (between different sides and corners compared to each other) with most of my lenses (notably not the 60-250), that simply shouldn't be there, so maybe the differences are more than a few hundred lw/ph or not. That would be the point of the test, to fix differences that people could see outside of a rigorous testing environment, like that jumped out within a few seconds of viewing at 1:1. I'd be good with a lens having 2962 lines vs. 3000 in the spec, so it's not like some tolerances shouldn't be allowed.

I would pay $100 more for a lens that was guaranteed perfectly consistent across the frame, within my non-scientific ability to measure. It would save me testing, shipping back and forth - four times for just one model. Ugh. The question is whether the testing would cost $100 alone, or whether that would cover testing, remediation, and testing again.

[normhead]

Even on my K5 with its AA filter I can see differences across the frame (between different sides and corners compared to each other) with most of my lenses (notably not the 60-250), that simply shouldn't be there, so maybe the differences are more than a few hundred lw/ph or not. That would be the point of the test, to fix differences that people could see outside of a rigorous testing environment, like that jumped out within a few seconds of viewing at 1:1. I'd be good with a lens having 2962 lines vs. 3000 in the spec, so it's not like some tolerances shouldn't be allowed.

I would pay $100 more for a lens that was guaranteed perfectly consistent across the frame, within my non-scientific ability to measure. It would save me testing, shipping back and forth - four times for just one model. Ugh. The question is whether the testing would cost $100 alone, or whether that would cover testing, remediation, and testing again.

I was thinking testing alone. If they decided to toss everything in the low end of the sample you could double the cost of the lens. According to lens rentals it would be extremely difficult to try and bring them all up to a higher spec.

[tibbitts]

I was thinking testing alone. If they decided to toss everything in the low end of the sample you could double the cost of the lens.

I would hope that they could determine what went wrong and eventually tweak the manufacturing to produce a reasonable number of compliant lenses. Twice the price might still be worth it, although you could also have a line of lenses made up of the rejects, I suppose.

[Wheatfield]

Pentax Distributors: "Hey Boss, we have a request for an inspection copy of the DFA 70-210 from Font-of-all-knowledge.com". Boss: "Great. Go into the store room and find the most de-centred copy we have to send them. And if you can't find a de-centred one, pick any and drop it from head height. Then send it off to them straight away; we want to see those MTF charts out as soon as possible."

Is that what really happens? It would seem so, given the regularity of the "they had a bad copy" mantra here on PF. As someone with a cupboard full of Pentax lenses and not one dud, I find it strains credulity that these review sites cop the bad copies so often. Indeed, surely distributors should test any lens before sending it for review, just to be safe.

More likely is that they are not properly calibrating the AF correction before they run their tests.
If what you are testing is to see what something does "out of the box", then you aren't testing the equipment at it's best.
I have yet to find a Pentax AF lens that couldn't stand a little bit of AF fine tuning, and even being out by one adjustment number will knock results off if you are pixel peeping.
An accurate test requires calibrated equipment. If they aren't running basic AF calibrations, they are not testing accurately.
Do any of these so called equipment testers actually say they are running an AF adjust prior to the test?
How about ensuring the cameras they are using have identical outputs?
All of this stuff matters. A real test requires changing one, and only one parameter at a time. I don't see how you can slap a Tamron lens on a Nikon D850 and a similar Pentax lens on a K1 and think that the comparison is going to be accurate.
I don't think there is any way in the world of getting accurate testing unless the exact same camera body is used for all the tests, and that simply isn't possible.

[tibbitts]

More likely is that they are not properly calibrating the AF correction before they run their tests.
If what you are testing is to see what something does "out of the box", then you aren't testing the equipment at it's best.
I have yet to find a Pentax AF lens that couldn't stand a little bit of AF fine tuning, and even being out by one adjustment number will knock results off if you are pixel peeping.
An accurate test requires calibrated equipment. If they aren't running basic AF calibrations, they are not testing accurately.
Do any of these so called equipment testers actually say they are running an AF adjust prior to the test?
How about ensuring the cameras they are using have identical outputs?
All of this stuff matters. A real test requires changing one, and only one parameter at a time. I don't see how you can slap a Tamron lens on a Nikon D850 and a similar Pentax lens on a K1 and think that the comparison is going to be accurate.
I don't think there is any way in the world of getting accurate testing unless the exact same camera body is used for all the tests, and that simply isn't possible.

I would think they would use live view for focusing.

[Wheatfield]

I would think they would use live view for focusing.

Maybe. Does the "tester" say? Does it show in his video?

[BROO]

Maybe. Does the "tester" say? Does it show in his video?

Li from the Cameraville said he used focus peaking in his latest video @8.04.

[monochrome]

I was thinking testing alone. If they decided to toss everything in the low end of the sample you could double the cost of the lens. According to lens rentals it would be extremely difficult to try and bring them all up to a higher spec.

I recall reading each DFA*50/1.4 was hand tested and some were sent back for tuning. Pentax was apparently unable to run their machines at production speed with sufficient tolersnces to do otherwise.

I would hope that they could determine what went wrong and eventually tweak the manufacturing to produce a reasonable number of compliant lenses. Twice the price might still be worth it, although you could also have a line of lenses made up of the rejects, I suppose.

I suspect many of their machines are elderly.

[clackers]

Your implication is that any lens that cannot meet the published MTF results is defective. I'm okay with that, but wouldn't that make almost all production lenses defective? Of course the ideal situation would be for manufacturers to test MTF as part of the production process and remanufacture any lenses not in compliance. I suspect part of the process might be some tweaking of that baseline MTF.

I'm not still sure you understand the concept, Tibbits.

It's like the frequency response curve of hi-fi for an amplifier, it's an upper bound. What you actually get out depends on your system and the speakers.

The end result, like the JPG, is a different story. A good photographer with a good camera and a good setup will get results someone else on the Internet fails to. And it should be possible to diagnose faults, like decentreing.

If you check out Roger Cicala's blog posts on Lens Rentals, he uses his optical bench to generate MTF curves and lenses can show massive variation between copies. All brands, all price levels - even Canon L series glass, he showed in one post.

[bdery]

Until @bdery in Quebec City is finished his review, I am disregarding all and any external reviews. This is the Age of Disinformation, where the free things in life are worth less than what you pay for them.

That's quite the compliment! To be fair, I'm still dependent on sample variation since I too get only one sample to test, but I try to be as thorough as possible.

[normhead]

I'm still taking my DA 18-135 out all the time irregardless of test scores. It's easily my most used lens all time for digital. You can buy or not by lenses based on them at your peril. I know every one thinks they will get Andy Warhol polaroid quality from a bad lens and Ansel Adams results with a great one, but that's just extra money in your pocket talking. Those dollars can get really loud.

[tibbitts]

I'm not still sure you understand the concept, Tibbits.

It's like the frequency response curve of hi-fi for an amplifier, it's an upper bound. What you actually get out depends on your system and the speakers.

The end result, like the JPG, is a different story. A good photographer with a good camera and a good setup will get results someone else on the Internet fails to. And it should be possible to diagnose faults, like decentreing.

If you check out Roger Cicala's blog posts on Lens Rentals, he uses his optical bench to generate MTF curves and lenses can show massive variation between copies. All brands, all price levels - even Canon L series glass, he showed in one post.

Honestly I tend to only look at one thing: edge/corner consistency. So I don't think I would care if my lens resolved a peak 2100 lines vs. somebody else's 2300 or whatever. I can see where not every copy would be exactly the same, but I think in a world of 50-60mp FF sensors, if I can see a difference in copies with my 16mp AA-equipped APS bodies, something is too far off. Of course some lenses are just horrible designs, and that might even include some current Pentax lenses, but the lenses mentioned here are certainly not horrible designs.

I read the LensRentals blog a while ago, and as you say it was very enlightening. I have rented a lens from them: a 60-250. I'd rented it to compare to the one I'd just bought, to decided whether to keep my new one.The LR model hit focus at 0 and my new one was a -10. And honestly the LR model was just slightly crisper all over - but very, very near the limit of what I could tell with 16mp and an AA filter. And as you point out, post-sharpening it was even harder to tell them apart. I always compare raw, although none of us are getting actual "raw" sensor data. Anyway I kept my new lens because I figured the -10 might be able to be fixed, and frankly I'd sent back up to 4 copies each of two other Pentax model lenses just trying to find one that was consistent across the frame .My new -10 lens and the LR copy both were. I think buying a lens and having the first copy be consistent across the frame is almost a miracle.

Oh, the LR copy had completely dead SDM out of the box - I had to play with it quite a bit to get it going, as is the case with SDM sometimes. Otherwise I would have been tempted to buy it and return the new copy, but nobody wants to start with dead SDM.

But there is no way with sharpening or anything else to fix one side being blurry and the other one not, and that's way more of an issue than peak MTF for me. I just bought another new lens a couple of months ago (not a Pentax brand), again stuck in my little world of 16mp and an AA filter. One side is ever so slightly off. I mean really, really slighly off. But I can tell, and I shouldn't be able to. It's even less noticable post-sharpening. As is usually the case, it's only off in that last 10-15% of the frame edge. But I have 16mp and I try to use every one. Anyway I kept the lens because from my unfortunate experiences with other lenses, it was as close to correct as I'd likely get in maybe five or ten copies. Eventually stores do get tired of sending you new copies of the same lenses (believe me, that does happen - I'm sure some of them have my name on a list now.)

Maybe it's just me but when I go to an exhibit of someone's photography - mostly I look at landscapes or other detailed photos - the very first thing I look at is all four corners and check for sharpness. I don't even think about whether I like the picture or not, because I know I could never get past a blurry edge or corner. And sure enough, even in giant enlargements from half a century ago, those corners are almost always solid - I can see really, really impressive sharpness right to the edges. How they did that with the technology of that day is amazing. Maybe they weren't the original image corners - maybe they'd cropped that 4x5 transparency down a bit when making that 16x20 and didn't tell me.But those prints almost never have one side or corrner that's less sharp than the others. So I don't want my otherwise less-impressive pictures to be that way. Everybody complains it's "pixel peeping" to look at a photo at 100%, but that's what you see when you make a reasonably large print from your 16mp image.