[northcoastgreg]

I have found it disturbing how common the bad copy phenomenon seems to be in regards to reviews of Pentax lenses.

It's not just Pentax lenses. It's not entirely uncommon for tests results online not to match up with user experiences, regardless of brand. And, oddly enough, it goes in both directions as well, with some lenses scoring ridiculously high on tests, others suspiciously low. Just look at ephotozine's test of the Olympus 14-150. No way does that lens have the edge sharpness that the tests show. Even the full rez sample images are not consistent with their test!

I find the same issue plaguing ephotozine's "test" of DA* 60-250. Look at the full rez sample photos. They are among the sharpest, and are easily the most impressive, of any of the sample images I've seen from any of ephotozine's tests, and I've gone through quite a number of their tests. The DA* 60-250 shots are quite sharp, with superb microcontrast. They are entirely inconsistent with ephotozine's own tests.

Is it really sample variation that's the problem here? Or is it tester error/incompetence? Why are tests from various sites so often at odds? Why are the tests at odds with sample images and user experiences? Is sample variation really that extreme and significant? If so, then these online tests are pretty much useless, because you'll never know whether they're testing a good or bad copy of the lens being tested.

[boriscleto]

It's all just measurbating. Go out and take pictures...

[clackers]

If so, then these online tests are pretty much useless, because you'll never know whether they're testing a good or bad copy of the lens being tested.

The good copy/bad copy phenomenon is real enough if you read through owners' experiences on these forums,

Fingers need to be crossed when buying a lens online, especially secondhand. :-)

As for reviews, they can only test the one in front of them, so reading multiple reports is due diligence!

[Rondec]

It's not just Pentax lenses. It's not entirely uncommon for tests results online not to match up with user experiences, regardless of brand. And, oddly enough, it goes in both directions as well, with some lenses scoring ridiculously high on tests, others suspiciously low. Just look at ephotozine's test of the Olympus 14-150. No way does that lens have the edge sharpness that the tests show. Even the full rez sample images are not consistent with their test!

I find the same issue plaguing ephotozine's "test" of DA* 60-250. Look at the full rez sample photos. They are among the sharpest, and are easily the most impressive, of any of the sample images I've seen from any of ephotozine's tests, and I've gone through quite a number of their tests. The DA* 60-250 shots are quite sharp, with superb microcontrast. They are entirely inconsistent with ephotozine's own tests.

Is it really sample variation that's the problem here? Or is it tester error/incompetence? Why are tests from various sites so often at odds? Why are the tests at odds with sample images and user experiences? Is sample variation really that extreme and significant? If so, then these online tests are pretty much useless, because you'll never know whether they're testing a good or bad copy of the lens being tested.

As I mentioned above, you even see variation in the same lens tested on different mounts. Obviously mtf will vary based on the sensor, but peak sharpness shouldn't, or vignetting, or CAs, but they do sometimes. The only solution from a user standpoint is to test a new lens, looking for defects/decentering, because I think all brands are prone to problems -- probably more so with higher end, wider aperture glass.

[normhead]

I won't dispute that the reviewer probably got a bad copy, but I have found it disturbing how common the bad copy phenomenon seems to be in regards to reviews of Pentax lenses. Maybe it is the same across brands*, but shouldn't they check for alignment and such before the lens goes out the door.

According to lens rentals, they're all about the same. Even buying a $10,000 plus dollar Nikon or Canon lens is no guarantee it gets to you working properly. For container shipping or aircraft, the lens has to be packed to survive a 6 foot drop. On a piece of equipment like a lens, it's probably a matter of luck if your lens comes aligned as it was when it left the factory.

I bet if you could fly to Vietnam and pick up your lens from the factory, these issues would be very rare, but I'm just guessing.

[bdery]

I won't dispute that the reviewer probably got a bad copy, but I have found it disturbing how common the bad copy phenomenon seems to be in regards to reviews of Pentax lenses. Maybe it is the same across brands*, but shouldn't they check for alignment and such before the lens goes out the door.

I think the conclusion is not that there are many bad lenses out there, but rather that lens testers (myself included, I've written a few reviews for pentaxforums) don't apply statistical analysis to reviews. It shames me because it's part of what I do on my day job, but it's simply not convenient to order 3, 5 or even better 10 lenses for testing. But that's how you build a valid statistic and minimize standard deviation impacts.

So the best way to interpret what you find on the web is to:

1-consider that all tests are elements of one larger sample, and draw conclusions regarding average performances
2-evaluate qualitatively if sample variation seems important or not, thus evaluating the standard deviation (are almost all reviews in agreement, or are results all over the place?)

For what it is worth, I have had good copies of the lenses I have purchased out of the box, although I don't own the 60-250.

All the lenses I own or have owned have been good copies. And that's quite a lot of lenses.