[joe.penn]

To clarify, I find PZ's measured data to conform to my experience. I don't give a damn about their opinions.

Ditto! I have purchased a lot (and I mean a lot) of lenses and always check mft charts from both pz and lenstip - I have not purchased a bad underperforming lens as of yet and this is attributed to their mft data.

[tuco]

I have no idea about lab tests but if they are testing a lens sharpness across its entire frame with equipment other than a cropped sensor camera, then they might be seeing things Pentax digital shooter do not on FF coverage lenses.

[monochrome]

I guess I just don't care any more about what someone else thinks - even a highly skilled technical reviewer whose opinions I read from time to time the way I read political blogs. Maybe I'm growing up or maybe I've run out of money or energy and I'm fooling myself. I certainly have grown tired of trying to defend my choices against the tide of popular opinion. The tide will rise and fall regardless of whether I want to.

As long as I avoid the true dogs, I likes what I owns and I owns what I likes. And I don't really care if Canon or Nikon or Sigma makes a "better" lens in a FL I already own since I don't own any of their cameras. Or whether a camera body that I like isn't very well received by the rest of the community, either.

I'd end with a, "So there!" but I like you guys and read your posts and they're interesting most of the time. But when I want or (think I) need something new I look here and at Flickr for examples of the product. That's all that matters to me.

[KDAFA]

It seems that some have misunderstood the main thrust of what I was saying.

Nowhere in my post did I question the accuracy of PZ's measured data, or their measuring methods.

Again, nowhere did I say that PZ (or other technical measurement sites) are useless - on the contrary, more than once I said that technical measurements have their time and place, and their usefulness.

But the key thing I was saying is that measurements alone are insufficient in revealing the full optical nature of a lens.

To try and illustrate that point, I showed that not one but three highly regarded Limited lenses (Pentax's premium glass) failed to impress PZ. (Yes, I am aware of other Pentax lenses PZ is impressed with, eg. the FA31 and DA70 Limiteds).

As I kept on stressing in my post, do we see a disconnect somewhere? In other words, how is it that great lenses like those three (and that's not just my isolated opinion - many, many others have seen it, and concur) fail to register on PZ? Clearly something is "missing" - there is a disconnect.

And that leads to my other main point: Actual user experience, formed over time in real-life usage situations is every bit as weighty as numbers and charts, and often more relevant. Technical tests need to be supplemented.

Lytrytyr pretty much sums it up:
For an informed decision,
I've found it best to factor in
a mix of (subjective) user reviews
and (objective, but sometimes flawed) technical tests.

Talking now about "pixie dust", Civiletti wrote: "Perhaps fairy dust is in the mind of the beholder. I have seen images on this site from these Pentax lenses that supposedly do something special in rendering. I do not see it."

And top-quark wrote: "Subjective reports of "pixie dust" and the like are utterly worthless - you might as well say it's "magic" and be done with it."

My response is that it is probably true that not everyone sees it. That's fine, and in so saying I mean no disrespect to anyone at all.

But some do see it.

There is an old saying that "where there's smoke, there's a fire". Subjective reports of pixie dust are not utterly worthless - the fact that various people are reporting it should at least trigger us to the possibility that something just might be up. And end of the day if an individual simply does not see it, then like I said, that's fine.

But I was nowhere suggesting that we just call it "magic", and disregard all technical testing. I've already reiterated that technical tests have their use. But just because a certain quality cannot be readily measured doesn't mean it doesn't exist - we know that. And indeed things like colour rendering, 3D effect, bokeh quality etc lie beyond our ability to measure today.

So in short, while sites like PZ and the like are doing the best they can by way of objectively testing lenses, many things lie beyond the realm of measuring, and the plain fact is that their results are thus inadequate evaluations of a lens. Hence the need to supplement with the experience of users in the field.

Incidentally, I wasn't zeroing in on PZ alone - I was speaking in general of the whole lens measurement thing. But PZ is the site I'm most familiar with (I guess many others are familiar with it too).

I need to add here that in my post I neglected to mention that price is always a consideration for most of us, myself included. I wanted to mention it but just forgot, and If I have offended any, please accept my apologies. Certainly I understand that finances are often limited, and cost is definitely a deciding factor in choosing lenses - or anything else, for that matter.

Top-quark wrote: "As far as I'm concerned, it tells me that I can better spend £500 on something else which is a good thing to know - when it comes to that sort of money I don't necessarily need to find out for myself."

Very true - 500 is a lot of money. But I wasn't suggesting that you should have to find out for yourself whether a lens is suitable by paying for it and trying it out, only to be dissatisfied - that would indeed be a costly experiment. What I meant was to check the technical reviews, and also leverage off the field experience of others - a friend who owns said lens would be ideal, of course.

In my own case, when I was researching between the DA70 Limited and the FA77 Limited, one of the first things I did was - would you guess it? - go to Photozone! After digesting, it was clear to me that the DA70 had more even sharpness across the frame and less Purple Fringing; and I knew that I would have to forego this if I chose the FA77. Like many here, I too found the FA77 expensive compared to the DA70. But then, I read user comments, which talked about that magical factor of the FA77 and I began to hunt for pictures that could substantiate that claim. Eventually I was sold - I was prepared to sacrifice some evenness across the frame, and tolerate the occasional PF, to obtain the "look" generated by the FA77, something I was hard-pressed to find in the pictures taken by the DA70 which I scrutinised.

I share my experience not to put down the DA70 Limited - it is a good lens in its own right, and truly I like the look of the pictures it produces too! (Certainly I love the "look" of all DA Limited family lenses). But for my personal taste, it was the FA77 which would serve best.

Rather, I share the above to illustrate that I made use of technical reviews, plus reports of users' field experience, plus scrutinised actual photos showing those "intangible, unmeasurable traits", and made my decision.

But since technical review sites eg. PZ form their conclusions mainly from measurements, it stands to reason that a given lens whose technical readings per se do not quite measure up to expectations will receive a so-so rating - something which seems to happen quite a bit to Pentax glass, it seems to me. And the tragedy lies in the fact that said lens could in fact be returning superb performance in practical use, seen in the real-world pictures taken! This was the main contention of my original post - are Pentax lenses being misrepresented? Of course, the same could be happening to non-Pentax glass, but that's for another forum.

---

Last edited by KDAFA; 05-01-2013 at 01:02 PM.

[Docrwm]

None of this matters because Pentax is killing itself. They just announced another round of price increases on lenses. Insanity is not a defense in bankruptcy court by the way.
https://www.pentaxforums.com/forums/pentax-news-rumors/223517-pentax-lens-pri...-go-again.html

[Paul Tipton]

I agree with the double blind tests. It would be hard pressed to find a difference. I can see a difference sometimes with certain conditions. I just woork around it. I have the DA 18-250 and use it extensively in the field. No lens changes and the quality of the photos is what matters. I like small and light, and I don't have to carry a ton of equipement in the field.

[Timd]

I think the key point here is that there are lens characteristics that one cannot measure. The lens performance in real life is more than sharpness, flare resistance, distortion, etc. For want of a title we call it pixie dust. No one believes the lens is magical. It just perfoms better than the "simple" tests indicate it should. Do we condem the lens because the tests say it is bad or do we look for better tests? The classic case that according to the theory of flight at the time, it was proven impossible for a bumblebee to fly. Did that make the bumblebee fly magically? No, it just meant that the theory was incomplete (turns out they were using fixed wing aerodynamics).

[KDAFA]

I think the key point here is that there are lens characteristics that one cannot measure. The lens performance in real life is more than sharpness, flare resistance, distortion, etc. For want of a title we call it pixie dust. No one believes the lens is magical. It just perfoms better than the "simple" tests indicate it should. Do we condem the lens because the tests say it is bad or do we look for better tests? The classic case that according to the theory of flight at the time, it was proven impossible for a bumblebee to fly. Did that make the bumblebee fly magically? No, it just meant that the theory was incomplete (turns out they were using fixed wing aerodynamics).

+1.

And the trouble is, lens test-sites are issuing verdicts with far-reaching influence. As mentioned in my OP, I wrote mainly to counter said effects, because sometimes great lenses are getting misrepresented.

[KDAFA]

Hi utak,

Actually, it was not my intention to specially "promote" the FA77, nor the other two lenses. Certainly the FA77 is expensive. I was just using the FA77 as an example to highlight an alarming observation - that PZ, for all the testing they did, seemed quite unable to discern what many end-users already know: Here is an outstanding lens, with a well-deserved reputation for imbuing remarkable character seen in the beautiful real-life images it can produce.

But PZ have been unable to detect this. Which makes one question why there is this disconnect. And that in turn challenges us to re-examine our foundational philosophy of evaluating a given lens.

Simply put, technical measurements alone are clearly unable to tell the whole story of a lens; PZ and others like it are ample evidence of this problem. And they issue authoritative verdicts, which can in effect misrepresent a lens.

Talking again about the cost, you can see in my later post (#34 in this thread) that I too groaned at the FA77's asking price. I have not had the privilege of using the Takumar 55 nor the M85, and if you say you find them also capable of similar "magic", I take your word for it, and am sincerely happy that there are cheaper alternatives available!

[KDAFA]

Though, at least the interviews from when the 77mm and the 43mm were designed tells a story that they weren't aiming to correct the lenses for flat field resolution testing, but optimizing them for "real life use" instead. There is something true to this judging by the pics I've seen as they have a different look to them compared to conventional lenses.

You mentioned seeing a "different look" given by these two lenses. Some may genuinely not see it, and that's fine. Others do see it, and highly value that special rendering. (Obviously I belong to the 2nd group).

There are a fair number of lens reviews online which praise the Limiteds (whether DA or FA) for having excellent all-metal construction, being compact and lightweight. All this is true and good, but it seems to me that few mention this "look" you spoke of, this unique rendering.

Which seems a shame to me, because surely that is one of the major reasons to use a Limited lens. In fact, if hypothetically speaking the Limiteds were made from polycarbonate, personally it wouldn't really bother me that much. While metal construction is nice to have, it's not crucial - just don't touch that rendering!

[utak]

[The 77mm] Its not a $1,000 lens IMHO.

It is now!!


KDAFA: "Actually, it was not my intention to specially "promote" the FA77, nor the other two lenses."

That's a relief because for a minute I was thinking you were a Pentax plant blogging to pre-sell yesterday's price increases. The 77mm is a good lens capable of outstanding images. Enjoy!

[Na Horuk]

But its not just Klaus and PZ, is it?
Often in gear review magazines and websites you can see a clear difference in the language they use for Pentax gear (and other stigmatized brands, like Sony some time ago) and the discourse used for Canikon (and even more grandiose language for premium brands like Hasselblad and Leica). You can try this yourself, read a review of Pentax gear in some medium other than a Pentax forum and you will notice that there is a metaphorical "...too bad its a Pentax" somewhere in there. It will always be disguised, though. Sometimes this affects the item's rating, sometimes its just words in the text. Oh, and every so often, when the quality is undeniable, they will say "I can't believe its a Pentax!" in some way, thus implying that Pentax has no business making good products, that in fact Pentax products are or at least should be inferior.
To sum up, reviewers are very important when it comes to notifying potential customers about the value and worth of products. And reviewers are human and will often have some type of bias. And bias is rarely beneficial to outliers, it usually works for the mainstream, the popular; and punishes the others. Sony struggled a lot with this, but Sony has enough money for PR and advertising that they are slowly turning this around. Pentax doesn't appear to be doing that, at least outside Japan.

[normhead]

One of the reasons I take reviews so lightly are the lack of double blind testing. There was a taste test done with beer many years ago, where the participants were all brand loyal drinkers who swore their beer was better than any other and that they could tell the difference. When given a taste of the 5 top selling brews they came it at exactly random choice levels. They couldn't tell the difference. The evidence is pretty much overwhelming, people can't make these kinds of decision accurately. Even a guy like Klaus over at PZ, can't make these kinds of decision accurately, even though he has the benefit of having 2D tests to help form his opinion.

If there's one thing you learn it's strength of conviction about accuracy does not mean one's opinion is more accurate. In fact those who have the strongest opinions are often those who have excluded the most relevant information in their research, focusing on one narrow aspect. Buy reducing a lens to an MTF number it's possible to ignore a lot of other relevant data, some of which are not even measurable. If psychology has taught us anything, it's that because something has some aspects that are quantifiable, doesn't mean the quantifiable aspect is the most important. It may be the most studied and tested, but that doesn't make it any more important. That just makes it the most talked about.

The one thing everyone can tell you, is if you buy the lens, can you get what you want to do done with it. Not maybe whether it's the best lens you can get for what you do, or if it will give you the most pleasing results based on your subjective analysis.

What most people want to know is if a certain lens matches their style. A reviewer probably can't tell you that.

[VisualDarkness]

I have an unproven theory about that. What we human Pentaxians see as "sharpness" is often the excellent micro contrasts that Pentax lenses tend to have. (Or rather a mix of excellent micro contrast and high resolution.) The already often mentioned DA 35 ltd, but also the DA 15 ltd, are both excellent examples of that. The actual resolution of these on test charts may really just be mediocre, but that sharpness is then corrected in PP. Off course, micro-contrast can be added in PP too, resolution charts don't measure micro-contrast. Now if PRI would actually measure and publish the micro-contrasts, then things would become different.

This would all be big fun if these bad test-results kept the prices down too, but no luck. And that brings me to the Pentax bang-for-the-buck-ratio, for which they are known. For the money spent on a Pentax lens with very noisy AF, you can buy one at the other brands that really scores in the benchmarks, with it's own quiet and fast motor, possibly with it's own dedicated OS.

Well, "sharpness" is a combination of contrast and resolution on most review sites. Klaus often write about sharpness in the text in addition to the MTF graphs and they are not the same thing, as he often states.

[normhead]

Well, "sharpness" is a combination of contrast and resolution on most review sites. Klaus often write about sharpness in the text in addition to the MTF graphs and they are not the same thing, as he often states.

Exactly, I love Klaus's numbers, I don't think his opinions are any more intelligent than anyone else's. I wouldn't buy a lens without checking Klaus' numbers. It's a great service he performs. His decisions on the "value" of a lens, well that's subjective, if the lens has what you want, it's value is amazing, if it doesn't have what you want, it's worthless. You can't say a lens is a good value, just because it's cheap. At least I can't, apparently others can. Ignore Klaus's star rating system and you'll be fine. His tests are bang on.