It seems that some have misunderstood the main thrust of what I was saying.
Nowhere in my post did I question the accuracy of PZ's measured data, or their measuring methods.
Again, nowhere did I say that PZ (or other technical measurement sites) are useless - on the contrary, more than once I said that technical measurements have their time and place, and their usefulness.
But the key thing I was saying is that measurements alone are insufficient in revealing the full optical nature of a lens.
To try and illustrate that point, I showed that not one but three highly regarded Limited lenses (Pentax's premium glass) failed to impress PZ. (Yes, I am aware of other Pentax lenses PZ is impressed with, eg. the FA31 and DA70 Limiteds).
As I kept on stressing in my post, do we see a disconnect somewhere? In other words, how is it that great lenses like those three (and that's not just my isolated opinion - many, many others have seen it, and concur) fail to register on PZ? Clearly something is "missing" - there is a disconnect.
And that leads to my other main point: Actual user experience, formed over time in real-life usage situations is every bit as weighty as numbers and charts, and often more relevant. Technical tests need to be supplemented.
Lytrytyr pretty much sums it up:
For an informed decision,
I've found it best to factor in
a mix of (subjective) user reviews
and (objective, but sometimes flawed) technical tests.
Talking now about "pixie dust", Civiletti wrote: "Perhaps fairy dust is in the mind of the beholder. I have seen images on this site from these Pentax lenses that supposedly do something special in rendering. I do not see it."
And top-quark wrote: "Subjective reports of "pixie dust" and the like are utterly worthless - you might as well say it's "magic" and be done with it."
My response is that it is probably true that not everyone sees it. That's fine, and in so saying I mean no disrespect to anyone at all.
But some do see it.
There is an old saying that "where there's smoke, there's a fire". Subjective reports of pixie dust are not utterly worthless - the fact that various people are reporting it should at least trigger us to the possibility that something just might be up. And end of the day if an individual simply does not see it, then like I said, that's fine.
But I was nowhere suggesting that we just call it "magic", and disregard all technical testing. I've already reiterated that technical tests have their use. But just because a certain quality cannot be readily measured doesn't mean it doesn't exist - we know that. And indeed things like colour rendering, 3D effect, bokeh quality etc lie beyond our ability to measure today.
So in short, while sites like PZ and the like are doing the best they can by way of objectively testing lenses, many things lie beyond the realm of measuring, and the plain fact is that their results are thus inadequate evaluations of a lens. Hence the need to supplement with the experience of users in the field.
Incidentally, I wasn't zeroing in on PZ alone - I was speaking in general of the whole lens measurement thing. But PZ is the site I'm most familiar with (I guess many others are familiar with it too).
I need to add here that in my post I neglected to mention that price is always a consideration for most of us, myself included. I wanted to mention it but just forgot, and If I have offended any, please accept my apologies. Certainly I understand that finances are often limited, and cost is definitely a deciding factor in choosing lenses - or anything else, for that matter.
Top-quark wrote: "As far as I'm concerned, it tells me that I can better spend £500 on something else which is a good thing to know - when it comes to that sort of money I don't necessarily need to find out for myself."
Very true - 500 is a lot of money. But I wasn't suggesting that you should have to find out for yourself whether a lens is suitable by paying for it and trying it out, only to be dissatisfied - that would indeed be a costly experiment. What I meant was to check the technical reviews, and also leverage off the field experience of others - a friend who owns said lens would be ideal, of course.
In my own case, when I was researching between the DA70 Limited and the FA77 Limited, one of the first things I did was - would you guess it? - go to Photozone! After digesting, it was clear to me that the DA70 had more even sharpness across the frame and less Purple Fringing; and I knew that I would have to forego this if I chose the FA77. Like many here, I too found the FA77 expensive compared to the DA70. But then, I read user comments, which talked about that magical factor of the FA77 and I began to hunt for pictures that could substantiate that claim. Eventually I was sold - I was prepared to sacrifice some evenness across the frame, and tolerate the occasional PF, to obtain the "look" generated by the FA77, something I was hard-pressed to find in the pictures taken by the DA70 which I scrutinised.
I share my experience not to put down the DA70 Limited - it is a good lens in its own right, and truly I like the look of the pictures it produces too! (Certainly I love the "look" of all DA Limited family lenses). But for my personal taste, it was the FA77 which would serve best.
Rather, I share the above to illustrate that I made use of technical reviews, plus reports of users' field experience, plus scrutinised actual photos showing those "intangible, unmeasurable traits", and made my decision.
But since technical review sites eg. PZ form their conclusions mainly from measurements, it stands to reason that a given lens whose technical readings per se do not quite measure up to expectations will receive a so-so rating - something which seems to happen quite a bit to Pentax glass, it seems to me. And the tragedy lies in the fact that said lens could in fact be returning superb performance in practical use, seen in the real-world pictures taken! This was the main contention of my original post - are Pentax lenses being misrepresented? Of course, the same could be happening to non-Pentax glass, but that's for another forum.
