[cyberjunkie]

First, a few words which i hope will clarify my mindset and explain the logical reasons behind this post.
I appreciate the efforts of anybody who takes the time to write a lens review.
I have nothing personal against anybody and, i repeat, i appreciate any contribution. This forum is reasonably free from flaming, brand wars, endless personal conflicts, and the like...
I don't want to fuel any confrontation. I'm just asking other forum members if some basic guidelines are needed (or not).

Not all the casual lurkers, or even the forum members, are very experienced.
This simple fact is confirmed, practically every day, by the number of naive or VERY basic questions are posted on this forum.
That's perfectly natural, nobody's born informed/experienced!
For example, i started taking pictures when i was barely 10, and now i'm 56. In the meantime i dedicated a lot of time to taking pictures and learning about photographic techniques.
Then i developed an interest for optics and the history of photography, and amassed a huge collection of vintage large format equipments (mostly lenses, at least 200).
Other people may not have the time, or the interest, to delve in such subjects. They just look for advice, or basic informations. That's perfectly fine.
BUT... recently i've seen many reviews referring to lenses which can't even be properly connected to a Pentax camera, digital or analog!
When i see a vote of 9 or even 10 given to a lens which can be barely force-fitted in a Pentax body, and which can't be focused farther than 3 or 4 meters... isn't it potentially misleading for the casual reader?
AND, given that ALL the casual readers are actually reading the text (which i think is pure utopia), what's the rationale behind such high votes given to lenses which can't be even focused to a full figure distance?
I appreciate the ingenuity behind such attempts, though i question the worthiness of having such lenses reviewed TOGETHER other glasses which can be correctly fitted to a Pentax camera body (via an adapter, or without it).
AFAIK there are no rules for lens reviews. Isn't it the right time to introduce at least a very basic guideline?
The existing reviews should be transferred to a new section, name it as you like... "NO-infinity lenses" could be a reasonable one.
If new reviews of the same kind are posted, they should go to the same section.
It would be simple, and should prevent misunderstandings.
In my opinion, even lenses equipped with a Leitax adapter ring (or the chinese copycat) should be clearly labeled as such: it could be a new a field which reads something like "Needs an adapter: YES/NO" That would be easy enough for the programmer who takes care of this site (opposed to creating a new group and transferring the reviews).

I won't comment on the substance of the reviews themselves. I wrote that i don't want to be confrontative (not even a little..), so i won't comment about the rationale behind those reviews, and the opinions expressed.
If somebody else comes forward and speaks his/her own mind, then i will add a couple of lines expressing my own opinions

cheers

Paolo

[Adam]

I agree- there should be a better way to tell which lenses have what mount. There are way too many M42 and universal mount lenses out there.

[Sorver]

Ok speaking as a newbie I feel it would be helpful as I now come here as first port of call for help if I need it and tend to research the death out of anything I plan to buy so the above mentioned would be very helpful but I understand that this also takes time on behalf of the moderater who runs this for the pure passion of Pentax photography so don't expect them to have to revise and check each lens review put up either must say I can see it from both sides. That's my thoughts at the risk of being accused a fence sitter............ I want the world to love me hehe.

[interested_observer]

Good (early) Morning,

This is a topic that I have thought about from time to time, and - actually started a thread once but dumped it before pushing submit. I don't have an answer, but some partial suggestions (not even suggestions - just comments). First a quick story.

I was once involved in a large program - worth several billion. The customer absolutely demanded the use of commercially available software everywhere - where possible. We all became very aware that the display functionality was the most critical component (either make the entire program a success or a failure). Selecting that capability was dropped in my lap. After several months of research of all the product candidates, I traded all of them, developed a scoring methodology, scored them and presented the results internally. Internally, the choice was not popular (are you on mind altering substances - drugs, are you drinking your lunch down at the bar, etc.). With the partners it was even less so (how stupid are you - you are a moron). The customer was very happy (as they liked the selected product). The scoring was the key. After several weeks of meetings, we wound up developing several additional evaluation templates and scoring approaches. We then scored them by committee (this took days for each candidate). Bottom line, each of the methodologies came to the same end conclusion/result - which was good, as it turned out to be a validating sensitivity analysis of the original approach and down selection. (PS - the program was a very successful - because of the display approach - and the customer has been ecstatic, and its now 10 years old.) What was funny was the total scores for each template and scoring methodology. For example, one was 9.9/10, another approach (completely different) was 72/100. How can you have such a wide disparity in evaluation of the same product, all based on the same basic performance requirements? - more on this later.

Ok, where am I going with this. A couple thoughts - not even suggestions, probably more along the lines of comments.

• Perhaps writing a couple of lens review templates (one short, one medium, one long) - for folks to follow. I had asked Rico, before his eye sight started to fail, if he would be able to write up a couple of reviews on some of the more off the beaten path lenses he was using so successfully.

• A scoring template/approach. Its probably too late now, since so many reviews have already been done. It also is probably impractical due the the large number of folks writing and scoring a review - where large variations would be reflected in the interpretation of approach. Also, Adam has implemented a set of scoring categories of sorts, which was a excellent step forward. Going back to the story for a moment - I struggled with a problem initially. A consistent scoring approach. If something is excellent/perfect, it should be a 10/10 right! Well, what about meeting a requirement - should that be a 10 too? After a lot of all day (and night) meetings with vigorous discussions, the best approach we found was - meeting a requirement 5/10. And then pulsing up, for the implementation approach/results, or how it was done/accomplished/etc. For instance - meeting a requirement 5/10, the implementation adding additional capability beyond the base need - adding up to an additional 5/10. Essentially, lets say that a FA 31 Ltd or 77 Ltd could come up with a score in the 70's/100, rather than 9.9/10. That would generate questions of how a great lens would generate scores of 70's - something must be wrong with either the evaluation or the scoring. Additionally, you would need to take in to account folks wanting to compare a 77/1.8 Ltd against a 85/1.4. Which leads to a case of where two readers wanting to apply the review results are looking at the review. One wanting to do portraits with bokeh (and not cuttingly sharp features), while the other wanting to use the lens for landscapes, wanting ultra sharpness with a deep DoF, and 3D rendering.

So, now after writing a wall of words, to me - the current scoring approach, appears to be pretty good (probably as good as it can be across a large group of folks with a very wide variety of backgrounds). Folks are not going to really read and understand a defied scoring approach (plus it will be interpreted with a lot of personal bias added in for good measure) . If they feel something is excellent - they are going to put down a 10 regardless (and if they feel its a great lens, they will adjust the component scoring to have the summary come out to be a 10). Averaging across the individual reviews - will take care of the outliers. The scoring methodology has already become somewhat self defining.

Rather than concentrate on a (strong or rigid) scoring approach, probably a descriptive template approach might be better. The scoring valuation would probably lead the readers to reading the descriptive evaluation which is probably the best approach. A template helping reviewers to write descriptions across a set of categories - and giving some examples/suggestions, might be the best possible approach.

I don't know - I got up to get a drink of water - so now I am going to go back to bed, as its 3.50 in the morning - O dark hundred.

[Driline]

How about a simple disclaimer? Read these at your own risk . Was I too long winded?

Right or Wrong the lens reviews are one of my favorite sections.

[PJ1]

interested_observer makes a valid point. But there seem to be a couple of issues rattling around in this thread already.

1 cyberjunkie says there are reviews of lenses which cannot (or can barely) be fitted to a Pentax body. Maybe the lens reviews need to be moderated in some way so that this kind of issue goes through to the keeper. Most of us probably don't recognise those lenses and don't look at those reviews. But I think it is useful to have reviews of lenses that Pentax users might dabble with. They just need to be sound (and fair) assessments. Which brings me to the next point.

2 Adam says

There are way too many M42 and universal mount lenses out there.

This is the first time I can't agree with Adam. Too many lenses is never enough! All those lenses are there to be played with by Pentax users (a major Australian photography chain advertises that there are "25 million lenses" that can be used with Pentax bodies). The value of having the reviews is that if one comes across a seemingly "bargain" lens, there is a ready reference to see what others think (or don't think). But the miscellaneous section of the reviews could be better organised - or more easily searchable. It does get confusing when the same lens has been sold under many different names.

3 interested_observer has a point - not every evaluation will be that same, but the law of averages suggests that lots of reviews of a lens should produce a balanced assessment. This falls over when there are not enough reviews to give a sound average. Personally, I have a problem with the kind of review that says "this lens came with the body and I don't like it" without any indication that the lens was given an objective trial. There are also reviews which go competetly against the trend (like a "4" amongst a run of eights and nines, or "this lens is soft" when the general consensus is that it is sharp). This often suggests a bad copy of a lens or (more likely) a problem with the person holding the camera. There is even one review in there where the reviewer admits the lens was never used but scores the manufacturer as a "5" for completely irrelevant reasons - and that score is averaged into the final analysis! When there are only four reviews of the lens (and the other three are "9") that skews the assessment. There are also problems when the reviewer "compares apples with oranges" - "this zoom lens is not as sharp as a prime" or "this f4 lens is not as fast as an f2". Maybe that is where some guidelines could be developed - and a moderator could pull a review and ask for it to be revised!
Personally, the reviews that I find most useful are those that include some images too demonstrate what the reviewer is talking about. Like anything else, this can have its problems. But the more experienced among us (not me!) will be able to tell the difference between "soft" and "out of focus".

So I would say build on the system and improve it. It may not be perfect but it is working well for the majority. And let's face it - we do see some pretty basic questions being asked sometimes, so for those who want to learn, we need to be able to point them towards sound data in the reviews.

[mikemike]

Designing effective survey instruments and analyzing the results is a very interesting field.

The way the lens reviews are here does not lend itself to producing a valuable statistics. The qualitative stuff in the body of the reviews is basically the only thing that is useful to me.

To improve the value of the statistics, adding better "anchors" than poor to excellent. The anchors would probably be different by the class of lens
An example of what I mean would be for zoom lenses the anchors on sharpness would probably be:

1 - Soft unless stopped down 2+ stops throughout the range
2 -
3 - Soft unless stopped down 1 stops throughout the range (on par with kit lens)
4 -
5 - Ok wide open, better stopped down a little
6 -
7 - As sharp as a prime at the same f-stop at its best focal length
8 -
9 - As sharp as a prime throughout 80% of its range
10 - As sharp as a prime throughout 100% of its range

Or it might be effective to even have the ability to rate certain aspects of a zoom lens at the widest, mid-range (or even a couple of mid-ranges for superzooms), and longest focal lengths.

Changing the camera used to a drop down and the ability to put cameras used and a way to say whether the lens kept up with your expectations as you moved up in skill and cameras.

And another field instead of just pros and cons - a good, bad, and ugly. I would say that DA* lenses especially suffer from punitive ratings based on ugly experiences with SDM failure. It is even possible that the same lens might be rated okay by the first owner who had it and had the SDM fail and replaced under warranty (a con or ugly experience depending on the person), sold it to a second owner who rated it well because they never had any SDM problems, sold it to a third owner who had it fail and had to pay out of pocket for it to be replaced only to have it fail again (very ugly), they sold it to someone else who disabled the SDM and now they love it.

A lot of those overall ratings are driven by delivering on expectations and everyone's expectations are different and they are different for each lens.

Another thing that would be useful is a couple of questions about reviewer's background years of experience with photography, ILCs, and # of lenses owned. And a couple of questions that might expose sample variation problems as well as an opt-out to expose reviews by noobs like:
Required microfocus adjustment with a choice for "I don't know what this is" and "I had to send it in for service adjustment"

When someone adds a new review, you might want to hit them up to see if their impression has changed on the lenses they have previously reviewed. And to give a comparative ranking of their lenses.




Sorry if that was a lot to read

[marcusBMG]

As a busy and inexpert contributor to the lens reviews I can strongly concur with cyberjunkies suggestion towards stronger reviewing guidelines.

When I started acquiring vintage lenses to use on my dslr/csc cameras, and then offering some opinion on them here I didn't know what I was talking about. Now a couple of years and much browsing and reading later, I can safely say I still don't know what I am talking about (really), but I have read enough, seen enough different lenses, and am methodical enough that I have a certain confidence I am able to offer useful information in a review. At least I now know the difference between resolution and contrast, lateral and longitudinal CA and can now work with RAW images so that processing biases can be recognised. But overall I do not assume, particularly when it comes to an opinion on IQ, that my comments are anything other than subjective and relative. In one or two self evident cases I have declared, for example "this lens is rubbish" otherwise its always "I felt that...it seemed to me...". There are just too many variables, I have learned from experience that if I want to get a sense of a comparison between a couple of lenses the only way is a suite of test pics taken at the same time under the same circumstances (lighting). On more than one occasion I have simply declared that I can't distinguish ( I only have 12MPx on my K-r). And I have no intention (at the mo) of imaging test charts and counting lpmm.

So what about all the, how shall I put this... less than thoughtful reviews that clutter up the listings? In a community forum like this to a large extent one has to shrug ones shoulders,. We swapped pm's, Adam, over the ones contributed prolifically by a guy, at the time many of his offerings seemed little better than clutter, however it does seem that guy is trying to give an honest opinion, however out of sync. And there's always going to be a quota of contributions that really are little more than a tweet that they are pleased with their latest acquisition.

I'm entirely with you, interested_observer, the whole thing is fraught with difficulty, as you describe so well with your personal example, and the best thing that can be done is offer some guidance. Guidance that subscribes to the fundamental KISS rule!

I think you're on the right lines mikemike, but what you propose is, for me, too complicated. I rather like the 1-5 ratings on dyxums, 1-10 doesn't really work for me, its really a:- substandard-7-8-9-10 rating scale not a 1-10 where (I suggest):

10 (A) = top, pro lens, best. Zeiss planar, pentax limiteds etc etc
9 (B) = very good lens, better than average, better than kit, "prosumer", quality primes...
8 (C) = good lens. Kit lens, modern zooms etc
7 (D) = worthwhile lens, something to offer.
E = May have positive aspects/something to offer but overall doesn't make the grade

I agree with you re experience level. Your descriptive criteria could be part of the guidance underlying this simple schema.

Pretty much everyone can understand this, it ties in with a general consumer understanding of things. Add some specific information and instruction re rating categories, some example good reviews, and thats the beginnings of a template.

As far as moderating the duff entries, the best way IMO is a mechanism for that to be done by the community. Maybe what we need is a thumbs-down criticise option to go with the thumbs-up like. Enough criticisms - gets dumped, or at least marked as unhelpful.

Just my 2 cents on the matter, compliments to you gents for your thoughtful contributions.

[johnyates]

Maybe what we need is a thumbs-down criticise option to go with the thumbs-up like

That would be my suggestion as well. I know there have been a few times when reading a review I've looked for such a button.

[stevebrot]

BUT... recently i've seen many reviews referring to lenses which can't even be properly connected to a Pentax camera, digital or analog!
When i see a vote of 9 or even 10 given to a lens which can be barely force-fitted in a Pentax body, and which can't be focused farther than 3 or 4 meters... isn't it potentially misleading for the casual reader?

I guess I missed those. Are we talking rangefinder glass or simply mounts other than M42 or Pentax-K/variants?

Edit: Found 'em. I agree that third-party reviews should probably be limited to lenses that have native mount support to Pentax cameras (including vintage) or can be legitimately adapted without mount replacement (i.e. Leitax) at the same registration distance. Those would include Pentax 6x7, Pentax 645, Pentax-K, Ricoh P (with caution), M42 (with caution), and M39(Zenit). This could be enforced with a gentle reminder page prior to allowing a new lens to be entered into the database.

I would also add my particular pet peeve that lenses without actual reviews should be removed.


Steve

[mikemike]

In defense of some of these dubious reviews, some people might legitimately love them for some kind of bonkers personal project like some of these crazy macro setups I have seen or they love it because they are tinkerers and love just screwing with them and getting them to work. Maybe a criteria like consistency of results and ease of use or is this lens usable by the mass market.... i.e. could you hand your camera equipped with this lens and set to green mode to someone and ask them to take a quick snap for you?

[Adam]

A thumbs-down button might be a good idea for the future, though I'm going to hold off for now as I'm planning on revamping the review system in the future.

I've now added a "mount type" field so that the correct mount (i.e. KAF3, KAF2, KA, K, Third-party (adapter required), M42 screwmount, non-Pentax, KR) can be selected at the time the lens is added. Now I just need a volunteer (or volunteers) to go through the database and edit lenses to set this field. PM me if you have some spare time!

[cyberjunkie]

Hi Adam!
I'd happily volunteer, not for the entire review base... that would be a bit too much, but i'd happily do part of it.
I'm currently in Thailand, but i will soon move back to Europe, where i will have plenty of spare time, till about mid September..
If you can't find somebody who can start right now, feel free to contact me in a couple of weeks.
EDIT: WTF, i heard about it now: military coup, martial law, constitution suspended, at home by 10PM... i guess it's time to go back to the old continent

Now, just a few comments about the posts i've read:
1) When i decided to throw the stone, and ask for the opinions of fellows members, my intention was NOT to criticize the current review system. My idea was just to have CLEAR label for the lenses which can just be force fitted, and which can't be focused to infinity.
2) being an IT specialist, i know what a complete change of the review system would mean! better to leave everything as is, and just add a clear warning that a certain lens can't be properly fitted to a Pentax body.
3) The number of the lenses is NEVER too big. I added myself a few lenses i own, and which were missing from the database. ANY M42 or T2 lens, which can be fitted via a simple adapter, actually belongs to the lens database, and ALL the relative reviews should be welcome.
4) Any change in the review system (i.e. from A to E) would be prone to the same problems which affect the current system. All the reviews are subjective, can't be otherwise. I would never, ever, introduce new limitations.
There will always be somebody, experienced or unexperienced, who buys a lens "with a name" for plenty of money, and posts a rave review, even if the lens is a nothing special
With a grain of salt it's possible to tell the meaningful review from the unworthy one.

cheers

Paolo

[marcusBMG]

to go through the database and edit lenses to set this field.

I'll do the adaptalls

[lytrytyr]

I would also add my particular pet peeve that lenses without actual reviews should be removed.

HD Pentax-DA 35mm F2.8 Limited Macro Reviews - DA Prime Lenses - Pentax Lens Reviews & Lens Database ?

https://www.pentaxforums.com/lensreviews/smc-pentax-q-08-wide-zoom-3.8-5.9mm-f3.7-4.html ?