The reliability of lens ratings.

**BigMackCam** · 03-10-2018, 02:45 PM

Originally posted by Lens tester

When buying Pentax K, M and A lenses, I took into account the grades included in this forum.
Unfortunately, in some cases they are very different from those which I obtained by testing the lenses.

User reviews and ratings on any website are highly subjective, and influenced by a number of factors - user experience, ability, expectation, use case, other lenses owned, price paid, copy variation etc.

Even dedicated review sites will often show considerably divergent results because of different test methodologies, criteria, priorities and test equipment.

The best you can do is to read as much as possible on a particular lens and then decide if it's worth trying out for yourself. Typically, I look at three different review websites that I rate reasonably highly, then I read the user reviews here and on other forums. That's normally a good cross-section of information to help me make my choice.

Ronald Oakes · 03-10-2018, 03:24 PM

How many times have we heard on this Forum ?...."Lens Reviews Are Subjective" !
And the bottom line is.....Yes they are !
Take most reviews with little more than a grain of salt.

If 2 people review a inexpensive lens and give it a 10 , and a 9 , that makes it a 9.5 rated lens !
I largely ignore their review.

However if 50 people review a lens ? That gives a much better idea how well it performs.
I believe the rating somewhat.

stevebrot · 03-10-2018, 04:47 PM - **1 Like**

Numeric user reviews are generally (almost always?) inflated by either inexperience or else a perspective that "I don't buy junk, so it must be good". Here are a few considerations/notes regarding reviews on this site:

The numeric scores often reflect a "its the best lens I have" when it is the only lens they have
There is some uncertainty about subscores, particularly "aberrations"
Look for reviews that discuss the lens in detail in the discussion and which discuss both pros and cons
Be suspicious of reviews by users with low post counts and or who flood the site with multiple reviews immediately after registering. Often this is accompanied by auction or marketplace offerings of the items reviewed.
My personal practice is to read the comments and ignore the scores
My personal practice when writing reviews:
- Reserve total scores above 8 for truly exceptional lenses
- Reserve sub-scores above 8 for where the lens truly stands out in that regard
- Include a detailed description including build quality
- Include pros
- Include cons
- Include pertinent comments on field experience (handling, ease of use, optical quirks)

I hope this helps. BTW, I noticed you posted 11 reviews today, all of which are fairly terse.

Edit: Terse is OK, though

BTW...Welcome to the Pentax Forums!

Steve

vonBaloney · 03-10-2018, 05:10 PM

Originally posted by stevebrot:

BTW, I noticed you posted 11 reviews today, all of which are fairly terse.

Which are now gone? Don't see any from this user.

As far as the reviews, you can't "unsee" the numbers of course, but I just read them and pay attention to the ones that sound like they were written by someone who thinks like I do. So many are obviously off-base, or just or praising or criticizing things I don't care about. For modern lenses, there are also other sources where you can look up resolution testing data, etc.

stevebrot · 03-10-2018, 05:13 PM

Originally posted by vonBaloney

Which are now gone? Don't see any from this user.

Lens tester's Reviews - Pentax Lens Reviews & Lens Database

(...not my usual practice, I was just curious...)

Steve

vonBaloney · 03-10-2018, 05:20 PM

Originally posted by stevebrot

Lens tester's Reviews - Pentax Lens Reviews & Lens Database

(...not my usual practice, I was just curious...)

But how do you find that? If I go to your profile, there is a "latest reviews" tab and I can see your reviews. But not on his profile -- no reviews. If I search in the reviews for lens tester, no results. Maybe that db is only updated once a day or something?

Ronald Oakes · 03-10-2018, 11:11 PM

Welcome to the forum Lens Tester !
I just read your review's in the lens section , and see that you Don't recommend the SMC 50 1.4 ? Also see you do not recommend the SMC 135 3.5 either ?
Those are two quite reputable lenses , yet they didn't pass your "test" ?
Could you disclose your lens testing methods , and possibly post a few supporting images ? Cheers !

---------- Post added 03-11-18 at 12:13 AM ----------

Originally posted by vonBaloney

But how do you find that? If I go to your profile, there is a "latest reviews" tab and I can see your reviews. But not on his profile -- no reviews. If I search in the reviews for lens tester, no results. Maybe that db is only updated once a day or something?

Go to the Lens review section , all the latest reviews are there on the right of the main page. I'm sure you will find the latest reviews quite enlightening.

nobody · 03-10-2018, 11:29 PM - **1 Like**

In my opinion the number of reviews is the most important thing in the review reliability. The more reviewers the more reliable overall result is. It is because of differences between particular reviewers. Differences within expectation, experience etc. For example some Prinzflex lens has 3 reviews and scores 9.3 points-that is not review I would belive in. On the other hand if SMC Pentax A 50 f1.7 scores 8.99 on the basis of 125 reviews, the score it is reliable for me. It is due to statistics-the bigger sample the more accurate results. No matter how different the samples are.

Unregistered User 8 · 03-10-2018, 11:35 PM

Originally posted by nobody

In my opinion the number of reviews is the most important thing in the review reliability. The more reviewers the more reliable overall result is. It is because of differences between particular reviewers. Differences within expectation, experience etc. For example some Prinzflex lens has 3 reviews and scores 9.3 points-that is not review I would belive in. On the other hand if SMC Pentax A 50 f1.7 scores 8.99 on the basis of 125 reviews, the score it is reliable for me. It is due to statistics-the bigger sample the more accurate results. No matter how different the samples are.

Excellent points, because whether the reviews are " good " or " bad," they become a part of the average, and ALL opinions valid with regards to reviews.

Sandy Hancock · 03-10-2018, 11:47 PM

Take five off the average rating. Make that a score out of five. That will get you closer to reality.

But as others have said, it's still subjective. Read all the reviews and ignore the best and the worst. Then take off five and make it out of five

nobody · 03-10-2018, 11:48 PM

Originally posted by pjv

Excellent points, because whether the reviews are " good " or " bad," they become a part of the average, and ALL opinions valid with regards to reviews.

Exactly.

Unregistered User 8 · 03-11-2018, 12:07 AM - **1 Like**

Originally posted by nobody

Exactly.

And mocking a review ( or getting others to do it ) because it is not agreed with it, is not the function of the review section. Proving results is not a part of the reviewing process either. Opinions of lenses do not have to be justified - or agreed with. If somebody feels that the " reviewer " is not qualified to make the review, then so be it, and disregard the review. BUT, to be disrespectful to the reviewer because you disagree with them is not in the spirit of the forums IMO. There is enough room here for ALL opinions.

**aslyfox** · 03-11-2018, 01:33 AM

if you want a true review of a lens - one you will agree to every time - and you live in the US -

spend the cash and rent it if possible

if you can't rent the lens, I hope you can find a review from a photographer who has posted photos here at the forum that you have liked

good luck

beholder3 · 03-11-2018, 02:05 AM

Any "rating" is a "judgement". It is done by a biased single person with opinions about everything. So the rating is purely subjective as well.
Objectivity is simply a myth.
That is why I want people to show me full res images made by the lens. I want to have a look myself. And I always suggest people do the same.

The older guys here will remember the "review" fun with "reviewers" making a big story about the existence of a dedicated video button on cameras. Boy, was I missing the pink camera body color as absolutely essential must have feature.

**Rondec** · 03-11-2018, 02:35 AM - **2 Likes**

Reviews are tough when they are crowd sourced. Steve got it right in his post.

No one gives scores of less than six or seven so essentially (as Sandy says) it turns into a five point scale -- even though ten points are possible. People compare it to their existing lenses. My first "good" lens was an FA 50 f1.4 prime -- way better than my FA-J 28-80 variable aperture zoom and so when I did a review on it, I said it was awesome. I review lenses harder now because I have used more lenses.

People also subconsciously include the price in the equation. If you bought a lens for a thousand dollars, you tend to review it harder than if you got it for a couple hundred or it was twenty-five dollars at a thrift store.

To me, the best part of a review is the narrative -- particularly if it includes photos illustrating the reviewers points. The narrative is the only way to tell whether a reviewer's shooting style mirrors yours and what they really think the strengths and weaknesses of a lens truly are.

03-10-2018, 03:24 PM	#17
Ronald Oakes Veteran Member Join Date: Jul 2017 Location: Virginia Posts: 1,582	How many times have we heard on this Forum ?...."Lens Reviews Are Subjective" ! And the bottom line is.....Yes they are ! Take most reviews with little more than a grain of salt. If 2 people review a inexpensive lens and give it a 10 , and a 9 , that makes it a 9.5 rated lens ! I largely ignore their review. However if 50 people review a lens ? That gives a much better idea how well it performs. I believe the rating somewhat. Last edited by Ronald Oakes; 03-10-2018 at 03:33 PM.

03-10-2018, 04:47 PM - 1 Like	#18
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Numeric user reviews are generally (almost always?) inflated by either inexperience or else a perspective that "I don't buy junk, so it must be good". Here are a few considerations/notes regarding reviews on this site: The numeric scores often reflect a "its the best lens I have" when it is the only lens they have There is some uncertainty about subscores, particularly "aberrations" Look for reviews that discuss the lens in detail in the discussion and which discuss both pros and cons Be suspicious of reviews by users with low post counts and or who flood the site with multiple reviews immediately after registering. Often this is accompanied by auction or marketplace offerings of the items reviewed. My personal practice is to read the comments and ignore the scores My personal practice when writing reviews: Reserve total scores above 8 for truly exceptional lenses Reserve sub-scores above 8 for where the lens truly stands out in that regard Include a detailed description including build quality Include pros Include cons Include pertinent comments on field experience (handling, ease of use, optical quirks) I hope this helps. BTW, I noticed you posted 11 reviews today, all of which are fairly terse. Edit: Terse is OK, though BTW...Welcome to the Pentax Forums! Steve Last edited by stevebrot; 03-10-2018 at 07:22 PM.

03-11-2018, 12:07 AM - 1 Like	#27
Unregistered User 8 Closed Account Join Date: Mar 2015 Photos: Gallery Posts: 8,694	Originally posted by nobody Exactly. And mocking a review ( or getting others to do it ) because it is not agreed with it, is not the function of the review section. Proving results is not a part of the reviewing process either. Opinions of lenses do not have to be justified - or agreed with. If somebody feels that the " reviewer " is not qualified to make the review, then so be it, and disregard the review. BUT, to be disrespectful to the reviewer because you disagree with them is not in the spirit of the forums IMO. There is enough room here for ALL opinions. Last edited by Unregistered User 8; 03-11-2018 at 02:19 AM. Reason: Typo

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Aperture block repair reliability	Cerebum	Repairs and Warranty Service	19	04-13-2018 02:48 AM
Question on repairs to DA 17-70 SDM motors and 16-85 reliability	jddwoods	Pentax SLR Lens Discussion	16	10-29-2017 03:08 PM
Reliability of pentax DSLRs?	pentax360	Pentax DSLR Discussion	70	09-09-2017 07:09 PM
Reliability: K-S1, K-S2 vs K-70?	runswithsizzers	Pentax DSLR Discussion	9	10-24-2016 07:52 AM
K50 reliability issues??	hal55	Pentax DSLR Discussion	16	05-11-2016 02:05 AM

03-10-2018, 02:45 PM	#16
BigMackCam Digitiser of Film Join Date: Mar 2010 Location: North East of England Posts: 20,661	Originally posted by Lens tester When buying Pentax K, M and A lenses, I took into account the grades included in this forum. Unfortunately, in some cases they are very different from those which I obtained by testing the lenses. User reviews and ratings on any website are highly subjective, and influenced by a number of factors - user experience, ability, expectation, use case, other lenses owned, price paid, copy variation etc. Even dedicated review sites will often show considerably divergent results because of different test methodologies, criteria, priorities and test equipment. The best you can do is to read as much as possible on a particular lens and then decide if it's worth trying out for yourself. Typically, I look at three different review websites that I rate reasonably highly, then I read the user reviews here and on other forums. That's normally a good cross-section of information to help me make my choice.

03-10-2018, 05:10 PM	#19
vonBaloney Pentaxian Join Date: Oct 2011 Location: Albuquerque, NM Posts: 6,029	Originally posted by stevebrot: BTW, I noticed you posted 11 reviews today, all of which are fairly terse. Which are now gone? Don't see any from this user. As far as the reviews, you can't "unsee" the numbers of course, but I just read them and pay attention to the ones that sound like they were written by someone who thinks like I do. So many are obviously off-base, or just or praising or criticizing things I don't care about. For modern lenses, there are also other sources where you can look up resolution testing data, etc.

03-10-2018, 05:13 PM	#20
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by vonBaloney Which are now gone? Don't see any from this user. Lens tester's Reviews - Pentax Lens Reviews & Lens Database (...not my usual practice, I was just curious...) Steve

03-10-2018, 05:20 PM	#21
vonBaloney Pentaxian Join Date: Oct 2011 Location: Albuquerque, NM Posts: 6,029	Originally posted by stevebrot Lens tester's Reviews - Pentax Lens Reviews & Lens Database (...not my usual practice, I was just curious...) But how do you find that? If I go to your profile, there is a "latest reviews" tab and I can see your reviews. But not on his profile -- no reviews. If I search in the reviews for lens tester, no results. Maybe that db is only updated once a day or something?

03-10-2018, 11:11 PM	#22
Ronald Oakes Veteran Member Join Date: Jul 2017 Location: Virginia Posts: 1,582	Welcome to the forum Lens Tester ! I just read your review's in the lens section , and see that you Don't recommend the SMC 50 1.4 ? Also see you do not recommend the SMC 135 3.5 either ? Those are two quite reputable lenses , yet they didn't pass your "test" ? Could you disclose your lens testing methods , and possibly post a few supporting images ? Cheers ! ---------- Post added 03-11-18 at 12:13 AM ---------- Originally posted by vonBaloney But how do you find that? If I go to your profile, there is a "latest reviews" tab and I can see your reviews. But not on his profile -- no reviews. If I search in the reviews for lens tester, no results. Maybe that db is only updated once a day or something? Go to the Lens review section , all the latest reviews are there on the right of the main page. I'm sure you will find the latest reviews quite enlightening.

03-10-2018, 11:29 PM - 1 Like	#23
nobody Senior Member Join Date: Aug 2017 Photos: Gallery Posts: 118	In my opinion the number of reviews is the most important thing in the review reliability. The more reviewers the more reliable overall result is. It is because of differences between particular reviewers. Differences within expectation, experience etc. For example some Prinzflex lens has 3 reviews and scores 9.3 points-that is not review I would belive in. On the other hand if SMC Pentax A 50 f1.7 scores 8.99 on the basis of 125 reviews, the score it is reliable for me. It is due to statistics-the bigger sample the more accurate results. No matter how different the samples are.

03-10-2018, 11:35 PM	#24
Unregistered User 8 Closed Account Join Date: Mar 2015 Photos: Gallery Posts: 8,694	Originally posted by nobody In my opinion the number of reviews is the most important thing in the review reliability. The more reviewers the more reliable overall result is. It is because of differences between particular reviewers. Differences within expectation, experience etc. For example some Prinzflex lens has 3 reviews and scores 9.3 points-that is not review I would belive in. On the other hand if SMC Pentax A 50 f1.7 scores 8.99 on the basis of 125 reviews, the score it is reliable for me. It is due to statistics-the bigger sample the more accurate results. No matter how different the samples are. Excellent points, because whether the reviews are " good " or " bad," they become a part of the average, and ALL opinions valid with regards to reviews.

03-10-2018, 11:47 PM	#25
Sandy Hancock Moderator Site Supporter Join Date: Feb 2012 Location: Adelaide Hills, South Australia Photos: Gallery \| Albums Posts: 11,272	Take five off the average rating. Make that a score out of five. That will get you closer to reality. But as others have said, it's still subjective. Read all the reviews and ignore the best and the worst. Then take off five and make it out of five

03-10-2018, 11:48 PM	#26
nobody Senior Member Join Date: Aug 2017 Photos: Gallery Posts: 118	Originally posted by pjv Excellent points, because whether the reviews are " good " or " bad," they become a part of the average, and ALL opinions valid with regards to reviews. Exactly.

03-11-2018, 01:33 AM	#28
aslyfox Seeker of Knowledge Join Date: Aug 2016 Location: Topeka, Kansas Photos: Gallery \| Albums Posts: 24,581	if you want a true review of a lens - one you will agree to every time - and you live in the US - spend the cash and rent it if possible if you can't rent the lens, I hope you can find a review from a photographer who has posted photos here at the forum that you have liked good luck

Thread Tools	Search this Thread
	Search this Thread: Advanced Search

03-11-2018, 02:05 AM	#29
beholder3 Pentaxian Join Date: Dec 2011 Posts: 3,112	Any "rating" is a "judgement". It is done by a biased single person with opinions about everything. So the rating is purely subjective as well. Objectivity is simply a myth. That is why I want people to show me full res images made by the lens. I want to have a look myself. And I always suggest people do the same. The older guys here will remember the "review" fun with "reviewers" making a big story about the existence of a dedicated video button on cameras. Boy, was I missing the pink camera body color as absolutely essential must have feature.

03-11-2018, 02:35 AM - 2 Likes	#30
Rondec Loyal Site Supporter Join Date: Mar 2009 Location: Gladys, Virginia Photos: Gallery Posts: 27,650	Reviews are tough when they are crowd sourced. Steve got it right in his post. No one gives scores of less than six or seven so essentially (as Sandy says) it turns into a five point scale -- even though ten points are possible. People compare it to their existing lenses. My first "good" lens was an FA 50 f1.4 prime -- way better than my FA-J 28-80 variable aperture zoom and so when I did a review on it, I said it was awesome. I review lenses harder now because I have used more lenses. People also subconsciously include the price in the equation. If you bought a lens for a thousand dollars, you tend to review it harder than if you got it for a couple hundred or it was twenty-five dollars at a thrift store. To me, the best part of a review is the narrative -- particularly if it includes photos illustrating the reviewers points. The narrative is the only way to tell whether a reviewer's shooting style mirrors yours and what they really think the strengths and weaknesses of a lens truly are.