The Lens Review Section is Borderline Worthless

secateurs · 11-24-2010, 05:04 PM

Originally posted by GeneV

It seems to me like 30 years ago, when you bought an off-brand lens, which tested as a less than stellar performer, there was more of a chance that the performance was truly abysmal so as to get in the way of your photos. Now, it seems like I can take a lens that legitimately tests "poorly" compared to others, and have to work hard to find the situation in which the difference in the lens resolution or vignetting or CA makes a difference in my photo.

So it seems that you are saying that the user ratings of 8-9 across the board are actually accurate. If there is such little difference between lenses, it must come down to personal preference. So why should anyone ever go for a more expensive lens over a cheaper one? (except maybe for WR?)

Originally posted by Lowell Goudge

- sample/test shots some people want sample shots, but others insist they must be either of subjects they (the reader) like, or museum/art gallery quality to make the review worthwhile and credible - everything is rated 8 or higher. True, but if all people want is gallery/museum quality shots this would be the case also.

I still feel i'm being misrepresented here. In my initial posts I was saying I used sample shots not to judge for myself the quality and issues, but to judge the experience of the photographer/reviewer.

Experience is important to me. A newbie to photography will tend to rate a cheap lens well because that's all they know. An experienced photographer will rate a more expensive/exotic lens well because they know the difference it makes.

Originally posted by Lowell Goudge

Your list above is a good starting point, and when the dust settles from this thread maybe what will come out of it is a guide on writing lens reviews, to be posted as a sticky, in the review database,

I think a Review Guide would be great. Some sort of pro former that can be followed, to make the reviews more uniform. Maybe even a form that is filled out which automatically formats the resulting review.

My favourite suggestion for improving the review section is the ability for users to rate reviews or reviewers, Amazon style.

vario · 11-24-2010, 05:20 PM

I certainly do not know how to fix this issue, but, after spending a week reading review after review of different len's to try and get up to speed on what i may like on my new K5, i can tell you i do not know what to get. When a reviewer tells you that the lens hunts so bad he cannot get the shot, when indoors, then gives the lens a 9 rating. Whats that all about?
Joe

**Ash** · 11-24-2010, 06:30 PM

There's nothing that necessarily needs to be 'fixed' IMO.

If there were individual ratings for value for money, IQ, build quality and utility (incorporating size/weight and handling), then an overall rating could be averaged out from these sub-ratings and get perhaps a more representative overall score for the lens. This may have been mentioned before....

But as with every subjective measure, the worth of a lens is an individual score, and it is up to each user and potential-buyer to consider these things before a reasonable judgement can be made about the value and utility of a lens.

**robbiec** · 11-24-2010, 06:49 PM

Originally posted by vario

I certainly do not know how to fix this issue, but, after spending a week reading review after review of different len's to try and get up to speed on what i may like on my new K5, i can tell you i do not know what to get. When a reviewer tells you that the lens hunts so bad he cannot get the shot, when indoors, then gives the lens a 9 rating. Whats that all about?
Joe

That would be generally down to the camera used. You'll find the K-5 very decisive in that regard. The 9 might be because when he /she does manage to get it into focus it delivers a pleasing image

**Lowell Goudge** · 11-24-2010, 07:12 PM

Sacateurs

Regarding rating the reviews the only way I see is helpful on a scale 1-5 as you indicate, or to show the reviewers rating points

Regarding the photos if you only want super experienced reviewers I am afraid you will limit the reviewers so badly that it will also limit the lenses reviewed

secateurs · 11-25-2010, 12:04 AM

Originally posted by Lowell Goudge

Regarding the photos if you only want super experienced reviewers I am afraid you will limit the reviewers so badly that it will also limit the lenses reviewed

Good point. I don't necessarily want only "proven" reviewers. I will just weigh their opinions more heavily.

Inexperienced photographers / reviewers have to start somewhere!

Adam · 11-25-2010, 12:12 AM

FYI, we will be bringing in a lot more professional reviews, so you'll have both a in-depth review available alongside consumer reviews for many lenses

Adam
PentaxForums.com Webmaster (Site Usage Guide | Site Help | My Photography)

PentaxForums.com server and development costs are user-supported. You can help cover these costs by donating or purchasing one of our Pentax eBooks. Or, buy your photo gear from our affiliates, Adorama, B&H Photo, KEH, or Topaz Labs, and get FREE Marketplace access - click here to see how! Trusted Pentax retailers:

stevebrot · 11-25-2010, 08:57 AM

Originally posted by Lowell Goudge

+/- compared to a "gold standard". Perhaps setting the K series pentax lenses at 5.

I guess that would result in no lens being rated higher than a 5.5 with most being rated a 3

Steve

**Lowell Goudge** · 11-25-2010, 09:12 AM

Originally posted by stevebrot

I guess that would result in no lens being rated higher than a 5.5 with most being rated a 3

Steve

Yeah, I hear you

given the present rating practices where the K's are all 9's and no one rates greater than 9.5, or lower than about 7.,

But the idea is to leave the scale open ended so that people could expand just a little between 9.0 and 9.5

That is, of course they don;t have tunnel vision

Nick Siebers · 11-25-2010, 09:56 AM

Or perhaps we should shift the ratings, so you can range a lens from 9.0 to 10 by 0.1s. A lens of 9.3 sounds so much better than a 3! ;-)

bobmaxja · 11-25-2010, 11:03 AM

It's human nature to never said I made a bad choice because... peoples want to sell it and they dont want the new buyer to know

**Ash** · 11-25-2010, 11:35 AM

Lens ratings do not and should not need to fit any normal distribution of scores. The bell curve is useful for standardisation, but these are all absolute not relative ratings of user satisfaction.

**Ash** · 11-25-2010, 11:52 AM

Originally posted by secateurs

So it seems that you are saying that the user ratings of 8-9 across the board are actually accurate. If there is such little difference between lenses, it must come down to personal preference. So why should anyone ever go for a more expensive lens over a cheaper one? (except maybe for WR?)

Lots of reasons to go for a 'more expensive' lens, which should be obvious to the user.
Anyone who thinks two lenses that equally score 9 averages but differ greatly in market price are the same probably has little appreciation of what the more expensive lens can do, so the cheaper one would suffice for him/her quite fine.

DA 55-300 currently at 8.95 vs. Sigma 70-200 f/2.8 II HSM currently at 8.58 - two totally different lenses at different price points despite overlapping focal ranges.

Then you have two optically similar lenses like the 18-55 II and 18-55 WR that score quite differently - the WR apparently less appealing according to the average rating!

That's why I suggest individual ratings for value, IQ, build and utility - then average these for each user's review and then average each of these averages and there may be a more representative score for each lens.

**Ron Boggs** · 11-25-2010, 11:57 AM

Not so sure the intent of the reviews has ever been to provide a statistical comparison that can be graphed and analyzed.

To me, as a forum-based review section it's really just a compilation of opinions and impressions that may provide some personal insight for readers.

Most of the reviews I've added were of somewhat uncommon lenses that folks would like to know more about. My impressions have been offered as impressions not as tests or objective measures. The more rare the lens, the more important it is that somebody out there reports on it.

**Marc Langille** · 11-25-2010, 12:00 PM

Originally posted by Ron Boggs

Not so sure the intent of the reviews has ever been to provide a statistical comparison that can be graphed and analyzed.

To me, as a forum-based review section it's really just a compilation of opinions and impressions that may provide some personal insight for readers.

Most of the reviews I've added were of somewhat uncommon lenses that folks would like to know more about. My impressions have been offered as impressions not as tests or objective measures. The more rare the lens, the more important it is that somebody out there reports on it.

+1....

Ron, hope you are doing well!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
K-7 Speculations (moved over from review section)	sctybear	Pentax DSLR Discussion	15	05-07-2010 05:48 PM
Suggestion Bag Review Section	lurchlarson	Site Suggestions and Help	11	04-24-2010 01:07 PM
Suggestion Lens Review Database section	Ivan Glisin	Site Suggestions and Help	9	05-13-2008 05:55 PM
Lens review section: adding DA 18-55 II?	Ivan Glisin	Site Suggestions and Help	2	04-04-2008 05:06 AM
DPreview launches lens review section - starts with Pentax (and some other misc brand	mattdm	Pentax News and Rumors	14	01-30-2008 04:48 AM

11-24-2010, 05:04 PM	#136
secateurs Senior Member Join Date: Oct 2010 Location: Adelaide, Australia Photos: Albums Posts: 250	Originally posted by GeneV It seems to me like 30 years ago, when you bought an off-brand lens, which tested as a less than stellar performer, there was more of a chance that the performance was truly abysmal so as to get in the way of your photos. Now, it seems like I can take a lens that legitimately tests "poorly" compared to others, and have to work hard to find the situation in which the difference in the lens resolution or vignetting or CA makes a difference in my photo. So it seems that you are saying that the user ratings of 8-9 across the board are actually accurate. If there is such little difference between lenses, it must come down to personal preference. So why should anyone ever go for a more expensive lens over a cheaper one? (except maybe for WR?) Originally posted by Lowell Goudge - sample/test shots some people want sample shots, but others insist they must be either of subjects they (the reader) like, or museum/art gallery quality to make the review worthwhile and credible - everything is rated 8 or higher. True, but if all people want is gallery/museum quality shots this would be the case also. I still feel i'm being misrepresented here. In my initial posts I was saying I used sample shots not to judge for myself the quality and issues, but to judge the experience of the photographer/reviewer. Experience is important to me. A newbie to photography will tend to rate a cheap lens well because that's all they know. An experienced photographer will rate a more expensive/exotic lens well because they know the difference it makes. Originally posted by Lowell Goudge Your list above is a good starting point, and when the dust settles from this thread maybe what will come out of it is a guide on writing lens reviews, to be posted as a sticky, in the review database, I think a Review Guide would be great. Some sort of pro former that can be followed, to make the reviews more uniform. Maybe even a form that is filled out which automatically formats the resulting review. My favourite suggestion for improving the review section is the ability for users to rate reviews or reviewers, Amazon style.

11-24-2010, 05:20 PM	#137
vario Senior Member Join Date: Mar 2010 Photos: Albums Posts: 114	I certainly do not know how to fix this issue, but, after spending a week reading review after review of different len's to try and get up to speed on what i may like on my new K5, i can tell you i do not know what to get. When a reviewer tells you that the lens hunts so bad he cannot get the shot, when indoors, then gives the lens a 9 rating. Whats that all about? Joe

11-24-2010, 06:30 PM	#138
Ash Loyal Site Supporter Join Date: Mar 2007 Location: Toowoomba, Queensland Photos: Gallery \| Albums Posts: 23,920	There's nothing that necessarily needs to be 'fixed' IMO. If there were individual ratings for value for money, IQ, build quality and utility (incorporating size/weight and handling), then an overall rating could be averaged out from these sub-ratings and get *perhaps* a more representative overall score for the lens. This may have been mentioned before.... But as with every subjective measure, the worth of a lens is an individual score, and it is up to each user and potential-buyer to consider these things before a reasonable judgement can be made about the value and utility of a lens.

11-24-2010, 06:49 PM	#139
robbiec Site Supporter Join Date: Feb 2009 Location: Cork Photos: Albums Posts: 1,882	Originally posted by vario I certainly do not know how to fix this issue, but, after spending a week reading review after review of different len's to try and get up to speed on what i may like on my new K5, i can tell you i do not know what to get. When a reviewer tells you that the lens hunts so bad he cannot get the shot, when indoors, then gives the lens a 9 rating. Whats that all about? Joe That would be generally down to the camera used. You'll find the K-5 very decisive in that regard. The 9 might be because when he /she does manage to get it into focus it delivers a pleasing image

11-24-2010, 07:12 PM	#140
Lowell Goudge Site Supporter Join Date: Jan 2007 Location: Toronto Photos: Gallery \| Albums Posts: 17,886	Sacateurs Regarding rating the reviews the only way I see is helpful on a scale 1-5 as you indicate, or to show the reviewers rating points Regarding the photos if you only want super experienced reviewers I am afraid you will limit the reviewers so badly that it will also limit the lenses reviewed

11-25-2010, 12:04 AM	#141
secateurs Senior Member Join Date: Oct 2010 Location: Adelaide, Australia Photos: Albums Posts: 250	Originally posted by Lowell Goudge Regarding the photos if you only want super experienced reviewers I am afraid you will limit the reviewers so badly that it will also limit the lenses reviewed Good point. I don't necessarily want only "proven" reviewers. I will just weigh their opinions more heavily. Inexperienced photographers / reviewers have to start somewhere!

11-25-2010, 12:12 AM	#142
Adam Administrator Site Webmaster Join Date: Sep 2006 Location: Arizona Photos: Gallery \| Albums Posts: 51,594	FYI, we will be bringing in a lot more professional reviews, so you'll have both a in-depth review available alongside consumer reviews for many lenses Adam PentaxForums.com Webmaster (Site Usage Guide \| Site Help \| My Photography) PentaxForums.com server and development costs are user-supported. You can help cover these costs by donating or purchasing one of our Pentax eBooks. Or, buy your photo gear from our affiliates, Adorama, B&H Photo, KEH, or Topaz Labs, and get FREE Marketplace access - click here to see how! Trusted Pentax retailers:

11-25-2010, 08:57 AM	#143
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by Lowell Goudge +/- compared to a "gold standard". Perhaps setting the K series pentax lenses at 5. I guess that would result in no lens being rated higher than a 5.5 with most being rated a 3 Steve

11-25-2010, 09:12 AM	#144
Lowell Goudge Site Supporter Join Date: Jan 2007 Location: Toronto Photos: Gallery \| Albums Posts: 17,886	Originally posted by stevebrot I guess that would result in no lens being rated higher than a 5.5 with most being rated a 3 Steve Yeah, I hear you given the present rating practices where the K's are all 9's and no one rates greater than 9.5, or lower than about 7., But the idea is to leave the scale open ended so that people could expand just a little between 9.0 and 9.5 That is, of course they don;t have tunnel vision

11-25-2010, 09:56 AM	#145
Nick Siebers Veteran Member Join Date: Jun 2008 Location: Madison, WI Photos: Gallery \| Albums Posts: 2,165	Or perhaps we should shift the ratings, so you can range a lens from 9.0 to 10 by 0.1s. A lens of 9.3 sounds so much better than a 3! ;-)

Thread Tools	Search this Thread
	Search this Thread: Advanced Search

11-25-2010, 11:03 AM	#146
bobmaxja Veteran Member Join Date: Mar 2010 Location: Laval, Quebec Canada Posts: 2,171	It's human nature to never said I made a bad choice because... peoples want to sell it and they dont want the new buyer to know

11-25-2010, 11:35 AM	#147
Ash Loyal Site Supporter Join Date: Mar 2007 Location: Toowoomba, Queensland Photos: Gallery \| Albums Posts: 23,920	Lens ratings do not and should not need to fit any normal distribution of scores. The bell curve is useful for standardisation, but these are all absolute not relative ratings of user satisfaction.

11-25-2010, 11:52 AM	#148
Ash Loyal Site Supporter Join Date: Mar 2007 Location: Toowoomba, Queensland Photos: Gallery \| Albums Posts: 23,920	Originally posted by secateurs So it seems that you are saying that the user ratings of 8-9 across the board are actually accurate. If there is such little difference between lenses, it must come down to personal preference. So why should anyone ever go for a more expensive lens over a cheaper one? (except maybe for WR?) Lots of reasons to go for a 'more expensive' lens, which should be obvious to the user. Anyone who thinks two lenses that equally score 9 averages but differ greatly in market price are the same probably has little appreciation of what the more expensive lens can do, so the cheaper one would suffice for him/her quite fine. DA 55-300 currently at 8.95 vs. Sigma 70-200 f/2.8 II HSM currently at 8.58 - two totally different lenses at different price points despite overlapping focal ranges. Then you have two optically similar lenses like the 18-55 II and 18-55 WR that score quite differently - the WR apparently less appealing according to the average rating! That's why I suggest individual ratings for value, IQ, build and utility - then average these for each user's review and then average each of these averages and there may be a more representative score for each lens.

11-25-2010, 11:57 AM	#149
Ron Boggs Site Supporter Join Date: Feb 2008 Location: MT Photos: Gallery Posts: 1,349	Not so sure the intent of the reviews has ever been to provide a statistical comparison that can be graphed and analyzed. To me, as a forum-based review section it's really just a compilation of opinions and impressions that may provide some personal insight for readers. Most of the reviews I've added were of somewhat uncommon lenses that folks would like to know more about. My impressions have been offered as impressions not as tests or objective measures. The more rare the lens, the more important it is that somebody out there reports on it.

11-25-2010, 12:00 PM	#150
Marc Langille Site Supporter Join Date: Dec 2006 Location: NW Arkansas, USA Posts: 4,708	Originally posted by Ron Boggs Not so sure the intent of the reviews has ever been to provide a statistical comparison that can be graphed and analyzed. To me, as a forum-based review section it's really just a compilation of opinions and impressions that may provide some personal insight for readers. Most of the reviews I've added were of somewhat uncommon lenses that folks would like to know more about. My impressions have been offered as impressions not as tests or objective measures. The more rare the lens, the more important it is that somebody out there reports on it. +1.... Ron, hope you are doing well!