The Lens Review Section is Borderline Worthless

paperbag846 · 11-18-2010, 01:29 PM

Originally posted by Hannican

I don't care if a lens costs $10,000 or $100 though, I want to know which one performs the best!

What you say makes a lot of sense... a 100 dollar lens that does not let you do what you want to do is far more expensive than a 1000 dollar lens you get use out of every day!

I find the only real solution is to painstakingly read all the reviews... and look at sample photos taken by each lens. It's much more helpful than a numerical value. Besides... a lens might be a 10 for portraits, but only a 6 for landscapes. Or something.\

I think thats why we see so many "this lens vs. that lens" posts - it generates a far more helpful response (often) then the reviews do.

Ecaterin · 11-18-2010, 01:31 PM

Originally posted by atupdate

Don't think of the rating system as a lens quality rating. There is no standard for rating the quality against. Therefore, you need to consider it as a value rating (IQ for the price/weight/focal range).

Tim

This is my perception as well. And many reviews state that explicitly - "In it's class, it's fantastic. So what if it's not a Limited Prime, I didn't want to spend $500." It hardly seems fair to rate the FA100-300 against the DAF100WR - but for their respective prices, that FA100-300 is a great little lens. Apples and oranges can't be judged on the same rating scale - IQ related to value makes sense.

When it comes to any review system, I give only a passing glance to the ratings (throwing out the highest and lowest ratings automatically) and read each review and scare up as many photos from that lens as possible (pixel-peeper=lots of samples). Even if someone rates a lens a "9," there's no counting on whether they value what I value. There are many reviews out there proclaiming a lens sharp! contrasty! purple fringe, but software cures that! I HATE PF, I won't use a lens with it, no matter how gorgeous it is otherwise. So that '9' isn't very meaningful to me.

Descriptions & photos with the lens. That's what I want - and our review section tends to have nice, long, specific descriptions. Makes me happy

opfor · 11-18-2010, 01:48 PM

As has been pointed out, what I might think of as a good lens may not be the same for you. For the reviews I have done, I have always tried to include pics. I think pics say more than numbers....

Ira · 11-18-2010, 02:05 PM

Originally posted by paperbag846

The ratings themselves basicially tell you if the owner is happy, not whether lens A is better than lens B for your needs. That would be gathered from the comments.

For example, the K55 1.8 has pretty much the same rating as the FA 43 LTD. That is preposterous. However if you were to read the comments, it would not take you long to figure out in which areas the FA 43 pulls far ahead of the K55.

User satisfaction also has a lot to do with price... I doubt the K55 would have a similar rating as the FA 43 if they cost the same amount.

Very, very well said.

**robbiec** · 11-18-2010, 02:40 PM

Originally posted by paperbag846

The ratings themselves basicially tell you if the owner is happy, not whether lens A is better than lens B for your needs. That would be gathered from the comments.

For example, the K55 1.8 has pretty much the same rating as the FA 43 LTD. That is preposterous. However if you were to read the comments, it would not take you long to figure out in which areas the FA 43 pulls far ahead of the K55.

User satisfaction also has a lot to do with price... I doubt the K55 would have a similar rating as the FA 43 if they cost the same amount.

The K55 punches way above its weight price wise IMO and it actually has nicer bokeh than the 43

I have both the FA43 and K55 and both have their uses and about equal time on the camera. I gave the 55 a 9 and the 43 a 10, conversely I gave the FA50 f/1.4 a 7 because it really needs a good hood and the build and tactile feel is horrible. The thread made me rethink some of my reviews and I dropped the Super Takumar 85 to a 7 because its getting too expensive compared to an FA77 at an easy 10 which is only a couple of hundred dollars more 2nd hand, harsh? perhaps but that is reality.

GhoSStrider · 11-18-2010, 02:51 PM

Originally posted by robbiec

The thread made me rethink some of my reviews and I dropped the Super Takumar 85 to a 7 ...

Oh man...you might be burned in effigy for even suggesting such a thing!

paperbag846 · 11-18-2010, 02:58 PM

Originally posted by robbiec

The K55 punches way above its weight price wise IMO and it actually has nicer bokeh than the 43

Wow! I feel lucky

. I knew the K55 outperformed it's price class, but I am surprised to hear you say the bokeh of the K55 is nicer than the 43! I REALLY like the bokeh of the K55 up till around f4, but the highlights turn into strong hexagons at that point, which is why I prefer more aperture blades.

paperbag846 · 11-18-2010, 03:00 PM

Originally posted by robbiec

Super Takumar 85 to a 7 because its getting too expensive compared to an FA77

Now I have an idea... why don't we all go in and rate the really good lenses 2's. This forum's review section is the first thing that pops up when you type in "pentax lens review", so maybe that will drive the prices of these really good lenses down

.

**robbiec** · 11-18-2010, 03:17 PM

Originally posted by paperbag846

Quote Originally posted by robbiec Quote
Super Takumar 85 to a 7 because its getting too expensive compared to an FA77
Now I have an idea... why don't we all go in and rate the really good lenses 2's. This forum's review section is the first thing that pops up when you type in "pentax lens review", so maybe that will drive the prices of these really good lenses down .

Instead of punishing your brain thinking up witty sarcasm how about following my lead and making the lens review section more real time. Super Takumars generally are going up in price, perhaps due to the Takumar thread, perhaps due to the ease of getting adapters, higher prices mean that they now should be scored harder. e.g. a Super Takumar 85 that was €200 last year verses €350 this year should be rated more critically.
And it is a really good lens, just not a great one.

KxBlaze · 11-18-2010, 03:50 PM

Originally posted by GhoSStrider

I think part of the problem is that it's not an absolute scale for people. For example. I think the tendency is to rate a superzoom against the superzoom category. The 18-250 seems to get a lot of love and rates very well. Is it even remotely close to the performance of an FA 31, though? Not a chance. So as an absolute, it's rating should probably be at least a few points lower than the 8.6 average it's currently holding. However, compared to the superzoom class, I think the rating fits pretty well, and that's how I read it.

While it's not a perfect system, I think it works at least for me. I don't look at the numerics too much anyways. The comments are of far greater value to me.

Agreed. All of the reviews I have done are based on the "class" of the lens and the $ I paid for it. Perfect example if the Tamron 70-300. There are a ton of much better telephoto lenses out there but they're all 100s of dollars more (some even 7-8x as much) so for $150 I gave that lens an 8. I don't see anything wrong with that rating. You need to look at the rating and then read the persons comments to find out why they gave it that rating. A rating alone means nothing, but with a description is it very beneficial. It would be impossible to have a universal rating system for every single lens ever made. Exactly like the quote above me stated, the DA 18-250 cannot even come close to the IQ of the FA31 but does that mean the 18-250 should get a rating of 5 while the FA31 gets a 10? I think the review section is good the way it is, people just need to know how to read it.

Parallax · 11-18-2010, 04:07 PM

Originally posted by robbiec

............... The thread made me rethink some of my reviews and I dropped the Super Takumar 85 to a 7 ..............

Somebody get a rope!

**rparmar** · 11-18-2010, 04:36 PM

The review section is like anything else done through "democratic" principles: the results are fictive, inflated and even damaging. But the participants get that warm glow of satisfaction from "having their say", as if that, in and of itself, is an accomplishment.

**robbiec** · 11-18-2010, 04:38 PM

Originally posted by Parallax

Somebody get a rope!

hehehe, I knew that might ruffle a few feathers... so far a burning effigy and now a rope...

What would I get if I said I preferred the f/1.9 version over the f/1.8 version and that is even more expensive and going totally nuts price wise... worth a 6 maybe?

**gofour3** · 11-18-2010, 04:45 PM

I have done over 30 lens reviews in the K series prime section.
The rating I assigned for each lens is based on the optics, value and overall usage against other K series primes. I think the lowest rating I gave was an 8, as all K series are very good lenses and well built.

However rating a K55/1.8 against a newer lens made for a DSLR would be pretty hard as one was designed for a FF film camera and one for APS-C sensor.

Phil.

Nick Siebers · 11-18-2010, 04:46 PM

Well, reviewers have taken their time to provide you with information, that you get for free via Pentax forums. It seems like poor form to bad mouth the service. "I don't like these free cookies!" --> then don't eat them.

I can see how someone might want to encourage reviewers to be more helpful to them, but a positive approach might be more successful. Even so, good luck.

I try to avoid reviewing lenses while I own them, unless I really love them. I would hate to drag down the rating of a lens I might sell, and wouldn't want to inflate my rating to serve myself. I am sure these factors matter to others as well.

11-18-2010, 02:40 PM	#20
robbiec Site Supporter Join Date: Feb 2009 Location: Cork Photos: Albums Posts: 1,882	Originally posted by paperbag846 The ratings themselves basicially tell you if the owner is happy, not whether lens A is better than lens B for your needs. That would be gathered from the comments. For example, the K55 1.8 has pretty much the same rating as the FA 43 LTD. That is preposterous. However if you were to read the comments, it would not take you long to figure out in which areas the FA 43 pulls far ahead of the K55. User satisfaction also has a lot to do with price... I doubt the K55 would have a similar rating as the FA 43 if they cost the same amount. The K55 punches way above its weight price wise IMO and it actually has nicer bokeh than the 43 I have both the FA43 and K55 and both have their uses and about equal time on the camera. I gave the 55 a 9 and the 43 a 10, conversely I gave the FA50 f/1.4 a 7 because it really needs a good hood and the build and tactile feel is horrible. The thread made me rethink some of my reviews and I dropped the Super Takumar 85 to a 7 because its getting too expensive compared to an FA77 at an easy 10 which is only a couple of hundred dollars more 2nd hand, harsh? perhaps but that is reality. Last edited by robbiec; 11-18-2010 at 02:50 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
K-7 Speculations (moved over from review section)	sctybear	Pentax DSLR Discussion	15	05-07-2010 05:48 PM
Suggestion Bag Review Section	lurchlarson	Site Suggestions and Help	11	04-24-2010 01:07 PM
Suggestion Lens Review Database section	Ivan Glisin	Site Suggestions and Help	9	05-13-2008 05:55 PM
Lens review section: adding DA 18-55 II?	Ivan Glisin	Site Suggestions and Help	2	04-04-2008 05:06 AM
DPreview launches lens review section - starts with Pentax (and some other misc brand	mattdm	Pentax News and Rumors	14	01-30-2008 04:48 AM

11-18-2010, 01:29 PM	#16
paperbag846 Veteran Member Join Date: Jul 2010 Photos: Gallery Posts: 2,395	Originally posted by Hannican I don't care if a lens costs $10,000 or $100 though, I want to know which one performs the best! What you say makes a lot of sense... a 100 dollar lens that does not let you do what you want to do is far more expensive than a 1000 dollar lens you get use out of every day! I find the only real solution is to painstakingly read all the reviews... and look at sample photos taken by each lens. It's much more helpful than a numerical value. Besides... a lens might be a 10 for portraits, but only a 6 for landscapes. Or something.\ I think thats why we see so many "this lens vs. that lens" posts - it generates a far more helpful response (often) then the reviews do.

11-18-2010, 01:31 PM	#17
Ecaterin Veteran Member Join Date: Nov 2009 Location: Seattle Photos: Gallery \| Albums Posts: 723	Originally posted by atupdate Don't think of the rating system as a lens quality rating. There is no standard for rating the quality against. Therefore, you need to consider it as a value rating (IQ for the price/weight/focal range). Tim This is my perception as well. And many reviews state that explicitly - "In it's class, it's fantastic. So what if it's not a Limited Prime, I didn't want to spend $500." It hardly seems fair to rate the FA100-300 against the DAF100WR - but for their respective prices, that FA100-300 is a great little lens. Apples and oranges can't be judged on the same rating scale - IQ related to value makes sense. When it comes to any review system, I give only a passing glance to the ratings (throwing out the highest and lowest ratings automatically) and read each review and scare up as many photos from that lens as possible (pixel-peeper=lots of samples). Even if someone rates a lens a "9," there's no counting on whether they value what I value. There are many reviews out there proclaiming a lens sharp! contrasty! purple fringe, but software cures that! I HATE PF, I won't use a lens with it, no matter how gorgeous it is otherwise. So that '9' isn't very meaningful to me. Descriptions & photos with the lens. That's what I want - and our review section tends to have nice, long, specific descriptions. Makes me happy

11-18-2010, 01:48 PM	#18
opfor Senior Member Join Date: Dec 2008 Location: Oxford, UK Photos: Albums Posts: 276	As has been pointed out, what I might think of as a good lens may not be the same for you. For the reviews I have done, I have always tried to include pics. I think pics say more than numbers....

11-18-2010, 02:05 PM	#19
Ira Inactive Account Join Date: Mar 2009 Location: Coral Springs, FL Photos: Gallery \| Albums Posts: 5,216	Originally posted by paperbag846 The ratings themselves basicially tell you if the owner is happy, not whether lens A is better than lens B for your needs. That would be gathered from the comments. For example, the K55 1.8 has pretty much the same rating as the FA 43 LTD. That is preposterous. However if you were to read the comments, it would not take you long to figure out in which areas the FA 43 pulls far ahead of the K55. User satisfaction also has a lot to do with price... I doubt the K55 would have a similar rating as the FA 43 if they cost the same amount. Very, very well said.

11-18-2010, 02:51 PM	#21
GhoSStrider Veteran Member Join Date: Jul 2009 Location: 14er Country Photos: Gallery Posts: 323	Originally posted by robbiec The thread made me rethink some of my reviews and I dropped the Super Takumar 85 to a 7 ... Oh man...you might be burned in effigy for even suggesting such a thing!

11-18-2010, 02:58 PM	#22
paperbag846 Veteran Member Join Date: Jul 2010 Photos: Gallery Posts: 2,395	Originally posted by robbiec The K55 punches way above its weight price wise IMO and it actually has nicer bokeh than the 43 Wow! I feel lucky . I knew the K55 outperformed it's price class, but I am surprised to hear you say the bokeh of the K55 is nicer than the 43! I REALLY like the bokeh of the K55 up till around f4, but the highlights turn into strong hexagons at that point, which is why I prefer more aperture blades.

11-18-2010, 03:00 PM	#23
paperbag846 Veteran Member Join Date: Jul 2010 Photos: Gallery Posts: 2,395	Originally posted by robbiec Super Takumar 85 to a 7 because its getting too expensive compared to an FA77 Now I have an idea... why don't we all go in and rate the really good lenses 2's. This forum's review section is the first thing that pops up when you type in "pentax lens review", so maybe that will drive the prices of these really good lenses down .

11-18-2010, 03:17 PM	#24
robbiec Site Supporter Join Date: Feb 2009 Location: Cork Photos: Albums Posts: 1,882	Originally posted by paperbag846 Quote Originally posted by robbiec Quote Super Takumar 85 to a 7 because its getting too expensive compared to an FA77 Now I have an idea... why don't we all go in and rate the really good lenses 2's. This forum's review section is the first thing that pops up when you type in "pentax lens review", so maybe that will drive the prices of these really good lenses down . Instead of punishing your brain thinking up witty sarcasm how about following my lead and making the lens review section more real time. Super Takumars generally are going up in price, perhaps due to the Takumar thread, perhaps due to the ease of getting adapters, higher prices mean that they now should be scored harder. e.g. a Super Takumar 85 that was €200 last year verses €350 this year should be rated more critically. And it is a really good lens, just not a great one.

11-18-2010, 03:50 PM	#25
KxBlaze Veteran Member Join Date: Feb 2010 Location: California Photos: Gallery Posts: 1,602	Originally posted by GhoSStrider I think part of the problem is that it's not an absolute scale for people. For example. I think the tendency is to rate a superzoom against the superzoom category. The 18-250 seems to get a lot of love and rates very well. Is it even remotely close to the performance of an FA 31, though? Not a chance. So as an absolute, it's rating should probably be at least a few points lower than the 8.6 average it's currently holding. However, compared to the superzoom class, I think the rating fits pretty well, and that's how I read it. While it's not a perfect system, I think it works at least for me. I don't look at the numerics too much anyways. The comments are of far greater value to me. Agreed. All of the reviews I have done are based on the "class" of the lens and the $ I paid for it. Perfect example if the Tamron 70-300. There are a ton of much better telephoto lenses out there but they're all 100s of dollars more (some even 7-8x as much) so for $150 I gave that lens an 8. I don't see anything wrong with that rating. You need to look at the rating and then read the persons comments to find out why they gave it that rating. A rating alone means nothing, but with a description is it very beneficial. It would be impossible to have a universal rating system for every single lens ever made. Exactly like the quote above me stated, the DA 18-250 cannot even come close to the IQ of the FA31 but does that mean the 18-250 should get a rating of 5 while the FA31 gets a 10? I think the review section is good the way it is, people just need to know how to read it.

11-18-2010, 04:07 PM	#26
Parallax Senior Moderator Loyal Site Supporter Join Date: Apr 2008 Location: South Dakota Photos: Gallery Posts: 19,332	Originally posted by robbiec ............... The thread made me rethink some of my reviews and I dropped the Super Takumar 85 to a 7 .............. Somebody get a rope!

Thread Tools	Search this Thread
	Search this Thread: Advanced Search

11-18-2010, 04:36 PM	#27
rparmar Site Supporter Join Date: Jan 2008 Photos: Gallery Posts: 8,817	The review section is like anything else done through "democratic" principles: the results are fictive, inflated and even damaging. But the participants get that warm glow of satisfaction from "having their say", as if that, in and of itself, is an accomplishment.

11-18-2010, 04:38 PM	#28
robbiec Site Supporter Join Date: Feb 2009 Location: Cork Photos: Albums Posts: 1,882	Originally posted by Parallax Somebody get a rope! hehehe, I knew that might ruffle a few feathers... so far a burning effigy and now a rope... What would I get if I said I preferred the f/1.9 version over the f/1.8 version and that is even more expensive and going totally nuts price wise... worth a 6 maybe?

11-18-2010, 04:45 PM	#29
gofour3 Site Supporter Join Date: Oct 2008 Location: Vancouver, Canada Posts: 8,088	I have done over 30 lens reviews in the K series prime section. The rating I assigned for each lens is based on the optics, value and overall usage against other K series primes. I think the lowest rating I gave was an 8, as all K series are very good lenses and well built. However rating a K55/1.8 against a newer lens made for a DSLR would be pretty hard as one was designed for a FF film camera and one for APS-C sensor. Phil.

11-18-2010, 04:46 PM	#30
Nick Siebers Veteran Member Join Date: Jun 2008 Location: Madison, WI Photos: Gallery \| Albums Posts: 2,165	Well, reviewers have taken their time to provide you with information, that you get for free via Pentax forums. It seems like poor form to bad mouth the service. "I don't like these free cookies!" --> then don't eat them. I can see how someone might want to encourage reviewers to be more helpful to them, but a positive approach might be more successful. Even so, good luck. I try to avoid reviewing lenses while I own them, unless I really love them. I would hate to drag down the rating of a lens I might sell, and wouldn't want to inflate my rating to serve myself. I am sure these factors matter to others as well.