The Lens Review Section is Borderline Worthless

pinholecam · 11-20-2010, 06:29 AM

It cuts both ways.
Some reviews put a lens down cause 'There is no 10 in my books'; '9+ is revered for the FA31ltd' ; etc does not make sense as well.
Put down my M85/2 from 9 to 7 because its "not an AF lens"; "Not f1.4" is plain silly imho. I am reviewing a lens in its own context and not wrt others.

GeneV · 11-20-2010, 06:40 AM

I feel that information about the price or the rating is useful in an individual review. The rating helps understand something about how enthusiastic the reviewer may be, and the price paid may shed some light on why. More information is always better.

However, where the price and ratings may go astray and become "borderline worthless" is in the averages. You can't really have a very meaningful average or even a median score from very different individuals with very different needs and uses. You can't have a meaningful average or median price where different individuals are purchasing the lens in different decades and on different continents. Perhaps some of the concerns could be alleviated by simply eliminating or de-emphasizing the averages.

Blue · 11-20-2010, 06:46 AM

Originally posted by Hannican

At least the part about "Rating" has become essentially a waste of time. Why does everyone keep giving all of their lenses 9s and 10s? Look at the K mount Primes, or the DA Primes sections. EVERYTHING is a 9 or higher.

People will even complain about things within the comments, but then give their lenses a 9 or a 10 anyway. I don't get it... if we're not going to tell the truth about our lenses, or if every lens really IS a 9 or a 10, then what's the point of even continuing to review them in this way?

The commentary part, and especially the sample pics that people post, are still excellent, and incredibly useful, but I think it's time we bring a little more truth back to this section (I don't remember it being this way a couple years back... there were lots of 7s and 8s two years ago).

A Concerned Forum User,

-Tim

Actually, you are incorrect. There are also idiots that talk about how good a lens is then give it a 5 or 6. I guess they have ego driven complexes that makes them think that being over critical makes them some sort of internet bad ass. There are plenty of 7 and 8 ratings in there. Plus, there are more than one review. While photozone does in depth reviews on lenses, it is done on a single specimen. Plus, they don't always compare lenses properly. For, example, they torture tested the FA 77mm/1.8 and got purple fringing. They did not torture test the DA 70mm/2.4 and proclaimed it to be superior in that regard even though even the Zeiss 85/1.4 would p.f. under that same torture test.

I look at the lens review system here as a starting point and realize it is based on peoples experiences and opinions. The other issue is what are you rating a lens type and class to? (Rhetorical)

You do thinks your way and I'll do things my way and one of us will be happy.

sarcasm

**robbiec** · 11-20-2010, 07:28 AM

Originally posted by robbiec

hehehe, I knew that might ruffle a few feathers... so far a burning effigy and now a rope...

What would I get if I said I preferred the f/1.9 version over the f/1.8 version and that is even more expensive and going totally nuts price wise... worth a 6 maybe?

Originally posted by Parallax

Are you familiar with the phrase "castration with a rusty razor blade"?

Put the 2 Super Takumars 85s at 8. But I did have a dig at the pricing and compared them unfavourably with the FA77, (same telephoto class) which I believe I am allowed to do, so you can put away the lynch mob!

**Lowell Goudge** · 11-20-2010, 07:53 AM

Originally posted by pinholecam

It cuts both ways.
Some reviews put a lens down cause 'There is no 10 in my books'; '9+ is revered for the FA31ltd' ; etc does not make sense as well.
Put down my M85/2 from 9 to 7 because its "not an AF lens"; "Not f1.4" is plain silly imho. I am reviewing a lens in its own context and not wrt others.

While I don't disagree with this point I.e. The review and rating is all within context then if I pay $200 for a super tak 85F1.9 in context of price performance I might rate it a 9 but if I pay$400 for it maybe it is only a 7?

To me that is part of the equation

macTak · 11-20-2010, 06:55 PM

Originally posted by Docrwm

It might improve things on the Review section if there were 3 ratings: Build Quality, Optical Quality, and Value for Money.

I agree that this would make some sense. Of course, some people already do it and state it in their reivews. But overall, numbers simply don't mean much without context or a fixed scale (not a scale that differs with every reviewer). The bulk of the review section is in the written reviews themselves, which often are very helpful and informative. It's the quickest, easiest way to get information on the earlier AF and all the MF Pentax lenses, so don't worry about the ratings too much--you cant really compare lenses with them (heck, the F50/1.4 is rated 9.57 and the FA50/1.4 is rated 8.81), but the reviews themselves often do.

fotaki · 11-21-2010, 04:00 AM

Originally posted by Lowell Goudge

While I don't disagree with this point I.e. The review and rating is all within context then if I pay $200 for a super tak 85F1.9 in context of price performance I might rate it a 9 but if I pay$400 for it maybe it is only a 7?

To me that is part of the equation

Good point but purchase price could adversely bias the value.

In the review I made I pointed out that I bought the lens from a charity shop for £10. It had some fungus which I cleaned up producing quite a nice sample. I mentioned this in the review but obviously didn't quote this in the purchase price section since it would have influenced true value of the lens.

I think we must be careful how value versus price paid is implemented

macTak · 11-21-2010, 01:12 PM

You also raise the point fotaki that for many of these older lenses have been bought used and been subjected to who knows what, and indeed some have (or had) fungus, scratches, etc, all of which makes consistant numbering and pricing difficult.

Pentaxor · 11-21-2010, 01:40 PM

actually I find the lens review section useful. atleast from a general standpoint we are given the idea what lens to look for. anything with a rating of 8 or 9 is worth considering. the ratings and reviews (whether short summary or in-depth) are based on either scientific (objective), subjective preference , or combination of both. so it's up to the reader to take things with a lil grain of salt and determine what are considered as useful information.

primarily, ratings are based on what the reviewer feels about the lens. if it meets his/her expectations and understood what the lens' pros and cons are. differences in opinion is unavoidable. what is acceptable for one person may not be good to the other. this is reflected on the ratings. the bandwagon phenomenon may seem atrocious but does exist. however, there is some degree of truth as to why people follow or say the same thing about the lens as the others as they find the information sufficient enough to influence their perception of the lens and are happy that it met their standards.

I would say personally that the review section maybe insufficient, but far from being useless. as I mentioned it gives the person a general idea on what to expect from a lens. if a person is really interested in knowing a particular lens, he won't just stop there. that is why we see a lot of discussions or consensus in the forum whenever a fellow member asks for a particular lens if he/she finds the info insufficient and wants to be certain. I think it's just human nature to ask people for more assurance rather than base solely on reviews. point is, there is no define or well-known figure that people would simply believe or follow without asking questions.

because of this dilemma, it is up to the person to understand this and to look for ways to gather useful information that would influence his purchase. there are several steps and school of thought that one may use.

with regards to ratings, the reader should understand why a person gave that lens a certain score. it may occur that a "7" is the new "4" or is a low rating equivalent. it is also possible that the only way that a lens score below 5 is because the lens is useless or is in a really bad state. the parameters for ratings varies from person to person, so it is important that the reader should read further and understand what the reviewer sees as a hindrance.

samples are necessary as this would give an idea on what to get from a lens and what it could do. this is also an effective way to entice the reader even more, or at some instance, lose interest. however, there is also a danger here especially for those who are just starting in photography. because certain people might think that owning a lens would make them instant great photographers and able to produce great images with the lens. optimal usage of the lens is another challenge or user skill and knowledge could influence or affect the lens' status.

thinking out of the box. this is the most effective way of gathering information about the lens. some people would resort to forum discussions inquiring about the lens. some gather resources from several lens review websites on the net. comparing reviews and looking for consistency in the findings is the best way of a lens' real capability. besides, at the end of the day, it is you that would pass the final judgment.

Ira · 11-22-2010, 04:05 PM

Originally posted by timo

However - I do wonder how many people are going to give a really stinking review to a lens that they might one day want to sell. Cynical, moi?

This is why I can never sell my S-Tak 105 2.8 here:

While I have a few lenses I don't care about, I absolutely hate this one, and have made it well known. Yet it's a highly sought-after lens.

wlachan · 11-22-2010, 04:21 PM

Objective reviews from individuals are just plain difficult and almost impossible because there are so much excitement and emotion involved. The importance of user reviews is not how awesome the products are being praised, but the repeated issues been raised by different individuals. Try to get a pattern and those are something to look out.

Adam · 11-22-2010, 07:31 PM

Originally posted by Hannican

At least the part about "Rating" has become essentially a waste of time. Why does everyone keep giving all of their lenses 9s and 10s? Look at the K mount Primes, or the DA Primes sections. EVERYTHING is a 9 or higher.

People will even complain about things within the comments, but then give their lenses a 9 or a 10 anyway. I don't get it... if we're not going to tell the truth about our lenses, or if every lens really IS a 9 or a 10, then what's the point of even continuing to review them in this way?

The commentary part, and especially the sample pics that people post, are still excellent, and incredibly useful, but I think it's time we bring a little more truth back to this section (I don't remember it being this way a couple years back... there were lots of 7s and 8s two years ago).

A Concerned Forum User,

-Tim

All those lenses are good, especially if the numbers are adding up! The comments are probably More important than the numbers, though.

Pentaxor · 11-22-2010, 07:35 PM

Originally posted by wlachan

Objective reviews from individuals are just plain difficult and almost impossible because there are so much excitement and emotion involved. The importance of user reviews is not how awesome the products are being praised, but the repeated issues been raised by different individuals. Try to get a pattern and those are something to look out.

the cons can be considered as dealbreakers. I think these are also taken into consideration.

stevebrot · 11-22-2010, 09:36 PM

If a grade of 1 is a Holga and 10 is a Summicron, then a grade of 8 or 9 for most Pentax and Asahi primes is entirely correct and appropriate.

Nuff said...

Steve

secateurs · 11-22-2010, 10:03 PM

Originally posted by Lowell Goudge

It would be useful to have specific prompts, so that people can consider all the points, making the reviews more consistent. little things, that people often forget.

If you want a scientific review, why not look for one done by a reputable reviewer? User reviews aren't meant to be scientific.

I think the ratings are based on what the user knows. If all they've ever used is the kit lens that came with their k-x, they will think it's pretty good.

It's a function of user experience, in my opinion. Far more than price. How to judge the user's experience and knowledge... there is the real problem. I tend to look through the samples the user has posted. If they're shots of the pet dog in the backyard, i mentally decrease the rating. If there are 1 in a million shots in the mix, I will tend to stick with what the user rates it.

Maybe the user ratings could be automatically adjusted by some factor? Maybe linked to the user's forum rep? Or a short survey which each user completes, which asks about which lenses the user owns and has used.

11-21-2010, 01:40 PM	#69
Pentaxor Veteran Member Join Date: May 2009 Location: Vancouver, B.C. Photos: Gallery Posts: 6,513	actually I find the lens review section useful. atleast from a general standpoint we are given the idea what lens to look for. anything with a rating of 8 or 9 is worth considering. the ratings and reviews (whether short summary or in-depth) are based on either scientific (objective), subjective preference , or combination of both. so it's up to the reader to take things with a lil grain of salt and determine what are considered as useful information. primarily, ratings are based on what the reviewer feels about the lens. if it meets his/her expectations and understood what the lens' pros and cons are. differences in opinion is unavoidable. what is acceptable for one person may not be good to the other. this is reflected on the ratings. the bandwagon phenomenon may seem atrocious but does exist. however, there is some degree of truth as to why people follow or say the same thing about the lens as the others as they find the information sufficient enough to influence their perception of the lens and are happy that it met their standards. I would say personally that the review section maybe insufficient, but far from being useless. as I mentioned it gives the person a general idea on what to expect from a lens. if a person is really interested in knowing a particular lens, he won't just stop there. that is why we see a lot of discussions or consensus in the forum whenever a fellow member asks for a particular lens if he/she finds the info insufficient and wants to be certain. I think it's just human nature to ask people for more assurance rather than base solely on reviews. point is, there is no define or well-known figure that people would simply believe or follow without asking questions. because of this dilemma, it is up to the person to understand this and to look for ways to gather useful information that would influence his purchase. there are several steps and school of thought that one may use. with regards to ratings, the reader should understand why a person gave that lens a certain score. it may occur that a "7" is the new "4" or is a low rating equivalent. it is also possible that the only way that a lens score below 5 is because the lens is useless or is in a really bad state. the parameters for ratings varies from person to person, so it is important that the reader should read further and understand what the reviewer sees as a hindrance. samples are necessary as this would give an idea on what to get from a lens and what it could do. this is also an effective way to entice the reader even more, or at some instance, lose interest. however, there is also a danger here especially for those who are just starting in photography. because certain people might think that owning a lens would make them instant great photographers and able to produce great images with the lens. optimal usage of the lens is another challenge or user skill and knowledge could influence or affect the lens' status. thinking out of the box. this is the most effective way of gathering information about the lens. some people would resort to forum discussions inquiring about the lens. some gather resources from several lens review websites on the net. comparing reviews and looking for consistency in the findings is the best way of a lens' real capability. besides, at the end of the day, it is you that would pass the final judgment. Last edited by Pentaxor; 11-22-2010 at 06:26 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
K-7 Speculations (moved over from review section)	sctybear	Pentax DSLR Discussion	15	05-07-2010 05:48 PM
Suggestion Bag Review Section	lurchlarson	Site Suggestions and Help	11	04-24-2010 01:07 PM
Suggestion Lens Review Database section	Ivan Glisin	Site Suggestions and Help	9	05-13-2008 05:55 PM
Lens review section: adding DA 18-55 II?	Ivan Glisin	Site Suggestions and Help	2	04-04-2008 05:06 AM
DPreview launches lens review section - starts with Pentax (and some other misc brand	mattdm	Pentax News and Rumors	14	01-30-2008 04:48 AM

11-20-2010, 06:29 AM	#61
pinholecam Pentaxian Join Date: Oct 2009 Location: North Photos: Gallery Posts: 4,709	It cuts both ways. Some reviews put a lens down cause 'There is no 10 in my books'; '9+ is revered for the FA31ltd' ; etc does not make sense as well. Put down my M85/2 from 9 to 7 because its "not an AF lens"; "Not f1.4" is plain silly imho. I am reviewing a lens in its own context and not wrt others.

11-20-2010, 06:40 AM	#62
GeneV Veteran Member Join Date: Oct 2008 Location: Albuquerque NM Photos: Albums Posts: 9,830	I feel that information about the price or the rating is useful in an individual review. The rating helps understand something about how enthusiastic the reviewer may be, and the price paid may shed some light on why. More information is always better. However, where the price and ratings may go astray and become "borderline worthless" is in the averages. You can't really have a very meaningful average or even a median score from very different individuals with very different needs and uses. You can't have a meaningful average or median price where different individuals are purchasing the lens in different decades and on different continents. Perhaps some of the concerns could be alleviated by simply eliminating or de-emphasizing the averages.

11-20-2010, 06:46 AM	#63
Blue Moderator Site Supporter Join Date: Jun 2008 Location: Florida Hill Country Photos: Gallery \| Albums Posts: 17,377	Originally posted by Hannican At least the part about "Rating" has become essentially a waste of time. Why does everyone keep giving all of their lenses 9s and 10s? Look at the K mount Primes, or the DA Primes sections. EVERYTHING is a 9 or higher. People will even complain about things within the comments, but then give their lenses a 9 or a 10 anyway. I don't get it... if we're not going to tell the truth about our lenses, or if every lens really IS a 9 or a 10, then what's the point of even continuing to review them in this way? The commentary part, and especially the sample pics that people post, are still excellent, and incredibly useful, but I think it's time we bring a little more truth back to this section (I don't remember it being this way a couple years back... there were lots of 7s and 8s two years ago). A Concerned Forum User, -Tim Actually, you are incorrect. There are also idiots that talk about how good a lens is then give it a 5 or 6. I guess they have ego driven complexes that makes them think that being over critical makes them some sort of internet bad ass. There are plenty of 7 and 8 ratings in there. Plus, there are more than one review. While photozone does in depth reviews on lenses, it is done on a single specimen. Plus, they don't always compare lenses properly. For, example, they torture tested the FA 77mm/1.8 and got purple fringing. They did not torture test the DA 70mm/2.4 and proclaimed it to be superior in that regard even though even the Zeiss 85/1.4 would p.f. under that same torture test. I look at the lens review system here as a starting point and realize it is based on peoples experiences and opinions. The other issue is what are you rating a lens type and class to? (Rhetorical) You do thinks your way and I'll do things my way and one of us will be happy.

11-20-2010, 07:28 AM	#64
robbiec Site Supporter Join Date: Feb 2009 Location: Cork Photos: Albums Posts: 1,882	Originally posted by robbiec hehehe, I knew that might ruffle a few feathers... so far a burning effigy and now a rope... What would I get if I said I preferred the f/1.9 version over the f/1.8 version and that is even more expensive and going totally nuts price wise... worth a 6 maybe? Originally posted by Parallax Are you familiar with the phrase "castration with a rusty razor blade"? Put the 2 Super Takumars 85s at 8. But I did have a dig at the pricing and compared them unfavourably with the FA77, (same telephoto class) which I believe I am allowed to do, so you can put away the lynch mob!

11-20-2010, 07:53 AM	#65
Lowell Goudge Site Supporter Join Date: Jan 2007 Location: Toronto Photos: Gallery \| Albums Posts: 17,891	Originally posted by pinholecam It cuts both ways. Some reviews put a lens down cause 'There is no 10 in my books'; '9+ is revered for the FA31ltd' ; etc does not make sense as well. Put down my M85/2 from 9 to 7 because its "not an AF lens"; "Not f1.4" is plain silly imho. I am reviewing a lens in its own context and not wrt others. While I don't disagree with this point I.e. The review and rating is all within context then if I pay $200 for a super tak 85F1.9 in context of price performance I might rate it a 9 but if I pay$400 for it maybe it is only a 7? To me that is part of the equation

11-20-2010, 06:55 PM	#66
macTak Veteran Member Join Date: Apr 2010 Photos: Gallery Posts: 759	Originally posted by Docrwm It might improve things on the Review section if there were 3 ratings: Build Quality, Optical Quality, and Value for Money. I agree that this would make some sense. Of course, some people already do it and state it in their reivews. But overall, numbers simply don't mean much without context or a fixed scale (not a scale that differs with every reviewer). The bulk of the review section is in the written reviews themselves, which often are very helpful and informative. It's the quickest, easiest way to get information on the earlier AF and all the MF Pentax lenses, so don't worry about the ratings too much--you cant really compare lenses with them (heck, the F50/1.4 is rated 9.57 and the FA50/1.4 is rated 8.81), but the reviews themselves often do.

11-21-2010, 04:00 AM	#67
fotaki Veteran Member Join Date: Aug 2008 Location: Greece and UK Photos: Gallery \| Albums Posts: 309	Originally posted by Lowell Goudge While I don't disagree with this point I.e. The review and rating is all within context then if I pay $200 for a super tak 85F1.9 in context of price performance I might rate it a 9 but if I pay$400 for it maybe it is only a 7? To me that is part of the equation Good point but purchase price could adversely bias the value. In the review I made I pointed out that I bought the lens from a charity shop for £10. It had some fungus which I cleaned up producing quite a nice sample. I mentioned this in the review but obviously didn't quote this in the purchase price section since it would have influenced true value of the lens. I think we must be careful how value versus price paid is implemented

11-21-2010, 01:12 PM	#68
macTak Veteran Member Join Date: Apr 2010 Photos: Gallery Posts: 759	You also raise the point fotaki that for many of these older lenses have been bought used and been subjected to who knows what, and indeed some have (or had) fungus, scratches, etc, all of which makes consistant numbering and pricing difficult.

11-22-2010, 04:05 PM	#70
Ira Inactive Account Join Date: Mar 2009 Location: Coral Springs, FL Photos: Gallery \| Albums Posts: 5,216	Originally posted by timo However - I do wonder how many people are going to give a really stinking review to a lens that they might one day want to sell. Cynical, moi? This is why I can never sell my S-Tak 105 2.8 here: While I have a few lenses I don't care about, I absolutely hate this one, and have made it well known. Yet it's a highly sought-after lens.

11-22-2010, 04:21 PM	#71
wlachan Veteran Member Join Date: Sep 2006 Location: Canada Photos: Albums Posts: 2,625	Objective reviews from individuals are just plain difficult and almost impossible because there are so much excitement and emotion involved. The importance of user reviews is not how awesome the products are being praised, but the repeated issues been raised by different individuals. Try to get a pattern and those are something to look out.

11-22-2010, 07:31 PM	#72
Adam Administrator Site Webmaster Join Date: Sep 2006 Location: Arizona Photos: Gallery \| Albums Posts: 51,597	Originally posted by Hannican At least the part about "Rating" has become essentially a waste of time. Why does everyone keep giving all of their lenses 9s and 10s? Look at the K mount Primes, or the DA Primes sections. EVERYTHING is a 9 or higher. People will even complain about things within the comments, but then give their lenses a 9 or a 10 anyway. I don't get it... if we're not going to tell the truth about our lenses, or if every lens really IS a 9 or a 10, then what's the point of even continuing to review them in this way? The commentary part, and especially the sample pics that people post, are still excellent, and incredibly useful, but I think it's time we bring a little more truth back to this section (I don't remember it being this way a couple years back... there were lots of 7s and 8s two years ago). A Concerned Forum User, -Tim All those lenses are good, especially if the numbers are adding up! The comments are probably More important than the numbers, though.

11-22-2010, 07:35 PM	#73
Pentaxor Veteran Member Join Date: May 2009 Location: Vancouver, B.C. Photos: Gallery Posts: 6,513	Originally posted by wlachan Objective reviews from individuals are just plain difficult and almost impossible because there are so much excitement and emotion involved. The importance of user reviews is not how awesome the products are being praised, *but the repeated issues been raised by different individuals*. Try to get a pattern and those are something to look out. the cons can be considered as dealbreakers. I think these are also taken into consideration.

11-22-2010, 09:36 PM	#74
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	If a grade of 1 is a Holga and 10 is a Summicron, then a grade of 8 or 9 for most Pentax and Asahi primes is entirely correct and appropriate. Nuff said... Steve

11-22-2010, 10:03 PM	#75
secateurs Senior Member Join Date: Oct 2010 Location: Adelaide, Australia Photos: Albums Posts: 250	Originally posted by Lowell Goudge It would be useful to have specific prompts, so that people can consider all the points, making the reviews more consistent. little things, that people often forget. If you want a scientific review, why not look for one done by a reputable reviewer? User reviews aren't meant to be scientific. I think the ratings are based on what the user knows. If all they've ever used is the kit lens that came with their k-x, they will think it's pretty good. It's a function of user experience, in my opinion. Far more than price. How to judge the user's experience and knowledge... there is the real problem. I tend to look through the samples the user has posted. If they're shots of the pet dog in the backyard, i mentally decrease the rating. If there are 1 in a million shots in the mix, I will tend to stick with what the user rates it. Maybe the user ratings could be automatically adjusted by some factor? Maybe linked to the user's forum rep? Or a short survey which each user completes, which asks about which lenses the user owns and has used.