[Lowell Goudge]

If you want a scientific review, why not look for one done by a reputable reviewer? User reviews aren't meant to be scientific.

no, but there can be many specific operational things that reviewers could easily put down, which would help a prospective purchaser. For example
- F stops in half or full stops
- focusing collor rotation
- front element moves/turns when focusing
- zoom creep if presnet

I think the ratings are based on what the user knows. If all they've ever used is the kit lens that came with their k-x, they will think it's pretty good.

and in the context of purpose, which perhaps should be stated it really is one of the best lenses for the value

It's a function of user experience, in my opinion. Far more than price. How to judge the user's experience and knowledge... there is the real problem. I tend to look through the samples the user has posted. If they're shots of the pet dog in the backyard, i mentally decrease the rating. If there are 1 in a million shots in the mix, I will tend to stick with what the user rates it.

that should be irrelevant. the reason someone shoots their pet dog, or cat, or flowers or ducks..... is that they are easily recognisable and there are details that in a 100% crop show the quality and specifically sharpness of a lens. When you go out and get your new toy, you do some quick shots to see how it works. By considering someone who posts a 1 in a million shot with a lens review, and think them a credible reviewer, is like making a false god.

Maybe the user ratings could be automatically adjusted by some factor? Maybe linked to the user's forum rep? Or a short survey which each user completes, which asks about which lenses the user owns and has used.

The former is a little like a popularity contest, and should be avoided, the latter has some merit.

But the biggest point is, that as a reviewer you always have to consider the context of the review and your reasons for writing the review. The survey you propose, and to some extent the purchase price which is already tracked, develop that context. Unfortunately, I think it is too late for the survey unless you can have retroactive updates, so people go back and add their data, or the survey is linked somehow to the personal data in your account. i.e. equipment lists, length of time shooting, etc...

[paperbag846]

that should be irrelevant. the reason someone shoots their pet dog, or cat, or flowers or ducks..... is that they are easily recognisable and there are details that in a 100% crop show the quality and specifically sharpness of a lens. When you go out and get your new toy, you do some quick shots to see how it works. By considering someone who posts a 1 in a million shot with a lens review, and think them a credible reviewer, is like making a false god.

I'm going to have to disagree strongly with this one. I see a lot of people claiming superb IQ sharpness etc., but the pictures they post aren't interesting to me, so I don't take their opinion as seriously as someone who takes some images I'm really interested in.

A perfect example are people with superfast lenses who post a large volume of botanical photography. I have no interest in botanical photography, and I've come to realize that while you would think a floral arrangement would serve as a good bokeh test... when you go and take the same lens and shoot a subject you are interested in, the bokeh is completely different. I think it is wise to look to the pictures to base your opinion. A lens that is a real stunner for one person (because they like shooting flowers) might not be the best choice for me (who has no interest in it).

"False gods..." is a harsh word. I've been into guitar equipment far longer than photography, and let me tell you, lots of people who are unable to play anything worth listening to have lots of very nice gear. They buy, and test, like many people here do with photos of their dogs and cats. I honestly don't understand your point about looking for sharpness in an "familiar object". Sharpness is sharpness, you can see it on anything.

The point I'm trying to make is, when I am looking at a new guitar, I want to hear some samples of someone playing it. If all they show me is heavy metal, who is to say that this guitar would fit my musical needs (i.e., very far away from heavy metal). A sample is good if it shows the potential buyer the kind of shots that he or she wants to eventually take.

Thats why lens testing can be such bullocks in my humble opinion. You can compare and contrast lens X and Y with pictures of grass blades, but who cares at ALL about blades of grass? That's a boring photograph and I don't need to buy a new lens to take a picture of grass. I think that user samples of good photography (amongst whom I am *not* placing myself with) are the MOST valuable resource a potential buyer can have. And since good photography is largely subjective, it is good to be able to see a wide-variety of images, of all sorts of subjects.

It instills confidence that the reviewer knows his or her tools well, and is using them for a purpose that is similar to your own goals. An actual *photographer*, not just someone comparing specifications (even in the non-numerical sense). I will admit that I don't give much credence to someone who shoots their cat or some flowers either... unless it's actually good enough to stand out from the thousands of other cat/flower pictures.

EDIT:

How about an optional profile? It is simple - anyone can create a "reviewer profile", it's basically a second signature. In this signature you list the lenses and other equipment you use. Maybe this list could be limited to lenses that the owner has already reviewed. These only appear at the end of reviews.

Therefore someone could say "well, this guy really loves lens X, and he would know because he's tried all of these!" another could say "well, this guy really loves lens Y, but he only has the kit lens, so his opinion is not as refined".

This, in conjunction with good sample pictures, would make it much easier to interpret the ratings.

[liveincontrast]

How about an optional profile? It is simple - anyone can create a "reviewer profile", it's basically a second signature. In this signature you list the lenses and other equipment you use. Maybe this list could be limited to lenses that the owner has already reviewed. These only appear at the end of reviews. Therefore someone could say "well, this guy really loves lens X, and he would know because he's tried all of these!" another could say "well, this guy really loves lens Y, but he only has the kit lens, so his opinion is not as refined".

I think this is one of the best ideas I've heard to provide context and credibility to lens reviews.

[Lowell Goudge]

I'm going to have to disagree strongly with this one. I see a lot of people claiming superb IQ sharpness etc., but the pictures they post aren't interesting to me, so I don't take their opinion as seriously as someone who takes some images I'm really interested in.

A perfect example are people with superfast lenses who post a large volume of botanical photography. I have no interest in botanical photography, and I've come to realize that while you would think a floral arrangement would serve as a good bokeh test... when you go and take the same lens and shoot a subject you are interested in, the bokeh is completely different. I think it is wise to look to the pictures to base your opinion. A lens that is a real stunner for one person (because they like shooting flowers) might not be the best choice for me (who has no interest in it).

"False gods..." is a harsh word. I've been into guitar equipment far longer than photography, and let me tell you, lots of people who are unable to play anything worth listening to have lots of very nice gear. They buy, and test, like many people here do with photos of their dogs and cats. I honestly don't understand your point about looking for sharpness in an "familiar object". Sharpness is sharpness, you can see it on anything.

there are lots of people who take pictures with the same attitude, buying the most expensive of everything, thinking it makes them better also, buit.....

overall, I will play devils advocate here, if you can see sharpness on anything, then why does it matter the subject. if I take a shot of a cat or dog from 3-5 feet away, using a portrait "focal length" lens, the sharpness you see in that shot should equate directly to the sharpness you see in a portrait, the bokeh will also be the same, as yo fade from in focus to out of focus and background. while subjects may not be interesting to you that does not mean the photograph does not demonstrate the usefulness of the lens for others. I will however, give you some ground here, a photo of a cat or dog at 3-5 feet is useless in evaluating the performance of a lens when focused to 30 feet or infinity. It is not the subject that is important here but the focusing distance.

The point I'm trying to make is, when I am looking at a new guitar, I want to hear some samples of someone playing it. If all they show me is heavy metal, who is to say that this guitar would fit my musical needs (i.e., very far away from heavy metal). A sample is good if it shows the potential buyer the kind of shots that he or she wants to eventually take.

I agree here, and that is part of my response above also, that there are many things other than sharpness at portrait distance that are important, tonal quality, mid range sharpness edge sharpness, etc all come into it, but many times the important things are not always detectable by a great photo either.

Thats why lens testing can be such bullocks in my humble opinion. You can compare and contrast lens X and Y with pictures of grass blades, but who cares at ALL about blades of grass? That's a boring photograph and I don't need to buy a new lens to take a picture of grass. I think that user samples of good photography (amongst whom I am *not* placing myself with) are the MOST valuable resource a potential buyer can have. And since good photography is largely subjective, it is good to be able to see a wide-variety of images, of all sorts of subjects.

true, test shots need to be varried, not just one subject, to see the full range of capabilities.

It instills confidence that the reviewer knows his or her tools well, and is using them for a purpose that is similar to your own goals. An actual *photographer*, not just someone comparing specifications (even in the non-numerical sense). I will admit that I don't give much credence to someone who shoots their cat or some flowers either... unless it's actually good enough to stand out from the thousands of other cat/flower pictures.

but that is not necessairly judged by the 1 in a million shot, because anyone can take one like that, even a complete fool. What is important, when doing test photos, in my opinion is to do the photos to demonstrate a particular quality of a lens, and these are best done in comparison to another lens, not stand alone. the problem is that no one person has every lens so side by side comparisons are difficult

EDIT:

How about an optional profile? It is simple - anyone can create a "reviewer profile", it's basically a second signature. In this signature you list the lenses and other equipment you use. Maybe this list could be limited to lenses that the owner has already reviewed. These only appear at the end of reviews.

Therefore someone could say "well, this guy really loves lens X, and he would know because he's tried all of these!" another could say "well, this guy really loves lens Y, but he only has the kit lens, so his opinion is not as refined".

This, in conjunction with good sample pictures, would make it much easier to interpret the ratings.

A reviewer profile is not a bad idea, but you disqualify a ton of people who have perhaps bought really good lenses, take excellent photos (the 1 in a million you want to see in reviews) and have never tried anything else, hence they don't get the credit the review is really worth evem if their likes and dislikes are valid because they don't have another similar lens or have owned on in the past. You will generate a high score as an equipment geek because you review ton's of lenses but perhaps, you are useless at photography. I fail to see how that serves the people who read the reviews to make decisions either.

[Pentaxor]

I'm going to have to disagree strongly with this one. I see a lot of people claiming superb IQ sharpness etc., but the pictures they post aren't interesting to me, so I don't take their opinion as seriously as someone who takes some images I'm really interested in.

A perfect example are people with superfast lenses who post a large volume of botanical photography. I have no interest in botanical photography, and I've come to realize that while you would think a floral arrangement would serve as a good bokeh test... when you go and take the same lens and shoot a subject you are interested in, the bokeh is completely different. I think it is wise to look to the pictures to base your opinion. A lens that is a real stunner for one person (because they like shooting flowers) might not be the best choice for me (who has no interest in it).

"False gods..." is a harsh word. I've been into guitar equipment far longer than photography, and let me tell you, lots of people who are unable to play anything worth listening to have lots of very nice gear. They buy, and test, like many people here do with photos of their dogs and cats. I honestly don't understand your point about looking for sharpness in an "familiar object". Sharpness is sharpness, you can see it on anything.

The point I'm trying to make is, when I am looking at a new guitar, I want to hear some samples of someone playing it. If all they show me is heavy metal, who is to say that this guitar would fit my musical needs (i.e., very far away from heavy metal). A sample is good if it shows the potential buyer the kind of shots that he or she wants to eventually take.

Thats why lens testing can be such bullocks in my humble opinion. You can compare and contrast lens X and Y with pictures of grass blades, but who cares at ALL about blades of grass? That's a boring photograph and I don't need to buy a new lens to take a picture of grass. I think that user samples of good photography (amongst whom I am *not* placing myself with) are the MOST valuable resource a potential buyer can have. And since good photography is largely subjective, it is good to be able to see a wide-variety of images, of all sorts of subjects.

It instills confidence that the reviewer knows his or her tools well, and is using them for a purpose that is similar to your own goals. An actual *photographer*, not just someone comparing specifications (even in the non-numerical sense). I will admit that I don't give much credence to someone who shoots their cat or some flowers either... unless it's actually good enough to stand out from the thousands of other cat/flower pictures.

this is exactly one of the dangers that I had mentioned on my previous posts regarding the sample images that are posted. it creates some degree of fallacy on lens use or simply form misconceptions. we cannot fault the reviewer or the one that is posting images as samples. the reviewer might only be posting the images that are readily available to him/her and besides, he/she is not paid to do someone else's bidding. people should know better than that. I mean, just because the samples does not seem to fit one's own taste can one simply form a conclusion that this lens is only good for cats, this is a dogs only lens, flower lens, people lens, etc... it's a fallacy logic.

I understand the point that is raised where the samples are insufficient or the samples were taken by a new user/owner. however, if the person loves to shoot dogs, cats, flowers, etc... that is another story and more into subject preference rather than a generalization of a lens' capability. heck, I could use my DA40 for flower shots only, and post images on the review. does it make useless for other types of photography? again, the fallacy logic.

either way, it doesn't give the lens justice with that kind of thinking. it is true that some images are dull and not helpful. some images do not interest others' type of photography. some reviews are one-liners. they are insufficient. but again, these are user reviews that are given freely. so it's better to take this with a grain of salt or cut people some slack. I mean, it's better than nothing at all because I feel that there are other lenses that are missing user reviews and info with. as for someone who really is interested in a particular lens, would the review section of the board be his last stop? or will he look somewhere else to gather info? personally, the lens review section only gives a general idea of what to expect from a lens in the eyes of the user.

[myeates]

Paperbag has made some very valid points, I would wholeheartedly agree with him on the analogy with music.

I have been on other sites one in particular springs to mind which is there to worship at the alter of leica, in particular the M9 et al. I do not doubt for one moment that given that kit and some of there gorgeous glass I could produce some wonderful images.

The trouble is most of the folk posting there cant! They buy this exotic kit and for what they produce and where it will be shown (web based sights-Flikr, facebook argh!) and other such accounts they may as well have used a nikon D4o and kit zoom. The other issue which has always made me chuckle is the burgeoning amount of lens babay,holga,diana and you name it type shots that abound on the web. Folk have found this new toy and try to use it to inject something worthwhile into their mundane record shot imagery.

The problem with this approach is that just because it's fuzzy dreamy and whatever else one would like to describe it, it does not necessarily make a worthy image.

I have found this site useful in helping me decide what new and old pentax glass I will buy, what helps me the most is the inclusion of some quality pics to highlight what they are eulogising about.

On my long photographic journey I have used in the past Pentax kit, MX and various lenses 28mm, 50 1.7, and the 100mm and loved every minute of doing so. I then moved onto Nikon before dipping a toe into the digital age with an Olympus E1 with 14-54 lens (superb combination). I still loved film though so purchased a canon A1 with 28, 50 and 85 lenses. Now I have bought the K5 I suspect I will swap the canon for the Pentax MX again..I will definitely post some feedback when I get the kit and will also attach pics or links to.

That's enough of my soap box happy snapping.

[myeates]

I talk to the trees at Olympus, Ricoh & Me


I attach the above to show what the marvelous compact the ricoh GRD3 can produce, this camera is for me at least the modern day version of the leica rangefinder, I love it, if you want to bother trawling through my blog you will find other examples taken with it .

---

Last edited by myeates; 11-23-2010 at 01:07 PM. Reason: image did not load on the first post

[Lowell Goudge]

Paperbag has made some very valid points, I would wholeheartedly agree with him on the analogy with music.

I have been on other sites one in particular springs to mind which is there to worship at the alter of leica, in particular the M9 et al. I do not doubt for one moment that given that kit and some of there gorgeous glass I could produce some wonderful images.

The trouble is most of the folk posting there cant! They buy this exotic kit and for what they produce and where it will be shown (web based sights-Flikr, facebook argh!) and other such accounts they may as well have used a nikon D4o and kit zoom. The other issue which has always made me chuckle is the burgeoning amount of lens babay,holga,diana and you name it type shots that abound on the web. Folk have found this new toy and try to use it to inject something worthwhile into their mundane record shot imagery.

The problem with this approach is that just because it's fuzzy dreamy and whatever else one would like to describe it, it does not necessarily make a worthy image.

I have found this site useful in helping me decide what new and old pentax glass I will buy, what helps me the most is the inclusion of some quality pics to highlight what they are eulogising about.

On my long photographic journey I have used in the past Pentax kit, MX and various lenses 28mm, 50 1.7, and the 100mm and loved every minute of doing so. I then moved onto Nikon before dipping a toe into the digital age with an Olympus E1 with 14-54 lens (superb combination). I still loved film though so purchased a canon A1 with 28, 50 and 85 lenses. Now I have bought the K5 I suspect I will swap the canon for the Pentax MX again..I will definitely post some feedback when I get the kit and will also attach pics or links to.

That's enough of my soap box happy snapping.

While I don't disagree with him, I think he needs to understand photos take for artistic purposes, and photos taken to demonstrate something. while an artistic photo will possibly demonstrate what you want to show, it is not always the case, but it is much more common to have photos that demonstrate a specific lens characteristic be less than artistic.

Your reference to Leica is noted, but I disageee with your comment "I do not doubt for one moment that given that kit and some of there gorgeous glass I could produce some wonderful images. " it is not the glass that makes the photograph, but the photographer. Ansil Adams didn't have a leica, did he?

[patk]

as a user with a vested interest in the lens review database continuing to exist (as in, i do refer to it when looking at purchases), i have a suggestion. there seems to be a lot of theory being expounded about how reviews should be structured, etc. but no one has yet demonstrated that things are dreadfully amiss.

so i have a suggestion. why don't those who think the reviews are "borderline useless" take a few of the reviews of the lenses they own and demonstrate why the existing reviews of those lenses are wrong or don't work. presumably they can show why a lens rated as an 8 on average is either not worthy or is underrated.

maybe then there can be some substantiation of the claim about uselessness to begin with.

if that proves to be too much to ask, then tell me why anyone should assume whatever they propose to replace the present system is any better (as in, more valid, more worthy of acceptance, etc).

[Lowell Goudge]

as a user with a vested interest in the lens review database continuing to exist (as in, i do refer to it when looking at purchases), i have a suggestion. there seems to be a lot of theory being expounded about how reviews should be structured, etc. but no one has yet demonstrated that things are dreadfully amiss.

so i have a suggestion. why don't those who think the reviews are "borderline useless" take a few of the reviews of the lenses they own and demonstrate why the existing reviews of those lenses are wrong or don't work. presumably they can show why a lens rated as an 8 on average is either not worthy or is underrated.

maybe then there can be some substantiation of the claim about uselessness to begin with.

if that proves to be too much to ask, then tell me why anyone should assume whatever they propose to replace the present system is any better (as in, more valid, more worthy of acceptance, etc).

You mean those who think the reviews are useless do a review of their lenses and use the useless tool to post them?

Good idea.

[paperbag846]

I agree with you Lowell/Pentaxor. I don't think that the 1 in a million shots are useless though. That is a factor of a skilled vs. lucky photographer. However, anyone's best pictures will show what a lens is capable of. I also find that "tests" accentuate lens characteristics in a way that you simply don't see in normal photography, so it can be misleading, albeit more scientific. So your observation that photography is artwork is correct... I suppose I prefer artwork to judge by as opposed to test numbers.

Another music analogy. When I set about acquiring some special effect pedals, I went online to see what my favorite musicians were using on stage. I was often suprised, "wow!", I thought, "they get amazing tone from that cheap delay?". The more I researched these things, the more I realized that artists will often take sub-par, or "technically imperfect" things to create great art with. Sometimes, the deficiencies of the gear actually contribute to the work, instead of detracting from it.

Now the deeper you get into it, you actually start to realize that you could swap gear A for gear B and get the same result... but in a situation where you are still learning, emulating someone else's gear choice is often a very easy way to get yourself into the ballpark.

So this is why I prefer real photos. I want to take a picture like you. You used this lens. Good enough for me!

[paperbag846]

A reviewer profile is not a bad idea, but you disqualify a ton of people who have perhaps bought really good lenses, take excellent photos (the 1 in a million you want to see in reviews) and have never tried anything else, hence they don't get the credit the review is really worth evem if their likes and dislikes are valid because they don't have another similar lens or have owned on in the past.

Individual readers could decide whether someone is qualified or not based on their gear, this would not result in a hard and fast rule.

Look at it this way. Someone with the FA trio goes and reviews the 18-135. He/she gives it an 8. You COULD say "well, he/she does not really know much about zooms" OR you could say "that zoom must be good as this user clearly has high standards!"

It's a nice way to give each reviewer a "resume" if you will, and the individual reader can decide whether their opinion is worth a damn. Since it does not require us to remove any of the existing data, and it's only an opt-in solution, I don't see how it could hurt.

[Pentaxor]

You mean those who think the reviews are useless do a review of their lenses and use the useless tool to post them?

Good idea.

I laughed on this one. I don't think a mere user could even present a review that is purely objective. this is why it's a user review. although as long as the consistend basis or trend lends towards saying the lens is really good, and cite whatever shortcomings the lens may have, and made it aware to the reader, it should already help the reader what to expect from the lens. I think there are two things that the reader should consider. "what makes a good lens and what makes a bad one".

just a room for argument and a little twist, what if one review says that the DA40 is a bad lens optically? does it invalidates his/her review? in other words, reviews that are seem out of character or different from the others.

[paperbag846]

Well if the person was comparing it to all sorts of wonderful Ziess glass, maybe it would seem reasonable. If not, one might conclude there is something wrong with the reviewers DA 40...

[Pentaxor]

I agree with you Lowell/Pentaxor. I don't think that the 1 in a million shots are useless though. That is a factor of a skilled vs. lucky photographer. However, anyone's best pictures will show what a lens is capable of. I also find that "tests" accentuate lens characteristics in a way that you simply don't see in normal photography, so it can be misleading, albeit more scientific. So your observation that photography is artwork is correct... I suppose I prefer artwork to judge by as opposed to test numbers.

Another music analogy. When I set about acquiring some special effect pedals, I went online to see what my favorite musicians were using on stage. I was often suprised, "wow!", I thought, "they get amazing tone from that cheap delay?". The more I researched these things, the more I realized that artists will often take sub-par, or "technically imperfect" things to create great art with. Sometimes, the deficiencies of the gear actually contribute to the work, instead of detracting from it.

Now the deeper you get into it, you actually start to realize that you could swap gear A for gear B and get the same result... but in a situation where you are still learning, emulating someone else's gear choice is often a very easy way to get yourself into the ballpark.

So this is why I prefer real photos. I want to take a picture like you. You used this lens. Good enough for me!

the truth is, not everything is made equal. if it were only possible, everybody would be happy to use only the kitlens.

like, what makes the DA40 special? if both the kitlens and DA40 could be swapped and get the same result, are you going to get rid of the DA40 and use the kitlens at 40mm? the zoom has a more flexible focal range, so that would make it the better choice, or not?