Forgot Password
Pentax Camera Forums Home
 
Log in or register to remove ads.

Showing results 1 to 2 of 2 Search:
Forum: Photographic Industry and Professionals 03-16-2014, 04:50 AM  
The problem with "Professional" Reviews
Posted By Little Ray
Replies: 21
Views: 2,658
Yep,

I fully admit to hyperbolizing the whole thing; and yes, Pentax is probably too much for me anyway. Most of the cameras in this class are. I do have aspirations though.

Cheers,

Little Ray
Forum: Photographic Industry and Professionals 03-16-2014, 03:36 AM  
The problem with "Professional" Reviews
Posted By Little Ray
Replies: 21
Views: 2,658
Hello All,

What I wanted to comment on is the disparity I see between professional reviews of cameras and real life experience of consumers. Not so much regarding sterile studio test reviews versus actually going outside, taking pictures and producing photographs; but rather, the often glaring disparity between professional reviews and end user findings regarding a product defect. How often do we see glowing reviews for products that make no mention of well-known manufacturing defects (hardware or software) discovered by consumers. Not even in follow up reviews if there are any. Examples include spots on the sensor of the Nikon 7000 (maybe it was the 7100) and perhaps Pentax had a similar issue with the K5 maybe. Sample variation is one explanation, as I am sure manufacturers do their best to hand pick the models sent to the reviewer. Also, pre-production models are sent out as well (we have all seen the Pentax K3 with Pentax logo on the back).

I understand some issues may not reveal themselves immediately; and only after some time (excluding normal wear and tear) will a defective part be revealed. This might not be noticed by an initial review and that makes sense, but even so should there not be some expectation of a long term review? We do see these types of long-term reviews for sporting equipment- camping gear is a good example.

Perhaps I am trying to come to terms with my own expectations of a product review. I find these reviews often carry themselves out like extended advertisements for a product as opposed to independent investigations. Reviewers tell us they are not biased and carry out their sterile tests (wearing white lab coats I am sure) and we are expected to take this as some standard of objectivity. Even the most well meaning reviewers I think are fooling themselves along with the public they hope to inform. The ceaseless drone and battery of test results carried out almost prove nothing anymore. That well-documented, repeatable and crippling product defects discovered by endusers do not make into reviews is damning to the credibility of any review, and any reviewer worth their salt should be ashamed.

What should be the standard for review of a camera? Should it be accepted what we have so far from DPreview, EPhotozine, CNet and the like? Conclusions of a reviewer for a camera (hand-picked by the manufacturer) based solely on graphs, charts and the same studio images over and over again? Or even worse, Youtube video statements from some person (read Senior Managing Photo Reviewer) merely holding a camera and occasionally pushing a button or navigating a menu with some air of authority? What is the point of going through these tedious tests if the product has some major manufacturing flaw rendering it a de facto paper weight?

If you are still reading, I apologize. I cannot give you back those 2 minutes.

There is a criticism among some educators regarding standardized tests. I remember a million years ago when I was in elementary or middle school we had to take the California Achievement Test. The test is suppose to measure against some standard of reading, writing, and arithmetic aptitude. One of the criticisms is that the teachers are restricted in their curriculum and end up teaching the student how to pass the test as opposed to educating us. Assessment based on multiple guess arithmetic problems is questionable if no work is shown, and teachers are right I think for this kind of criticism. Likewise reviews from the likes of DPreview and so on, if there is no evaluation beyond the standard test they perform, then it is only half a job at best.


I would like to think the following should become standard for any review- 1) sample variation testing should be a part of a review; 2) Long-term testing should be performed to ferret out any product defects over time. And honestly, some folks notice issues straight out of the box and so I wonder how is it the professional reviewer missed it. Additionally, any review for a product based on a pre-production model should be automatically disregarded by us, the consumer; and, any reviewer with any integrity should not accept such a sample to be used as the subject for a review. And, Any review that is a mind-numbing reiteration of the product overview page from the manufacturer can also be thrown out. Any review released the day of production release should also be ignored as nothing other than free advertising for the manufacturer.



Yes, I can choose to ignore all the reviews and buy based on my own assessment of the promises of the manufacturer, but these reviews are meant to be on our side. However, currently, they seem to be run as a business. DPreview is run by Amazon and reviews are made to and prioritized to products that sell or traditionally have the marketshare. If that is how they want to run themselves then that is their business, but we should not accept any review they carry out. They have lost credibility. This is not specifically a diatribe against Dpreview as such (though it sounds like one a bit). I do not care specifically if DPReview ever does a review or not on any particular camera or manufacturer, but we should be wary of these reviews.


Caveat Emptor – Let the buyer beware. This is no longer a caution to the consumer against the manufacturer; but equally well, we should be cautious to what the reviewer is selling, especially if they are leaving a job undone. That reviewers want to run a camera through a battery of lab tests is fine, go for it. Obviously people love that stuff, and it can be useful information especially in evaluating performance against manufacturer claims. I would like to see at least some more inspired subjects, but that is neither here nor there I suppose. Reminds me though during a programming course I followed at university. The professor said while testing code that if we did not find any issue with our code there was something wrong with our test. All products these days are going to have some issue, I understand. The job of the reviewer, however, is suppose to find them and pass them along to the potential consumer. There is an implied trust in a product review between the reviewer and the consumer; and for that, we should hold the reviewer to some degree accountable for their review. I think it is time we invoke the principle Caveat venditor, Let the seller beware, and apply it to not only the manufacturer for false advertising claims (as does happen sometimes) but to the "professional' reviewer as well for their negligence at the very least. That it is solely left up to the consumer to find fault and carry out the due diligence to document, reproduce, corroborate with others to try to get the manufacturer to correct the issue. We should expect some support from our trusted professional reviewers.



For the purposes of full disclosure, this started because of my slavish obedience to the sea of reviewers selling me on the virtues of the Pentax K3 and my discover through teh good folks here on the forum that there does seem to be an issue regarding the mirror flap. I was able to back out of my purchase before potentially being stuck with a dud; but I find it interesting that I have yet to come across any reviewer - DPreview, Ephotozine, Pentax Forums, Digital Resource who had this issue either during initial testing or at least in some kind of follow-up statement. Correction, it seems Adorama briefly discussed this in a conclusion section; but obviously they did not do their homework as they reported it was resolved from a firmware fix; but we know that is not true and they should know better to make such a statement.


Apologies again for wasting everyone’s time.
Search took 0.00 seconds | Showing results 1 to 2 of 2

 
Forum Jump


All times are GMT -7. The time now is 03:20 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top