Originally posted by falconeye cputeq, you're right for a tester with average (read insufficient) qualification.
Let me give an example from the testing methodology of German
computer magazine c't.
IT news, features and forums - heise online UK
and
c't - magazin fr computertechnik
They have an outstanding reputation, not met by any photo magazine, or any other computer magazine I am aware of, on a worldwide scale...
What do they do?
First, they live with the products long enough to know them.
Second, they reconfigure the products to bring them to their best value, even sometimes making the manufacturer give them an updated model following their input. This means, exchanging drivers, running updates, maybe exchanging too loud fans etc. They do what an actual buyer would have done.
Third, they judge the individual aspects. They do
not give an overall score!
I don't see why a photo mag couldn't do this.
Yes, I'm aware of c't, I used to read them all the time
The problem with your analogy is that w're probably dealing with slightly different demographics between the examples.
Like Ziff-Davis publications (think: "PC World"), I don't think Popphoto is really catering to the hard core photography market. Most of their reviews are extremely cursory in nature - offering fairly superficial reviews of the cameras in question. They give an overall synopsis of the performance of the camera, and any significant problems or awesomeness they find.
Heck, they don't even quote resolutions for lenses, etc, but instead give each lens a "quality / print size" grid according to aperatures -- What the hell is a B+ photo? Where is this explained? I haven't found it, but I would imagine they're just rough things like "the photo looks okay".
They're targeting the newbish photography market (one would think) -- So their articles aren't going to delve into too many details. I read them mostly for their previews of upcoming hardware and photography tips, not really for their hardware reviews (though I do pay attention to them), and I'm certainly not ashamed to say so -- Many of their photo and Photoshop tips are awesome, while their reviews are a bit bland and cookie-cutter.
-------
Juxtapose this with c't, which goes into every minutea of every detail with regards to their articles. C't is catering towards the Tom's Hardware / Maximum PC crowd of computer geeks (like myself), not the mass-market group like Ziff-Davis would.
Popphoto probably doesn't have the time to get initmately familiar with everything they review, and even if they did, I would doubt one would see such an in-depth hardware review like one could find at
DPcanikonreview.com
---------
I'll go out on a limb and bet that many of the writers at Popphoto are simply photography jourlalists (not photojournalists), just like nearly everyone at MaximumPC are computer journalists, with very many real computer nerds/experts (as once pointed out by their former editor).
They write about a given subject, but aren't neccesarily top experts in the fields, which is okay given their target audiences.
I believe if one makes that distinction between highly technical and mass-market, Popphoto can be seen for what it is, not what we think it
should be. Given that, a review of a camera using factory defaults is the only fair method for the segment Popphoto caters to, and for the types of articles Popphoto writes.