Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
06-14-2008, 04:46 AM   #1
Veteran Member




Join Date: Jan 2008
Location: Ste-Anne des Plaines, Qc., Canada
Posts: 2,014
Pop Photo July issue

In the july issue, Pop Photo has a comparison test of high ISO cameras. In the test results, they rated the noise level of the K20D at ISO 3200 and 6400 as unacceptable. What ticks me off is the fact (stated in the article) that they left the NR off on the K20D while all the other contenders had high ISO N.R. on. It shows that Pop Photo has a strong dislike of Pentax cameras to use such tricks to discredit a model. They should use the same conditions for all the cameras they test for a particular issue. They implied that they left the camera in the state it comes in the box (default setting), but they turned the high ISO on on the Olympus E-3. If you look only at the results without reading the whole article, it could mislead a lot of readers. Why turn high ISO n.r. on for the Olympus and not do it for the K20D? POP PHOTO, YOU STINK

06-14-2008, 05:31 AM   #2
Lens Buying Addict
monochrome's Avatar

Join Date: Mar 2008
Location: Kirkwood (St. Louis) MO
Photos: Gallery | Albums
Posts: 19,483
I don't take their (or any) photography magazine for just this reason, but I'd bet Pentax doesn't advertise enough (pay enough bribes) to be treated fairly - this is a form of blackmail.
06-14-2008, 05:52 AM   #3
Veteran Member




Join Date: Nov 2006
Location: Hong Kong
Posts: 1,934
What's the big deal here?

So, what's the big deal?

Even Dpreview do tests at the default factory settings. It is just impossible to try each combination for they to carry out the review. Does an individual setting require an individual test report?

If it is the factory default, it is fair enough to believe that it is the best setting as recommended by the maker, isn't it?

Anyway, I would say those noise tests nowadays by the measurbators are not so meaningful as long as noises are measured along instead of the Signal-to-Noise (aka S/N) ratio.

Take the following DPR posted high ISO shootout image for example:-

https://www.pentaxforums.com/forums/pentax-dslr-discussion/29317-14-mpix-high-iso-shoot-out.html

I must say I am impressed with the details and rich colours retained by the K20D at high ISO, although it is obviously noisy from ISO 800 already. (The 450D and A350 are far less noisy, but there are no good IQ at all (as Phil Askey and his guy seem to be right in what they are presenting))

QuoteOriginally posted by flyer Quote
In the july issue, Pop Photo has a comparison test of high ISO cameras. In the test results, they rated the noise level of the K20D at ISO 3200 and 6400 as unacceptable. What ticks me off is the fact (stated in the article) that they left the NR off on the K20D while all the other contenders had high ISO N.R. on. It shows that Pop Photo has a strong dislike of Pentax cameras to use such tricks to discredit a model. They should use the same conditions for all the cameras they test for a particular issue. They implied that they left the camera in the state it comes in the box (default setting), but they turned the high ISO on on the Olympus E-3. If you look only at the results without reading the whole article, it could mislead a lot of readers. Why turn high ISO n.r. on for the Olympus and not do it for the K20D? POP PHOTO, YOU STINK
06-14-2008, 05:55 AM   #4
Veteran Member




Join Date: Nov 2006
Location: Hong Kong
Posts: 1,934
Also, we actually don't need to read the new issue to compare their *old* results. A quick research on the old published issue will give us the info (which I did that already):-

RiceHigh's Pentax Blog: First K20D Production Camera Formal Test


QuoteOriginally posted by flyer Quote
In the july issue, Pop Photo has a comparison test of high ISO cameras. In the test results, they rated the noise level of the K20D at ISO 3200 and 6400 as unacceptable. What ticks me off is the fact (stated in the article) that they left the NR off on the K20D while all the other contenders had high ISO N.R. on. It shows that Pop Photo has a strong dislike of Pentax cameras to use such tricks to discredit a model. They should use the same conditions for all the cameras they test for a particular issue. They implied that they left the camera in the state it comes in the box (default setting), but they turned the high ISO on on the Olympus E-3. If you look only at the results without reading the whole article, it could mislead a lot of readers. Why turn high ISO n.r. on for the Olympus and not do it for the K20D? POP PHOTO, YOU STINK


06-14-2008, 06:28 AM   #5
Pentaxian
jsherman999's Avatar

Join Date: Dec 2007
Photos: Gallery
Posts: 8,228
QuoteOriginally posted by RiceHigh Quote

...
Anyway, I would say those noise tests nowadays by the measurbators are not so meaningful as long as noises are measured along instead of the Signal-to-Noise (aka S/N) ratio.
...

You're right about that, RH, so many people seem to not understand the S/N ratio concept - and I seriously mean really do not understand it. I'm starting to come to the dismaying conclusion that even some semi-pro reviewers for publications like Pop Photo don't quite get the concept - which would be equivalent to a writer for Car & Driver not quite getting the concept of, say, fuel injection.

If you're a reviewer, it's your responsibility to know this stuff, and to not misrepresent measured results by using out-of-context frames of reference.

The S/N ratio of the K20D is nearly class-leading, if not at the very top of it's class. The fact that this has not been spelled out in big bold letters means that reviewers either a) discount S/N inappropriately, b) don't really fully understand the concept, and thus skim over it, or c) assume everyone already understands it, so it doesn't need to be discussed. Meanwhile, it continues to be misunderstood by a good portion of the buying public as well as a good number of secondary reviewers/ review referrers.




.

Last edited by jsherman999; 06-14-2008 at 06:37 AM.
06-14-2008, 06:38 AM   #6
Veteran Member
cupic's Avatar

Join Date: Apr 2008
Location: Australia-NSW
Photos: Gallery
Posts: 4,724
You cant compare apples to pears

cheers
06-14-2008, 06:41 AM   #7
Veteran Member




Join Date: Jan 2008
Location: Ste-Anne des Plaines, Qc., Canada
Posts: 2,014
Original Poster
QuoteOriginally posted by RiceHigh Quote
So, what's the big deal?
The BIG DEAL is that the Olypus E-3 comes off with N.R. off as the default setting and they turned it on. They can't even respect their own protocol, which makes them as biased as you. That is the point.
06-14-2008, 08:08 AM   #8
Senior Member




Join Date: Sep 2007
Location: Australia
Posts: 255
I never understood the point of this "TEST AT OUT OF BOX DEFAULT SETTINGS" business. Ok, from a practical point of view, they might not have time or ability to test EVERY setting, sure.

But its a DSLR. It is absolutely meant to have its settings changed, fiddled with, adjusted to suit the person, the shot, the place, the lighting, you name it. That's the entire point of the camera.

You find a major flaw, you go look if there is a setting to change. This is obvious, its what you do with anything you buy. Don't you change the setting of the seats in a new car? Fiddle with the settings on your new mobile phone?

And if its as blindingly obvious as a setting as noise reduction factor in a NOISE TEST, then you fail in life if you don't check that.

I hope they don't test waterproof cameras one day, and one is sent as factory default with the card door open or something.

The Underwatermagictron 2000 failed completely in our underwater testing, water poured in and destroyed the camera. Unacceptable!

06-14-2008, 08:11 AM   #9
Veteran Member




Join Date: Nov 2006
Location: Hong Kong
Posts: 1,934
QuoteOriginally posted by flyer Quote
The BIG DEAL is that the Olypus E-3 comes off with N.R. off as the default setting and they turned it on. They can't even respect their own protocol, which makes them as biased as you. That is the point.
So what then? Do you expect every measurebation test conducted by each measurbator to be the same each time? Ask yourself if the test results are favourable to Pentax, will you make the complaint? (which the case is just of no big deal IMO)

As long as the reviewers let you know about the settings, I think it is already fair enough. Whether their published results are meaningful or not is another story.
06-14-2008, 09:35 AM   #10
Pentaxian
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,862
QuoteOriginally posted by RiceHigh Quote
I would say those noise tests nowadays by the measurbators are not so meaningful as long as noises are measured along instead of the Signal-to-Noise (aka S/N) ratio.
RH, well said.

But things are improving, slowly.

Recent German magazine "Color Foto" decided they must redo all camera tests (29 DSLRs retested) because they "forgot" to notice the disappearance of texture detail in some models.

Some of those "preservation of texture" values read (bigger is better, max. 15, ISO 100/400/800/1600):

Pentax K20D(Samsung GX20): 15/15/15/14.5(15) (or 100%!)
Pentax K200D: 15/15/15/13.5
Pentax K100Dsp: 13/13.5/13.5/12
Nikon D300: 14.5/14.5/14/14 (or 98%)
Nikon D3: 14.5/14.4/14/14.5
Leica Digilux 3: 15/14.5/14.5/14.5
Canon EOS 40D: 10/10.5/10.5/8.5 (or 66%)
Sony Alpha 700: 9/8.5/9/10.5 (or 62%)

So, if you just would weight category "preservation of texture" high enough, Pentax would win the entire competition. Of course, they didn't. But they ended up at a very respectful position.
06-14-2008, 10:09 AM   #11
Veteran Member




Join Date: Jan 2008
Location: Ste-Anne des Plaines, Qc., Canada
Posts: 2,014
Original Poster
[QUOTE=RiceHigh;268196]So what then? Do you expect every measurebation test conducted by each measurbator to be the same each time? Ask yourself if the test results are favourable to Pentax, will you make the complaint? (which the case is just of no big deal IMO)

As long as the reviewers let you know about the settings, I think it is already fair enough. Whether their published results are meaningful or not is another story.[/QUOTE
Comparison tests, to be valid, should ALWAYS be done under the same conditions, not the conditions that suit the tester. By the way, what makes you an "expert" on the K10D/K20D. As far as I know, you never owned or used any of the two mentioned cameras, yet you keep bringing them down. If those cameras are as bad as you imply, how come the K10D won so many awards? As far as I'm concerned, you wouldn't know a good camera if it bit you on the a**.

P.S. Answer if you want' I don't care and won't answer back. You always need to be right, even when you're wrong.
06-14-2008, 09:10 PM   #12
Veteran Member
morfic's Avatar

Join Date: Feb 2008
Location: San Antonio, TX
Photos: Gallery
Posts: 428
falconeye: those are impressive numbers, no wonder i love the K20D so much over the 30D. Makes me wanna see if i can subscribe to Color Foto from the US.
06-14-2008, 10:35 PM   #13
Veteran Member




Join Date: Jun 2007
Location: Buffalo/Rochester, NY
Photos: Albums
Posts: 1,133
Testing with settings out of the box certainly shows that most of these "reviewers" are catering to the masses who think SLRs are nothing more than fancy point and shoots with lots of extra buttons.

Who in their right mind would assume that default settings are the best settings? Well, except maybe Microsoft.
06-15-2008, 07:03 AM   #14
Inactive Account




Join Date: Apr 2008
Posts: 464
I don't see a problem with this methodology.
------------------

Let's assume Popphoto does alter camera settings -- Guess what happens? You have people complaining "we all know that Setting X is better than Setting Y in the camera but they used the worse setting!"

PopPhoto is a mass-market photography magazine -- they are to cameras as Ziff-Davis publishing is to computers, so their reviews should be taken with a grain of salt.

That being said, testing cameras in factory defaults is about the only fair way to do things -- but, in a case such as this (all cameras having NR on except the Pentax), a special note should be mentioned.
06-15-2008, 11:45 AM   #15
Pentaxian
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,862
QuoteOriginally posted by cputeq Quote
That being said, testing cameras in factory defaults is about the only fair way to do things
cputeq, you're right for a tester with average (read insufficient) qualification.

Let me give an example from the testing methodology of German computer magazine c't.
IT news, features and forums - heise online UK
and
c't - magazin fr computertechnik
They have an outstanding reputation, not met by any photo magazine, or any other computer magazine I am aware of, on a worldwide scale...

What do they do?

First, they live with the products long enough to know them.

Second, they reconfigure the products to bring them to their best value, even sometimes making the manufacturer give them an updated model following their input. This means, exchanging drivers, running updates, maybe exchanging too loud fans etc. They do what an actual buyer would have done.

Third, they judge the individual aspects. They do not give an overall score!


I don't see why a photo mag couldn't do this.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
article, camera, cameras, dslr, iso, issue, k20d, olympus, photo, photography, pop, pop photo, test
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
K-7 has graced the cover of the September issue of Pop Photo as a "Best Buy!" Matjazz Pentax DSLR Discussion 33 11-04-2012 11:57 AM
Photo of the Week #122, July 11 - July 25, 2010 VaughnA Weekly Photo Challenges 47 07-24-2010 10:45 PM
Photo of the Week #121, July 4 - July 18, 2010 jmschrei Weekly Photo Challenges 50 07-18-2010 10:01 AM
Pop Photo K20d review jms698 Pentax News and Rumors 136 02-25-2008 01:00 PM
Pop Photo roy Pentax News and Rumors 1 01-30-2008 03:42 PM



All times are GMT -7. The time now is 10:20 PM. | See also: NikonForums.com, part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top