Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
12-12-2009, 12:19 PM   #31
Veteran Member
er1kksen's Avatar

Join Date: Oct 2008
Location: Forestville, NY
Photos: Albums
Posts: 1,801
QuoteOriginally posted by falconeye Quote



This isn't exactly true.

Don't look at the DxOMark. It is bullshit. BUT! Look at the individual graphs which, when read correctly, are the best lab results being published to date. Esp. the Full SNR curves (not available in compare mode!).

You will see that DxO derives almost identical graphs (SNR 18%, Dynamic Range) for K10D, K200D and K-m.

These graphs are relevant.


The DxOMark score is bullshit for an obvious reason: it is independent of sensor resolution! A higher resolving sensor tends to score worse (*) while it should score better. Otherwise, a 4 pixel sensor will win every DxOMark competition

__
(*) DxO normalizes for resolution. But this is not enough when measuring DR. They would have to do their measurements on downscaled images which they can't because they use the mosaiced raw data.

Careful, there. I said the scores are irrelevant, not the graphs. I agree that the graphs (when normalized for resolution using the print option) are probably the best available. I was just trying to answer the earlier posters who seem to be overreacting to the "dissapointing" scores.

I really wonder what it is they base the scores off of. The example of the E-410 vs. E-510 is what really perplexes me; their imaging hardware is exactly identical in every way, as is their image-processing firmware. There's probably more sample variation within each of the models than there is a difference on average between them. The dxomark graphs should be completely identical (I haven't checked them, though). Why are the scores so different?

12-12-2009, 12:41 PM   #32
Veteran Member
PentaxPoke's Avatar

Join Date: Jan 2008
Location: Oklahoma
Posts: 1,411
I disagree with some here that say that the scores are BS. Are they BS because our camera didn't score well? I guarantee that when the k-x comes in with scores that beat cameras over twice the cost, people will suddenly change their minds about the scores.

I actually think that the low-light ISO score is relevant and useful. It is better than a qualitative evaluation of noise. It is a score that has some real-world value because it essentially answers the question: "how high can I set ISO and still get an excellent image?" I have oversimplified it but here is the actual definition:

"An SNR value of 30dB reflects an excellent image quality. Low-Light ISO is then the highest ISO setting for the camera such that the SNR reaches this 30dB value while keeping a good dynamic range of 9EVs and a color depth of 18bits. "

So in other words, the K-7 hits this limit at an ISO of 536 and the 7D hits this limit at an iso of 809, and the k20d hits the limit at 639.

This confirms what many here said about the noise of the k20d vs the K-7. I didn't want to believe it myself, but now the evidence is overwhelming.

Look, I know we are are trying to be whistling past the graveyard here, but we have all known that the high-iso performance on the K-7 is not up to the level of the competition. Hence the number of threads here talking about how they should put the Sony sensor in the K-8. As a camera, the K-7 has some other excellent qualities that many of us appreciate. Let's not be blinded to one of its limitations.

Last edited by PentaxPoke; 12-12-2009 at 12:47 PM.
12-12-2009, 02:08 PM   #33
m8o
Veteran Member
m8o's Avatar

Join Date: Jul 2007
Location: 40°-55'-44" N / 73°-24'-07" W [on LI]
Photos: Gallery | Albums
Posts: 3,092
QuoteOriginally posted by JohnBee Quote
[analysis snipped]...We commonly shoot K20's at ISO3200 - 4000 with reliable results, whereas the K-7 couldn't provide us with the same workable output at such sensitivities...
Great analysis. One thing I need to know however. Is this in RAW or JPG?

DPReview as an example had stated similar, but also stated to shoot RAW to alleviate. Is your analysis based on JPEG output, or RAW + PP output? much Thanx.

My K10D doesn't AF for shiet anymore. I thought I was [finally] going to get a K7D soon. Hadn't been reading the news forum section for about a year. Then started reading all this. Amazing there are no K20ds to buy anymore; plus I suspect them to be the "returned lemons of the bunch" by this time.
12-12-2009, 02:09 PM   #34
Veteran Member
JohnBee's Avatar

Join Date: Sep 2008
Location: Newrfoundland
Photos: Albums
Posts: 4,667
QuoteOriginally posted by PentaxPoke Quote
Look, I know we are are trying to be whistling past the graveyard here, but we have all known that the high-iso performance on the K-7 is not up to the level of the competition.
lol


QuoteOriginally posted by m8o Quote
Great analysis. One thing I need to know however. Is this in RAW or JPG. DPReview as an example had stated similar, but also stated to shoot RAW to alleviate. Is your analysis based on JPEG output, or RAW + PP output? much Thanx.
RAW, I don't think our units have shot a single JPG frame since we've been shooting Pentax.
So a typical high ISO workflow(depending on the scene) would be, PEF > GBG TINGE REMOVER > ACR(0,0,0) > TIFF > NN(Noise Ninja) or CS4(Define 3) > OUTPUT.
Seems a little extensive but it's actually quite effortless when you get familiar with the types of noise and output objectives.

PS. we've really taken a liking to the latest version of Define's noise removal software. It leaves wavelet(noise ninja, neatimage etc) noise reductions technology in the dust(so to speak). and has put K20D ISO6400 in the map in a very big way.


Last edited by JohnBee; 12-12-2009 at 02:17 PM. Reason: Old age, dislexia and innadequate schooling
12-12-2009, 02:11 PM   #35
Pentaxian
gazonk's Avatar

Join Date: Feb 2009
Location: Oslo area, Norway
Photos: Gallery | Albums
Posts: 3,746
QuoteOriginally posted by PentaxPoke Quote
I
This confirms what many here said about the noise of the k20d vs the K-7. I didn't want to believe it myself, but now the evidence is overwhelming.
Isn't that because K-7 has less "cooked" high iso raw than the K20d?

Anyway, it's getting increasingly obvious that Pentax needs to come up with a "K-8" that has a quite different sensor than the current one. I would have passed the K-7 anyway (bought the K10d 2 years ago, and feel that I have only scratched its possibilites - and lenses are better investments anyway), so I'm already looking forward to the "K-8" :-)

In the meantime, Pentax seems to sell a lot of the K-x in Japan, they're in a good position now, 2010 is going to be a wonderful year for pentaxians, and certainly not only because of the 645D.
12-12-2009, 02:35 PM   #36
m8o
Veteran Member
m8o's Avatar

Join Date: Jul 2007
Location: 40°-55'-44" N / 73°-24'-07" W [on LI]
Photos: Gallery | Albums
Posts: 3,092
Thanx John.
12-12-2009, 04:21 PM   #37
Veteran Member
er1kksen's Avatar

Join Date: Oct 2008
Location: Forestville, NY
Photos: Albums
Posts: 1,801
QuoteOriginally posted by PentaxPoke Quote
I disagree with some here that say that the scores are BS. Are they BS because our camera didn't score well? I guarantee that when the k-x comes in with scores that beat cameras over twice the cost, people will suddenly change their minds about the scores.
I for one will feel the exact same way about the numeric scores regardless of what results the K-x gets. I've been saying the scores are meaningless since long before the K-7 scores were published. The graphs dxomark produces are useful and legitimate. The numerical scores and sensor rankings are a different store. For the most obvious example, see the scores of the E-420 and E-520 as placed on that chart in the OP. There are four other cameras in between them in ranking, including the K-7.

Just for reference, everything about the image-capture systems (hardware and firmware) is absolutely identical between the E-420 and E-520. The results should be pretty much identical as well.

Now look at the actual graphs. On the SNR graphs, both screen and print mode, they are almost exactly identical. The difference in numeric score and ranking is a bit questionable. Even stranger, when we add in the K-7 (which scored three places above the E-420 and one place below the E-420 on the chart), noise levels at a given ISO are almost identical in the "screen" graph, and a fair bit better on the "print" graph. Switching the K-7 for the Sony A230 (which also ranks between the E-420 and E-520), the A230 is better according to both the screen and print graphs.

Where do the numbers come from? The graphs show how the cameras perform. The numbers don't really show much of anything.

QuoteQuote:
Look, I know we are are trying to be whistling past the graveyard here, but we have all known that the high-iso performance on the K-7 is not up to the level of the competition. Hence the number of threads here talking about how they should put the Sony sensor in the K-8. As a camera, the K-7 has some other excellent qualities that many of us appreciate. Let's not be blinded to one of its limitations.
I really don't think most of us are "whistling past the graveyard." The K-7 is about as noisy as most of the competition according to the graphs on DXO and the RAW noise tests done at other review sites. Based on my experience with the K20D it's probably a little worse in real life than the review sites would suggest. But the fact is that people are getting upset over some pretty meaningless numbers, when what they really need to do are look at the graphs.

12-12-2009, 05:10 PM   #38
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by er1kksen Quote
Why are the scores so different?
DxO has a score system based on thresholds rather than integration of curves.

E.g., imagine two similiar and very flat curves of SNR vs. ISO, and a threshold where the curves crosses a SNR value, and the corresponding crosspoint ISO value enters the score (actually, this is what they do ...).

Then, the integral over both curves would be very similiar, yet the score would differ a lot.

Mathematically, this is known as an unstable function, where small variations of input can produce large variation of output. Sometimes known as the butterfly effect (when it occurs in iterated functions, not applicable here).

DxO does a decent job with their curves but screws it up with their score function when using a mathematically unstable function.

I've already thought about creating an alternative robust score function and publish an alternative DxOmark figure based on their curves...

QuoteOriginally posted by PentaxPoke Quote
I actually think that the low-light ISO score is relevant and useful.
It isn't wrong but can be very misleading. Cf. above.
I really recommend to use DR over ISO and tonal range over ISO curves as a first guide. If the curves have no flat portion then the low-light ISO score comparison can be useful as well.
12-12-2009, 10:15 PM   #39
Veteran Member
Otis FanOtis FanOtis FanOtis FanOtis FanOtis Fan
Rupert's Avatar

Join Date: Sep 2006
Location: Texas
Photos: Gallery | Albums
Posts: 25,123
THe ISO factor is important to many of us, and a step backward is not the way other camera makers are going, so how Pentax thought this would fly is beyond logic to me. I love my K20D but just have not been able to convince myself that a step backwards in IQ and low light performance is beneficial, even with the nice features in the K7. Yes, they should have used the Kx sensor, then they would have had a killer camera, but they didn't, did they.....

Combine this with the fact that in a small Metroplex comunity of 8 million where I live, there is not a retail outlet around where you can actually handle a K7, and it is a "no buy" for me. Maybe Pentax will get serious about cameras in the future, and about camera marketing, but until they do Iwill stick with my K20D and hope their executives stop smoking whatever it is they are smoking.....
Regards
12-12-2009, 11:59 PM   #40
Veteran Member
er1kksen's Avatar

Join Date: Oct 2008
Location: Forestville, NY
Photos: Albums
Posts: 1,801
QuoteOriginally posted by falconeye Quote
DxO has a score system based on thresholds rather than integration of curves.

E.g., imagine two similiar and very flat curves of SNR vs. ISO, and a threshold where the curves crosses a SNR value, and the corresponding crosspoint ISO value enters the score (actually, this is what they do ...).

Then, the integral over both curves would be very similiar, yet the score would differ a lot.

Mathematically, this is known as an unstable function, where small variations of input can produce large variation of output. Sometimes known as the butterfly effect (when it occurs in iterated functions, not applicable here).

DxO does a decent job with their curves but screws it up with their score function when using a mathematically unstable function.

I've already thought about creating an alternative robust score function and publish an alternative DxOmark figure based on their curves...
That makes sense. Its really unfortunate that people go placing so much weight in the resulting numbers without really understanding what the numbers are derived from.
12-13-2009, 01:15 AM   #41
Veteran Member
Kenn100D's Avatar

Join Date: Feb 2007
Location: Paranaque City, Philippines
Photos: Gallery
Posts: 697
its like differential calculus then integral. one wrong input = wrong answer. =)
12-13-2009, 05:50 AM   #42
juu
Veteran Member




Join Date: Sep 2008
Posts: 680
QuoteOriginally posted by falconeye Quote
I've already thought about creating an alternative robust score function and publish an alternative DxOmark figure based on their curves...
That would be great.
12-13-2009, 07:01 AM   #43
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by juu Quote
That would be great.
Thanks.

One obstacle is the retrieval of raw curve data from DxO's website. This data is currently fetched by a Flash applet and theoretically, one should be able to download the data by monitoring the TCP/IP connection. Anybody wanting to help?

BTW,
A good read is
Canon 500D/T1i vs. Nikon D5000
where DxO defends why the score can vary a lot when curves do not.

This link highlights the importance of Bayer matrix filter sensitivities as it affects the color sensitivities but NOT gray noise and dynamic range. Pentax is more like Nikon in this respect, leading to a better color sensitivity curve.
12-13-2009, 07:18 AM   #44
Veteran Member




Join Date: Mar 2009
Location: Perth Australia
Photos: Albums
Posts: 1,514
K7 performing worse than the k20d is the trade off for 4 channel readout on the sensor.... aka video.

People on this forum were speculating the 4 channel readout might result in this long before the k7 came out and high iso noise performance was confirmed a while ago.

the k7 is a stop gap, fixes a few issues with the k20d (af light, frame rate) and ads video at the price we see here.

we will see a k7x before long.

Last edited by WerTicus; 12-13-2009 at 07:30 AM.
12-13-2009, 08:25 AM   #45
Nubi
Guest




OK, let's go take some pictures.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
a500, benchmarks for k-7, color, depth, dxomark, iso, k-7, low-light, pentax news, pentax rumors, range
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
DxOMark rates Pentax K-r lexurian Pentax News and Rumors 72 11-09-2010 04:05 PM
DXOMark evaluates the 645D rawr Pentax News and Rumors 30 11-08-2010 02:59 PM
Pentax Kx Now on dxoMark taurus9 Pentax DSLR Discussion 15 03-10-2010 02:28 PM
(dxomark) K20D and K7. Reportage Pentax DSLR Discussion 20 01-13-2010 05:06 AM
DxOMark Sensor ranks yakiniku Photographic Technique 7 01-16-2009 11:01 PM



All times are GMT -7. The time now is 04:11 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top