Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
07-27-2014, 05:34 AM   #31
Veteran Member
bossa's Avatar

Join Date: Jul 2011
Location: Adelaide
Posts: 4,546
QuoteOriginally posted by Digitalis Quote
yeah there is something suspicious in that result though there is a possiblity that the K5 has an AA filter that is stronger than the D7000 - but with the same lens, even when taking sample variation into account it suggests that they may indeed use default camera settings, opposed to the settings that get the most out of the camera that most dedicated users will know about.
Sorry Doug I deleted that image.. I'll upload it again below:

Same Sensor, Same Lens (model) completely different results.

Previously deleted post below----------------------------------------------------------------------------------------------------------------------------

I think DxO must work with default NR settings because scores with the same lens on the same sensor (K-5 vs D7000) gets lower scores for the Pentax.

Sigma 35mm F1.4 DG HSM A Pentax versus Sigma 35mm F1.4 DG HSM A Nikon on Nikon D7000 - Side by side lens comparison - DxOMark

As I said in the post I removed, they must use default NR settings that lower resolution.

Attached Images
   

Last edited by bossa; 07-27-2014 at 05:48 AM.
07-27-2014, 05:56 AM   #32
Veteran Member




Join Date: Dec 2013
Photos: Gallery
Posts: 603
While we are at it, check the link : Sigma 35 mm F1.4 Art on K3 and D7100 . The data looks suspicious in this case, because according to it, D7100, performs better. Same lens, same sensor, different results.
07-28-2014, 03:42 PM   #33
Loyal Site Supporter
Loyal Site Supporter




Join Date: Aug 2009
Location: New England
Photos: Albums
Posts: 706
I found a good use for DXO the other day. I've always wanted to know if the earlier version of the Sigma 50-500mm lens (the non-OS version) was better or worse than the later (OS) version for how I shoot. I've seen a lot of vague comments from people and reviewers over time, but I found that the DXO database had both versions of the lens tested in Canon mount. At that point, it was just a matter of pulling up a comparison of those two lenses on the same body on their site and then really drilling down into the data (I could care less about their overall ratings). I primarily shoot the 50-500mm at >300mm and at f/8, and I found out that the older version I have is better at 400mm and 500mm at f/8 than the newer one, and not as good at 300mm. Since I care more about the longer end, that is exactly what I needed to know to stay with the version of the lens I have, and I was very thankful DXO performed those tests.

In an ideal world, a person would build their own user profile on DXO's site where they rate what they care about for how they'd shoot with lenses of given focal lengths, then that user's preferences would be factored in to generate personalized scores for lenses and bodies. Their attempt at a one-size-fits-all numerical rating is just laughable.
07-28-2014, 04:23 PM   #34
Pentaxian
normhead's Avatar

Join Date: Jun 2007
Location: Near Algonquin Park
Photos: Gallery | Albums
Posts: 40,442
DxO is clearly biased, they say so right on their website. They test in a basement with the type of lighting you'd have in a typical living room. Their testing is biased because they shoot in extraordinary light, and then pretend like their results are applicable to normal shooting conditions. If you bias your science, by shooting in abnormal conditions, that's scientific bias, as in bias built into the design of the experiment.

07-31-2014, 10:34 AM   #35
Loyal Site Supporter
Loyal Site Supporter
WPRESTO's Avatar

Join Date: Aug 2010
Location: Massachusetts
Photos: Gallery | Albums
Posts: 58,951
QuoteOriginally posted by GeneV Quote
Absolutely. I present evidence in court for a living, and I often deal with a certain "scientific" field. Some cases are very clear, but for many, I can pretty much tell you who to call in this field if you want it to be more likely that you will get "x" result. It will take the skill of another "scientist" biases in another direction to point out how the design of the test and the reporting favor that result.
IMHO as someone who is/was considered a "scientist." The core of science is the misunderstood "scientific method" the real core of the which is an "if--then" hypothesis or theory or prediction. Example: IF water is made of hydrogen and oxygen in a 2:1 ratio, THEN, every time water is broken down into it's components, we should obtain two parts hydrogen to one part oxygen. If sometimes we get 3:1, or 1:1, or sometimes fail to get hydrogen or oxygen, or both, the proposition/hypothesis/theory is incorrect. MEASUREMENT is a scientific tool, but in and of itself is not science. How we measure the amount of hydrogen & oxygen obtained is limited by measurement devices, methods, and even interpretation. If our measurements indicate 1.9999876 hydrogen to 1.00000000165 oxygen, is the hypothesis true or false? There are statistical rules for "within the margin of error," or "statistically insignificant deviation from expectation," etc. The testing of lenses & cameras is a process of measurement, NOT SCIENCE. The results may be "objectively" interpreted by machines, or subjectively by humans (as with evaluating a test chart image). Ultimately the critical question is: is the obtained image satisfactory? Insofar as it is possible to "over sharpen" an image, and soft lenses are used on purpose, there seems little point in worrying if someone else's lenses are better. I presume anyone deeply concerned about resolution would own nothing but prime lenses, and probably no focal length greater than about 200mm, and generally lenses with the widest possible f-stop insofar as diffraction is a function of diameter. If you look at most lens resolution tests, DXO or other, for a given focal length the highest rated lenses almost always the largest F-stop (f1.4 for lenses 24 to 85mm; f2 for lenses 100 to 200mm; f2.8 for 300 & 400mm).
08-01-2014, 05:50 AM   #36
Pentaxian
normhead's Avatar

Join Date: Jun 2007
Location: Near Algonquin Park
Photos: Gallery | Albums
Posts: 40,442
QuoteOriginally posted by WPRESTO Quote
If you look at most lens resolution tests, DXO or other, for a given focal length the highest rated lenses almost always the largest F-stop (f1.4 for lenses 24 to 85mm; f2 for lenses 100 to 200mm; f2.8 for 300 & 400mm).
hmmmmmm.... I know my DA*60-250 ƒ4 @200 is sharper than the DA* 200 2.8, but maybe I need bigger sample. Something to look into next time I'm injured and can't get out to take pictures. That's a very interesting take on the subject... I'm not sure I've encountered it before. Thanks for posting.

I've always said, until someone can explain to me why I like my 21 ltd images so much... sharpness is guideline not a pre-requisite. It's not the sharpest, it doesn't have the stellar CA control, but for me it's usually the first prime out of the bag. The 35 ƒ2.4 is faster and sharper, but I would never use it in a place where I could use the 21. I'm sure the laws of physics could explain that, but there isn't any person on earth that I know who could show me what's special about the implementation of those laws in the 21 that makes it special. There's certainly no clues in the test charts.

My guess is that lenses that lenses that render the way the eye sees are valued more than lenses that are more technical in nature. I love my Sigma 8-16 for architecture.. for people or scenery my 10-17 often takes much more appealing images.

QuoteQuote:
In an ideal world, a person would build their own user profile on DXO's site where they rate what they care about for how they'd shoot with lenses of given focal lengths, then that user's preferences would be factored in to generate personalized scores for lenses and bodies. Their attempt at a one-size-fits-all numerical rating is just laughable.
If you look at the sample variations as recored at lensrentals.com you realize that sites like DxO are essentially irresponsible until they are testing at least 10 samples of a given lens and averaging the conclusions. When comparing closely matched lenses, my guess would be a third of the time they get a good sample of an average lens design and a poor sample of a better designed lens and the lenses come out in the wrong order on their charts. I wouldn't base any decision on any ratings, where the results are within 200 lw/ph or where I don't have access to the images that were rated. That leaves DxO out, on both counts.

Last edited by normhead; 08-01-2014 at 06:24 AM.
08-01-2014, 07:49 AM   #37
Loyal Site Supporter
Loyal Site Supporter
WPRESTO's Avatar

Join Date: Aug 2010
Location: Massachusetts
Photos: Gallery | Albums
Posts: 58,951
QuoteOriginally posted by normhead Quote
hmmmmmm.... I know my DA*60-250 ƒ4 @200 is sharper than the DA* 200 2.8, but maybe I need bigger sample. Something to look into next time I'm injured and can't get out to take pictures. That's a very interesting take on the subject... I'm not sure I've encountered it before. Thanks for posting.

I've always said, until someone can explain to me why I like my 21 ltd images so much... sharpness is guideline not a pre-requisite. It's not the sharpest, it doesn't have the stellar CA control, but for me it's usually the first prime out of the bag. The 35 ƒ2.4 is faster and sharper, but I would never use it in a place where I could use the 21. I'm sure the laws of physics could explain that, but there isn't any person on earth that I know who could show me what's special about the implementation of those laws in the 21 that makes it special. There's certainly no clues in the test charts.

My guess is that lenses that lenses that render the way the eye sees are valued more than lenses that are more technical in nature. I love my Sigma 8-16 for architecture.. for people or scenery my 10-17 often takes much more appealing images.
An initial suspicion about your preference for the 21mm over the sharper 35mm is that the perspective may be more pleasing, tending to bring out a foreground subject and push the background away. As I wrote - and it's not a deep or original insight with me - what matters is whether a lens gives satisfactory results, not whether it is the sharpest optic of that focal length. And there is convenience. Zooms are enormously convenient, and may capture a fleeting moment lost to the process of changing SFL lenses. There are many trade-offs.
Not too surprised that the 60-250mm can at least match the 200mm f2.8. Zooms of the roughly 70-200mm range (pioneered, I believe, by the Vivitar Series 1), commonly f2.8, include some with excellent sharpness, matching or surpassing many SFL lenses within that zoom range. BUT, generally, (not always), the f2.8 zooms of 70~200mm have higher DXO ratings than f4.0 zooms of the same range from the same manufacturer.
I will confess to being at times obsessed by resolution, and making mistakes all ways, purchasing a lens with mediocre rating for convenience and finding it unsatisfactory, purchasing a lens with outstanding rating and finding no better than what I already owned, taking SFL lenses for quality in addition to a zoom and never taking the former out of the backpack. I must have 20 lenses now (afraid to count) some of which sit lonely and neglected for many months, some not used for years (should put them in PF for sale). Maybe some day.

08-01-2014, 07:57 AM   #38
Loyal Site Supporter
Loyal Site Supporter




Join Date: Mar 2009
Location: Gladys, Virginia
Photos: Gallery
Posts: 27,603
QuoteOriginally posted by normhead Quote
hmmmmmm.... I know my DA*60-250 ƒ4 @200 is sharper than the DA* 200 2.8, but maybe I need bigger sample. Something to look into next time I'm injured and can't get out to take pictures. That's a very interesting take on the subject... I'm not sure I've encountered it before. Thanks for posting.

I've always said, until someone can explain to me why I like my 21 ltd images so much... sharpness is guideline not a pre-requisite. It's not the sharpest, it doesn't have the stellar CA control, but for me it's usually the first prime out of the bag. The 35 ƒ2.4 is faster and sharper, but I would never use it in a place where I could use the 21. I'm sure the laws of physics could explain that, but there isn't any person on earth that I know who could show me what's special about the implementation of those laws in the 21 that makes it special. There's certainly no clues in the test charts.

My guess is that lenses that lenses that render the way the eye sees are valued more than lenses that are more technical in nature. I love my Sigma 8-16 for architecture.. for people or scenery my 10-17 often takes much more appealing images.



If you look at the sample variations as recored at lensrentals.com you realize that sites like DxO are essentially irresponsible until they are testing at least 10 samples of a given lens and averaging the conclusions. When comparing closely matched lenses, my guess would be a third of the time they get a good sample of an average lens design and a poor sample of a better designed lens and the lenses come out in the wrong order on their charts. I wouldn't base any decision on any ratings, where the results are within 200 lw/ph or where I don't have access to the images that were rated. That leaves DxO out, on both counts.
To me, they eye test is pretty crucial. I find that my DA *200 is pretty sharp at f2.8 and really sharp at f4. You sort of learn what sort of detail you can process out of images produced from a particular lens. On the other hand, the DA 55-300 needs to be shot at f8 to get the same sort of detail. It can be done, but a lot tougher to hand hold. But you are right. One copy does not tell the whole story, but if I have a lens that isn't giving the results I want, I will have Pentax look at it to see what can be done to fix it.

(DA *200 at f4).

08-01-2014, 10:40 AM   #39
Pentaxian




Join Date: Feb 2010
Location: Northern Michigan
Photos: Gallery | Albums
Posts: 6,155
QuoteOriginally posted by normhead Quote
I've always said, until someone can explain to me why I like my 21 ltd images so much... sharpness is guideline not a pre-requisite.
When you reach the better type of lenses, I would say that sharpness can no longer even be a reliable guideline, simply because it ceases to be of any real significance with a lens like the DA 21. What APS-C lenses, or what K-Mount lens for that matter, is significantly sharper at 21mm? The trouble with using sharpness for high-end lenses is that it causes people to obsess over small differences of sharpness. The fact that lens A is 20% sharper than lens B may seem, if we focus merely on numbers, significant. 20% can seem significant, particularly for those who equate lens performance with resolution. But what does it mean in terms of practical output? It means that you can print 20% larger and retain the same level detail (but only at print sizes that where that extra sharpness matters). So if the maximum for getting sufficient detail for lens A would yield a 20" by 20" print, lens B would allow one to make a 22" by 22" print and retain the same level of detail. How significant is two inches?

When sharpness doesn't come into play (or is largely insignificant), then other factors of lens performance rise in prominence. When I look at web-sized images of the DA 21, what I see and what is most apparent is not sharpness (at web size they're plenty of sharp, and therefore can't be differentiated, in terms of sharpness, from in-focus images taken with other lenses, most of which are also plenty sharp at web-size resolution), but the excellent contrast and color rendition of the DA 21. The DA 21 produces images that have superb contrast and bite, along with rich, vivid, aesthetically satisfying colors. While you could crank up the contrast and saturation in post to make up for these elements, the fact is the contrast and saturation you get from a lens usually looks better from what you get in post, especially in prints. Lens contrast is primarily micro-contrast, a higher quality type of contrast that can't really be duplicated in post. Micro-contrast allows you to preserve tonal gradients throughout the image (including details in highlights and shadows) while still producing a contrasty image.
08-09-2014, 04:15 PM   #40
Veteran Member
GeneV's Avatar

Join Date: Oct 2008
Location: Albuquerque NM
Photos: Albums
Posts: 9,830
QuoteOriginally posted by northcoastgreg Quote
The trouble with using sharpness for high-end lenses is that it causes people to obsess over small differences of sharpness. The fact that lens A is 20% sharper than lens B may seem, if we focus merely on numbers, significant. 20% can seem significant, particularly for those who equate lens performance with resolution. But what does it mean in terms of practical output?
I agree 100% about the obsession.

I also think we may be to such a fine point of performance that every test should average several copies.
08-09-2014, 04:32 PM   #41
Unregistered User
Guest




QuoteOriginally posted by Steve.Ledger Quote
Bias requires that evidence obtained through objective and scientific testing is ignored in favor of a personal preference. The question is therefore illogical.
Please explain to us how the question is illogical. Instead of merely blurting out smug yet meaningless retorts, it might be helpful if you would specify just what it is that you are trying to say - albeit in a highly clumsy manner. Regardless of the spin you will put to it, the fact remains that the question "Is DxO biased against Pentax products?" is not illogical. Indeed, logic carries no relevance whatsoever to the matter on hand.
08-09-2014, 04:36 PM   #42
Veteran Member




Join Date: Oct 2009
Location: Far North Qld
Posts: 3,301
Well as you ask so nicely.. No :P
08-09-2014, 04:59 PM   #43
Unregistered User
Guest




QuoteOriginally posted by hjoseph7 Quote
Is DxO biased against Pentax products ? When I read some of their reviews its like they consistently rate Pentax lenses low. Meanwhile I read other reviews about the same lens and the owners rate the lens pretty high. It seems like Nikon is given the green-light and consistently scores high on DXO's pixel peeping charts. Maybe they are not getting enough moola under the table from Pentax.
The question you should ask is the following:

Does the clout that big companies carry in terms of spitting money into the coffers of a media outlet per advertisement or a research institution have a bearing on editorial decisions and research findings?

The answer to this question is an unequivocal 'yes.'

Granted that Pentax do indeed do well in some of the the tests carried out by dxomark, would these minor triumphs make a practical difference in terms of increasing Pentax's sales figures? Doubtful.

Do a background check on DxOmark's board of writers and it shouldn't be hard to determine whose pocket they are in. The photography industry is relatively transparent; hence your task shouldn't be too difficult.

Last edited by Unregistered User; 08-09-2014 at 06:08 PM.
08-09-2014, 05:14 PM   #44
Pentaxian
normhead's Avatar

Join Date: Jun 2007
Location: Near Algonquin Park
Photos: Gallery | Albums
Posts: 40,442
I shoot in daylight, DxO doesn't test in daylight... end of story.
08-09-2014, 09:58 PM   #45
Veteran Member




Join Date: Dec 2007
Photos: Gallery
Posts: 8,237
QuoteOriginally posted by ElJamoquio Quote
They aren't biased at all.

Their algorithm basically considers sharp lenses and fast lenses at the cost of almost everything else (they might be considering CA and distortion to a minor degree).

If I were to make a $100, 100 gram, 10-600mm F/4 that had gorgeous bokeh, it would score poorly on DxO's test.

If I were to make a $10,000, 2000 gram, 50mm F/1.2 that had shitty bokeh and turned everything fluorescent green, it would score very well.


One more reasonable example - the 15mm Limited, which I think renders beautifully, is a joy to use, is nice and light, and as a 15mm doesn't really require that fast of an aperture, scores very poorly.
+1, also the 'score' is derived from across-the-frame sharpness, so a lens designed for center sharpness can rate softer even though it's brutally sharp in the center of the frame (where it potentially is used more often.)

DXO is actually very useful for sensor metrics and ratings - their lens scores have much less applicability in real-world situations.


Last edited by jsherman999; 08-09-2014 at 10:13 PM.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
dxo, k-mount, lens, pentax, pentax lens, rate, reviews, slr lens
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Biased Ratings in Lense Reviews Belgarchi Pentax SLR Lens Discussion 13 01-22-2013 05:56 AM
DxO are discounting DxO Optics Pro again rawr Digital Processing, Software, and Printing 4 11-09-2011 01:00 PM
DXO are discounting DXO Optics Pro again until 14 June rawr Digital Processing, Software, and Printing 17 06-25-2011 07:52 AM
Biased Review or Not? Eric Seavey Pentax K-5 & K-5 II 8 11-17-2010 05:56 AM
DXO are giving 30% off DXO Optics Pro until Dec 25 rawr Digital Processing, Software, and Printing 13 11-11-2010 01:22 PM



All times are GMT -7. The time now is 01:29 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top