This is such a typical "academic" type response. pointing out everything that might be wrong..... think court of law here... this is evidence. You may
have issues with the evidence, but, it's still evidence.
What I have always said is, you go to places like DxO and you see the numbers, but you don't know how to translate that information into what you're going to see in your pictures. You may have 1 stop more this or that, more noise or less noise or whatever, but what photographers are interested in would be producing images, not numerical analysis. I've been asking for over two years for someone with an FF to do an analysis like this. I don't want a bunch of mumbo jumbo. The fact is one stop difference is very small, and can usually be overcome. People say "one stop difference" as if that is some kind big deal. But you look at pictures like this, and you end up thinking, people have been wetting their pants over nothing.
And that's the thing here. What these images show more than anything else, is that under normal shooting conditions, there just isn't that mcuh difference. It explains to me why decent comparative images have never been offered, in support of theis imagined huge advantage FF is supposed to confer on it's users. The actual differences from FF to APS-c are only dramatic with numerical exaggeration. IN real life, they are pretty much insignificant. The whole FF thing that has gone on is based on hype , not images.
So, these guys have done a perfunctory investigation... I'd say, fine, I totally understand that, SO, show me something better. And please don't say wait for DxO. DxO don't make their images, on which they base their statistics available. In 60% of scientific studies the conclusions are not supported by the data. Without the data, you really can't check on the validity of the conclusions. When you deal with a site like DxO you have to have blind trust, that they got it right. Personally, I'd dismiss any site that says "trust me". On Imaging Resource, you can look and see for yourself, using the same images they used. And what they show is what these images show, not a lot of difference. SO based on previous experience, I'd put more stock in these images than whatever DxO publishes, and I'd put what IR publishes ahead of both of them, because they are shooting in a more controlled environment, and they share their data.
People who don't share their data, often have something to hide. IN DXO's case, what they seem to behiding is that they are not accounting for sample variation.
In the absence of evidence, one should be cautious, but one also has to evaluate what the evidence means. And this evidence means the average guy going out and taking the average images in an average uncontrolled environment can probably do well with either system. And I'm not seeing the FF advantage. That may not mean anything to the technically inclined, but criticism from the technically/academicly inclined can be petty, obscure and pretty much meaningless in the real world. And IMHO, often the technically inclined way over state miniscule differences in the interest of academic discussion.
So I'd say calling this a joke is a bit harsh. It's less than thorough, but in the two years I've been asking folks to come up with something, to which I've been referred to 5 year old articles using cameras that are no longer available, I'd say the evidence supporting the greater utility of FF cameras is more than a joke. I don't have an FF or I'd do it myself. I can not understand why, when we've had posts comparing prints made with D800, and saying the K-01 images are in some ways better, and so many other bits of non-numerically analyses images.
I'd say the actual photos, the empirical evidence including those shot in test conditions at IR, shows there is very little if any difference between FF and APS-c images at this point in time. Perhaps as pixels get smaller and APS-c approaches it's diffraction limit that may change, but to date, the theoretical claims make APS-c sound much worse than FF, and in these images, the pictures look very much the same.
The differences between APS-c and FF are greatly exaggerated and always have been. IMHO. And recently, comparing K5IIs images to Olympus E-M5 images, I'd say the same for 4/3 and APS-c. My guess is still, that doubling anything, be it aperture , sensor size, whatever, is relatively meaningless. I like to think you could notice the difference between 4/3 and FF, but I have no conclusive evidence that's true. It's a hunch. One that from my perspective has just as much merit as the pointless parade or numerical evaluations using artificial standards like ƒ-stops as applied to DoF.
Or as I used to say when I was teaching, if I wanted to devise a test so one student would pass and one would fail, I could do that. But what would that prove? Unfortunately many of the tests proving the advanced utility of FF fall into that category.
It's all part of reality. I guess when the IR results come out and show pretty much the same thing (as they did the with D7100 and D600), you'll find a reason to dismiss those too.