[Dartmoor Dave]

They would not dare to "cook" a RAW file, even just a little bit, would they? That would be naughty IMHO.

I've been doing a bit of reading, and the depressing (and to me shocking) answer is that, yes, they do. Apparently all Nikon low-end consumer cameras use lossy raw compression, while Canon uses lossless. And of particular relevance to the discussion above on this page is the news that all Sony cameras use lossy compression, even top-of-the-range ones. Inevitably, Pentax isn't mentioned in the article linked to below, but based on my experience with the K-S1 I'd be willing to bet that it uses lossy raw compression. And I'd also bet that the K10D doesn't.

I think some more research is needed to try to find out which Pentax cameras use which sort of raw compression.

Compressed vs Uncompressed vs Lossless Compressed RAW Options

[Andrew_Oid]

I've been doing a bit of reading, and the depressing (and to me shocking) answer is that, yes, they do. Apparently all Nikon low-end consumer cameras use lossy raw compression, while Canon uses lossless. And of particular relevance to the discussion above on this page is the news that all Sony cameras use lossy compression, even top-of-the-range ones. Inevitably, Pentax isn't mentioned in the article linked to below, but based on my experience with the K-S1 I'd be willing to bet that it uses lossy raw compression. And I'd also bet that the K10D doesn't.

I think some more research is needed to try to find out which Pentax cameras use which sort of raw compression.

Compressed vs Uncompressed vs Lossless Compressed RAW Options

Interesting information, Dave. And I can see some people get quite passionate about their RAW options. Banding is the only issue I'm concerned with. I don't recall running into it with the a850. But then maybe I didn't shoot with that camera under conditions (like low light) that would produce it.

[tim60]

It is a an unrelated issue, but in video I, and some collaborators looked at using a DSLR for video to then perform analysis of time variation from frame to frame. All the DSLRs that we found information for used the common time lossy video standard, rather than complete separate frame capture. We were trying to measure some pretty subtle effects and would have liked a camera that did not have the time integration effect, at least as a research reference point to determine the impact of that kind of compression.

[tim60]

Interesting information, Dave. And I can see some people get quite passionate about their RAW options. Banding is the only issue I'm concerned with. I don't recall running into it with the a850. But then maybe I didn't shoot with that camera under conditions (like low light) that would produce it.

But then, you do not want to be limited to taking only certain kinds of pictures because of artefactual problems caused by a compression algorithm.

[Andrew_Oid]

But then, you do not want to be limited to taking only certain kinds of pictures because of artefactual problems caused by a compression algorithm.

Except that I don't know what factors cause or contribute to it. It still needs some investigation as I said. It would seem that photo rendering has gotten more complex. The design and engineering of lenses and sensors will always involve some trade-offs and software is no doubt always evolving and (hopefully) improving.
The banding only occurs occasionally and from what I can see there are many ways of dealing with it though I only tried the one method. I'm very critical when it comes to image quality and the limitation of the a850 is, as I've said before, poor performance at high ISO. The a7 is much improved in that respect. This is a photo taken at ISO 800, f9 and 1/13th second hand-held. No discernible banding. I would've liked a sharper image but I had a Macro Revuenon 28mm f2.8 lens on the camera. The color had already faded considerably in the shot taken immediately after this one, so there was no time to adjust. (I have replaced the Revuenon with an SMC Takumar 28mm f3.5 lens, though. I hope I can get a repeat of conditions tonight.)

Twilight Splendor | Sony a7 with Macro Revuenon 28mm f2.8 le? | William Bolton | Flickr

[rayallen]

I've been doing a bit of reading, and the depressing (and to me shocking) answer is that, yes, they do. Apparently all Nikon low-end consumer cameras use lossy raw compression, while Canon uses lossless. And of particular relevance to the discussion above on this page is the news that all Sony cameras use lossy compression, even top-of-the-range ones. Inevitably, Pentax isn't mentioned in the article linked to below, but based on my experience with the K-S1 I'd be willing to bet that it uses lossy raw compression. And I'd also bet that the K10D doesn't.

I think some more research is needed to try to find out which Pentax cameras use which sort of raw compression.

Compressed vs Uncompressed vs Lossless Compressed RAW Options

Thanks for that link, Dave. I had a read of that article and, while some cameras allow you to specify different types of RAW compression, I have not seen such an option on either my K10D or K-3.
However, I recall noticing some years back that on the K10D there was a size difference between PEF and DNG. We Pentax owners are fortunate that we can choose either RAW format. Many other brands do not provide that option.
So, I decided to do a test and the results astounded me. I just stood at my front door and took two shots in fairly quick succession changing the RAW file format between shots. And then I did the same with my K-3.

K10D, 3872X2592, 10 MP DNG = 16.1 MB PEF = 8.78 MB Less than 10MP?? Compressed? Yes, I think so.

K-3, 4016X4000 24.1 MP DNG = 31.4 MB PEF = 31.0 MB

I don't know what to make of those numbers. Would anybody care to try to make some sense of it? Just how raw is the PEF on the K10D? Could others please do a similar test to see if you get numbers close to mine?

[Dartmoor Dave]

So, I decided to do a test and the results astounded me. I just stood at my front door and took two shots in fairly quick succession changing the RAW file format between shots. And then I did the same with my K-3. K10D, 3872X2592, 10 MP DNG = 16.1 MB PEF = 8.78 MB Less than 10MP?? Compressed? Yes, I think so. K-3, 4016X4000 24.1 MP DNG = 31.4 MB PEF = 31.0 MB I don't know what to make of those numbers. Would anybody care to try to make some sense of it? Just how raw is the PEF on the K10D? Could others please do a similar test to see if you get numbers close to mine?

Wow, that's a big difference in file size between DNG and PEF on the K10D, especially when the numbers for the K-3 are so close to each other. As long as the difference in the K10D files is down to lossless compression then it's not a problem, but I think I'd personally still stick to DNG just in case. My GX-10 version of the K10D only offers DNG without the PEF option, with the average DNG file size just over 18MB.

I confess that I haven't got a clue about how to make sense of these numbers, and the more I've tried to research the issue online the more confused I've got. I keep running into the usual internet problem of one person repeating a myth that they've read in a forum somewhere, then other people repeating that as if it's fact. And I really don't want to add to that mythology with misinformed theories of my own.

Fortunately I'll be spending Christmas with a man who spent his entire career designing computer graphics hardware. Namely, my father. He was a member of MPEG, the international body that specifies the various digital video formats, and he knows pretty much everything there is to know about graphics compression. I just hope I can get him to make some sense of things for me in a way that my distinctly non-mathematical brain can understand. I was the kid in the family who inherited my mother's arty-farty genes rather than my father's technically-minded ones. It'll probably end up with me sitting there looking stupid while my 9 year old maths genius niece laughs at me because she thinks it's so easy. . . but I'm willing to take one for the team.

[rayallen]

Wow, that's a big difference in file size between DNG and PEF on the K10D, especially when the numbers for the K-3 are so close to each other. As long as the difference in the K10D files is down to lossless compression then it's not a problem, but I think I'd personally still stick to DNG just in case. My GX-10 version of the K10D only offers DNG without the PEF option, with the average DNG file size just over 18MB.

I confess that I haven't got a clue about how to make sense of these numbers, and the more I've tried to research the issue online the more confused I've got. I keep running into the usual internet problem of one person repeating a myth that they've read in a forum somewhere, then other people repeating that as if it's fact. And I really don't want to add to that mythology with misinformed theories of my own.

Fortunately I'll be spending Christmas with a man who spent his entire career designing computer graphics hardware. Namely, my father. He was a member of MPEG, the international body that specifies the various digital video formats, and he knows pretty much everything there is to know about graphics compression. I just hope I can get him to make some sense of things for me in a way that my distinctly non-mathematical brain can understand. I was the kid in the family who inherited my mother's arty-farty genes rather than my father's technically-minded ones. It'll probably end up with me sitting there looking stupid while my 9 year old maths genius niece laughs at me because she thinks it's so easy. . . but I'm willing to take one for the team.

Dave,
I have been using PEF on my K10D for longer than I can remember mainly because the DNG files were larger. I don't recall the difference being so large but the file sizes change slightly depending on image content. So, the K10D will be set back to PEF. Your GX10 DNG files are even slightly larger than my K10D ones but only by a small margin but you don't have any other option.

Your father was a member of MPEG? Wow, I am impressed and you can tell him that. I hope you have a happy time with him at Christmas and I also expect that he might be able to shed some lignt on the issues we have been discussing. Please don't sell yourself short on your technical abilities as I think you have quite an analytical mind. Anyway, I will wait to hear the result of your deliberations with your dad.

[MJSteele]

Athabasca Falls, Jasper National Park, Canada. Oct 11, 2017. 3-frame panorama. K-1 + Super-Takumar 55/1.8 @ f/16.

Just recently purchased one of these fine lenses for 20 dollars US at a antique store. m42 mount, a little dirty on outside but cleaned up to almost new condition

[Andrew_Oid]

Getting back to my roots, black and white. How much quicker and easier it all is in digital!

[Jonathan Mac]

This discussion on RAW formats is quite interesting. I always shoot RAW plus jpeg and have always used DNG as it's a standard format but I never checked if PEF files take up the same space. As I always delete the RAW file after processing, maybe I should switch to PEF and save myself some SD card space.

I'm in favour of compression as long as it's not lossy, which would defeat the purpose of RAW. I imagine that companies use lossy RAW compression because it increases the number of images the camera can store in the buffer, one of the key specs that many people look at when considering a new camera.

[paulh]

I'm in favour of compression as long as it's not lossy, which would defeat the purpose of RAW

Exactly. Like Ray, I've known for awhile that the K10D .dng files are larger than the .pef files. For that reason I set my K5 to .pef as well, although I haven't really compared the two formats on that camera yet. Hopefully we'll get some answers regarding this troubling "lossy RAW" conundrum. Maybe Dave's dad can help shed some light on this as well!

Willows at the lake. Super Tak 28/3.5 on *ist D (which only does .pef)

[Clarkey]

I do not profess to know, but reading the technical DNG description, think that the size difference may be due (at least in part!) by the framework that allows DNG to interact/record applicable data from other camera makers/types.

[wkraus]

To make things more complicated, I found that running my .dng files from the K-5 II through Adobe's .dng converter made them noticeably smaller. Not that it makes much sense to do so...

[Andrew_Oid]

Another in the Breads of Bangkok series, "Linseed Bread". Shot at f16 and MFD.