[house]

One advantage DNG had over PEF in terms of archiving is that the DNG has the colour matrix embedded. This means the decoding software needs to know very little about your camera. This is also useful with new cameras that aren't supported by software yet. I've already taken advantage of this many times.

[newmikey]

There are examples of AV formats that have gone extinct without any possibility of being able to recovery data, is it really so far-fetched to think the same cannot happen to digital image files? Cameras that use nonstandard pixel formats and altered colour filter arrays* and alternate pixel pattern layouts** that may be difficult to correctly interpret with current software as the original processing software could. Early Kodak, Fuji,SONY, Nikon and Foveon cameras have raw files that for various and assorted reasons rather difficult to read correctly.

We were discussing rather run-of-the-mill Pentax PEF if I recall correctly. Similar to other mainstream camera models and their proprietary raw formats I indeed think it is far-fetched. I can still read 2006 K110D PEFs with any software I use today. I expect to be able to do so indefinitely. The fact that somewhat more "odd" or one-time nonstandard file formats may have become "difficult" to read today is really not a valid argument in a discussion about Pentax PEF format.

Your examples are also far-fetched:
The SONY DSC F828 predates the very first DNG format by almost a whole year. Last time I looked an SRF file can still be converted by existing software such as but not limited to DCRAW (and any software which uses it as a library to read raw files). The decryption code was published by the legendary Dave Coffin on Github and remains there for anyone to use (libopenraw/sony_clear.c at master · freedesktop/libopenraw · GitHub)

Sigma Foveon native files can be converted using RawTherapee and a plethora of other software packages. Later Foveon models included DNG as raw format.

I'll try the referred Super CCD HR and SR I and II, the Nikon D1X used non-square pixel elements later but again, rather far-fetched indeed.

---------- Post added 06-25-22 at 10:47 AM ----------

One advantage DNG had over PEF in terms of archiving is that the DNG has the colour matrix embedded. This means the decoding software needs to know very little about your camera. This is also useful with new cameras that aren't supported by software yet. I've already taken advantage of this many times.

Now, thàt indeed is a valid point I had not considered even though I use software which is frequently updated. Thanks for submitting a reasonable argument in this discussion. Appreciate it!


[EDIT]I just downloaded a sample SRF file from the raw file repository at raw.pixls.us and tried to open it. Please find results below for DarkTable which opens it without any trouble and with the appropriate WB - but I see why you referenced it when I tried it in various other raw converters. Luckily it was only a one-time attempt by Sony to use this format so I do not think it makes such a valid argument for today's discussion. I don't think many people used that particular raw format seeing how hard it was to obtain a sample.

[madison_wi_gal]

So to me, an app "has white blank box. Has no idea what to do with this" would be a sign to update my app, not to feed it a DNG unless it was a temporary measure to tide me over until the update.

Well, in my case, before I got DxO I'd use MacOS Photos to preview my card contents to see what I wanted to work on. Since DxO can preview SD contents in a grid and can show PEFs as well as DNGs I no longer have any strong feeling about it.

And since my "app" (in this case, MacOS Photos on Monterey) is part of the native OS apps, the app *is* updated.

[MarkJerling]

Sorry. Same old, same old argument doesn't fly with me and never did. Last time I looked even on Windows or Apple there weren't too many packages which read DNG but not PEF. On Linux that number goes down to zero. What is your definition of "broadly supported"?

PEF = "Pentax Electronic File"
DNG = "Digital Negative Image". "DNG is an open-source, royalty-free, highly compatible raw image format used by many different camera manufacturers."

I don't think there can be any argument that DNG is used by more camera brands than Pentax. It's also the preferred format for a number of archival systems, most notably the US Library of Congress.

Here's a little video that delves into this in more detail.

You Tube

You Tube

Of course, you'll notice the video is from 2010. Back then, there was probably less support on all operating systems for the PEF and other proprietary raw formats. DNG sought to address that problem.

I'd argue that it still offers advantages over PEF and other proprietary raw formats.

[biz-engineer]

I don't think there can be any argument that DNG is used by more camera brands than Pentax

Are we sure that Pentax (or other brand) DNG would be read-able if Pentax wouldn't be supported?

Personally, I'm not so sure about how generic are DNG files, and I think (maybe I'm wrong) that each camera manufacturer has it's own way of using the DNG container. If the DNG implementation is brand specific, that means there's no generic advantage over PEF.

[MarkJerling]

Are we sure that Pentax (or other brand) DNG would be read-able if Pentax wouldn't be supported?

Personally, I'm not so sure about how generic are DNG files, and I think (maybe I'm wrong) that each camera manufacturer has it's own way of using the DNG container. If the DNG implementation is brand specific, that means there's no generic advantage over PEF.

All the information I can find online (and through Adobe) points to DNG being based on the TIFF file format, which is an industry standard file format, to which a set of rules is then applied so that Adobe (or other software) can read a DNG file regardless of origin. There's also a particular (loss-less) compression algorithm as well as the ability to determine very easily if the file has been corrupted in some way.

Because DNG uses a specific file specification, all users should be using that specification. See: Digital Negative (DNG), Adobe DNG Converter | Adobe Photoshop
So far as I can tell, therefore, there is no evidence to suggest that the DNG specification varies from brand to brand.

[thibs]

All the information I can find online (and through Adobe) points to DNG being based on the TIFF file format, which is an industry standard file format, to which a set of rules is then applied so that Adobe (or other software) can read a DNG file regardless of origin. There's also a particular (loss-less) compression algorithm as well as the ability to determine very easily if the file has been corrupted in some way.

Because DNG uses a specific file specification, all users should be using that specification. See: Digital Negative (DNG), Adobe DNG Converter | Adobe Photoshop
So far as I can tell, therefore, there is no evidence to suggest that the DNG specification varies from brand to brand.

At least in the K5 era, PEF file had to be supported for each specific camera. With DNG you can always open the file, even if it may exhibit color calibration of some sorts etc. At least you're sure you can open it.
If there's any fear to have IMO it is against the PEF format, not the other way around.

[newmikey]

At least in the K5 era, PEF file had to be supported for each specific camera. With DNG you can always open the file, even if it may exhibit color calibration of some sorts etc. At least you're sure you can open it.
If there's any fear to have IMO it is against the PEF format, not the other way around.

You are all ignoring the elephant in the room: when presented with a DNG file, there is no obvious way to tell whether it is really a raw file or even whether it is actually produced by a specific camera respecting the DNG guidelines. It could be:
1) A straight out of camera raw file (from a camera manufacturer offering the DNG option to store raw in-camera)
2) A manipulated raw file stored by LR (or some other software)
3) The result of a conversion of a proprietary raw file to DNG via any of the many existing converters which may or may not result in the loss of proprietary Maker notes
4) Same as (3) but now in a lossy compression format which may even omit some sensor data besides Maker notes
5) A non-raw image file (post demosaicing, post WB or both) stored inside a DNG container
6) Any other variation I have not yet named above

This is not about fear but about certainty. The certainty that a PEF/NEF/CR2/ORF/ARW/etc file comes straight out of the camera which generated it, with only minimal adjustments allowed to its EXIF (such as copyright info or GPS coordinates)

[MarkJerling]

The point of a DNG file is not certainty that it's straight from camera or the other options you mention. The point is that it's a TIFF type file with all its parameters recorded in an industry agreed way. That's what makes it good for archival storage.

The original raw data in a DNG file is never altered. The DNG file is writable in certain segments, although not in the raw data segment. As such, with the correct tools, it should be possible to find what modifications, if any, has been made to a file. Certainly, people who work in the evidential image sector will be able to tell if a DNG file has been manipulated or not. This could be done by looking at factors such as whether they're looking at Raw DNG" or "Linear DNG" and other pointers, well beyond my understanding.

The whole point of DNG is to be a universally accessible raw format. I've yet to see evidence that the different parameters you mention cannot be found by people with the necessary skills. It's not an "elephant in the room" moment.

For more information about the DNG format and how it preserves it's original dataset without manipulation, see: https://opencommons.uconn.edu/cgi/viewcontent.cgi?article=1027&context=libr_pubs

The Library of Congress only uses DNG format and, for many reasons, views it superior to all other proprietary (manufacturer based) raw formats. It's also the archival master format at The National Gallery of Art, Washington, D.C. It is, in short, the most trusted raw format there is and can operate without the input of any specific supplier's camera / system specific software.

[newmikey]

To me, the nice thing about DNG is that I can continue using my old version of Lightroom --version 6.14 -- with newer cameras like the K-3 III while shooting RAW.

Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.

[house]

The whole point of DNG is to be a universally accessible raw format. I've yet to see evidence that the different parameters you mention cannot be found by people with the necessary skills. It's not an "elephant in the room" moment.

I know that DNG's can cause issues due to the scope of the standard. Programmers don't feel they can cover all bases and when they do some third party makes a different interpretation that fails to work. As you know it doesn't have to be raw data and it's apparently not straightforward to figure out.

But that isn't an issue for Pentax so far. DNG's are slightly "safer" for new cameras or old software.

[reh321]

Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.

I also don’t drive my auto to ‘it’s fullest capacity’.
I use what I need, no more or less.

[slartibartfast01]

Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.

Can you give an example of "any camera potential in the sensor data of newer camera models" that would benefit from later software versions? I don't think pixel shift is supported even now by Adobe.

[Rondec]

Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.

I'm trying not to take offense. There has been less improvement in actual RAW development in newer Lightroom versions than you might think. More than that, if you use third party apps like Topaz Suite or Nik Effects, you can bring a lot out of your RAW images. I post images here and I don't think my limitations are related to using older software.

For pixel shifting images I typically am using RAW Therapee which does a significantly better job of processing than does the newest version of Lightroom.

[house]

For pixel shifting images I typically am using RAW Therapee which does a significantly better job of processing than does the newest version of Lightroom.

Have you used it on a standard raw file? With dual demosaic (amaze or RCD/VNG4) and capture sharpening I find the results are cleaner and more detailed than any other software. Of course there are 10 further sharpening tools you might use but the aforementioned is enough for me. Lightroom just feels crude in comparison. I've only done quick comparisons though.