[Kerrowdown]

DNG vs PEF?

Interestingly even some Pentax cameras didn't natively support the Pentax PEF... for example K-500, K-30, K-01, Q-51, Q7, Q10, Q... but all supported DNG.

[Tas]

As a quick test, I took this picture in both DNG and PEF versions. The DNG files were 16 megabytes and the PEF files were 10 megabytes in size.

I then ran the PEF version through my absolute favourite (open source) editing tool i.e Darktable. No problems there, it works just fine. Now I don't know why I should still use DNG. One learns something new all the time.

Use the format that suits your needs I reckon.

I'm not sure what you did in PP other than look at file size and a direct comparison of how the image looked, so if you haven't done so already I'd recommend pushing and pulling on both files (PEF & DNG) a bit in post to see if there's any benefit to one over the other. This should let you see if the highlights blow out quickly or there's more noise when raising shadows, or if the colour in one responds to adjustments better etc.

Having said this, there's an interesting comparison video showing the different responses between Lightroom and On1 Photo RAW on a Fuji file. The comparison focused on the develop module so it only impacted exposure, tones and colour. However the difference each program made on the same file was quite interesting but it meant that the PP program added another variable to achieving the best possible results in an image. Neither program was conclusively better, they each had their own strengths/weaknesses in the comparison. All the more reason to get it right in camera.

Tas

[funktionsfrei]

As a quick test, I took this picture in both DNG and PEF versions. The DNG files were 16 megabytes and the PEF files were 10 megabytes in size.

I then ran the PEF version through my absolute favourite (open source) editing tool i.e Darktable. No problems there, it works just fine. Now I don't know why I should still use DNG. One learns something new all the time.

As a quick test, I took this picture in both DNG and PEF versions. The DNG files were 16 megabytes and the PEF files were 10 megabytes in size.

Apparently not a very new camera. And obviously someone has re-engineered the non-documented PEF format, so that your open source software is capable to understand it.

[boriscleto]

As a quick test, I took this picture in both DNG and PEF versions. The DNG files were 16 megabytes and the PEF files were 10 megabytes in size.

I then ran the PEF version through my absolute favourite (open source) editing tool i.e Darktable. No problems there, it works just fine. Now I don't know why I should still use DNG. One learns something new all the time.

Which camera? The K10D? That used an older version of DNG that didn't support lossless compression. PEF did support compression in that camera. Later Pentax DSLRs support a later version of DNG with lossless compression...

[newmikey]

Apparently not a very new camera. And obviously someone has re-engineered the non-documented PEF format, so that your open source software is capable to understand it.

Let's be straight and quick about this: the undocumented features in the PEF format are the exact same undocumented features in the DNG files generated by Pentax cameras. These are limited to certain metadata fields, not the image data itself (the raw sensor output) which is identical in both. Camera-specific DNG files are merely documented in known areas, regardless of the manufacturer involved. Those same undocumented metadata fields are equally hidden from interpretation in both formats.

This can easily be observed if you run a PEF and a DNG file generated by the same body through exiftool. The Adobe DNG format document says this about the subject: (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf)

Camera manufacturers may want to include proprietary data in a raw file for use by their own raw converter. DNG allows proprietary data to be stored using private tags, private IFDs, and/or a private MakerNote.
It is recommended that manufacturers use the DNGPrivateData and MakerNoteSafety tags to ensure that programs that edit DNG files preserve this proprietary data. See Chapter 4, “DNG Tags” on page 22 for more information on the DNGPrivateData and MakerNoteSafety tags.

The Adobe manufacturers guidance document (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/DNG_...ufacturers.pdf) clearly mentions the same thing:

Manufacturers can extend the DNG file with private metadata that is not publicly documented, so that they can embed special features that may only be unlocked by their own solutions. Therefore, even as the DNG specification enables the image information in raw files to be universally understood, it also lets manufacturers differentiate their products with special features that may go beyond what exists in the specification.

All DNG does is make sure that the known metadata ends up in one and the same place, regardless of camera-make or model but it doesn't do anything to disclose undocumented metadata - it doesn't suddenly become an open format by having the camera generate a DNG.

Worse than that is that Adobe's DNG converter actually strips out anything it doesn't already know and recognize, as Adobe clearly states in the manufacturerś primer referred to above, this time with my emphasis added:

DNG files that camera manufacturers directly create will always be preferable

and

It should be noted that the Adobe DNG Converter will not necessarily maintain all of the private metadata in certain camera-specific raw formats because this information is not publicly documented and therefore not available to Adobe. However, the Adobe DNG Converter will maintain all of the original image data as well as all of the metadata needed for a high-quality final conversion. Arguably, the private metadata is not really archival, regardless of the format used, simply because it is undocumented. Nevertheless, Adobe recommends that, when photographers use the Adobe DNG Converter for archival purposes, they should maintain both the resulting DNG file and the original camera-specific file. The DNG file offers greater assurance of longevity, but the camera-specific file may contain more metadata. This distinction does not exist, however, for DNG files created by camera manufacturers because they can include all of the private metadata within the Digital Negative.

So much for archival longevity of DNG files created in the PC with the Adobe DNG converter and so much for the fact that in-camera DNG would be better documented than in-camera manufacturer-proprietary format - they are both exactly equally undocumented and their usability will last exactly the same time, until the camera manufacturer drops support for that specific camera-model in its own proprietary software converters (such as DCU for Pentax).

What then remains is the question as to who is better at uncovering, unscrambling, documenting and utilizing private metadata and putting it to good use in the raw conversion software: Adobe or the Opensource community. I'd venture to say that would be the latter with Phil Harvey's exiftool uncovering a whole host of metadata tags which Adobe software remains oblivious to.

[Wasp]

Which cameperl-Image-ExifToolra? The K10D? That used an older version of DNG that didn't support lossless compression. PEF did support compression in that camera. Later Pentax DSLRs support a later version of DNG with lossless compression...

It is indeed a K10D. The compression on the PEF files make it a better choice as far as storage goes.

[rgknief60]

If you install the PEF codec. Viewers don't really read RAW files anyway. They mostly just display the JPEG preview. The exception would be FastRawViewer...

Faststone Image Viewer wll read raw files if you go into the settings and modify the raw tab's settings, or press the "A" key. Set the first two drop downs to on the RAW tab to "actual size - slow." And it is slow to load. I used this to extract and save one of the K-1's PS images from raw before I knew about DCRawPS's option to extract all four.

Now I use Raw Therapee for K-1 PS, as it has the best PS motion control options out there, including image brightness equalization of the 4 images, saveable visualization of the areas with motion, and the ability to choose which one of the 4 images is used for the motion correction.

My K-1 arrived today! I noticed it shoots .PEF in raw, I'm used to (and have codec support for) .DNG files, is there any difference between the two? They seem to take up the same amount of space/size...

Bruce,

Perhaps applicable to your original question: It appears that Adobe CS6 does modify my original DNGs when I make changes with camera raw. Have not been able to change that behavior to make it use XMP sidecar files as it did with older PEFs, even with the settings options in Bridge. Anyone else have a solution?

Reading K-1 PEFs is disabled/not supported in CS6, you have to use CC version. I assume that CCs behavior with DNG files is the same as CS6, but have not tried CC to verify. So modification of my DNG originals is the price I have to pay to use CS6 with the K1.

It is possible to use Adobe's DNG converter to convert PEF to DNG, and even embed the PEF into the DNG so it can be extracted later. Both options increase the use of storage space which the K-1 seems to slurp up anyway.

Don't know about LR6's behavior with K-1 DNGs, whether it modifies them or not. LR6 wil read K-1 PEFs and use XMP sidecars, I believe, but have not tried it, as use K-1 DNGs exclusively now.

Cheers!

Roger

[stevebrot]

The DNG files were 16 megabytes and the PEF files were 10 megabytes in size.

Yes, on the K10D, that is expected. On more recent models, the size is about the same.


Steve

[stevebrot]

And obviously someone has re-engineered the non-documented PEF format so that your open source software is capable to understand it.

You keep using that term as if it is some sort of indictment or is self-evident in some way.* You might want to consider doing some forensic examination of some PEF files, but first a definition of terms. PEF is not a format, it is a file type whose organization is TIFF/EP compliant in the same sense that DNG is TIFF/EP compliant. At the data level and EXIF level, the two file types are equivalent. I would have to confirm, but I believe that the PEF from my K10D have the same version stamp as those from my K-3.

As for darktable, its camera support follows RawSpeed's camera support. For more on the RawSpeed library and its camera support:

https://rawstudio.org/blog/?p=800
https://rawstudio.org/blog/?p=816

Let's be straight and quick about this: the undocumented features in the PEF format are the exact same undocumented features in the DNG files generated by Pentax cameras.

This point has been brought to this user's attention many times, but it does not seem to sink in. What's more, DNG from a Pentax camera is undocumented in the same sense as DNG from Adobe DNG creator though with the latter unreadable except for Adobe products (i.e. proprietary to Adobe).


Steve

* The tendency of Adobe product to not open PEF from new camera models is not related to the readability of files or data format; rather, it is the business decision to not provide backward profile compatibility to other than active product version...no profile -- no curves, no curves -- no processing. Curves are tool-specific even though bound to a particular camera.

[AJF]

If using RAW files with Lightroom and DCU it seems a PEF file will always remain fully compatible with DCU but after Lightroom saves changes to a DNG it may not be

[Wheatfield]

If using RAW files with Lightroom and DCU it seems a PEF file will always remain fully compatible with DCU but after Lightroom saves changes to a DNG it may not be

Lightroom is non destructive editing. It doesn't touch the raw file unless one actually opens it in an external editor using Lightroom adjustments or exports it from Lightroom as some other format.
In either of these situations the raw file is undisturbed and the file is saved as something else, be it a TIFF, PSD, Jpeg or any other format..

[Mikesul]

What Wheatfield said.

[Class A]

Lightroom is non destructive editing. It doesn't touch the raw file unless one actually opens it in an external editor using Lightroom adjustments or exports it from Lightroom as some other format.
In either of these situations the raw file is undisturbed and the file is saved as something else, be it a TIFF, PSD, Jpeg or any other format..

Lightroom won't change the image data contained in a DNG file, but it can write metadata into a DNG file.

There is an option in Lightroom with which one can choose to store metadata in a sidecar file or in the DNG file itself.

[newmikey]

Why is everyone resurrecting a 5+ yo thread?

[Mikesul]

To get more posts, no doubt.