Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
11-25-2014, 02:28 PM   #1
Site Supporter
Site Supporter
Spodeworld's Avatar

Join Date: Aug 2011
Location: New Joisey
Photos: Gallery | Albums
Posts: 1,249
PEF vs DNG in LR for K3

Is there any advantage using PEF over DNG for the K3 if you process in LR?

I know that DNG is universal, so that is better for longevity. But, does shooting the K3 in PEF offer an advantage over DNG if you use LR?

11-25-2014, 02:37 PM   #2
Site Supporter
Site Supporter
boriscleto's Avatar

Join Date: Dec 2009
Location: Liverpool, NY
Photos: Gallery
Posts: 12,898
There is no difference in the data. Only the wrapper.
11-25-2014, 02:49 PM   #3
Site Supporter
Site Supporter
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 27,698
QuoteOriginally posted by boriscleto Quote
There is no difference in the data. Only the wrapper.
Yep, the DNG spec is lenient in that regard. The two file types are functionally equivalent and treated by LR the same way with a (AFAIK) single exception. If you opt to save metadata to file and the imported file is DNG, the metadata is written directly into the XMP segment of that file. If the imported file is PEF, the metadata is written to a sidecar XMP file. The difference is subtle and is only a consideration if you use a service bureau or print shop and anticipate multiple iterations requiring tweaking of the processing for print and a lot of back and forth exchange of modified settings. LR does this for all proprietary RAW file types.


Steve
11-25-2014, 03:21 PM   #4
Veteran Member




Join Date: Jan 2012
Location: United States
Posts: 343
Is there any way to use the embedded color profile with a PEF in LR? Would you have to extract the profile from the DNG? Is the color profile the same no matter what processing you pick on camera (vivid, natural, etc?) Right now I'm using DNG so that I can use the embedded profile which i find better for subtle tones like sunsets and skin tones.

11-25-2014, 03:27 PM   #5
Site Supporter
Site Supporter
Spodeworld's Avatar

Join Date: Aug 2011
Location: New Joisey
Photos: Gallery | Albums
Posts: 1,249
Original Poster
Actually, I was kind of wondering that. I thought that maybe there were profile settings that would be available in PEF that are not available if you use DNG.

I know that when use my Nikon, LR gives me options to choose various embedded profiles that are not available when I use DNGs from my K30.

QuoteOriginally posted by y0chang Quote
Is there any way to use the embedded color profile with a PEF in LR? Would you have to extract the profile from the DNG? Is the color profile the same no matter what processing you pick on camera (vivid, natural, etc?) Right now I'm using DNG so that I can use the embedded profile which i find better for subtle tones like sunsets and skin tones.
11-25-2014, 03:42 PM   #6
Site Supporter
Site Supporter
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 27,698
QuoteOriginally posted by y0chang Quote
Is there any way to use the embedded color profile with a PEF in LR? Is the color profile the same no matter what processing you pick on camera (vivid, natural, etc?)
Yes, the color profile in LR always the same regardless of the camera's JPEG settings. LR works in a wide gamut (Adobe RGB or a variant of ProPhoto RGB) with final profile declared and/or embedded on export. As a result, there is no advantage when shooting RAW to using whatever ICC is embedded (sRGB vs. Adobe RGB). That being said, there is an embedded camera profile (not the same as color profile) that is present in the PEF and Pentax DNG that is available to LR. It can be set using the Camera Calibration section in Develop mode. The appearance on initial import is somewhat different than that of the Adobe Standard profile for the K-3.


Steve
11-25-2014, 04:28 PM   #7
Pentaxian
emalvick's Avatar

Join Date: Feb 2008
Location: Davis, CA
Photos: Gallery
Posts: 1,479
Going in a different direction, I think the biggest differences are with respect to file management. As mentioned earlier, using a PEF file is going to result in development settings and metadata being stored in external XMP files. Using DNG means all XMP data will be embedded into the file. Further, when using Adobe products the edits can be seen by embedding a JPG for the current state of the edits.

For file management, the PEF files and process give the benefit of you having RAW files and never touching them. There are less opportunities for corruption, and if you regularly back up your files, processing essentially only touches the XMP files. As a result, you end up only backing up XMP files as you go through multiple edits for files (you should back up the PEFs at least once). Backing up the XMPs means you are dealing with smaller files so the process is quicker.

The negative to PEF files is that you end up with essentially twice the files because you'll have an XMP file for each file you edit. You have to remember to back them up, and you have to remember to keep the XMP files with the PEF files when you move and copy them around.

DNG files are convenient in that all that info gets stored directly in the DNG file itself (remember that DNG is a container file for the RAW data and the metadata). Everytime you edit or process DNG files the info is written to the XMP within the file. As a result, everytime you back up files, you have to back up the full DNG file. While you aren't really messing with the raw data, editing the DNG files always brings up a small chance of corruption.

Choice between DNG and PEF, in my own opinion, should be a choice over what type of management you want to use or how much you want to worry about it. LR manages both choices well, but it helps if you know what is happening in case you ever need to access your files elsewhere or change file management routines.
11-25-2014, 07:06 PM - 1 Like   #8
Site Supporter
Site Supporter
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 27,698
QuoteOriginally posted by emalvick Quote
As mentioned earlier, using a PEF file is going to result in development settings and metadata being stored in external XMP files.
No, not really. I have many, many PEF files in my repository and not a single external XMP file. That is only if you opt to save metadata to file. Lightroom uses its Catalog for managing metadata and associated resources. Metadata written to file outside the Catalog is an optional activity.

Here are the default LR rules for metadata and supporting files. These are the same for all imported image files regardless of type or file extension:
  • Metadata include keywords, develop steps, select status, ratings, color codes, and so on
  • Supporting files include thumbnail images, catalog settings, and housekeeping database files
  • The Catalog is how LR keeps track of your images and your work
  • All metadata and supporting files are saved in the Catalog
  • The catalog contains many, many files, often several per image
  • You do not have to explicitly save metadata or your edits at any point. They are updated and saved in real time by the program in the Catalog.
  • All metadata is in the catalog, regardless of file type or whether a RAW file is proprietary or DNG
  • By default, metadata is NOT written back to the imported image file, regardless of format or type, nor is it written to any other file outside the Catalog
  • Catalog metadata may be written to the output file (JPEG, TIFF, DNG) on export. You can choose then how much to include.
In addition to the storage in the Catalog you may also choose to write a copy of the metadata to file outside the Catalog. The reasons for this vary, but usually have to do with sharing processing information with a third party or other software or a desire to have a second copy of the metadata. Here are the LR rules for metadata written to file outside the Catalog:
  • You may write a copy of the metadata out to file at any time using keyboard shortcuts or mouse right-click
  • If the imported image file is of the type JPEG, PNG, TIFF, PSD, or DNG, the metadata is written to the XMP portion of the imported file's header (exif). This will change the date stamp on the file itself, but not the capture information in the file header. Again, this action overwrites the file that was imported to Lightroom, which is often the original copy.
  • If the imported image file is of a proprietary RAW format (e.g. PEF), the metadata is written to a separate "side-car" file with the same file name, but having the extension .xmp. The metadata are in the XMP format and can be read with a standard text editor.
  • In ALL cases the Catalog remains the working repository for metadata. Anything written to image or side-car files is considered to be a snapshot for posterity and is external to the workings of LR.
  • Writing of metadata to file can be automated as a background process, though there is NO guarantee that the metadata is in sync at any point in time unless it is manually saved. In addition, the extra disk activity is resource intensive.
  • Again, the Catalog remains the master record and is what LR uses for its processes. The metadata that is saved to file is outside the system for all of LR's image processing and is NOT used for generating exported JPEGs, TIFFs and so on.
  • Again, writing metadata out to file is an optional operation and not part of regular LR workflow
As you might imagine, the various LR mavens suggest that you not save metadata to file as part of normal practice. This is particular true for automated saving to file. Instead, the conventional wisdom is that you regularly back up the catalog to a external location (cloud or external drive) and also back up both the catalog and image files as part of your computer system backup.

Lightroom includes various ways to manage metadata written to file as well as mechanisms to read metadata from (!) a file into the catalog. This last feature allows edits to the XMP by other tools to be migrated into LR.

More than you (the reader) ever wanted to know, eh? There is more, but you really don't want to know, right? Good!


Steve


Last edited by stevebrot; 11-25-2014 at 07:25 PM.
11-26-2014, 08:54 AM   #9
Pentaxian
emalvick's Avatar

Join Date: Feb 2008
Location: Davis, CA
Photos: Gallery
Posts: 1,479
QuoteOriginally posted by stevebrot Quote
No, not really. I have many, many PEF files in my repository and not a single external XMP file. That is only if you opt to save metadata to file. Lightroom uses its Catalog for managing metadata and associated resources. Metadata written to file outside the Catalog is an optional activity.

Here are the default LR rules for metadata and supporting files. These are the same for all imported image files regardless of type or file extension:
  • Metadata include keywords, develop steps, select status, ratings, color codes, and so on
  • Supporting files include thumbnail images, catalog settings, and housekeeping database files
  • The Catalog is how LR keeps track of your images and your work
  • All metadata and supporting files are saved in the Catalog
  • The catalog contains many, many files, often several per image
  • You do not have to explicitly save metadata or your edits at any point. They are updated and saved in real time by the program in the Catalog.
  • All metadata is in the catalog, regardless of file type or whether a RAW file is proprietary or DNG
  • By default, metadata is NOT written back to the imported image file, regardless of format or type, nor is it written to any other file outside the Catalog
  • Catalog metadata may be written to the output file (JPEG, TIFF, DNG) on export. You can choose then how much to include.
In addition to the storage in the Catalog you may also choose to write a copy of the metadata to file outside the Catalog. The reasons for this vary, but usually have to do with sharing processing information with a third party or other software or a desire to have a second copy of the metadata. Here are the LR rules for metadata written to file outside the Catalog:
  • You may write a copy of the metadata out to file at any time using keyboard shortcuts or mouse right-click
  • If the imported image file is of the type JPEG, PNG, TIFF, PSD, or DNG, the metadata is written to the XMP portion of the imported file's header (exif). This will change the date stamp on the file itself, but not the capture information in the file header. Again, this action overwrites the file that was imported to Lightroom, which is often the original copy.
  • If the imported image file is of a proprietary RAW format (e.g. PEF), the metadata is written to a separate "side-car" file with the same file name, but having the extension .xmp. The metadata are in the XMP format and can be read with a standard text editor.
  • In ALL cases the Catalog remains the working repository for metadata. Anything written to image or side-car files is considered to be a snapshot for posterity and is external to the workings of LR.
  • Writing of metadata to file can be automated as a background process, though there is NO guarantee that the metadata is in sync at any point in time unless it is manually saved. In addition, the extra disk activity is resource intensive.
  • Again, the Catalog remains the master record and is what LR uses for its processes. The metadata that is saved to file is outside the system for all of LR's image processing and is NOT used for generating exported JPEGs, TIFFs and so on.
  • Again, writing metadata out to file is an optional operation and not part of regular LR workflow
As you might imagine, the various LR mavens suggest that you not save metadata to file as part of normal practice. This is particular true for automated saving to file. Instead, the conventional wisdom is that you regularly back up the catalog to a external location (cloud or external drive) and also back up both the catalog and image files as part of your computer system backup.

Lightroom includes various ways to manage metadata written to file as well as mechanisms to read metadata from (!) a file into the catalog. This last feature allows edits to the XMP by other tools to be migrated into LR.

More than you (the reader) ever wanted to know, eh? There is more, but you really don't want to know, right? Good!


Steve
Actually I know all that stuff, but I think it is just a bit dangerous to rely on the LR catalog or any software for that matter to store all your metadata. I actually prefer to have everything in XMP, specifically keywords and location info. I actually use a different software for catalogging my images anyway. LR's catalog is just a front end for my development process and LR's catalog was too limited for my needs a while back.

I've observed the effects of having LR write XMP data continually vs. just keeping things in catalog, and I have no issues on my system. I do realize that will vary by computers. More importantly, XMP data is a bit more universal. It gets used in my web pages, and it will be the only way for my important image information to be transferred to a different software should LR ever go the route of Photoshop and become subscription based. I won't stick with the software. As it is, I find the XMP data quite useful for using different RAW editors which can access my keywords and such.

But I suppose that's too much digression and confusion... Of course the method of just using the LR catalog itself reduces the differences of using a PEF vs. DNG as basically negligible (which they kind of already are anyway).

Again, it just shows that there are really know right or wrong ways to deal with these issues. There are just the ways that work best for each user.
11-26-2014, 04:59 PM   #10
Pentaxian
Na Horuk's Avatar

Join Date: Feb 2012
Location: Slovenia, probably
Photos: Gallery | Albums
Posts: 10,934
I prefer DNG because it is the most likely to work with other software, as well.
Btw, with LR its important how you import. Do you "add DNG" or do you "copy as DNG"? One makes a copy of the original file and turns it into a dng, this has a little effect even if you are turning a dng into dng. Just adding the file is different.
And in threads like these I usually chime in that colour space is more important than pef vs dng. Colour space can be any, but usually you want it to be the same setting in the camera as well as in the software that you are using. The fewer conversions between colour space, the less chance for something to.. unexpectedly change. Most of the internet works on sRGB, especially if you upload photos to websites, so it can be a good idea to use sRGB colour space throughout your process.
11-27-2014, 12:22 AM   #11
Site Supporter
Site Supporter
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 27,698
QuoteOriginally posted by emalvick Quote
Actually I know all that stuff, but I think it is just a bit dangerous to rely on the LR catalog or any software for that matter to store all your metadata.
RAID 1 + incremental file backups + incremental partition backup + separate catalog backup. (I count 5+ copies, 2 in real time)

That is more than what I had for film work in the wet darkroom. In that case it was 3x5 card, one copy.


Steve

---------- Post added 11-26-14 at 11:33 PM ----------

QuoteOriginally posted by emalvick Quote
Of course the method of just using the LR catalog itself reduces the differences of using a PEF vs. DNG as basically negligible (which they kind of already are anyway).
Yep. That is about it. It is only when you want to actively manage the metadata that there is any functional difference with LR. BTW, I had to fall back on my copy of Evening's "The Adobe Photoshop Lightroom 5 Book" as a fact check for my understanding of how LR handles metadata. Evening devotes nine pages of fine print to the fine art of how to deal with metadata written out to file. I don't currently have a use case that requires writing out to file and for that I am glad.


Steve

Last edited by stevebrot; 11-27-2014 at 12:33 AM.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
advantage, backup, catalog, copy, dng, dng in lr, dslr, k-3, k3, lr, metadata, pef, pef vs dng, pentax k-3, steve
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
K3 - RAW support (PEF/DNG) by Apple (Aperture3, OSX) Uncle Pete Pentax K-3 20 03-21-2014 05:03 AM
K3 pef/dng readable in Lightroom 3.6? hoanpham Pentax K-3 5 12-21-2013 12:33 PM
WB issue in LR 5 with K-3 DNG Azzy Pentax K-3 21 12-16-2013 04:51 PM
RAW in Linux - PEF vs. DNG? danbob Digital Processing, Software, and Printing 55 07-26-2009 01:47 PM
PEF vs DNG Trub Digital Processing, Software, and Printing 7 06-15-2007 09:51 PM



All times are GMT -7. The time now is 11:53 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top