DNG Raw Files corrupted by Adobe products

JeaPea · 03-01-2018, 01:58 PM

Am I the only one? Being a Pentaxian it often feels like one is participating in a minority sport. Please read on and hopefully help a true believer...

Lightroom and Bridge are corrupting my DNG raw files as generated by four Pentax DSLR bodies, i,e. K-5 II, K-3, K-70 and Pentax K-1.

I can open and edit the DNG raw files from these Pentax cameras in Pentax DCU, DxO Photolab, Topaz Studio, and Skylum Luminar as long as they haven't been previewed in Lightroom CC Classic and/or Adobe Bridge CC 2018. Once they have been previewed, let alone edited, in Lightroom and/or Bridge, they become unusable in other software applications.

Upon previewing them in Lightroom and/or Bridge, both the thumbnails and image data become unusable. The previews have either a weird posterised look or have large sections covered in red diagonals on a white background.

Has any other Pentaxian experienced this? Better still, does anyone have a solution to this issue?

Thanks in anticipation of your help.

pschlute · 03-01-2018, 02:23 PM

Don't get this issue with LR and Silkypix DS Pro

Altheim · 03-01-2018, 02:27 PM

Have you checked for a possible plugin for Pentax format from Adobe. It could be that the native software is reading and then doing a writeback at close that corrupts the file. Just a thought.

stevebrot · 03-01-2018, 02:28 PM

Welcome to the Pentax Forums!

How are you doing your import into LR? Are you using the "Convert to DNG" feature?

Also, are you configured to automatically write metadata changes to XMP?

Either of those will rewrite the DNG.

Steve

**jatrax** · 03-01-2018, 02:42 PM

Originally posted by stevebrot

How are you doing your import into LR? Are you using the "Convert to DNG" feature? Also, are you configured to automatically write metadata changes to XMP? Either of those will rewrite the DNG.

^^^^^ This.
I use a number of different programs and all read my Pentax camera DNGs without issue. None of them should have an issue reading the Adobe converted DNGs either but the only thing I can think of changing the DNG is what Steve posted.

**Oakland Rob** · 03-01-2018, 02:56 PM

I use all those applications except Topaz, and many more, and have no issues with DNGs after doing all sorts of stuff to them in Lr (I auto-write changes back to the DNG). On a Mac.

Perhaps you could put one of the corrupted DNGs somewhere, maybe a before and after, so some of us could see if we can reproduce the issue.

It's especially odd cuz Bridge is just a browser. Did you run it through ACR in Bridge? I'm not 100% sure what you mean by "previewed."

JeaPea · 03-01-2018, 03:37 PM

Originally posted by stevebrot

Welcome to the Pentax Forums!

How are you doing your import into LR? Are you using the "Convert to DNG" feature?

Also, are you configured to automatically write metadata changes to XMP?

Either of those will rewrite the DNG.

Steve

The DNG files I am using are those that are natively produced by the Pentax bodies I own. They are "added" to the Lightroom catalogue or viewed dierctly from the folders that they are stored on an internal hard drive...

---------- Post added 03-01-18 at 03:43 PM ----------

Originally posted by Altheim

Have you checked for a possible plugin for Pentax format from Adobe. It could be that the native software is reading and then doing a writeback at close that corrupts the file. Just a thought.

Thanks for your comment. However, the DNG format is an Adobe file standard and Lightroom and Bridge should (and write) to these as they would any DNG file whether it results from a conversion or is generated by cameras such as Pentax or Leica

JeaPea · 03-01-2018, 03:40 PM

Originally posted by Oakland Rob

I use all those applications except Topaz, and many more, and have no issues with DNGs after doing all sorts of stuff to them in Lr (I auto-write changes back to the DNG). On a Mac.

Perhaps you could put one of the corrupted DNGs somewhere, maybe a before and after, so some of us could see if we can reproduce the issue.

It's especially odd cuz Bridge is just a browser. Did you run it through ACR in Bridge? I'm not 100% sure what you mean by "previewed."

Bridge (and Lightroom) both write any editing adjustments and update the embedded JPEG preview to the DNG file envelope. I have heard mention that these apps may be corrupting the TIFF headers (DNG files having a similar structure to TIFF files

---------- Post added 03-01-18 at 03:53 PM ----------

Originally posted by stevebrot

Welcome to the Pentax Forums!

How are you doing your import into LR? Are you using the "Convert to DNG" feature?

Also, are you configured to automatically write metadata changes to XMP?

Either of those will rewrite the DNG.

Steve

Thanks for your comments. These Adobe are supposed to write any changes and update the DNG file envelope thus avoiding the need for .xmp sidecar files. This corruption only happens with these two apps and not with other software that I have mentioned. So far, I have been referred to Adobe's third tier of Support. They can replicate the issue but are currently unable to identify why or offer a solution. The issues have only happened since updates to both apps from September 2017 onwards and would seemingly point to Adobe updates ince then.

**cpk** · 03-01-2018, 04:00 PM

Have you tried taking some photographs in the PEF format to see if the same problem occurs?

stevebrot · 03-01-2018, 04:34 PM - **2 Likes**

Originally posted by JeaPea

Thanks for your comment. However, the DNG format is an Adobe file standard and Lightroom and Bridge should (and write) to these as they would any DNG file whether it results from a conversion or is generated by cameras such as Pentax or Leica

This is not quite true. DNG is a published specification, but hardly a standard. It only guarantees interoperability between Adobe products. A good example would be a Pentax DNG generated by your camera. It is supported by Adobe and several other company's products. If, however, one opts to import that Pentax-generated DNG into LR using the option to convert to DNG*, the resulting DNG will NOT be readable by the camera or by the Pentax software that came with the camera. The same is true for PEF converted to DNG by Adobe tools capable of doing so. Adobe-generated or modified DNG offer limited, if any, compatibility outside of Adobe products.

After years of working with LR and DNG (over a decade now), my conclusion is that a DNG file generated by or modified by LR should be considered of limited usefulness in other, non-Adobe, tools.

Steve

* If all DNG are created equal, one may ask why Adobe offers this option for DNG from other entities.

stevebrot · 03-01-2018, 04:45 PM - **1 Like**

Originally posted by JeaPea

These Adobe are supposed to write any changes and update the DNG file envelope thus avoiding the need for .xmp sidecar files.

Martin Evening in his Light 5 book strongly advises against using that feature, suggesting instead to trust the catalog and back that up on a regular schedule. Creating duplicate copies of XMP (catalog and file) for the same cataloged image is not a good idea since the writes are not guaranteed to be synchronous (LR generally does the file writes on a time-available basis). I also suspect that the thumbnail preview images are also written into the file, which may explain the viewer issues you are encountering.

I treat the DNG as inviolate image source and may eventually take the additional step of marking those files read-only to avoid corruption by my tools.

Steve

pentaxfall · 03-01-2018, 05:23 PM

I make the DNG files read-only before any software touches it. I remember Adobe CS6 was not generating XMP files if the file is not in read only state. I do not edit regularly so this is not a headache for me.

**clackers** · 03-01-2018, 05:35 PM

Originally posted by JeaPea

Am I the only one? Being a Pentaxian it often feels like one is participating in a minority sport. Please read on and hopefully help a true believer...

Lightroom and Bridge are corrupting my DNG raw files as generated by four Pentax DSLR bodies, i,e. K-5 II, K-3, K-70 and Pentax K-1.

I can open and edit the DNG raw files from these Pentax cameras in Pentax DCU, DxO Photolab, Topaz Studio, and Skylum Luminar as long as they haven't been previewed in Lightroom CC Classic and/or Adobe Bridge CC 2018. Once they have been previewed, let alone edited, in Lightroom and/or Bridge, they become unusable in other software applications.

Upon previewing them in Lightroom and/or Bridge, both the thumbnails and image data become unusable. The previews have either a weird posterised look or have large sections covered in red diagonals on a white background.

Has any other Pentaxian experienced this? Better still, does anyone have a solution to this issue?

Thanks in anticipation of your help.

Not happening on my system, JP, sorry.

**lsimpkins** · 03-01-2018, 08:46 PM

Originally posted by JeaPea

Am I the only one? Being a Pentaxian it often feels like one is participating in a minority sport. Please read on and hopefully help a true believer...

Lightroom and Bridge are corrupting my DNG raw files as generated by four Pentax DSLR bodies, i,e. K-5 II, K-3, K-70 and Pentax K-1.

I can open and edit the DNG raw files from these Pentax cameras in Pentax DCU, DxO Photolab, Topaz Studio, and Skylum Luminar as long as they haven't been previewed in Lightroom CC Classic and/or Adobe Bridge CC 2018. Once they have been previewed, let alone edited, in Lightroom and/or Bridge, they become unusable in other software applications.

Upon previewing them in Lightroom and/or Bridge, both the thumbnails and image data become unusable. The previews have either a weird posterised look or have large sections covered in red diagonals on a white background.

Has any other Pentaxian experienced this? Better still, does anyone have a solution to this issue?

Thanks in anticipation of your help.

I am not seeing this issue.

I am using LR Classic CC with latest updates. My workflow is to apply Auto Tone during DNG file import, so all my images are processed to some degree. I have LR set up to write metadata changes to XMP.

I just opened a DNG file in LR that I know I heavily edited in LR. When I open the same file in Pentax DCU 5, what I see is the original unedited file (not even my crops). If I open the file in On1 Photo RAW 2017, I see the same thing as in PDCU.

As a SWAG, could this be yet another Graphics Processor related issue?

**Alex645** · 03-02-2018, 01:13 AM

Originally posted by stevebrot

Adobe-generated or modified DNG offer limited, if any, compatibility outside of Adobe products.

my conclusion is that a DNG file generated by or modified by LR should be considered of limited usefulness in other, non-Adobe, tools.

Steve

+1 with Steve here. I currently use both LR and PS CC and have no issues with any camera generated PEF or DNG as long as I donʻt try to then export them to other non-Adobe apps.

Itʻs already too easy to over process digital images and my overall recommendation is to find one post process work flow that suits your needs and stick with it. Manipulating files between applications from different developers is a bit of alchemy and with every update, what may have worked yesterday may become impossible tomorrow.

03-01-2018, 01:58 PM	#1
JeaPea New Member Join Date: Aug 2016 Posts: 6	DNG Raw Files corrupted by Adobe products Am I the only one? Being a Pentaxian it often feels like one is participating in a minority sport. Please read on and hopefully help a true believer... Lightroom and Bridge are corrupting my DNG raw files as generated by four Pentax DSLR bodies, i,e. K-5 II, K-3, K-70 and Pentax K-1. I can open and edit the DNG raw files from these Pentax cameras in Pentax DCU, DxO Photolab, Topaz Studio, and Skylum Luminar as long as they haven't been previewed in Lightroom CC Classic and/or Adobe Bridge CC 2018. Once they have been previewed, let alone edited, in Lightroom and/or Bridge, they become unusable in other software applications. Upon previewing them in Lightroom and/or Bridge, both the thumbnails and image data become unusable. The previews have either a weird posterised look or have large sections covered in red diagonals on a white background. Has any other Pentaxian experienced this? Better still, does anyone have a solution to this issue? Thanks in anticipation of your help.

03-01-2018, 04:34 PM - 2 Likes	#10
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by JeaPea Thanks for your comment. However, the DNG format is an Adobe file standard and Lightroom and Bridge should (and write) to these as they would any DNG file whether it results from a conversion or is generated by cameras such as Pentax or Leica This is not quite true. DNG is a published specification, but hardly a standard. It only guarantees interoperability between Adobe products. A good example would be a Pentax DNG generated by your camera. It is supported by Adobe and several other company's products. If, however, one opts to import that Pentax-generated DNG into LR using the option to convert to DNG, the resulting DNG will NOT be readable by the camera or by the Pentax software that came with the camera. The same is true for PEF converted to DNG by Adobe tools capable of doing so. Adobe-generated or modified DNG offer limited, if any, compatibility outside of Adobe products. After years of working with LR and DNG (over a decade now), my conclusion is that a DNG file generated by or modified by LR should be considered of limited usefulness in other, non-Adobe, tools. Steve If all DNG are created equal, one may ask why Adobe offers this option for DNG from other entities. Last edited by stevebrot; 03-01-2018 at 04:56 PM.

03-01-2018, 08:46 PM	#14
lsimpkins Site Supporter Join Date: Feb 2013 Location: SE Pennsylvania Photos: Gallery \| Albums Posts: 537	Originally posted by JeaPea Am I the only one? Being a Pentaxian it often feels like one is participating in a minority sport. Please read on and hopefully help a true believer... Lightroom and Bridge are corrupting my DNG raw files as generated by four Pentax DSLR bodies, i,e. K-5 II, K-3, K-70 and Pentax K-1. I can open and edit the DNG raw files from these Pentax cameras in Pentax DCU, DxO Photolab, Topaz Studio, and Skylum Luminar as long as they haven't been previewed in Lightroom CC Classic and/or Adobe Bridge CC 2018. Once they have been previewed, let alone edited, in Lightroom and/or Bridge, they become unusable in other software applications. Upon previewing them in Lightroom and/or Bridge, both the thumbnails and image data become unusable. The previews have either a weird posterised look or have large sections covered in red diagonals on a white background. Has any other Pentaxian experienced this? Better still, does anyone have a solution to this issue? Thanks in anticipation of your help. I am not seeing this issue. I am using LR Classic CC with latest updates. My workflow is to apply Auto Tone during DNG file import, so all my images are processed to some degree. I have LR set up to write metadata changes to XMP. I just opened a DNG file in LR that I know I heavily edited in LR. When I open the same file in Pentax DCU 5, what I see is the original unedited file (not even my crops). If I open the file in On1 Photo RAW 2017, I see the same thing as in PDCU. As a SWAG, could this be yet another Graphics Processor related issue? Last edited by lsimpkins; 03-01-2018 at 08:49 PM. Reason: add additional thoughts

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Managing unwanted RAW (DNG) files when shooting in RAW+	Weevil	Digital Processing, Software, and Printing	3	03-24-2013 07:43 PM
RAW DNG files corrupted - I need advice please!!	jpzk	Digital Processing, Software, and Printing	9	08-21-2012 01:29 PM
645D: Corrupted DNG Raw Files with Lightroom?	marktucker	Pentax Medium Format	24	07-30-2012 09:49 PM
K5 raw DNG files borked by Aperture/Apple Raw upgrade?	filmamigo	Digital Processing, Software, and Printing	4	02-22-2011 04:51 AM
Opening bad, corrupted, or unreadable RAW files	bdavis	Digital Processing, Software, and Printing	3	04-04-2007 02:54 PM

03-01-2018, 02:23 PM	#2
pschlute Moderator Loyal Site Supporter Join Date: Mar 2007 Location: Surrey, UK Photos: Gallery Posts: 8,198	Don't get this issue with LR and Silkypix DS Pro

03-01-2018, 02:27 PM	#3
Altheim New Member Join Date: Feb 2018 Posts: 17	Have you checked for a possible plugin for Pentax format from Adobe. It could be that the native software is reading and then doing a writeback at close that corrupts the file. Just a thought.

03-01-2018, 02:28 PM	#4
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Welcome to the Pentax Forums! How are you doing your import into LR? Are you using the "Convert to DNG" feature? Also, are you configured to automatically write metadata changes to XMP? Either of those will rewrite the DNG. Steve

03-01-2018, 02:42 PM	#5
jatrax Site Supporter Join Date: May 2010 Location: Washington Cascades Photos: Gallery \| Albums Posts: 12,991	Originally posted by stevebrot How are you doing your import into LR? Are you using the "Convert to DNG" feature? Also, are you configured to automatically write metadata changes to XMP? Either of those will rewrite the DNG. ^^^^^ This. I use a number of different programs and all read my Pentax camera DNGs without issue. None of them should have an issue reading the Adobe converted DNGs either but the only thing I can think of changing the DNG is what Steve posted.

03-01-2018, 02:56 PM	#6
Oakland Rob Loyal Site Supporter Join Date: Apr 2015 Location: Bay Area California Photos: Gallery \| Albums Posts: 798	I use all those applications except Topaz, and many more, and have no issues with DNGs after doing all sorts of stuff to them in Lr (I auto-write changes back to the DNG). On a Mac. Perhaps you could put one of the corrupted DNGs somewhere, maybe a before and after, so some of us could see if we can reproduce the issue. It's especially odd cuz Bridge is just a browser. Did you run it through ACR in Bridge? I'm not 100% sure what you mean by "previewed."

03-01-2018, 03:37 PM	#7
JeaPea New Member Join Date: Aug 2016 Posts: 6 Original Poster	Originally posted by stevebrot Welcome to the Pentax Forums! How are you doing your import into LR? Are you using the "Convert to DNG" feature? Also, are you configured to automatically write metadata changes to XMP? Either of those will rewrite the DNG. Steve The DNG files I am using are those that are natively produced by the Pentax bodies I own. They are "added" to the Lightroom catalogue or viewed dierctly from the folders that they are stored on an internal hard drive... ---------- Post added 03-01-18 at 03:43 PM ---------- Originally posted by Altheim Have you checked for a possible plugin for Pentax format from Adobe. It could be that the native software is reading and then doing a writeback at close that corrupts the file. Just a thought. Thanks for your comment. However, the DNG format is an Adobe file standard and Lightroom and Bridge should (and write) to these as they would any DNG file whether it results from a conversion or is generated by cameras such as Pentax or Leica

03-01-2018, 03:40 PM	#8
JeaPea New Member Join Date: Aug 2016 Posts: 6 Original Poster	Originally posted by Oakland Rob I use all those applications except Topaz, and many more, and have no issues with DNGs after doing all sorts of stuff to them in Lr (I auto-write changes back to the DNG). On a Mac. Perhaps you could put one of the corrupted DNGs somewhere, maybe a before and after, so some of us could see if we can reproduce the issue. It's especially odd cuz Bridge is just a browser. Did you run it through ACR in Bridge? I'm not 100% sure what you mean by "previewed." Bridge (and Lightroom) both write any editing adjustments and update the embedded JPEG preview to the DNG file envelope. I have heard mention that these apps may be corrupting the TIFF headers (DNG files having a similar structure to TIFF files ---------- Post added 03-01-18 at 03:53 PM ---------- Originally posted by stevebrot Welcome to the Pentax Forums! How are you doing your import into LR? Are you using the "Convert to DNG" feature? Also, are you configured to automatically write metadata changes to XMP? Either of those will rewrite the DNG. Steve Thanks for your comments. These Adobe are supposed to write any changes and update the DNG file envelope thus avoiding the need for .xmp sidecar files. This corruption only happens with these two apps and not with other software that I have mentioned. So far, I have been referred to Adobe's third tier of Support. They can replicate the issue but are currently unable to identify why or offer a solution. The issues have only happened since updates to both apps from September 2017 onwards and would seemingly point to Adobe updates ince then.

03-01-2018, 04:00 PM	#9
cpk Site Supporter Join Date: Feb 2010 Location: Montreal Posts: 638	Have you tried taking some photographs in the PEF format to see if the same problem occurs?

03-01-2018, 04:45 PM - 1 Like	#11
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by JeaPea These Adobe are supposed to write any changes and update the DNG file envelope thus avoiding the need for .xmp sidecar files. Martin Evening in his Light 5 book strongly advises against using that feature, suggesting instead to trust the catalog and back that up on a regular schedule. Creating duplicate copies of XMP (catalog and file) for the same cataloged image is not a good idea since the writes are not guaranteed to be synchronous (LR generally does the file writes on a time-available basis). I also suspect that the thumbnail preview images are also written into the file, which may explain the viewer issues you are encountering. I treat the DNG as inviolate image source and may eventually take the additional step of marking those files read-only to avoid corruption by my tools. Steve

03-01-2018, 05:23 PM	#12
pentaxfall Veteran Member Join Date: Sep 2017 Location: Bangalore, India Posts: 581	I make the DNG files read-only before any software touches it. I remember Adobe CS6 was not generating XMP files if the file is not in read only state. I do not edit regularly so this is not a headache for me.

03-01-2018, 05:35 PM	#13
clackers Otis Memorial Pentaxian Join Date: Jul 2013 Location: Melbourne Photos: Albums Posts: 16,397	Originally posted by JeaPea Am I the only one? Being a Pentaxian it often feels like one is participating in a minority sport. Please read on and hopefully help a true believer... Lightroom and Bridge are corrupting my DNG raw files as generated by four Pentax DSLR bodies, i,e. K-5 II, K-3, K-70 and Pentax K-1. I can open and edit the DNG raw files from these Pentax cameras in Pentax DCU, DxO Photolab, Topaz Studio, and Skylum Luminar as long as they haven't been previewed in Lightroom CC Classic and/or Adobe Bridge CC 2018. Once they have been previewed, let alone edited, in Lightroom and/or Bridge, they become unusable in other software applications. Upon previewing them in Lightroom and/or Bridge, both the thumbnails and image data become unusable. The previews have either a weird posterised look or have large sections covered in red diagonals on a white background. Has any other Pentaxian experienced this? Better still, does anyone have a solution to this issue? Thanks in anticipation of your help. Not happening on my system, JP, sorry.

03-02-2018, 01:13 AM	#15
Alex645 Site Supporter Join Date: May 2015 Location: Kaneohe, HI Photos: Albums Posts: 3,526	Originally posted by stevebrot Adobe-generated or modified DNG offer limited, if any, compatibility outside of Adobe products. my conclusion is that a DNG file generated by or modified by LR should be considered of limited usefulness in other, non-Adobe, tools. Steve +1 with Steve here. I currently use both LR and PS CC and have no issues with any camera generated PEF or DNG as long as I donʻt try to then export them to other non-Adobe apps. Itʻs already too easy to over process digital images and my overall recommendation is to find one post process work flow that suits your needs and stick with it. Manipulating files between applications from different developers is a bit of alchemy and with every update, what may have worked yesterday may become impossible tomorrow.