Originally posted by newmikey I would say Pentax made an outing into using DNG as it standard raw format but abandoned that later on, keeping PEF as standard in all newer models (K-70, KP, K-1) with DNG for compatibility. Reading the manual for my particular model, the K-70, on page 25 it literally states under the menu listings the default format as PEF.
This defuses your statement as if "DNG is now the Pentax main and consistent raw file format, and PEF is an optional extra for historical purposes." - I'd say it is just the other way around. The reasons do not really matter to us but fact of the matter is they literally state the opposite in the camera manual so I must ask where your statement came from and what it was based upon?
Having worked as an engineer in the IT industry, I beleieve there is a hierarchy of technical decision-making in Pentax:
Higher-up, there is a decision about what raw file formats are available for Pentax cameras to use. For a long time this was something like "always use DNG, optionally use PEF". We can deduce that from the raw file formats that new Pentax cameras actually used over many years. What we can't deduce is whether that decision has now changed. The last DNG-only camera I know of was announced in 2014. There have only been 6 Pentax cameras announced since then).
Below that higher-level there is a specific decision made for every camera. Perhaps the higher level policy has now changed to "DNG and PEF", or perhaps it is just an accident of history that we haven't seen any more DNG-only cameras since 2014.
Then at the lowest level there are people setting the defaults in the camera. The defaults are somewhat arbitrary. When I get a new camera I go through the menu setting the camera as I want it. I don't take the defaults seriously, because they tend to be chosen by people who don't do photography the way I do. In both the K-1ii and the K-3ii there doesn't appear to be any preference in the operating manual for one or the other. I can't remember which way they were set by default, and I don't care because they are set to DNG now!
Originally posted by newmikey When Adobe designed DNG as a format, they actually did one thing right: they listened to camera manufacturers and left specific room for them to add their own undisclosed metadata right inside the DNG itself without the need for those manufacturers to disclose the use of that "secret sauce" to Adobe.
It wasn't camera manufacturers who put pressure on Adobe. It was users. I was a user of DNG from 10 days after it was launched at the end of September 2004. The concern of a number of us was that, while the Adobe software at the time didn't need the Makernote which the DNG Converter had used to generate the DNG Metadata, things may change in future. Perhaps a later DNG Converter could exploit more the Makernote? Perhaps software other than that from Adobe could exploit some of the "secret auce"? (The DNG Converter can read a DNG, of course, so a thought was that if the Makernote was preserved, it could be further exploited by the latest DNG Converter).
I kept my PEFs until I was confident that they had been converted properly, and especially until the Makernote was preserved as DNGPrivateData. I was able to reconvert where relevant. Then I deleted all my PEFs. From that point on, (in 2005, about 6 months or so after I started to use DNG), after checking that a batch of conversions had worked, I had the confidence to delete my PEFs. That never caused me a problem.
Which camera manufacturers would be concerned and have sufficient influence to cause Adobe to provide DNGPrivateData? Obviously not those who had their own raw file formats and their own software to process them. Canon and Nikon hadn't made a decision use DNG in-camera and supply software that could process DNG and exploit any "secret sauce" their cameras might have put there. DNG was at best irrelevant to them, and at worst an irritant to be suppressed.
Leica was the first rumoured camera maker to use DNG in-camera, and one of the first to do so. I believe they packaged Photoshop Elements with their first cameras to use DNG. And Photoshop Elements didn't exploit any "secret sauce"in DNGs. Like other Adobe software processing DNGs, it uses the published DNG metadata. So Leica had no reason to put pressure on Adobe.
Ricoh's GR series was the first compact camera to use DNG, and in fact Ricoh exploit DNG more than most. They use DNG 1.3.0.0 in order to use the lens-correction opcodes. (The K-1-series "only" uses DNG 1.2.0.0).
It is worth noting that, when I stopped counting in 2010, there were 14 cameras makers using DNG, 47 camera models using DNG, 240+ software products able to process DNG to some or a full extent, and 290+ Adobe-Convertible raw image formats. And about 7 or 8 alternative DNG converters to cater for niche or hacked cameras that the Adobe DNG Converter didn't cater for.
For comparison, during the first 5 years when about 38 camera models were launched that wrote DNG, Adobe software added support for about 21 Canon models, about 20 Nikon models, and about 22 Olympus models.
Originally posted by newmikey I don't really know where you get those ideas but all of my non-Adobe converters support DNG out of the box, as long as it is camera-generated DNG. It's the converted DNG's out of f.i. the Adobe DNG Converter they cannot stomach - that should tell you something about the quality of the conversion probably.
That says more about those raw converters than about DNG! DNG is comprehensive. It caters for more capability than a typical camera uses. After all, Adobe software manages with just the published DNG specification features of DNG, so it can be done.
Originally posted by newmikey As things stand today, there is no compelling reason to use DNG unless it is camera-generated and you use (and plan to continue to use into the future) Adobe software.
I take the converse view. PEF is irrelevant to me, so the fact that my Q-Series cameras didn't use PEF wasn't an issue to me. All I ask from a Pentax camera is "please use DNG; it doesn't matter whether you support PEF".
Originally posted by newmikey Having said that, the once so free and happy DNG format becomes a jail for your images once it has been touched by Adobe software (either converted from proprietary raw to dng, or written to by LR) as it will take away your freedom to use non-Adobe software thereafter.
Why?
I can take a K-1ii DNG which I've used in Lightroom, put it onto an SD Card, put it back into the K-1ii, and view it. In fact, a minute before I wrote this I did precisely this! It has not been changed by Lightroom. (I've also just used the DNG SDK to examine the DNG file and verify this).
I suspect you are talking about using Lightroom to store the catalogued metadata editting back into the DNG. Rather than say, storing it as an XMP file, or simply using the Lightroom catalogue and not interfere with the DNG. I don't do that.
Originally posted by newmikey There is the little thing of proof of ownership as well - a PEF file is pure raw proof of authorship as it is a true digital negative which does not get written to once it leaves the camera. DNG's might or might not be useful in proving you own a given file, depending on whether they were manipulated or converted.
I don't let my DNGs get changed by Lightroom. But PEFs can also be changed by software! To the best of my knowledge, there is no way of detecting whether a PEF has been changed since it left the camera.
---------- Post added 1st Nov 2018 at 02:41 PM ----------
Originally posted by BigMackCam Interestingly, with the exception of Adobe Lightroom (and, of course, Camera Raw), most raw processing software will not load the native .ARW files from my Sony-A99-based Hasselblad HV. But if I run them through AdobeDNGConverter, a range of different software will open and process them without difficulty - including both digiKam and Darktable (which form the core of my current workflow).
I mention this only as a curiosity rather than my belief that DNG is somehow better. For this camera, DNG (or rather, lower case ".dng") is my only viable option. I suspect that AdobeDNGConverter is either removing or normalising something in the Hasselblad files to make them look the same as the A99, but that's only a guess...
Interesting!
See my experience at the address below:
Support via DNG but not native raws
I one found that Phase One Capture One wouldn't support DNGs derived in any way from rival digital backs. So I used a hex-editor to change the camera/back name in the DNG to a same-size meaningless string of letters, and the software then worked!
It deliberately checked for, and rejected, DNGs that had the names of rivals in them!
---------- Post added 1st Nov 2018 at 02:47 PM ----------
Originally posted by newmikey Nor will they open in most any other 3rd-party converter. Seems a high price to pay for shaving off a few MB's of storage?
When I stopped counting in 2010, there were 14 cameras makers using DNG, 47 camera models using DNG, 240+ software products able to process DNG to some or a full extent, and 290+ Adobe-Convertible raw image formats. And about 7 or 8 alternative DNG converters to cater for niche or hacked cameras that the Adobe DNG Converter didn't cater for.
Products that support DNG in some way