Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
06-25-2022, 01:16 AM - 2 Likes   #16
Pentaxian




Join Date: May 2015
Photos: Albums
Posts: 2,998
One advantage DNG had over PEF in terms of archiving is that the DNG has the colour matrix embedded. This means the decoding software needs to know very little about your camera. This is also useful with new cameras that aren't supported by software yet. I've already taken advantage of this many times.

06-25-2022, 01:44 AM   #17
Site Supporter
Site Supporter
newmikey's Avatar

Join Date: May 2007
Photos: Gallery
Posts: 2,195
QuoteOriginally posted by Digitalis Quote
There are examples of AV formats that have gone extinct without any possibility of being able to recovery data, is it really so far-fetched to think the same cannot happen to digital image files? Cameras that use nonstandard pixel formats and altered colour filter arrays* and alternate pixel pattern layouts** that may be difficult to correctly interpret with current software as the original processing software could. Early Kodak, Fuji,SONY, Nikon and Foveon cameras have raw files that for various and assorted reasons rather difficult to read correctly.
We were discussing rather run-of-the-mill Pentax PEF if I recall correctly. Similar to other mainstream camera models and their proprietary raw formats I indeed think it is far-fetched. I can still read 2006 K110D PEFs with any software I use today. I expect to be able to do so indefinitely. The fact that somewhat more "odd" or one-time nonstandard file formats may have become "difficult" to read today is really not a valid argument in a discussion about Pentax PEF format.

Your examples are also far-fetched:
The SONY DSC F828 predates the very first DNG format by almost a whole year. Last time I looked an SRF file can still be converted by existing software such as but not limited to DCRAW (and any software which uses it as a library to read raw files). The decryption code was published by the legendary Dave Coffin on Github and remains there for anyone to use (libopenraw/sony_clear.c at master ∑ freedesktop/libopenraw ∑ GitHub)

Sigma Foveon native files can be converted using RawTherapee and a plethora of other software packages. Later Foveon models included DNG as raw format.

I'll try the referred Super CCD HR and SR I and II, the Nikon D1X used non-square pixel elements later but again, rather far-fetched indeed.

---------- Post added 06-25-22 at 10:47 AM ----------

QuoteOriginally posted by house Quote
One advantage DNG had over PEF in terms of archiving is that the DNG has the colour matrix embedded. This means the decoding software needs to know very little about your camera. This is also useful with new cameras that aren't supported by software yet. I've already taken advantage of this many times.
Now, thŗt indeed is a valid point I had not considered even though I use software which is frequently updated. Thanks for submitting a reasonable argument in this discussion. Appreciate it!


[EDIT]I just downloaded a sample SRF file from the raw file repository at raw.pixls.us and tried to open it. Please find results below for DarkTable which opens it without any trouble and with the appropriate WB - but I see why you referenced it when I tried it in various other raw converters. Luckily it was only a one-time attempt by Sony to use this format so I do not think it makes such a valid argument for today's discussion. I don't think many people used that particular raw format seeing how hard it was to obtain a sample.
Attached Images
 

Last edited by newmikey; 06-25-2022 at 04:00 AM.
06-25-2022, 05:16 AM   #18
Reboot before calling IT
Loyal Site Supporter
madison_wi_gal's Avatar

Join Date: Sep 2016
Location: Madison WI
Photos: Gallery | Albums
Posts: 597
QuoteOriginally posted by newmikey Quote
So to me, an app "has white blank box. Has no idea what to do with this" would be a sign to update my app, not to feed it a DNG unless it was a temporary measure to tide me over until the update.
Well, in my case, before I got DxO I'd use MacOS Photos to preview my card contents to see what I wanted to work on. Since DxO can preview SD contents in a grid and can show PEFs as well as DNGs I no longer have any strong feeling about it.

And since my "app" (in this case, MacOS Photos on Monterey) is part of the native OS apps, the app *is* updated.
06-28-2022, 07:44 PM - 2 Likes   #19
Moderator
Site Supporter
Loyal Site Supporter
MarkJerling's Avatar

Join Date: May 2012
Location: Wairarapa, New Zealand
Photos: Gallery | Albums
Posts: 19,533
QuoteOriginally posted by newmikey Quote
Sorry. Same old, same old argument doesn't fly with me and never did. Last time I looked even on Windows or Apple there weren't too many packages which read DNG but not PEF. On Linux that number goes down to zero. What is your definition of "broadly supported"?
PEF = "Pentax Electronic File"
DNG = "Digital Negative Image". "DNG is an open-source, royalty-free, highly compatible raw image format used by many different camera manufacturers."

I don't think there can be any argument that DNG is used by more camera brands than Pentax. It's also the preferred format for a number of archival systems, most notably the US Library of Congress.

Here's a little video that delves into this in more detail.


Of course, you'll notice the video is from 2010. Back then, there was probably less support on all operating systems for the PEF and other proprietary raw formats. DNG sought to address that problem.

I'd argue that it still offers advantages over PEF and other proprietary raw formats.

06-28-2022, 09:24 PM   #20
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 11,135
QuoteOriginally posted by MarkJerling Quote
I don't think there can be any argument that DNG is used by more camera brands than Pentax
Are we sure that Pentax (or other brand) DNG would be read-able if Pentax wouldn't be supported?

Personally, I'm not so sure about how generic are DNG files, and I think (maybe I'm wrong) that each camera manufacturer has it's own way of using the DNG container. If the DNG implementation is brand specific, that means there's no generic advantage over PEF.
06-29-2022, 02:19 AM - 2 Likes   #21
Moderator
Site Supporter
Loyal Site Supporter
MarkJerling's Avatar

Join Date: May 2012
Location: Wairarapa, New Zealand
Photos: Gallery | Albums
Posts: 19,533
QuoteOriginally posted by biz-engineer Quote
Are we sure that Pentax (or other brand) DNG would be read-able if Pentax wouldn't be supported?

Personally, I'm not so sure about how generic are DNG files, and I think (maybe I'm wrong) that each camera manufacturer has it's own way of using the DNG container. If the DNG implementation is brand specific, that means there's no generic advantage over PEF.
All the information I can find online (and through Adobe) points to DNG being based on the TIFF file format, which is an industry standard file format, to which a set of rules is then applied so that Adobe (or other software) can read a DNG file regardless of origin. There's also a particular (loss-less) compression algorithm as well as the ability to determine very easily if the file has been corrupted in some way.

Because DNG uses a specific file specification, all users should be using that specification. See: Digital Negative (DNG), Adobe DNG Converter | Adobe Photoshop
So far as I can tell, therefore, there is no evidence to suggest that the DNG specification varies from brand to brand.
06-29-2022, 11:23 PM   #22
Pentaxian
thibs's Avatar

Join Date: Jun 2007
Location: Belgium
Photos: Albums
Posts: 6,947
QuoteOriginally posted by MarkJerling Quote
All the information I can find online (and through Adobe) points to DNG being based on the TIFF file format, which is an industry standard file format, to which a set of rules is then applied so that Adobe (or other software) can read a DNG file regardless of origin. There's also a particular (loss-less) compression algorithm as well as the ability to determine very easily if the file has been corrupted in some way.

Because DNG uses a specific file specification, all users should be using that specification. See: Digital Negative (DNG), Adobe DNG Converter | Adobe Photoshop
So far as I can tell, therefore, there is no evidence to suggest that the DNG specification varies from brand to brand.
At least in the K5 era, PEF file had to be supported for each specific camera. With DNG you can always open the file, even if it may exhibit color calibration of some sorts etc. At least you're sure you can open it.
If there's any fear to have IMO it is against the PEF format, not the other way around.

06-30-2022, 12:11 AM   #23
Site Supporter
Site Supporter
newmikey's Avatar

Join Date: May 2007
Photos: Gallery
Posts: 2,195
QuoteOriginally posted by thibs Quote
At least in the K5 era, PEF file had to be supported for each specific camera. With DNG you can always open the file, even if it may exhibit color calibration of some sorts etc. At least you're sure you can open it.
If there's any fear to have IMO it is against the PEF format, not the other way around.
You are all ignoring the elephant in the room: when presented with a DNG file, there is no obvious way to tell whether it is really a raw file or even whether it is actually produced by a specific camera respecting the DNG guidelines. It could be:
1) A straight out of camera raw file (from a camera manufacturer offering the DNG option to store raw in-camera)
2) A manipulated raw file stored by LR (or some other software)
3) The result of a conversion of a proprietary raw file to DNG via any of the many existing converters which may or may not result in the loss of proprietary Maker notes
4) Same as (3) but now in a lossy compression format which may even omit some sensor data besides Maker notes
5) A non-raw image file (post demosaicing, post WB or both) stored inside a DNG container
6) Any other variation I have not yet named above

This is not about fear but about certainty. The certainty that a PEF/NEF/CR2/ORF/ARW/etc file comes straight out of the camera which generated it, with only minimal adjustments allowed to its EXIF (such as copyright info or GPS coordinates)
06-30-2022, 02:21 AM - 4 Likes   #24
Moderator
Site Supporter
Loyal Site Supporter
MarkJerling's Avatar

Join Date: May 2012
Location: Wairarapa, New Zealand
Photos: Gallery | Albums
Posts: 19,533
The point of a DNG file is not certainty that it's straight from camera or the other options you mention. The point is that it's a TIFF type file with all its parameters recorded in an industry agreed way. That's what makes it good for archival storage.

The original raw data in a DNG file is never altered. The DNG file is writable in certain segments, although not in the raw data segment. As such, with the correct tools, it should be possible to find what modifications, if any, has been made to a file. Certainly, people who work in the evidential image sector will be able to tell if a DNG file has been manipulated or not. This could be done by looking at factors such as whether they're looking at Raw DNG" or "Linear DNG" and other pointers, well beyond my understanding.

The whole point of DNG is to be a universally accessible raw format. I've yet to see evidence that the different parameters you mention cannot be found by people with the necessary skills. It's not an "elephant in the room" moment.

For more information about the DNG format and how it preserves it's original dataset without manipulation, see: https://opencommons.uconn.edu/cgi/viewcontent.cgi?article=1027&context=libr_pubs

The Library of Congress only uses DNG format and, for many reasons, views it superior to all other proprietary (manufacturer based) raw formats. It's also the archival master format at The National Gallery of Art, Washington, D.C. It is, in short, the most trusted raw format there is and can operate without the input of any specific supplier's camera / system specific software.
06-30-2022, 04:17 AM   #25
Site Supporter
Site Supporter
newmikey's Avatar

Join Date: May 2007
Photos: Gallery
Posts: 2,195
QuoteOriginally posted by Rondec Quote
To me, the nice thing about DNG is that I can continue using my old version of Lightroom --version 6.14 -- with newer cameras like the K-3 III while shooting RAW.
Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.
06-30-2022, 06:00 AM   #26
Pentaxian




Join Date: May 2015
Photos: Albums
Posts: 2,998
QuoteOriginally posted by MarkJerling Quote
The whole point of DNG is to be a universally accessible raw format. I've yet to see evidence that the different parameters you mention cannot be found by people with the necessary skills. It's not an "elephant in the room" moment.
I know that DNG's can cause issues due to the scope of the standard. Programmers don't feel they can cover all bases and when they do some third party makes a different interpretation that fails to work. As you know it doesn't have to be raw data and it's apparently not straightforward to figure out.

But that isn't an issue for Pentax so far. DNG's are slightly "safer" for new cameras or old software.
06-30-2022, 07:09 AM   #27
Pentaxian
reh321's Avatar

Join Date: Nov 2014
Location: South Bend, IN, USA
Photos: Gallery | Albums
Posts: 22,146
QuoteOriginally posted by newmikey Quote
Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.
I also donít drive my auto to Ďitís fullest capacityí.
I use what I need, no more or less.
06-30-2022, 07:33 AM   #28
Pentaxian




Join Date: May 2015
Photos: Gallery
Posts: 1,865
QuoteOriginally posted by newmikey Quote
Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.
Can you give an example of "any camera potential in the sensor data of newer camera models" that would benefit from later software versions? I don't think pixel shift is supported even now by Adobe.

06-30-2022, 07:47 AM - 4 Likes   #29
Loyal Site Supporter
Loyal Site Supporter




Join Date: Mar 2009
Location: Gladys, Virginia
Photos: Gallery
Posts: 26,080
QuoteOriginally posted by newmikey Quote
Thanks, that clears up where you stand. You don't really care about all of the software improvements and technical experience gained since 2017 and you have no interest in availing yourself of any camera potential in the sensor data of newer camera models since that same date. I will never understand the fact people buy the latest and greatest DSLR body only to subsequently not use it to the fullest of its capability and/or the fullest of newer software developments available.
I'm trying not to take offense. There has been less improvement in actual RAW development in newer Lightroom versions than you might think. More than that, if you use third party apps like Topaz Suite or Nik Effects, you can bring a lot out of your RAW images. I post images here and I don't think my limitations are related to using older software.

For pixel shifting images I typically am using RAW Therapee which does a significantly better job of processing than does the newest version of Lightroom.
06-30-2022, 09:54 AM - 1 Like   #30
Pentaxian




Join Date: May 2015
Photos: Albums
Posts: 2,998
QuoteOriginally posted by Rondec Quote
For pixel shifting images I typically am using RAW Therapee which does a significantly better job of processing than does the newest version of Lightroom.
Have you used it on a standard raw file? With dual demosaic (amaze or RCD/VNG4) and capture sharpening I find the results are cleaner and more detailed than any other software. Of course there are 10 further sharpening tools you might use but the aforementioned is enough for me. Lightroom just feels crude in comparison. I've only done quick comparisons though.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
argument, camera, color, color profiles, container, converters, dng, files, format, formats, icc, pef, pentax, photography, photoshop, post, processes, profile, profiles, question, software, type, user
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
PEF vs. DNG - Is there any loss of data having the camera save in DNG RAW? Selvadero Troubleshooting and Beginner Help 41 08-12-2020 07:21 AM
Managing unwanted RAW (DNG) files when shooting in RAW+ Weevil Digital Processing, Software, and Printing 3 03-24-2013 07:43 PM
Microsoft releases RAW Codec with support for Pentax PEF raw files NKK Digital Processing, Software, and Printing 16 08-27-2012 06:00 AM
PEF files converted to DNG with Lightroom not readable by Raw Therapee? klh Digital Processing, Software, and Printing 13 08-08-2012 06:13 AM
UFRAW and K5 raw files (DNG and PEF) sterretje Pentax K-5 & K-5 II 3 11-23-2010 11:39 PM



All times are GMT -7. The time now is 07:39 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top