Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
08-01-2012, 10:39 AM   #1
Forum Member




Join Date: Aug 2012
Photos: Albums
Posts: 50
DNG format issues

Befor and after purchase of new K-30 model I was initially happy about the DNG format, because no more AdobeDNGConverting is required and a couple of DAM and Editing tools prefer DNG over some proprietary RAW formats. However I identified two major issue:
  1. Missing option to define size of preview.
  2. Limited rendering display and export with a couple of software tools and severe issues with CaptureOne.
For point 1 the preview size definition is quite important. First to reduce file size. But more important, second to easily check that third party software is not using preview instead of raw date for processing.

For point 2 the issue is, that embedded DNG preview or JPG from Pentax look different from the rendering display and export result with most software. With IDimager and MediaPro the resulting images are darker, with SilkyPix it looks a litle bit pale and with CaptureOne the result is awfull wrong.
See forum.phaseone.com/En/viewtopic.php?f=41&t=12629 for a more detailed description and a couple of images showing the issue.

There is even a big difference when displaying the Pentax JPG and DNG files with all standard tools under Windows (e.g. Irfanview, WinFaxViewer, PicasaViewer, ...). When removing or modifying the DNG preview with AdobeDNG Converter from original Pentax DNG, there are additional variations of displayed results visible. That is quite confusing and I never experienced such variation with DNGs up to now.

I checked also the DNG file and JPG from another posting in this forum:
www.pentaxforums.com/forums/pentax-k-30/191598-example-raw-file.html
If you compare JPG and DNG this shows in some software tools clearly some issues with skin tone.


Last edited by Plentax; 08-06-2012 at 01:31 AM.
08-01-2012, 11:03 AM   #2
Veteran Member
Docrwm's Avatar

Join Date: Sep 2010
Location: Somewhere in the Southern US
Photos: Gallery | Albums
Posts: 11,275
Welcome aboard. The JPGs are processed with a variety of alterations based on the camera's settings and the RAWs are just that - RAW data that can be altered to suit your desires with post-processing.

Here's a link that does a good job of showing the differences:
Learn about RAW, JPEG, and TIFF with the digital photography experts at Photo.net.
08-01-2012, 11:31 AM   #3
Veteran Member
Philoslothical's Avatar

Join Date: May 2011
Posts: 1,723
QuoteOriginally posted by Plentax Quote
For point 1 the preview size definition is quite important. First to reduce file size. But more important, second to easily check that third party software is not using preview instead of raw date for processing.
In my opinion, file size isn't important at all. Disk space is cheaper than time and effort these days. If you think your editor is using the preview for processing, then that editor is garbage, throw it away and use something better. Seriously.

QuoteOriginally posted by Plentax Quote
For point 2 the issue is, that embedded DNG preview or JPG from Pentax look different from the rendering display and export result with most software. With IDimager and MediaPro the resulting images are darker, with SilkyPix it looks a litle bit pale and with CaptureOne the result is awfull wrong.
Again, there would be something very wrong if the raw data looked the same as the camera processed jpeg. Each raw editor applies its own default settings when initially rendering the raw data. Better editors allow you to fully customize these settings/profiles. This explains the variation you're seeing between editors.

Your best bet would be to settle on the editor you're most comfortable with, establish a profile or initial settings for it that you like, and go from there. It's far too easy to start looking for problems, and when you do, you usually will find them - even if the problem is just not fully understanding the programs or formats you're using.
08-01-2012, 02:12 PM   #4
Veteran Member
Anvh's Avatar

Join Date: Sep 2011
Posts: 4,616
I wonder what you other experiences are?
I've it with the K10D and the K5, doesnt matter if i used PEF or DNG the difference is simply RAW vs JPEG

08-01-2012, 04:02 PM   #5
Forum Member




Join Date: Aug 2012
Photos: Albums
Posts: 50
Original Poster
QuoteOriginally posted by Philoslothical Quote
...again, there would be something very wrong if the raw data looked the same as the camera processed jpeg. Each raw editor applies its own default settings when initially rendering the raw data. Better editors allow you to fully customize these settings/profiles. This explains the variation you're seeing between editors...
Thank you Philoslothical,

some of your comments are helpful, but some sound a little bit sarcastical. From PhaseOne I received information that DNG format of Pentax files is not according to standard and quite proprietary. Due to this the PhaseOne rendering in CaptureOne RAW editor seems not be able to create something useful from K-30 DNG files. And this is not about small variations! Did you really took a look on the images from the PhaseOne link I referenced in first posting?

Here is the original JPG from Pentax K-30 (left) and the displayed DNG file from CaptureOne (right) as 320x211 thumbnails:
(the exported JPG from C1 does look identical to the displayed DNG image, that is why I did not use ascreenshot)

The original 4928x3264 image files are:
imgp0097_from_pentax-k30.jpg (6.89 MByte), imgp0097_from_capture-one.jpg (13.1 MByte)
imgp0097_from_pentax-k30.dng (16.2 MByte)

In the posting www.pentaxforums.com/forums/pentax-k-30/192180-any-dng-available.html there is a similar issue visible.
I downloaded both the JPG and the DNG. The skin tones between original JPGs from K-30 and JPGs generated from original DNG are quite different.
This is exactly the same effect, which I see on all my K-30 photos with human faces.

QuoteOriginally posted by Philoslothical Quote
...your best bet would be to settle on the editor you're most comfortable with, establish a profile or initial settings for it that you like, and go from there. It's far too easy to start looking for problems, and when you do, you usually will find them - even if the problem is just not fully understanding the programs or formats you're using.
Very good tip! I like to settle for CaptureOne and with simple edits for IDimager. But what profile can be used in C1 to correct the drastic color deviations (e.g. from blue to violett or yellow to green)?

PhaseOne supports PEF files, but these files are not generated from K-30. What a pitty. PhaseOne even tried to simulate PEF files from DNG, which failed. So what is the benefit with DNG if it creates more trouble than PEF. With PEF I could use AdobeDNGConverter of C1 or both together, with strange DNGs there seems to be no solution in C1.

Last edited by Plentax; 08-03-2012 at 03:39 AM.
08-01-2012, 04:11 PM   #6
Veteran Member
Anvh's Avatar

Join Date: Sep 2011
Posts: 4,616
PEF files are connect to model so the K10D create other PEF files then the K20D so even if the K30 could make PEF files you couldn't use them unless C1 support the camera.
Beside you would have the same problems.


Anyway there should be somewhere option about colour profiles most likely there should be some to pick from, most RAW editors have that option so good luck finding it since i dont know C1.
You can also make your own profile and there are ways to automaticly make some with the right tools (x-rite colorchecker)

About DNG, all the problems i've run into is the support for the file and not a fault in the files themself, if you use another program you would have very different result. I tried C1 3 years ago and it didnt like DNG at all in those days, i got very freaky colour shifts while the other programs worked quite fine.
08-01-2012, 04:25 PM   #7
Forum Member




Join Date: Aug 2012
Photos: Albums
Posts: 50
Original Poster
Thank you Anvh,

but with PEF I could convert from PEF to DNG with AdobeDNGConverter. Isn't it? And such files should work with DNG compatible software, isn't it?

I am not the DNG expert, but I have confidence in other experts. Here is a quote from PhaseOne forum:
QuoteOriginally posted by P1-forum:
...due to the DNG file tag, CaptureOne is trying to read it as if it were a supported Adobe DNG file but clearly it is not. Further, the file cannot be converted by Adobe converter into a standardized DNG., so that if nothing else Capture One could properly read it. Ultimately the file is it's own proprietary RAW format despite the "DNG" tag, which will require specialized support be built for it in CaptureOne...

Last edited by Plentax; 08-03-2012 at 03:40 AM.
08-01-2012, 04:53 PM   #8
Pentaxian




Join Date: Mar 2010
Location: Frankfurt am Main
Posts: 1,064
Please keep in mind that the DNG format defines just the structure of a DNG file. It cannot do it better, as the sensors of cameras even of the same brand are very different, and the data, no matter whether stored as DNG or PEF or whatsoever, don't mean the same from different bodies/firmware. The interpreting software MUST know details about how the camera handles its RAW data (as a first step the sensor values must be normalized). In this sense, not only the Pentax DNG is unique, but also the DNGs of every other brand's cameras.
As the K30 is new, not all software companies will have updated their software yet. And some may do it only for their next version, to force customers to spend some money on upgrades (instead updates for free). I heard this was a problem in the past also with other brands than Pentax.

Another problem may be the processing of the normalized data to display a picture. They must be handled just as if you would create a JPEG file (without 8 bit conversion and JPEG compression). The parameters used can be taken from data the camera stored in the DNG, or from a profile the software has in its database, or the software uses some default setting. These data decide about ISO, exposure, white balance, saturation, colour shift, contrast, basic sharpening, etc.

If you use a similar profile with these programs, the pictures displayed shouldn't look too much different. If they do, you would have to find out which profile they are using. If they don't have a appropriate one, you may have to create your own.

Edited: Some points better explained.


Last edited by RKKS08; 08-01-2012 at 05:06 PM.
08-01-2012, 04:55 PM   #9
Pentaxian




Join Date: Jun 2009
Location: GMT +10
Photos: Albums
Posts: 10,720
The supposed expert you quote on the PhaseOne forum is spouting rubbish.

Capture 1 had exactly the same issue with K-x DNG's for a while. It took until v v6.3.3, I think, for it to correctly render K-x (and K-5) DNG's. I remember sending them a K-x DNG that was going dramatically wrong to illustrate the problem.

Eventually they fixed the issue, despite their claim that they only support PEF for K-x/K-5.

So C1 do respond to user concerns, and the issue is totally fixable by them. In fact the C1 rendering of K-5 DNG's in particular is really, really excellent now. They did a great job and their support response was excellent.

I think that since the K-30 only shoots DNG, they will eventually step up to the plate on K-30 too. Just keep reminding them. They do listen and respond.

The pix below illustrate what the problem used to be with K-x DNG's and C1.

The first pic is a K-x JPG, the second pic is what a K-x DNG used to look like under the default rendering of C1 Express.

K-x JPG:



K-x DNG in C1 prior to v 6.3.3


Last edited by rawr; 08-01-2012 at 05:07 PM.
08-02-2012, 01:47 AM   #10
Forum Member




Join Date: Aug 2012
Photos: Albums
Posts: 50
Original Poster
Thank you very much RKKS08 and RAWR,

apparently it is easy to create confusion with DNG-"Standard" and the end consumer is trapped between camera manufacturers, image software and Adobe standardization. And finally DNG is not (yet) an official standard. I know this situation well from a couple of other industry (quasi) standards. But also JPEG standard is affected. E.g. with 4:4:4 or 4:2:2 instead of popular 4:2:0 color subsampling. A lot of devices and image software are not able to display or process something else than 4:2:0 JPG correctly.

QuoteOriginally posted by RKKS08 Quote
...as the K30 is new, not all software companies will have updated their software yet...
A lot of software is able to display the DNGs from K-30 more or less good without update. E.g. FastPictureViewerCodec, SilkyPix, IDimager show only a minor color deviation, which does look correctable (see P1 forum link in the first posting). Why does just C1 has this kind of problems? Because the rendering quality of C1 is excellent only for supported files? May be it is really worth the effort to wait (and pay) for an update from PhaseOne. Thank you RAWR for your K-x images from C1 and the explanation. That helps!

QuoteOriginally posted by rawr Quote
The supposed expert you quote on the PhaseOne forum is spouting rubbish. Capture 1 had exactly the same issue with K-x DNG's for a while. It took until v v6.3.3, I think, for it to correctly render K-x (and K-5) DNG's...
I hesitated with such statement, because the quote (public) is not from a P1 forum member, but from a very helpful and friendly P1 staff. From P1 helpdesk I received additional (non public) information, which I would not like to quote without authorization. You are right, the P1 support is good, qualified and fast. So I am still optimistic. May be my decision for K-30 was too early, but my discussion with you and the C1 issue is paving the way for further K-30 users.

Questions:
  • If direct support of DNG from cameras is so difficult, what about proprietary RAWs from cameras converted with AdobeDNGConverter to DNG?
  • Are the camera DNG different from converted Adobe DNG?
  • If the user has to wait for SW updates with DNG format, is finally the camera DNG nothing else than a proprietary RAW format?
  • Which tool is able to show all relevant DNG properties? Up to now I use and like PhotoME, www.photome.de
  • If there is no preview in the AdobeDNG (setting in AdobeDNGConverter) how are previews generated so fast and good from other tools like FPVC, www.fastpictureviewer.com/codecs

Last edited by Plentax; 08-03-2012 at 03:40 AM.
08-02-2012, 02:44 AM   #11
Pentaxian




Join Date: Jun 2009
Location: GMT +10
Photos: Albums
Posts: 10,720
QuoteOriginally posted by Plentax Quote
because the quote (public) is not from a P1 forum member, but from a very helpful and friendly P1 staff.
Ooops! I'm sorry about that. I have great respect for the P1 staff, but that just sounded wrong.
QuoteOriginally posted by Plentax Quote
Are the camera DNG different from converted Adobe DNG?
Yes, for the same reasons Pentax Digital Camera Utility, Silkypix, DxO etc render DNG's differently from Lightroom (which uses the same engine as Adobe's DNG Converter).

QuoteOriginally posted by Plentax Quote
Which tool is able to show all relevant DNG properties?
Phil Harvey's EXIFtool is always the most up to date, and is probably the most comprehensive EXIF utility available, and is always more up to date than PhotoMe. Give it a try.

QuoteOriginally posted by Plentax Quote
how are previews generated so fast and good from other tools like FPVC
DNG, like PEF, has built in JPG previews. Maybe it uses them. Without knowing more about FPVC, perhaps the RAW convertor they employ (dcraw?) is just very efficient and also kept up to date?
08-02-2012, 06:20 AM   #12
New Member




Join Date: Jun 2012
Posts: 15
Plentax, I endorse your views about Capture One having a hard time with the K30 DNGs. They sure look horrible.
I like using Capture One and have served my apprenticeship on it such that I would not like to move on.
I always used PEFs but was surprised to see Capture One seems to have fixed the DNG problem with the KX DNGs so that I will confidently swap to DNG.
It sounds like the Capture One crew are not going to find it a push over making the K30 DNGs work and then we will have to load in a whole new program that may cost money rather than just add a new profile.
I do not understand how DNG can become a universal standard if it is so proprietary. That defeats the whole purpose and our great grand-kids are not going to bother looking at our photos in some weird format.
08-02-2012, 06:54 AM   #13
Pentaxian




Join Date: Jun 2009
Location: GMT +10
Photos: Albums
Posts: 10,720
These images are some examples of straight-out-of-camera K-5 DNG's processed by C1 Express.

I do not think it should be too hard for Phase One to successfully support K-30 DNG's:


Camel girls

Juvenile Pied Butcherbird portrait 1
08-02-2012, 07:44 AM   #14
Forum Member




Join Date: Aug 2012
Photos: Albums
Posts: 50
Original Poster
Thank you RAWR,
QuoteOriginally posted by rawr Quote
...DNG, like PEF, has built in JPG previews. Maybe it uses them. Without knowing more about FPVC, perhaps the RAW convertor they employ (dcraw?) is just very efficient and also kept up to date?
DNG created with AdobeDNGConverter v7.1 have not always previews. They can have no, medium or full preview. In my case with 4928x3264 images from K-30 I just converted the Pentax DNG with AdobeDNGConverter in three addtional AdobeDNG files. As far as I know, this does not make sense from RAW point of view. But the preview is modified as configured (either no preview, or with 1024x678 medium or with 4928x3264 full dimension in my case).

Please see below a screenshot from Win7 FIlemanager (ExtraLargeIcon View). Of course the DNGs are displayed only because of FastPictureViewerCodec v3.3.0.66. But you see the difference of AdobeDNGs with or without preview. Even without preview FPVC is able to display a thumbnail, which is just a little bit darker than the other thumbnails.


For original image files see: www.ey-com.de/captureone/k-30/

I will number the pictures in my following text with 1-2-3 (first row) and 4-5-6 (second row)
You can identify the cropping well on the left side of images (where the red X marks the small car)
If you open the files for example with Irfanview 4.33 you detect the following:
  1. This is just again the image visible or exported with CaptureOne v6.4.3. Bad as reported but no cropping. Of course C1 displays such bad color also for AdobeDNG images 4-6, because the preview is not used for display or conversion.
  2. This is the original Pentax DNG. Note the cropping. Dimensions are 4928x3264
  3. This is the original Pentax DNG. Looks fully identical to "2". Dimensions are 4928x3264
  4. This AdobeDNG is a little bit darker. No cropping. Dimensions are 4928x3264
  5. This AdobeDNG does look identical to "4", but with lower resolution. Dimensions are 1024x678.
  6. This AdobeDNG is even more darker. Dimension are 2468x1636 (?). Don't know how this preview is generated in WIndows/FPVC and Irfanview.
The variations of cropping and dimensions (4960x3300, 4936x3272 or 4928x3264 in my case) among various other tools are a little bit confusing but perhaps not really relevant.
More important is the color and with a couple of tools (IDimager, Irfanview, FPVC, ...) I have the impression, that they display Pentax DNG and convert to JPG quite properly. And this without any update! I cannot imaging that all tools without CaptureOne are prepared for Pentax K-30 DNGs. There must be some other reason.

Last edited by Plentax; 08-05-2012 at 04:09 PM.
08-02-2012, 08:39 AM   #15
Forum Member




Join Date: Aug 2012
Photos: Albums
Posts: 50
Original Poster
QuoteOriginally posted by kevcol Quote
...it sounds like the Capture One crew are not going to find it a push over making the K30 DNGs work and then we will have to load in a whole new program that may cost money rather than just add a new profile...
It is not only about money, but also about number of tools and time you have to spend for the effective workflow with them.

QuoteOriginally posted by kevcol Quote
...I do not understand how DNG can become a universal standard if it is so proprietary. That defeats the whole purpose and our great grand-kids are not going to bother looking at our photos in some weird format.
Exactly! Imagine the grand-kids want to use your negatives. With analog film this is obvious and simple. With digital DNGs this looks from perspective of today like a disaster. The poor kids have to look for appropriate tools, for profiles, for ICC, for converters, ...

May be it is better to create three (R+G+B) black & white analog high resolution microfilms from all your digital photographs? I know about some funded academic & industry research projects about this serious long term archival topic.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
couple, dng, issues, jpg, k-30, k-50, pentax, pentax k30, pentax k50, preview, size, software, tools
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
PEF or DNG file format consiglieri Troubleshooting and Beginner Help 12 11-11-2010 10:13 AM
K200d raw format: dng vs. pef tibbitts Pentax DSLR Discussion 7 03-04-2009 11:01 AM
DNG in-camera vs DNG from Lightroom? rfortson Digital Processing, Software, and Printing 9 11-02-2008 11:06 PM
K10D DNG format. Useless? Edvinas Pentax DSLR Discussion 8 09-19-2006 03:03 PM



All times are GMT -7. The time now is 01:57 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top