[texandrews]

.... An out-of-camera DNG lacks rendered image data and just has voltages from the sensor. (This is the difference between RAW and image files.) ....


Steve

What do you mean by this exactly? Because my DNG's OOC from my Pentax cameras have all the image data. I must be severely misunderstanding something here...

Part of all this as a sidebar is my current work deployment on IPTC metadata for a whole lot of images to be finally ingressed into DAMS...

[acoufap]

What do you mean by this exactly? Because my DNG's OOC from my Pentax cameras have all the image data. I must be severely misunderstanding something here...

Part of all this as a sidebar is my current work deployment on IPTC metadata for a whole lot of images to be finally ingressed into DAMS...

The raw image data isn't RGB image data but captured output voltages of the sensor cells plus RGB filter info according to the Bayer pattern and some more info like white balance. There's also a small embedded preview image - usually 720 x 480px - created by the camera built-in raw converter using standard parameter you can modify by color custom profiles in camera. The preview image we can view on the cameras back display.

[stevebrot]

There's also a small embedded preview image - usually 720 x 480px

With Pentax, there is also a full-size (though lossy-compressed) preview JPEG.


Steve

[stevebrot]

What do you mean by this exactly? Because my DNG's OOC from my Pentax cameras have all the image data. I must be severely misunderstanding something here...

The raw image data isn't RGB image data but captured output voltages of the sensor cells

This ^ ^ ^ is the basic explanation. Neither a DNG nor a PEF file from the camera contains a readable image per se. By image data, I mean brightness values for the three color channels organized into pixels. It is a little hard to explain, but pixels are a logical construct and can be made any number of ways. Camera sensors provide data (voltages) that may be assembled into pixels and pixels may be used to drive a display, but technically neither hardware has "pixels". RAW capture data are simply voltages from the sensor's detector sites written to file with minimal processing by the camera. Interpreting those voltages into a visually pleasing viewable collection of pixels is the job of the RAW converter, either in-camera or by a computer.


Steve

[acoufap]

With Pentax, there is also a full-size (though lossy-compressed) preview JPEG.


Steve

Thanks for this info - I didn't know that!

Do you know how to extract it using Exiftool? - I only know how to extract the 720 x 480px preview.

[texandrews]

This ^ ^ ^ is the basic explanation. Neither a DNG nor a PEF file from the camera contains a readable image per se. By image data, I mean brightness values for the three color channels organized into pixels. It is a little hard to explain, but pixels are a logical construct and can be made any number of ways. Camera sensors provide data (voltages) that may be assembled into pixels and pixels may be used to drive a display, but technically neither hardware has "pixels". RAW capture data are simply voltages from the sensor's detector sites written to file with minimal processing by the camera. Interpreting those voltages into a visually pleasing viewable collection of pixels is the job of the RAW converter, either in-camera or by a computer.


Steve

Ah, of course---you were limiting yourself solely to sensor data, not camera data....

[stevebrot]

Do you know how to extract it using Exiftool? - I only know how to extract the 720 x 480px preview.

ExifTool can do this, but it is a pain.* PDCU is able to do it using menu --> tools --> Extract JPEG...


Steve

* Meaning that I figured it out at one time, but don't remember anymore how its done.

[stevebrot]

Ah, of course---you were limiting yourself solely to sensor data, not camera data....

Yep! Minimally processed sensor voltages and metadata (set ISO, process protocol, white balance multipliers, and so forth) along with several preview JPEGs are the only thing in the file.


Steve

[acoufap]

ExifTool can do this, but it is a pain.* PDCU is able to do it using menu --> tools --> Extract JPEG...


Steve

* Meaning that I figured it out at one time, but don't remember anymore how its done.

PDCU is a pain for me - on MacOS!

I use Exiftool all the time - on the command line. Have built some bash-scripts. They work like a charm.

[stevebrot]

PDCU is a pain for me - on MacOS!

I use Exiftool all the time - on the command line. Have built some bash-scripts. They work like a charm.

Well, I guess you will figure it out

The "pain" is figuring out how ExifTool references the previews (IIRC integer, zero-based) and getting the one you want. I suppose one could simply run an incrementing loop until error and see what pops out.


Steve

[photoptimist]

What do you mean by this exactly? Because my DNG's OOC from my Pentax cameras have all the image data. I must be severely misunderstanding something here...

Part of all this as a sidebar is my current work deployment on IPTC metadata for a whole lot of images to be finally ingressed into DAMS...

How cameras work: interpolation explains much of the issue of why the DNG isn't really a proper image even if if contains all the sensel data from the sensor.

One key point that the article glosses over as "tedious math with vibrant results" is the problem that the "red" sensels in the camera also have some sensitivity to green and blue light. The same is true for the "green" and "blue" sensels of the camera. The effect of this cross-talk between the colors is that the "raw" colors of a DNG are quite muted*. Getting true colors from the RAW image means trying to subtract the amount of mixed in light in each color -- e.g., subtracting the estimated amount of green and blue light picked up by the "red" sensel and doing the same for green and blue.

*One exception might be the Phase One Trichromatic that uses much more saturated color filters on the CFA to reduce the cross-talk between the colors but the camera pays for it with a 1 stop loss of light compared to a traditional formulation of the CFA.

[texandrews]

How cameras work: interpolation explains much of the issue of why the DNG isn't really a proper image even if if contains all the sensel data from the sensor.

One key point that the article glosses over as "tedious math with vibrant results" is the problem that the "red" sensels in the camera also have some sensitivity to green and blue light. The same is true for the "green" and "blue" sensels of the camera. The effect of this cross-talk between the colors is that the "raw" colors of a DNG are quite muted*. Getting true colors from the RAW image means trying to subtract the amount of mixed in light in each color -- e.g., subtracting the estimated amount of green and blue light picked up by the "red" sensel and doing the same for green and blue.

*One exception might be the Phase One Trichromatic that uses much more saturated color filters on the CFA to reduce the cross-talk between the colors but the camera pays for it with a 1 stop loss of light compared to a traditional formulation of the CFA.

Hmmmm....but from what I've read here and elsewhere is that there's no appreciable difference between the DNG and PEF's coming out of Pentax cameras....

[jbinpg]

exiftool -a extracts all preview images from a raw file.

exiftool -b -PreviewImage extracts just the preview image.

exiftool -b -ThumbnailImage extracts just the thumbnail image

[acoufap]

exiftool -a extracts all preview images from a raw file.

exiftool -b -PreviewImage extracts just the preview image.

exiftool -b -ThumbnailImage extracts just the thumbnail image

This command I use all the time and it works - since years:

exiftool -b -previewimage -w previews/%d%f_.jpg -ext DNG .


The usual command to get a full sized jpg should be the following:

exiftool -b -JpgFromRaw -w thumbsAndPreviews/%d%f_.jpg -ext DNG .

But this doesn't work.


Of course I tried your other suggestions using DNG files from K-5, KP and K-1.

They simply don't work ... I tested them as follows:

exiftool -b -ThumbnailImage -w thumbnails/%d%f_.jpg -ext DNG .

=> no image creation

exiftool -a -w thumbsAndPreviews/%d%f_.jpg -ext DNG .

=> The -a command simply dumps the existing meta data


In which way do you use these commands exactly using raw files from which Pentax DSLR?

[BruceBanner]

Lightroom and the standalone DNG converter can do this, but the DNG is just a wrapper for the rendered TIFF image data (pixels). An out-of-camera DNG lacks rendered image data and just has voltages from the sensor. (This is the difference between RAW and image files.) The bottom line is that once a RAW file is processed and saved to JPEG, TIFF, or whatever, you can't change it back. If you make a TIFF, the characteristics of that file are what you have to work with.


Steve

Hmm... ok, I wonder why the presets get greyed out then when looking at Tiff files rather than DNG or Jpgs. Yes Jpgs.... they are fine, and even with those bought for paid presets there is a preset tool for jpgs that yer supposed to click and apply if working with jpgs rather than raws (it seems to dim things down and lower exposure). I know the presets like to know the camera model and make, I will get emails from said preset vendor saying "congrats, Canon bla bla is not supported, download the update here" etc. Could it be that the issue is not Tiff per se but metadata missing that the DNG and Jpgs have... Perhaps if the Tiff file had the right metadata then the presets will work kinder with it and perhaps not even be greyed out

ExifTool can do this, but it is a pain.* PDCU is able to do it using menu>tools>Extract JPEG...


Steve

* Meaning that I figured it out at one time, but don't remember anymore how its done.

Can you not also use the camera? I have taken RAW before and then used the in camera Raw Development and just pulled the Jpg out of the RAW (highest quality), and do it for all files at once (batch).