[Mark DeB]

Hi, another beginner's question for you. Today I was comparing the embedded JPG in a PEF file with the one generated by DCU 5 where you just open the file in Laboratory and save, making no changes to the parameters. I would have thought that the images might be close to identical; previous discussion on the forums suggests that they would be (I guess because it's the manufacturer's software either way). But the difference is really not subtle. My question is, should I be surprised? I can post images but if the answer is basically that everybody sees this already, then never mind. This is with my *ist DS2. (I also see a difference between the embedded JPG and the output of PPL.) I got the embedded JPG using exiftool, and it appears identical to the one extracted by irfanview (when I open the PEF in irfanview). I am wondering, is there something I am doing wrong, or some control I have overlooked that is making the difference?

Thanks,

Mark

[kypfer]

My experience with these sort of symptoms is that DCU "over-reacts" on occasion, applying excessive noise reduction from the start, thereby causing blurring/smearing and/or bad colours in some areas.
This doesn't just happen with "older" cameras, like the *istD models, but at least one instance when I was "pushing the boundaries" with my K-5 and an infra-red filter.
I will emphasise at this point that this is not "usual" behaviour, but sometimes seen with, if not "poorly" exposed originals, certainly images that might be considered to need at least some noise reduction … high ISO, low light levels etc.
So, first check the Noise Reduction tab in DCU, disable anything that is set (making notes if necessary), see what the differences are and then re-apply the various sliders to suit your requirements.
This behaviour may or may not be "by design", I certainly consider it to be less than optimal, but it happens so rarely and it's so easy to remedy I simply live with it
Good luck

[Kobayashi.K]

Many camera's have a quality range of 1..3 stars (among them K3-III), while DCU has a range of 1..4 stars when the JPEG is exported. This could mean two things: DCU produces higher quality JPEG's, or the maximum quality (minimum compression) is the same but the total range is subdivided differently.

So I think, to avoid uncertainties, the best test is to compare the embedded JPEG's with the JPEG's saved in-camera with RAW+, for all the possible qualities.
Doing this in a quick test shows that the file size of the embedded JPEG is virtually equal to the one-star (*) JPEG saved in-camera (K3-III).

Besides file size (compression) there might be other settings in the embedded JPEG that are changed, which can also be checked in DCU, and there may be differences between PEF and DNG.

In my test the RAW's were saved as PEF and the JPEG's as Custom image 'Flat'. The embedded JPEG's were saved from DCU by selecting the Browser tab, selecting the RAW, and use menu 'Extract JPEG' with the right mouse button.

[acoufap]

Different camera models not only mostly use different sensors but also different algorithms to develop the jpgs based on a specified color mode (natural, bright...).

I don't think that DCU uses camera specific developing algorithms, i.e. for every camera model / color mode the algorithms of these different camera models.

So to me it seams pretty clear that there will be recognisable differences between the camera generated jpgs and the ones generated by DCU.

When a raw converter changes the behaviour of raw development tools with newer versions, then you will often see differences from the new image rendering compared to the rendering of the older version - even if you don't change any parameters.

[Kobayashi.K]

I might be misinterpreting your answer, and I'm not informed about the details of this process, but it is plausible (and a matter of good programming) that the algorithms (the classes in the camera firmware and in DCU) which are generating the JPEG's are always the same. I suppose what differs is the preprocessing in each camera, but the output is always the same standardized format which can be fed to the various classes for the Custom Images.

I would be a heck of a job if Pentax needs to change all the Custom Image classes each time they bring out a new camera, not to mention the problems you get with adapting DCU. So surely this process is split into a preprocessing part, which is built into the camera, and a standardized set of classes for generating a JPEG output. These classes are used in all cameras alike and also in DCU.

Of course there are a lot of parameters that differ between camera's and lenses but I suppose this is also a standardized set which is stored additionally in the RAW to make the format portable, and adjustable in DCU via sliders and checkboxes.

The objective is that the various Custom Images all have the same look, no matter from which camera they come. The different characteristics between the Custom Images are sometimes already very subtle, so I think for the average user any variations there might be between cameras is only a negligible fraction of it.

[acoufap]

The objective is that the various Custom Images all have the same look, no matter from which camera they come.

Of course. - Congratulations to Pentax if they never need to optimize any of their rendering functions. PDCU is at least 12 years old … even classical music tends to be interpreted differently over time.

Somewhere I read that their camera jpg engine got a lot better with the latest camera models. If so - what does this mean for PDCU? No need to answer this question.

[pschlute]

Embedded jpeg is produced by the camera (from raw). DCU jpeg (from raw) is produced by different software.

I fail to see the point in trying to compare one with the other. if you use raw files you should process them to give you the output you want.

[Mark DeB]

My experience with these sort of symptoms is that DCU "over-reacts" on occasion, applying excessive noise reduction from the start, thereby causing blurring/smearing and/or bad colours in some areas.
This doesn't just happen with "older" cameras, like the *istD models, but at least one instance when I was "pushing the boundaries" with my K-5 and an infra-red filter.
I will emphasise at this point that this is not "usual" behaviour, but sometimes seen with, if not "poorly" exposed originals, certainly images that might be considered to need at least some noise reduction … high ISO, low light levels etc.
So, first check the Noise Reduction tab in DCU, disable anything that is set (making notes if necessary), see what the differences are and then re-apply the various sliders to suit your requirements.
This behaviour may or may not be "by design", I certainly consider it to be less than optimal, but it happens so rarely and it's so easy to remedy I simply live with it
Good luck

OK thank you! In the example that follows I turned off the Spurious color signal reduction; none of the boxes are checked for Noise Reduction. (Highlight Adjustment is applied, compensation 50.)

Many camera's have a quality range of 1..3 stars (among them K3-III), while DCU has a range of 1..4 stars when the JPEG is exported. This could mean two things: DCU produces higher quality JPEG's, or the maximum quality (minimum compression) is the same but the total range is subdivided differently.

So I think, to avoid uncertainties, the best test is to compare the embedded JPEG's with the JPEG's saved in-camera with RAW+, for all the possible qualities.
Doing this in a quick test shows that the file size of the embedded JPEG is virtually equal to the one-star (*) JPEG saved in-camera (K3-III).

Besides file size (compression) there might be other settings in the embedded JPEG that are changed, which can also be checked in DCU, and there may be differences between PEF and DNG.

In my test the RAW's were saved as PEF and the JPEG's as Custom image 'Flat'. The embedded JPEG's were saved from DCU by selecting the Browser tab, selecting the RAW, and use menu 'Extract JPEG' with the right mouse button.

Good point about the JPEG quality, but I don't think that's it. I took a JPG with *** quality immediately after (no RAW+ on my camera, as far as I know) and I don't see a difference. I could post it if you want. I think that whatever I am seeing makes a bigger difference than the JPEG quality difference. I didn't know you can get the embedded JPG from DCU, thanks for that!

Different camera models not only mostly use different sensors but also different algorithms to develop the jpgs based on a specified color mode (natural, bright...).

I don't think that DCU uses camera specific developing algorithms, i.e. for every camera model / color mode the algorithms of these different camera models.

So to me it seams pretty clear that there will be recognisable differences between the camera generated jpgs and the ones generated by DCU.

When a raw converter changes the behaviour of raw development tools with newer versions, then you will often see differences from the new image rendering compared to the rendering of the older version - even if you don't change any parameters.

Ah, thanks!

So here, from a new shot I took today, are the output from DCU and the embedded JPG. (In that order, I think; the DCU is 327.1 KB and extracted is 342.5 KB. Sorry, not sure how to label them. Both attachments are apparently reduced in file size from what they are on my computer.) In the camera, Saturation, Contrast, and Sharpness are all set to 0. When I load the file into Laboratory, DCU tells me it is using in-camera settings (I think; the Camera setting icon is greyed out) and also shows Contrast and Sharpness one tick to the right of center (not exactly sure why they aren't 0, but OK). The other sliders on the Custom Image page are in the center. The Image Tone was set to Bright in the camera, and it looks to me like the setting in DCU for Image Finishing is Bright. White Balance set to camera setting. Everything in Exposure/Tone is 0 or off.

I am not sure whether it is possible to see the relevant difference between the pictures here, but I can definitely see it when I go back and forth between them in Irfanview on my computer. I am wondering, what are they (the camera and DCU) doing differently, what correction?

Thank you!

---------- Post added 01-23-23 at 04:29 PM ----------

Embedded jpeg is produced by the camera (from raw). DCU jpeg (from raw) is produced by different software.

I fail to see the point in trying to compare one with the other. if you use raw files you should process them to give you the output you want.

Yes indeed, good point; what prompted the question is this. I was reading, in the forum, discussion about the relative merits of sticking with the jpegs that come straight out of the camera vs. working with RAW files. Some say that Pentax does a pretty good job with JPG. So, is there a way to have the camera produce the RAW file, but also to know what it would have produced had it been set on JPG (when there is no RAW+ on the camera)? Some posts on the forum, going back a few years, suggest that one can either look at the embedded JPG or the DCU output when no parameters are changed, and that they would give the same result. But I don't see how they have the same result. So, if I want to know what the JPG would have looked like (while still choosing RAW), would it be the embedded JPG? Is the embedded JPG identical, in all respects, to what the camera would have produced on the JPG (highest quality) setting? Thanks.

[Kobayashi.K]

.... I didn't know you can get the embedded JPG from DCU, thanks for that!

So how can you talk about the embedded image all the time. When you open a RAW in DCU the photo you see is NOT the embedded JPEG. It is the Custom Image reproduced with the Parameters list from the RAW. If you directly save this photo with 'File | Save As with Image processing' you get exactly the same image as if it was saved from the camera with RAW+.

[pschlute]

So, is there a way to have the camera produce the RAW file, but also to know what it would have produced had it been set on JPG

Sorry, but I fail to see the relevance of this. Either work with the raw file, or work with the jpeg.

[Kobayashi.K]

Sorry, but I fail to see the relevance of this. Either work with the raw file, or work with the jpeg.

There are scenario's when you are only interested in JPEG and still save as RAW only. For example if you want to have both a color photo and a monochrome photo without wanting to convert the color photo into monochrome in post (for example by de-saturation). This is obviously not an option the camera allows. So the strategy is to save only RAW, open it in DCU, and generate a 'Flat' and a 'Monochrome' Custom JPEG image from it. This guarantees the same quality as if the JPEG's were saved in-camera.

[Mark DeB]

So how can you talk about the embedded image all the time. When you open a RAW in DCU the photo you see is NOT the embedded JPEG. It is the Custom Image reproduced with the Parameters list from the RAW. If you directly save this photo with 'File | Save As with Image processing' you get exactly the same image as if it was saved from the camera with RAW+.

OK, in the meantime I went back to an older discussion and found this written by stevebrot in 2019:

If one desires a high-quality JPEG from DNG/PEF that retains all of the camera's custom image, lens correction, HDR, and special effect features without doing the processing in-camera, the easiest way is to use PDCU with the default settings direct from the "Browser" view. Simply select one or a range of files and "Save as..." or "Process Multiple Files..." to JPEG. The results will be image-identical to making a file using either JPEG or RAW+ as the capture type.

So, if I understand correctly, he is saying that, if you take a picture in RAW, and want to get a JPG that would have resulted if you had selected JPG on the camera, you can do that by saving from the Browser as he specifies.

And, if I understand correctly, what you say implies that, in order to get a JPG that would have resulted if you had selected JPG on the camera, you can do it by opening the RAW file in Laboratory and saving with image processing.

Am I tracking so far?

So a consequence of both of these is that the two operations will give the same results, correct? It's just that what I am seeing so far is telling me that what I think you are saying will give the same result as JPG SOOC does not give the same result. I took a picture of a scene using JPG at the highest setting. Then, a few seconds later, I took a picture with the RAW setting (same exposure). I opened the PEF file in DCU. I changed nothing and saved it with image processing. It is darker than the JPG SOOC and there are lots of other clearly visible differences. Maybe I am misunderstanding! Or maybe I'm doing something wrong. Thank you for your patience.

Mark

[sbh]

In my experience DCU die the same processing s as the camera.

Shooting RAW+ will give you the exact same jpeg result in camera as processing the raw in DCU with the same settings (for example Bright or Flat).

[Kobayashi.K]

....So a consequence of both of these is that the two operations will give the same results, correct? It's just that what I am seeing so far is telling me that what I think you are saying will give the same result as JPG SOOC does not give the same result. I took a picture of a scene using JPG at the highest setting. Then, a few seconds later, I took a picture with the RAW setting (same exposure). I opened the PEF file in DCU. I changed nothing and saved it with image processing. It is darker than the JPG SOOC and there are lots of other clearly visible differences. Maybe I am misunderstanding! Or maybe I'm doing something wrong. Thank you for your patience. Mark

The difference between 'Save As' and 'Save As with image processing' is that the function of the latter is to convert the RAW to JPEG (or TIFF) including changes you have made on the Laboratory tab. But when no changes are made I assumed it was identical to 'Save As'.
I did a little test myself with a PEF from my K3-III to get to the bottom of it and indeed I also see a slight difference in brightness and other differences in texture when pixel peeping at 150%. However, if you take a look at the histograms, the shapes are identical except that the processed one has slightly lower values (in my case a mean value 139 instead of 140). On the other hand, the file saved with 'Save as' has no visible differences compared to a JPEG saved in-camera with RAW+, at 150%. So, after this I can imagine you say they are not exactly the same. Just pick your option.

Notice that in both cases the default compression factor is always one star (*), which you can change, and there is a discrepancy when the camera has a 3-star range (*ist DS2) instead of the 4-star range in DCU. A test I did some time ago revealed that if the ranges are different you can never get back the original compression (check the file sizes).

In practice though, in most cases you might have shot a batch of RAW files you want to convert to JPEG. So, after opening the directory with the RAW's you select more of them for conversion (with the Shift or Ctrl key), but then the only menu item which is available is 'File | Process multiple files'. The 'Save As' and 'Save As with image processing' menu items are dimmed. From the 'process' word you can derive that changes on the Laboratory tab are also saved. So, theoretically in that case your quest to save to a really identical JPEG is thrown out of the window. However, as stated above, one can assume when you have made no changes on the Laboratory tab the JPEG's are very close except for the compression factor, and they are definitively representative for the selected Custom Image style.

[Kobayashi.K]

In my experience DCU die the same processing s as the camera.
Shooting RAW+ will give you the exact same jpeg result in camera as processing the raw in DCU with the same settings (for example Bright or Flat).

I fully agree, but obviously it is always possible to find differences, depending how critical you are. Imagine for example the Pentax team is using a different language for the firmware than the Silkypix team that produces the code for DCU. Then it is very difficult to exactly reproduce the results from the camera in DCU. But for everyday use, and with the limitations the JPEG already has, I think they did a good job.