[normhead]

Dear norm, please reread the initial post properly and do not skip. I outline the processes involved. What you are seeing is a DNG file with the SAME exposure settings applied as the Jpg file that was generated from within the Camera body of the K-1 (with absolutely all Jpg processing options turned off). They both have the same Tone Curves, yet the DNG does appear to be nearing blown out highlights. That's the entire purpose of the thread, I thought if anything it would be the other way around, odd indeed. I am not applying a different Exposure or Highlight level from one image to the next.

Dear Bruce
I don't care what the settings are, look at the images, they aren't the same. The jpeg is darker.
But no problem, ignore list.
I will never trouble you with my analysis again.
"Don't throw your parts before swine for they will trample them underfoot."

---------- Post added 11-17-18 at 08:44 AM ----------

I have been using this feature for quite some time, as I have found that I rarely need the raw data file and I rarely shoot in a burst mode. With the camera set to JPEG only, I capture the image for the scene, review it and its histogram on the rear screen, then decide whether to save the raw data as well.

Philip

That's a pretty cool tip.

[microlight]

@Philip - absolutely right. I didn’t include this as I’d already written enough, and it didn’t immediately spring to mind anyway as I shoot exclusively raw, so don’t routinely see it.

[TonyW]

There has been some mention of close to blowing/clipping but as far as I can see there is no evidence to support this in the DNG file which is the important file as it is the parent of the JPEG and contains all the camera captured data not just a small subset which is JPEG.

There are no clipped highlights in the DNG in fact no where near to blowing. It is even possible that you would not have clipped highlight detail if you had not applied minus 2EV exposure compensation in camera (at least for the raw - JPEG would probably have clipped)

What you have been looking at is just a default rendering from Adobe Camera Raw (that includes LR as ACR engine is identical) of a DNG image That image rendering includes hidden adjustments including 0.50EV baselineExposure adjustment for your camera at ISO 400.

The difference can be seen in the attached rendering comparison between Adobe Standard and one with the hidden adjustments removed.

Your dull JPEG is no where near being a raw file equivalent it is just a small section of the captured raw data and while you can push and pull that data there are limits before the data breaks - those limits much greater in the full data set in raw.

[awscreo]

If a JPEG is better than the RAW version, it is because either the RAW processor being used cannot reproduce the same look as the in-camera processing or the skill of the person doing the RAW processing is not up to the task.

Below is an in-camera JPEG of some wonderful Fall foliage at the Japanese Garden near my home. Try as I might, I could not reproduce the look in Lightroom with the version I had or the skills I had at the time (October 2009).


Pentax K10D, Pentax-FA 35/2

I took a Kodachrome slide* at the same time and working with the scan was similarly difficult despite the original being quite vibrant.


Pentax KX, Pentax-K 55/1.8, Kodachrome 64

...and finally, a somewhat less vivid version of a digital capture processed in LR. I am tempted to revisit this image with LR 6 to see if I can do a better job.


Pentax K10D, Pentax-FA 35/2


Steve

* I was one of the lucky few who managed to secure several rolls from the last production run of Kodachrome. Those were then processed in the last big batch through the machine at Dwaynes. What a privilege, eh?

Adobe is notorious for bad management of a red channel. I think image would look much closer to the jpeg with RawTherapee.

[TonyW]

Adobe is notorious for bad management of a red channel.....

Interesting comment, what evidence do you have to support this view?

[awscreo]

Interesting comment, what evidence do you have to support this view?

I'll find the article/test I read few years back (it should be saved on my computer at home), it was mainly talking about the then current version poorly converting Olympus raws, they provided very comprehensive analysis on that. I believe the author suggested that the color conversion of the less popular camera brands was quite poor. I notice blown out reds and blues a lot with lightroom, even with unprocessed raws. Rawtherapee doesn't exhibit the issue, or does to a much smaller extent.

[TonyW]

I'll find the article/test I read few years back (it should be saved on my computer at home), it was mainly talking about the then current version poorly converting Olympus raws, they provided very comprehensive analysis on that. I believe the author suggested that the color conversion of the less popular camera brands was quite poor. I notice blown out reds and blues a lot with lightroom, even with unprocessed raws. Rawtherapee doesn't exhibit the issue, or does to a much smaller extent.

Yes, I have heard complaints from Fuji X Trans users, seem to recall relating to ‘worms’? If I understood and recall correctly this issue could be the result of pushing detail slider too high (deconvolution) in combination with poor choice of radius (I can see worms too if I push to hard with other cameras). Cannot say I have noticed blown channels using Pentax or Nikon kit other than user error on my part.

[GUB]

There has been some mention of close to blowing/clipping but as far as I can see there is no evidence to support this in the DNG file

Dosen't your raw developer show this small green blip on the histogram if you darken half a stop?.(Darktable here)
Bruces first exported jpg from dng would have definitely clipped this and who knows what the slider combo did. That blip will be precisely the highlights on that frond.

[awscreo]

Yes, I have heard complaints from Fuji X Trans users, seem to recall relating to ‘worms’? If I understood and recall correctly this issue could be the result of pushing detail slider too high (deconvolution) in combination with poor choice of radius (I can see worms too if I push to hard with other cameras). Cannot say I have noticed blown channels using Pentax or Nikon kit other than user error on my part.

I think I can show some samples when I get home. I have few images (like flowers) that show red clipping with all other colors and overall exposure being correct.

Also, I definitely think color I get from rawrherapee (and capture one when I used it for a week or so) is better than lightroom. However, I'm way too used to the work flow at this point, and try to compensate in photoshop after initial work in lr.

[TonyW]

Dosen't your raw developer show this small green blip on the histogram if you darken half a stop?.(Darktable here)
Bruces first exported jpg from dng would have definitely clipped this and who knows what the slider combo did. That blip will be precisely the highlights on that frond.

No green blip until you start to push the exposure to +1.8 EV on the raw image. There is no blip either on the JPEG either Bruce's own or my export of the raw from JPEG. Any blips that occur are seemingly user generated and with a little highlight compression eliminated.

Attached shows raw image SOOC in RawTherapee, Default, Auto button (approx +1.85 Exposure-you can just see the blip forming) and with added highlight compression. Second row LR default (without any compensation for Adobe rendering of raw data), Auto button (+1.8 Exposure) with no attempt to pull back highlight data. Finally a RawDigger histo.

[jbinpg]

What you are seeing is a result of different default base curve implementations. Turn off the base curve and instead go to a linear input profile and you can then more accurately judge the recorded data via the histogram. Your image will be washed out but it is the "purest" base image with which to work.

[TonyW]

What you are seeing is a result of different default base curve implementations. Turn off the base curve and instead go to a linear input profile and you can then more accurately judge the recorded data via the histogram. Your image will be washed out but it is the "purest" base image with which to work.

Already briefly touched upon in post #33, but at least in Adobe ACR engine a linear curve alone will not fix it, you will need to get back to Process 2010, set custom curve to linear and subtract baseline exposure. You may find that you end up having made an exposure of up to -2EV less than optimal.

Are you talking about other than Adobe converter as it is not clear how you could turn off the base curve?

[jbinpg]

Sorry, should have mentioned that I have zero Adobe editor experience. I use darktable on linux and it is trivial to disable the base curve and change input profiles there. I note the posted rawdigger histograms show no signs of clipping.

I have a couple of observations about the JPEG's being produced - first, JPEG isn't an eight-bit standard; it's a twenty-four bit standard, eight bits per color channel, red, green, and blue. Secondly, it isn't necessarily a compressed image - there's no loss of data if you select the largest file size of JPEG the camera can store, because that's the "100%" standard (uncompressed). The K-1 records the same amount of data for both the TIFF version (whether stored as DNG or PEF), as it does for the uncompressed JPEG: L(36M:7360x4912); RAW: (36M:7360x4912) (Specifications | PENTAX K-1 | RICOH IMAGING). I have no reason to believe that it isn't the SAME data. Now, saving the largest uncompressed file size of either type will obviously still be more time consuming than storing compressed JPEGs, since the file sizes are much smaller in the latter, even given the extra processing time required to convert the sensor's data using a compression algorithm. However, it takes half as much time to store one or the other as it does to store both. I don't know why people keep talking about compressed JPEG data, or comparing that to RAW data, which, to my mind, are not relevant to this discussion at all. Comparing apples to oranges.

[BruceBanner]

Ok, I thank everyone for their input on this. A lot of this information is going over my head, but I just wanted to clear up a couple of things;

1) The photo itself. I am not trying to get help for editing this image, I am simply using this image as a demonstration of RAW vs JPG. Initially I was intending to show how superior RAW is to Jpg for 'editing breathing room', however I am currently finding the opposite is true.

2) The sliders and values I changed are purposeful, not for the intent of creating a flattering image but rather to explore which file 'cracks' first (in terms of reaching blown highlights/shadows).

3) -2EV was used for the shot as I felt this might prove how much better RAW can be vs Jpg in terms of clawing back shadows.

Apparently normhead didn't take too well to my 'dear norm' comment, for this I apologise. I was trying to be funny, not patronising, my bad. He's not wrong, the Jpg and RAW image when imported into LR do have different exposures before I have adjusted any values whatsoever, I just wanted to be clear that there was no exposure tweaking on my behalf, not anywhere along the process of extracting the Jpg out of the RAW file (from within the K-1).

I'm now pondering if it is this 'uneven' exposure balance between the two images that leads to the RAW struggling more in the below situation;

RAW


JPG


Same adjustments as before except I bumped the Whites from +28 to +60. As you can see the RAW is blown, the Jpg not (or hardly so). To someone trying to convince them to swap from RAW to Jpg this doesn't look very convincing argument does it?

My sliders may be wild and not atypical of an edit, but my understanding of what RAW files can do was of such that you can have the sliders anywhere you like (vs it's Jpg counterpart) and it will be 'struggling' less, coping better with highlights, shadows, whites etc etc.

So, if I'm hearing everyone correctly then what's going on is one of two things (or a combination of the two);

1) The RAW and Jpg are not even starting off on even footing, LR is interpreting the two files differently (upon import) and as such making global adjustments to both is unfair (depending upon the adjustments and which direction the adjustments are favouring the other). I should try RawTherapee (it's the only other RAW editor I have installed).

2) The RAW file may be 'encountering problems sooner' than the Jpg version as the extra data things it has over the Jpg are the things that are triggering the issues i.e. the additional bits of information that the RAW file has over the Jpg are the things we're seeing causing the problem. The Jpg can't trigger the same issue as it simply lacks the data in the file that could get 'triggered'. In some situations it may be beneficial to have those additional bits of information, just not in this case (if the sliders were actually the look the photographer was aiming for). This time the image is better suited to its lossy Jpg counterpart, where it's missing information is actually of help, not hindrance. Push the sliders differently and the opposite could be true.

Is this right?