[BruceBanner]

Ok ok... let's just get a few things straight before we go ogling at the images contained within this post. First and foremost, I have never been a Jpg shooter. Ever since I owned my first DLSR (Pentax K-50) I have shot RAW. Not because I am some purist, but because I was told (at the time, 2-3yrs ago) that RAW was better for editing and gave greater dynamic control. For a newbie shooter this was attractive as mistakes will be made as I learn this craft, RAW has more data, simple as that.
Now 2-3yrs later, as I have become more proficient (getting things more right in camera and correcting less exposure errors etc), I am shooting in some situations Jpg, usually for better buffer limits or longer high continuous sustained bursting and even possibly less work in PP (as the Jpg file I am using is somewhere more to where I want it to be than a 'dull' RAW file that needs a little more TLC).

A recent conversation on this forum, a member stated they were unconvinced that the Jpgs they were generating were in someway inferior, unable to be manipulated to the same degree as a RAW file. This prompted me to 'lay into them' about all the benefits of RAW (bla bla bla, yada yada yada). I mentioned how one time I saw this incredible shot from the K-1, a shot taken inside some old warehouse and included a large window with perfectly exposed trees, clouds and sky outside to be seen, this in addition to some wonderful well let interior items as well. It looked HDR to me, but the user indeed reported it was not, just proof of the fantastic dynamic range "of the RAW K-1 files".

I thought this would be a good example of proving to this forum member of how a Jpg cannot match a RAW file so I set out to do just that, recreate that kinda scenario.

What I discovered next came as a bit of a shock, I am trying to understand what is going on (hence this post), because my own interpretation of the file/s suggest to me that perhaps the Jpg somehow has a better recoverability of shadows and highlights than what the RAW file itself... which makes no sense at all.

Ok, so onto the image itself.

The EXIF first;

K-1+FA77
1/200
f1.8
ISO 400
EV -2.0
WB 5000k
Adobe RGB
Custom Image Natural, nothing changed, all values 0

It is important to note that all Jpg processing was off during the shot (DNG) and that no Jpg processing was turned on during the conversion in the camera of RAW > Jpg. (Yep... just to be clear the shot itself was just a single RAW DNG file, not a RAW+ where a Jpg is created at the time along with).

Now, I'd display the RAW file here, but of course it won't show, so this first image below represents the RAW DNG file imported into LR and exported straight away as a Jpg with no editing whatsoever. You can now understand the -2.0EV logic (I was intending to protect highlights in the shot, bring up exposure slightly and large boost to shadows to bring the interior detail up.

DNG file as a Jpg Export, no editing at all.

The actual RAW file can be found here; Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.


Next up I show you the Jpg of the RAW DNG file that has been converted to Jpg via the K-1 Camera. This Jpg hasn't even seen LR at all (so it's not a Jpg Import and Export Jpg from LR)

You can download this file here; Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.


Ok, so now I go back to the DNG file in LR. I make the following adjustments only;

Exposure +0.83
Contrast +43
Highlights -100
Shadows +100
Whites +29
Blacks 0

That's it, no more, no less. I put the Highlight warning on and felt that these adjustments almost started setting off a few off red blinkies (but not quite). The attempt was to very basically try and get an image that didn't ruin the content outside of the window but also brought some context to the interior (whilst pushing the limits of Highlights and Shadows range).

Here is the DNG file with those adjustments exported as a Jpg;

You can either make those same adjustments with the DNG file available above, or grab the Jpg version seen above from here; Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.


Next up. For the first time I import the Jpg that was generated from within the camera body of the K-1 into LR. I too then make the exact same adjustments, or use the 'Sync' feature. This is what it looks like;

You can download the file and have a look for yourself here; Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.


Now is it my imagination or does the Jpg look like it's managing to control highlights and shadows better than the actual RAW DNG file? Take a look at the brightest fern leaf/branch towards the edge of the window on the right. Which one looks closer to putting red blinkies on, the RAW file or the Jpg?


Let's take a closer look.

Here's a cropped version of the same DNG file above with those same adjustments on that fern branch that is brightest (exported as a jpg of course);

Download this file here if you like; Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.

And now the same crop this time the aforementioned In Camera Jpg version (with adjustments);

Download here; Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.


Now... I wasn't expecting this at all. To my eyes the Jpg is superior, it has more room to move in both in terms of Highlights and Shadows (I could increase Exposure or Whites to the overall image before reaching what the fern is looking like in the RAW file).

I'm not saying I am happy with the final image here, this was all just a test, but I really expected the opposite to be true. Actually... the very fact they are even this close is surprising to me!

So now I'm left wondering a few things;

1) Have I somehow skewed things to not being fair, somehow biased towards Jpg?

2) Have things progressed far better for Jpg manipulation?

3) Is my interpretation wrong, do you disagree with me?

4) Is the nature of my test shot somehow a bad example (-2.0 EV etc), would a more normal exposed shot be a better example of RAW's supposedly better superiority?


Right now this little experiment has me rethinking the whole RAW thing completely. There's been numerous times I have had to toggle out of RAW and into Jpg due to buffering issues or wanting a prolonged High Continuous burst that RAW just wasn't capable of. I'm not (at this point) even interested in tweaking any in camera Jpg processing, I'm happy with a dull and lifeless Jpg, just the very thought of quicker processing times, smaller file sizes, better buffer limits is actually really exciting...


Well sorry for the long post but I felt it was important.


TIA!

Bruce

[normhead]

Raw isn't necessarily superior, but it is for someone who has mastered their PP software. The first thing I notice is the difference in exposure values. In one of your DNGs (the bottom one you) actually have blown out your highlights. This is not a jpeg vs raw issue. As far as i can tell, it's an exposure issue. That's what levels and curves are for.

The jpeg conversion from modern cameras is pretty good. You have to have a bit of skill with your PP software to be better.

If you have a jpeg and DNG and you like the jpeg better. You probably should forgo raw until you have the skills to do better than jpeg. That's almost like the qualification to enter the raw club.

My advice would be work on that raw file until it's as good or better than the jpeg. If you currently like the jpeg better, start by matching the exposure values. All the information is there in the raw (and a lot more), you just need to learn to access it. Once you have the two files looking pretty much the same, then see what else you can do with the raw file, like maybe pulling up some detail in those dark areas.

Once you have an image looking the way you want, hopefully you can save what you did as preset, and apply it as the starting point for similar images. I almost never start working from scratch. You know your raws are going to be a little flat. Most of my presents are small adjustments to constrast, saturation nd definition before I even start to work on the image. I have 10 presets saved for different types of image, with between 5 to 10 adjustments as starting point. Without that kind of work flow, you're going to find raw very tedious.

Sometimes what you can do with raw is about the same as jpeg, but not with the images you posted. You can make those dark areas lighter and more realistic with raw. Working on your jpeg files to do the same will probably produce a real mess.

[lsimpkins]

I put the Highlight warning on and felt that these adjustments almost started setting off a few off red blinkies (but not quite).

No comment on your overall observations and conclusion, as I am on my laptop.

Just a tip to make Highlight and White adjustments easier and more accurate than looking for blinkies. When adjusting the Whites hold down the Alt key (Windows). This will make the screen black except for those areas that are blown out. If totally black, just increase the Whites until you see some spots of color or white appear. Then slowly back off until they are gone. I find this easier and faster to do than to describe.

[thazooo]

There's nothing wrong with shooting .jpg .The processing power of current cameras has advanced greatly since the idea of RAW processing was put out there (ist D time frame). I think that getting it right in camera, like you mentioned, has a lot to do with it. I rarely see weird artifacts any more in jpg OOC and Pixel Editors handle jpg images as nicely as RAW. IIRC the myth of degrading jpg images after several saves has pretty much been Debunked.
My work flow that has developed over the years is RAW and process to .tiff, save. I use Pixel editor for other adjustments. What I've noticed is less and less adjustments to the RAW file. I could probably switch over to jpg any time and not notice a difference.
I shoot a m4/3 system also, it's always been jpg and I wouldn't know how to develop RAW to match the jpg output. It's that good.
Good luck playing with it.

[08amczb]

tl;dr
I think you like the edit made by the camera + your own edit. The tone curve and adjustments made by the camera added to your edit. If you would like to replicate it in lightroom use the brush tool, select everything and make more adjustments to get over the +/-100 limit.

(When the adjustments of the camera is in the opposite direction of your taste it can really harm the image.)

[pschlute]

Difficult to make a comparison as the jpeg produced in-camera from the raw file is not being processed to the same parameters as the jpeg produced in LR from the raw file. Even if you produced both jpegs using say Pentax "Bright" profile, there may still be differences in exposure for example.

ps why are you processing in AdobeRGB for an image that you intend to show on the web ?

[DeadJohn]

The JPG looks like it was reduced in contrast; that's probably just how the JPG engine works. The reduced contrast brought down the highlights giving the impression that there's more headroom at the bright end of the histogram.

For my photos (a lot of low light and astro), I always use raw DNG for these reasons:

1. Lightening very dark shadows. JPG becomes noisy and loses details sooner than DNG.
2. Adjusting color casts. Auto WB doesn't always get things right and WB adjustments are more flexible with DNG. Setting manual WB isn't a workaround because local light conditions can dictate different WB settings and it's tough to tell until sitting at a full size monitor.
3. Picking up subtle color variations. JPG only offers 8-bits (256 levels) of each color channel. DNG offers greater bit depth, so less chance of stair-step color changes in a twilight sky.

[stevebrot]

If a JPEG is better than the RAW version, it is because either the RAW processor being used cannot reproduce the same look as the in-camera processing or the skill of the person doing the RAW processing is not up to the task.

Below is an in-camera JPEG of some wonderful Fall foliage at the Japanese Garden near my home. Try as I might, I could not reproduce the look in Lightroom with the version I had or the skills I had at the time (October 2009).


Pentax K10D, Pentax-FA 35/2

I took a Kodachrome slide* at the same time and working with the scan was similarly difficult despite the original being quite vibrant.


Pentax KX, Pentax-K 55/1.8, Kodachrome 64

...and finally, a somewhat less vivid version of a digital capture processed in LR. I am tempted to revisit this image with LR 6 to see if I can do a better job.


Pentax K10D, Pentax-FA 35/2


Steve

* I was one of the lucky few who managed to secure several rolls from the last production run of Kodachrome. Those were then processed in the last big batch through the machine at Dwaynes. What a privilege, eh?

[TonyW]

There is no reason why you should not get a really great image out of shooting JPEG but your original premise of raw being superior is absolutely correct, at least technically. Image content will play an important role here but you will loose headroom by shooting JPEG over raw - it may or may not matter.

JPEG are 8 bit which limits camera captured colour down an 8 bit pipeline
JPEG SOOC you will have no control over what colour information is discarded
JPEG will compress image data
JPEG contain less dynamic range compared to raw - thereby limiting exposure recovery
Anything applied in camera settings e.g. sharpening, noise reduction will be impossible to recover back to its raw state

Had a quick look at the DNG and cannot get anywhere near clipping in ACR with your settings. My suspicion is that what you are seeing is just down to the way you have processed your raw data and there are probably at least a dozen different ways that you could play with to get a result as good as any OOC JPEG

An example of a gross edit of your JPEG image with the same edits applied to the raw version. Look at the green artifacting showing on lamp, window frame, around the tree/plant on the right vs the clean raw below

A JPEG is a raw file that the camera has converted. Can you convert a RAW file better than the camera (which has been programmed by experts) can?

After many laborious experiments in PS, I find that I usually can't, exceptions being very high dynamic range and certain white balance situations. I certainly can't get any better noise reduction. I save any sharpening for the very last step in post-processing (I have minimum sharpening set on my camera JPEGS).


I have looked at numerous "RAW is better articles" and the examples they use are usually poorly exposed in the first place. Most of them showed little real-world improvement. In-camera processing has made great strides, a trend which is accelerating with cell phones.


As always, Your Mileage May Vary.

[dsmithhfx]

which has been programmed by experts

...to look like what they think joe average camera buyer wants their OOC pictures to look like. If that's how you want your pictures to look too, you're all set. Otherwise...

[house]

Raw vs jpg is perhaps best compared to shooting full auto vs manual. (or any mode which gives you creative control). The jpg engine just can't know exactly what you are after and it does limit your flexibility. A 8-bit jpeg just doesn't contain all the information of a 14-bit raw. Weather those extra bits are useful to you is another issue and dependent on how you chose to expose your shot.

Generally speaking you can get a lot more detail out of a raw file. Similarly noise reduction can be much more cpu intensive and generally better when using a competent raw developer. I've never heard of the opposite being true and compared to the Pentax engine you can get a lot more detail if you use rawtherapee.

A lot of work by skilled people have gone into the jpeg engines however. It's not easy to outdo them and for many shots you'll struggle to achieve better overall look working from raw.

I know from other people that LR defaults include a lot of tweaks. So you are sort of half way back to shooting auto again. That LR raw has lots of embedded assumptions about what you want to do to the file. This right out of the bat. You can probably disable some curve to avoid crushing the blacks and overexposing the highlights and work from there.

OSo now I'm left wondering a few things;1) Have I somehow skewed things to not being fair, somehow biased towards Jpg?2) Have things progressed far better for Jpg manipulation?3) Is my interpretation wrong, do you disagree with me?4) Is the nature of my test shot somehow a bad example (-2.0 EV etc), would a more normal exposed shot be a better example of RAW's supposedly better superiority?

1. No you set up sounds good to me
2. No, the limits on the jpeg are the same as they ever was. Jpeg development hasn't moved faster than raw development (they aren't different generally just less and more baked data in jpeg. The whole demosaic work is also already done)
3. Yes. You can't tweak a jpeg to the same extent of a raw there's simply more data in the raw. The jpeg engine may however create a better base than you are able to due to the skill difference between you and the developers.
4. No the example is fine. It's just one case but still useful example.

Basically the Pentax in camera jpeg engine bakes the jpeg with a basecurve (or similar) that is andvantageous compared to the LR basecurve in these specific circumstances. I'd be very surprised if you can't get better results that that using LR. Might be a bit more involved that's all.

---------- Post added 11-16-18 at 12:24 PM ----------

the myth of degrading jpg images after several saves has pretty much been Debunked.

It's not a myth and it's not debunked. Perhaps jpeg code has become better but it's still lossy (it compresses the file by degrading quality) and every save recalculates the compression based on available data. After each save the available data is further from the original.

[stevebrot]

Can you convert a RAW file better than the camera (which has been programmed by experts) can?

Define "better". I do know that I generally do a better job of realizing my intent for the capture than the expert programming.

The in-camera processing on my K-3 is very good, but also very limited. The resultant JPEGs have no ability to adjust errors of white-balance correction and significantly limited capacity to absorb changes to contrast or color saturation and hue without artifact. I shoot RAW and could use the JPEG preview*, but it almost never reflects my intent for the image. I suppose I could use PDCU with the in-camera settings as a start point, but why bother.

That being said, sometimes I will refer back to the in-camera version as an option, but never with the intent to work from the JPEG. Instead, I will use the camera or PDCU to process the RAW to TIFF with subsequent PP in LR and/or Affinity Photo.


Steve

* Both DNG and PEF from Ricoh/Pentax cameras include the full-resolution JPEG processed to camera settings at time of exposure. This image is easily extracted, though it is arguably easier and better to simply use PDCU to process the RAW using the camera settings embedded in the file metadata and output to 16-bit TIFF.

[stevebrot]

...to look like what they think joe average camera buyer wants their OOC pictures to look like. If that's how you want your pictures to look too, you're all set. Otherwise...

I think you mean joe-average Ricoh expert programmer.


Steve

[stevebrot]

I know from other people that LR defaults include a lot of tweaks.

Definitely, though I don't know that I would call them tweaks. While the actual data (voltages) from the sensor may be used as a start point for RAW processing, no regular RAW processing tool does so. A TIFF from a so-called linear RAW conversion is almost opaque to further processing using a tool such as Lightroom or Photoshop. (Yes, I have tried it.)

You can probably disable some curve to avoid crushing the blacks and overexposing the highlights and work from there.

The easiest way to do that is with a custom camera profile using the DNG Profile Editor.


Steve