...to look like what they think joe average camera buyer wants their OOC pictures to look like. If that's how you want your pictures to look too, you're all set. Otherwise...

My Pentaxes offer me a wide variety of ways to set up a JPEG, I can have my OOC images any way I like.

---------- Post added 11-16-18 at 15:01 ----------

Define "better".

A result that I find most suitable for my purposes.

---

Last edited by Unregistered User; 11-16-2018 at 02:03 PM. Reason: syntax

[dsmithhfx]

I can have my OOC images any way I like

You're all set!

[DeadJohn]

Can you convert a RAW file better than the camera (which has been programmed by experts) can?

Yes, I can. Not because I'm great, but because the JPG software developers were constrained to working with limited information. They didn't know what I was photographing. In-camera JPG is similar to loading a DNG in Lightroom and clicking the "auto" button. Either technique gives a reasonably exposed photo but it's not always what the photographer intended.

JPG works fine for many photographers. It would probably work fine for 90% of my photos. That other 10%, though, are some of my favorite photos and need processing because of tricky lighting.

[LensBeginner]

The RAW detail is too saturated. That happens to me as well with the greens and ACR... don't know who's the "culprit", nor I care.
Go into the HSL panel, Saturation tab, Green -20 (roughly, don't have time to check now). Done. ;-)

[BruceBanner]

Raw isn't necessarily superior, but it is for someone who has mastered their PP software. The first thing I notice is the difference in exposure values. In one of your DNGs (the bottom one you) actually have blown out your highlights. This is not a jpeg vs raw issue. As far as i can tell, it's an exposure issue. That's what levels and curves are for.

The jpeg conversion from modern cameras is pretty good. You have to have a bit of skill with your PP software to be better.

If you have a jpeg and DNG and you like the jpeg better. You probably should forgo raw until you have the skills to do better than jpeg. That's almost like the qualification to enter the raw club.

My advice would be work on that raw file until it's as good or better than the jpeg. If you currently like the jpeg better, start by matching the exposure values. All the information is there in the raw (and a lot more), you just need to learn to access it. Once you have the two files looking pretty much the same, then see what else you can do with the raw file, like maybe pulling up some detail in those dark areas.

Once you have an image looking the way you want, hopefully you can save what you did as preset, and apply it as the starting point for similar images. I almost never start working from scratch. You know your raws are going to be a little flat. Most of my presents are small adjustments to constrast, saturation nd definition before I even start to work on the image. I have 10 presets saved for different types of image, with between 5 to 10 adjustments as starting point. Without that kind of work flow, you're going to find raw very tedious.

Sometimes what you can do with raw is about the same as jpeg, but not with the images you posted. You can make those dark areas lighter and more realistic with raw. Working on your jpeg files to do the same will probably produce a real mess.

Dear norm, please reread the initial post properly and do not skip. I outline the processes involved. What you are seeing is a DNG file with the SAME exposure settings applied as the Jpg file that was generated from within the Camera body of the K-1 (with absolutely all Jpg processing options turned off). They both have the same Tone Curves, yet the DNG does appear to be nearing blown out highlights. That's the entire purpose of the thread, I thought if anything it would be the other way around, odd indeed. I am not applying a different Exposure or Highlight level from one image to the next.

Please see the attachment.

No comment on your overall observations and conclusion, as I am on my laptop.

Just a tip to make Highlight and White adjustments easier and more accurate than looking for blinkies. When adjusting the Whites hold down the Alt key (Windows). This will make the screen black except for those areas that are blown out. If totally black, just increase the Whites until you see some spots of color or white appear. Then slowly back off until they are gone. I find this easier and faster to do than to describe.

Thanks, yes I think I knew that but had forgotten. What led me to forget I think was that I actually just found it pretty easy to turn on the Highlight warning, zoom in on what I perceived to be the brightest thing in the image and watch for a red bit to emerge hehe. Your way is probably best, thanks for reminding me!

There's nothing wrong with shooting .jpg .The processing power of current cameras has advanced greatly since the idea of RAW processing was put out there (ist D time frame). I think that getting it right in camera, like you mentioned, has a lot to do with it. I rarely see weird artifacts any more in jpg OOC and Pixel Editors handle jpg images as nicely as RAW. IIRC the myth of degrading jpg images after several saves has pretty much been Debunked.
My work flow that has developed over the years is RAW and process to .tiff, save. I use Pixel editor for other adjustments. What I've noticed is less and less adjustments to the RAW file. I could probably switch over to jpg any time and not notice a difference.
I shoot a m4/3 system also, it's always been jpg and I wouldn't know how to develop RAW to match the jpg output. It's that good.
Good luck playing with it.

Well that's where my heads currently at. Like I said, in the beginning I used to make a ton of exposure mistakes, I thought I was being smart using RAW for recovering these images. These days I am mastering my K-1 a lot more, I make less messes and get more things to a nice editing place to begin with. I'm wondering if I now need that RAW dynamic range, because what I do feel I miss more is being able to fire off a long continuous high burst shot without hitting the buffer so quickly!

What is the oddest about all of this is that my samples actually seem to contradict what the masses and general consensus is saying. Just look at the RAW vs Jpg, after those adjustments the Jpg is WINNING. :/ Wth...

tl;dr
I think you like the edit made by the camera + your own edit. The tone curve and adjustments made by the camera added to your edit. If you would like to replicate it in lightroom use the brush tool, select everything and make more adjustments to get over the +/-100 limit.

(When the adjustments of the camera is in the opposite direction of your taste it can really harm the image.)

Please elaborate.

Be sure to read the initial post carefully. I quote;

"It is important to note that all Jpg processing was off during the shot (DNG) and that no Jpg processing was turned on during the conversion in the camera of RAW > Jpg. (Yep... just to be clear the shot itself was just a single RAW DNG file, not a RAW+ where a Jpg is created at the time along with)."

The Jpg 'edit' the camera (K1) made is as close to the RAW file itself, no Jpg processing on at all. My own 'edit' was not biased to one image or the other, just extreme Highlight and Shadow recovery made to both files, with some slight exposure boost;

Exposure +0.83
Contrast +43
Highlights -100
Shadows +100
Whites +29
Blacks 0

My issue is not about Jpg with tweaks in the camera vs RAW (which is often the battle), my issue is a DULL non processed Jpg from camera that is as close to being a 'RAW jpg' as you can get, would appear to have greater room to breathe in terms of recovering exposure, highlights, shadows and details. It doesn't make a lot of sense.

Difficult to make a comparison as the jpeg produced in-camera from the raw file is not being processed to the same parameters as the jpeg produced in LR from the raw file. Even if you produced both jpegs using say Pentax "Bright" profile, there may still be differences in exposure for example.

ps why are you processing in AdobeRGB for an image that you intend to show on the web ?

Just to clarify something, perhaps you missed it.

1) At the time the shot was taken, I was shooting RAW only, not RAW+. However every Jpg processing tool in the camera was turned off.
2) The Jpg was created not from the RAW file in LR, but from the K-1 doing a RAW>Jpg conversion in the camera body (spits out the Jpg onto the sd card). The processing of RAW>Jpg allowed me to turn on some Jpg Processing but once again I refused any of such. The aim was to try and get as close to a 'RAW Jpg' as possible.


I mention AdobeRGB, it's what my Pentax K-1 has set (I think this is default), it's under Menu Item 2>Image Capture Settings>Color Space. What should I have it set at then? sRGB better?

Thanks!

The JPG looks like it was reduced in contrast; that's probably just how the JPG engine works. The reduced contrast brought down the highlights giving the impression that there's more headroom at the bright end of the histogram.

For my photos (a lot of low light and astro), I always use raw DNG for these reasons:

1. Lightening very dark shadows. JPG becomes noisy and loses details sooner than DNG.
2. Adjusting color casts. Auto WB doesn't always get things right and WB adjustments are more flexible with DNG. Setting manual WB isn't a workaround because local light conditions can dictate different WB settings and it's tough to tell until sitting at a full size monitor.
3. Picking up subtle color variations. JPG only offers 8-bits (256 levels) of each color channel. DNG offers greater bit depth, so less chance of stair-step color changes in a twilight sky.

Aha! I think you're right and onto something!

Despite my greatest efforts to produce a 'RAW Jpg' (that is... spit out a jpg that has no in camera processing options turned on, try and match a RAW file etc), it still can't really be done because perhaps 2 things;

1) As you say 'just how the JPG engine works'

or..

2) The RAW DNG file is being darkened/stronger contrast from LR, it's being placed in a worst 'starting point' to edit the shot vs the Jpg.

I mean, I know I can load the RAW DNG file in RawTherapee and the two Images (LR vs RawTherapee) look very very different...

Are Jpgs then less open to different interpretation from various softwares vs RAW files?

And as for your reasons to use RAW I couldn't agree more, I've seen that before. What I found interesting was just this particular example shot I took and how Jpg appeared to be winning, at least at this current edit level.

More and more however I am starting to see the argument for RAW being something more useful for landscapers than portrait or sports shooters. Sure we can mess the skin up a bit with Jpg and have less room to recover, however if you get WB right first it does become slightly less of an issue. I guess it starts to come down to (once again) what you're shooting, does the RAW dynamic range matter more vs better buffer rates and longer bursts.

I believe we can set certain User Modes to shoot Jpg only and others RAW? I'll have to check on that.

If a JPEG is better than the RAW version, it is because either the RAW processor being used cannot reproduce the same look as the in-camera processing or the skill of the person doing the RAW processing is not up to the task.

Below is an in-camera JPEG of some wonderful Fall foliage at the Japanese Garden near my home. Try as I might, I could not reproduce the look in Lightroom with the version I had or the skills I had at the time (October 2009).


Pentax K10D, Pentax-FA 35/2

I took a Kodachrome slide* at the same time and working with the scan was similarly difficult despite the original being quite vibrant.


Pentax KX, Pentax-K 55/1.8, Kodachrome 64

...and finally, a somewhat less vivid version of a digital capture processed in LR. I am tempted to revisit this image with LR 6 to see if I can do a better job.


Pentax K10D, Pentax-FA 35/2


Steve

* I was one of the lucky few who managed to secure several rolls from the last production run of Kodachrome. Those were then processed in the last big batch through the machine at Dwaynes. What a privilege, eh?

Your wording of 'better' is subject tho in your case is it not? In my example of the fern, I am talking purely about same adjustment values being made to as close to 'RAW Jpg' as you can achieve vs the actual RAW and what appears to be the Jpg coping better... that I find odd. However as Deadjohn points out perhaps there never can be a 'fair fight' between Jpg and RAW, because well that is the nature of things. There is a difference between my OOC Jpg vs RAW, even before any LR sliders are touched. Perhaps sometimes this difference favours DNG files more than Jpgs...

A JPEG is a raw file that the camera has converted. Can you convert a RAW file better than the camera (which has been programmed by experts) can?

After many laborious experiments in PS, I find that I usually can't, exceptions being very high dynamic range and certain white balance situations. I certainly can't get any better noise reduction. I save any sharpening for the very last step in post-processing (I have minimum sharpening set on my camera JPEGS).


I have looked at numerous "RAW is better articles" and the examples they use are usually poorly exposed in the first place. Most of them showed little real-world improvement. In-camera processing has made great strides, a trend which is accelerating with cell phones.


As always, Your Mileage May Vary.

Wait up... In the camera, if you choose to shoot Jpg only, you get higher buffer limits and longer high continuous bursting capabilities. If Jpgs are always from RAWs then how is this possible? You can't set the camera to shoot Jpg and then choose to get the 'RAW'' out of it (only Tiff or Jpg can be achieved)...

The RAW detail is too saturated. That happens to me as well with the greens and ACR... don't know who's the "culprit", nor I care.
Go into the HSL panel, Saturation tab, Green -20 (roughly, don't have time to check now). Done. ;-)

I don't use ACR, I do use LR>PS and have seen the PS Saturation boost it can often apply, so I do hear what you're saying.

However in this particular example, reducing Green Saturation by -20 still leaves brighter 'more prone' to highlight clipping than the Jpg. I don't think this issue has anything to do with colour levels.

[DeadJohn]

Wait up... In the camera, if you choose to shoot Jpg only, you get higher buffer limits and longer high continuous bursting capabilities. If Jpgs are always from RAWs then how is this possible? You can't set the camera to shoot Jpg and then choose to get the 'RAW'' out of it (only Tiff or Jpg can be achieved)...

JPG files are smaller than raw files, so a burst of JPG takes less time to save to the memory card. The camera has some fast, internal memory for running basic operations and storing a few photos (the buffer). Creating a jpg from raw is done within the fast memory. Writing to the memory card is considerably slower.

Why do JPG files use less storage space?

1. JPG has less bit-depth than raw. JPG is 8-bit, which only allows 256 levels each of red, blue, green. Raw files use more bits and offer smaller steps between brightness levels.
2. JPG often uses lossy compression. Mathematical rules are applied to shrink the data before saving it, and those rules are aggressive enough to permanently lose some information. You can see it if you open a JPG, save it, close it, reopen, save it again, etc. After a few iterations the image quality can be noticeably degraded. Raw files can also use compression, but it's a lossless compression that's not as aggressive and doesn't save as much space.

Lossy compression - Wikipedia
Image compression - Wikipedia

[stevebrot]

There is a difference between my OOC Jpg vs RAW, even before any LR sliders are touched. Perhaps sometimes this difference favours DNG files more than Jpgs...

That is to be expected because different RAW processors were used in the creation of the two images, in-camera (shared with PDCU) and ACR (used by Lightroom). When a JPEG or TIFF is brought into Lightroom, it comes in fresh with no changes being made. RAW captures on the other hand, are just a collection of numbers (voltages) and have nothing that may be reasonably interpreted as pixels*, the stuff digital images are made of. Those pixels are crafted from the data by the RAW processor in several steps, each of which applies elements of nuance of interpretation through mathematical curve transformation and inspired invention.

When a RAW capture is displayed in LR, the initial display image in the Develop module is the base output of the converter interpreted according to (usually) the Adobe Standard camera profile (for PEF and DNG from supported cameras) or the manufacturer-supplied embedded profile (DNG for unsupported cameras). Note:

• The image (millions of pixels) displayed in the main pane in LR for a DNG or PEF was made FROM the data and is NOT the data themselves. The data are not and will never be pixels.
• The displayed image for a DNG or PEF (all those pixels) exists in memory only as an interpretation of something (the capture data) that cannot be displayed.
• The above two points should bring home a central truth, that being that THERE IS NO SUCH THING AS A RAW IMAGE, only interpretations of the capture data (voltages). When comparing a JPEG (any JPEG) to the displayed ACR RAW output in Lightroom, one is comparing the output of RAW processing engines, one saved to file in the past and the other rendered directly to memory in real time.
• In reference to digital photography, there is no such thing as a non-processed JPEG or TIFF; all started out as sensor data (voltages) subject to interpretation on the path to becoming structured arrays of pixel-mapped data in a computer file (an image file).

By analogy, the RAW capture corresponds to the latent image (chemical data) on exposed/undeveloped photographic film, while a JPEG or TIFF corresponds to a viewable image (either negative or positive) created from that latent image. One can't do a slide show with undeveloped Fujichrome.


Steve

* Despite common usage, a pixel is a virtual construct defining the smallest element of a digital image. A pixel has, as attributes, numeric values for each color channel and not a whole lot more. Pixels are created by software processes and don't exist in the physical world. Camera sensors do not have pixels, nor do they have physical structures that map directly to pixels. Image files (JPEG, TIFF, GIF, PNG, etc.) have data structures that map directly to pixels.

---

Last edited by stevebrot; 11-16-2018 at 10:17 PM.

[BruceBanner]

JPG files are smaller than raw files, so a burst of JPG takes less time to save to the memory card. The camera has some fast, internal memory for running basic operations and storing a few photos (the buffer). Creating a jpg from raw is done within the fast memory. Writing to the memory card is considerably slower.

Why do JPG files use less storage space?

1. JPG has less bit-depth than raw. JPG is 8-bit, which only allows 256 levels each of red, blue, green. Raw files use more bits and offer smaller steps between brightness levels.
2. JPG often uses lossy compression. Mathematical rules are applied to shrink the data before saving it, and those rules are aggressive enough to permanently lose some information. You can see it if you open a JPG, save it, close it, reopen, save it again, etc. After a few iterations the image quality can be noticeably degraded. Raw files can also use compression, but it's a lossless compression that's not as aggressive and doesn't save as much space.

Lossy compression - Wikipedia
Image compression - Wikipedia

I'm referring to the fact that @Cipher is saying all Jpgs come from RAW files, but this doesn't make sense to me as you can shoot Jpg only (and derive all the benefits of such) and cannot retrieve the RAW file information.

[stevebrot]

If Jpgs are always from RAWs then how is this possible?

To summarize:

• Normal in-camera processing always involves creation of an image-in-memory regardless of what type of file is written to the card. This image is used for the instant review on the rear LCD and for file creation. The image-in-memory is always the result of in-camera RAW processing.
• JPEG output is the writing of the image-in-memory to JPEG format on file
• DNG/PEF output is a bundle of the sensor data along with a JPEG version of the image-in-memory written to file
• The JPEG image on file (JPEG or DNG/PEF files) is used for the image review/playback on the rear LCD
• In a very real way, a JPEG image is created and saved to file for each exposure

I hope this helps.


Steve

[GUB]

Exposure +0.83
Contrast +43
Highlights -100
Shadows +100
Whites +29
Blacks 0

Geeeez Bruce get your sliders under control! After brightening the image you increased contrast then decreased contrast!. Cruel and unusual treatment of an image!. Use your exp comp slider and black point slider to keep the histo unclipped and do the rest with the colour curve tool.
Took a detail within the window pane on your first and second image posted and here are the histos in a screen shot. Don't know which one is which but as far as I am concerned that doesn't matter because they are near enough to identical. The variation you achieved must have been downstream of here.
If you open that dng again as unchanged (trash the side files) and wind the exposure back about a stop you will see an unusual extra green spur on the bright end of the histo that will have been clipped when you exported the dng unchanged. That will be pixels of highly reflective highlights on that fern. You need to keep those suckers under control.
If you want to get results as a raw user you need to think in terms of that raw. Here when you visualised that image you knew it was going to be a challenge of dynamic range so why did you set your camera to iso 400 and throw away 2 stops of DR? The way I see it best practise here would have been to go with iso100 and be 2 stops under exposed that you can recover in the developer. But only achievable in raw. Of course you could probably gotten away with 1/100 rather than 1/200 but that is another story.

[microlight]

Yes, a raw file is not a picture file itself but is a 1:1 data record of what the sensor sees when you take a photo, plus some settings information and a thumbnail jpg that you can actually see. A raw file has to be processed so that the picture becomes visible. Shooting a jpg in camera performs this processing of the sensor’s output data to jpg against fixed settings, rather than the photographer doing it in post. Which does it better is respectfully not the point here!

The point is that in creating a jpg in camera, the sensor data are not retained but are dumped once the jpg is written to the card. This doesn’t however explain why shooting jpgs doesn’t fill up the buffer so quickly - on the surface it seems paradoxical because the camera is processing the data and writing the file. However, this all happens in a tiny fraction of a second in memory (unless you turn all of the camera’s processing on!) before the file is written to the card, which is the process bottleneck.

For me, the key to understanding this critical part of the process is something that’s only been alluded to in the discussion so far, in the reference to lossy compression. Yes, the in-camera jpg is created from the raw output from the sensor, but the important point is that the jpg standard protocol then throws away up to two-thirds of the raw data before saving, as it was designed to do. So when the camera saves the jpg file, it’s much smaller than the equivalent raw file, which retains all of the data from the sensor, and therefore saves much faster, which is why the buffer doesn’t fill up so quickly.

So the jpg you get from the camera is an approximation of the original image (because you now don’t have all the data), the trade-off being the size reduction and saving speed. Don’t lose sight of the fact that even at the best quality setting, you effectively throw away a lot of the original picture data when you shoot direct to jpg. This is why in jpgs, dynamic range is more limited and recovery from dark and bright areas produces more noise, as you can’t recover data that you’ve already thrown away. And yes, every time you process a jpg into another jpg you degrade it further. A benefit of post-processing raw files is that you can choose to save your final edit as a jpg where you will still lose data, or something like tiff where you won’t. The trade-off is again, file size.

Think of the analogy with sound recordings. Converting a lossless WAV to even the best quality lossy mp3 throws away two thirds of the audio information, and you can hear the difference in comparison. Edit a mp3 and the sound quality degrades because you had less information there to start with, and then you’re throwing away more. It’s all physics.

[GUB]

So the jpg you get from the camera is an approximation of the original image (because you now don’t have all the data), the trade-off being the size reduction and saving speed. Don’t lose sight of the fact that even at the best quality setting, you effectively throw away a lot of the original picture data when you shoot direct to jpg. This is why in jpgs, dynamic range is more limited and recovery from dark and bright areas produces more noise, as you can’t recover data that you’ve already thrown away

Yep that is the way i see it. If you set the incamera jpg parameters correctly for the image then the result is as good as the edited raw. But if you have to reach out for more information later it is not there.

[LensBeginner]

I don't use ACR, I do use LR>PS and have seen the PS Saturation boost it can often apply, so I do hear what you're saying.

However in this particular example, reducing Green Saturation by -20 still leaves brighter 'more prone' to highlight clipping than the Jpg. I don't think this issue has anything to do with colour levels.

Ok... I've found the time to download the crops and the actual RAW.
By scanning back and forth between the two crops one can see that there's a difference in exposure, and not only in the greens but also in the yellows/oranges.

I second the comment on the sliders, though I don't think it will affect the image much, as I believe the engine will just cancel-out the opposed commands.


The biggest issues that I can see are a slight OoF in the diagonal branch, a light amount of glow and some light fringing.
All in all, the lack of detail looks more like a demosaicing issue to me... perhaps you could try using another RAW converter?

[pschlute]

I mention AdobeRGB, it's what my Pentax K-1 has set (I think this is default), it's under Menu Item 2>Image Capture Settings>Color Space. What should I have it set at then? sRGB better?

The colour space will affect only jpegs produced in-camera (and raw previews).

The default setting will be sRGB. This is because sRGB is the standard for web viewing. Anyone viewing sRGB images on a non colour-managed browser on a standard computer monitor (most people) will see the images colours broadly as you intended. To see AdobeRGB colours as the author intended requires the web browser to be colour-managed and the images viewed on a AdobeRGB compliant monitor.

Your jpeg produced in LR has sRGB colour space (the default LR setting).

You should be using just one colour space for this comparison and I suggest you use sRGB

[MrB1]

The point is that in creating a jpg in camera, the sensor data are not retained but are dumped once the jpg is written to the card.....

.....the important point is that the jpg standard protocol then throws away up to two-thirds of the raw data before saving, as it was designed to do.

.....Don’t lose sight of the fact that even at the best quality setting, you effectively throw away a lot of the original picture data when you shoot direct to jpg....

For our Pentax cameras, if the file format is set to JPEG only, the image file is written to the card but the raw data stays in the buffer, until either another shot is taken or the camera is switched off. That is why it is possible to save the raw data file of a JPEG image while reviewing it on the rear screen immediately after taking the shot. For example, with the K-3 review screen set to show the basic image capture parameters (i.e. the first in the set of five screen views), there is also an indication in the top right corner to press the AE-L button in order to save the raw data file as well. (In the k-70 the +/- button is used.) Of course, this is only useful for single-shot captures - when the camera is in one of the burst modes, in addition to all the JPEG files, the raw data of only the last file in the burst would still be available for saving.

I have been using this feature for quite some time, as I have found that I rarely need the raw data file and I rarely shoot in a burst mode. With the camera set to JPEG only, I capture the image for the scene, review it and its histogram on the rear screen, then decide whether to save the raw data as well.

Philip