[itshimitis]

I'll upload the blue island DNG when I get to a computer.

Can I politely ask we focus on the issue instead of processing ethics? I have tried rescuing images by fixing that area but it's impossible with gradient skies - it'll do your head in.

Lastly, I have one of the largest collections of NZ images in the world. I've grown up dealing with the huge dynamic range our clear atmosphere & harsh sun angles creates. I've shot everything from KM25/Velvia & all kinds of scanners and a long history of digital cameras. I have NEVER had so much difficulty as I have with the 645z in this area. I'm really scratching my head. It shouldn't be this hard it this point in digital development.

Otherwise I love the 645z (some of the glassware and CA not so much though).

It wasn't my intention to start a meta-discussion about ethics, I was just saying that a stated particular solution may or may not be something you or I might consider as a work-around. Apologies if I skewed things.

[rob8888]

Hi all

There are a few red herrings appearing;
1) its not about jpgs - this shows up in 16 bit Tiff files (prophotoRGB or Adobe1998, or any other profile) - jpegs only make it worse/easier to see.
2) its not about clipping. This image is miles away from clipped.

Its a strange response in the blue channel I've never seen before. If you want to meet it, get the file below and dial in around contrast +50, exposure +60 in LR. You'll see it now. Then pull your blacks down and enjoy! Touch any vibrance or saturation and you're on your way to pop art.

This should not be happening surely - but it's common in these hues. You all know collectively a lot more that I do so I'd be really grateful for your deliberations please.

Here's the .DNG raw file download of the blue island image below:

[Kolor-Pikker]

Hi all

There are a few red herrings appearing;
1) its not about jpgs - this shows up in 16 bit Tiff files (prophotoRGB or Adobe1998, or any other profile) - jpegs only make it worse/easier to see.
2) its not about clipping. This image is miles away from clipped.

Its a strange response in the blue channel I've never seen before. If you want to meet it, get the file below and dial in around contrast +50, exposure +60 in LR. You'll see it now. Then pull your blacks down and enjoy! Touch any vibrance or saturation and you're on your way to pop art.

This should not be happening surely - but it's common in these hues. You all know collectively a lot more that I do so I'd be really grateful for your deliberations please.

Here's the .DNG raw file download of the blue island image below:

Well, I've played around with the exposure and specifically looked at the spots you marked on the image, but I'm having a harder time spotting the problem area unless I really crank the settings. I have no idea what the sky looked like originally, so I have no point of reference.

I really don't know what to say, I went though all my photos again and went to town with the settings, adding up to 50~100 clarity, saturation, vibrance and contrast to see if anything would break and nothing does, but both of the shots you uploaded do. The first one has that very obvious band going across the sky, and the blue shot if pushed really hard has a funky transition from the dark blue sky to the brighter area.

Maybe something about your camera is faulty after all? I don't think it can be any of the in-camera settings, because those aren't supposed to effect Raw capture, but just in case try to disable all of them anyway. It would be best if you find a way a of testing a second 645 to see if it is a problem with your camera, maybe someone you know has one? rent one for a day or two? I'm out of ideas.

[Royce Howland]

Rob, I looked at your blue island DNG file. The original is fine, not a thing wrong with it that I can see. I diagnose the green banding as follows. The situation you're running into is being triggered by a couple of things. First is that the image is very monochromatic in a particular range of blue hues that are very sensitive to contrast changes. When you're working on an RGB file, doing things that involve contrast changes will push and pull the R, G and B colour channels around. They don't typically change uniformly; the underlying math is trying to maintain a consistent perceptual "look", but sometimes when the colour channels move, they hit inflection points with each other where a hue shift can be caused. This is happening to this file.

Making curves edits in Photoshop to approximate the tonal appearance of your web site JPEG of this image, I can easily create green bands in the sky in the same area you pointed out. That's because by default, Photoshop curves (I believe like most Lightroom contrast edits such as the grad tool) work in colour mode, changing the R, G and B channels differently and producing hue shifts. In Photoshop, you can change the blending mode of a curves layer from "Normal" to "Luminosity", which attempts to restrict the tone curve to impacting only relative brightness of the image. But this only partially restricts the green banding from happening, because fundamentally in an RGB colour image when you impact contrast you also impact colour. The two are linked.

The second thing is then triggered when you convert to sRGB for posting on the web. Recall that the gamut of sRGB is more limited than Adobe RGB, and much more limited than ProPhoto RGB (a variation of which is being used internally within Lightroom while you're editing the file). It so happens that your range of blues here involve a degree of contribution from the R channel that gets absolutely crushed when you convert to sRGB. This exaggerates what was already happening from the contrast edits above, and really bakes in some channel inflection points that make dealing with the hue shifts extremely difficult after the conversion to sRGB.

I've attached a couple of screenshots here. The first shows the R, G and B histograms from your blue island DNG, where I converted to 16-bit TIFF / ProPhoto RGB with mild edits much as you described. (I use Photoshop, not LR, but that doesn't affect the underlying diagnosis.) All 3 histograms look normal, though clearly the R channel is biased to the darker tones, while the B channel is biased to the brighter tones.

The second screenshot shows the same R, G and B histograms after converting the edited file to sRGB, but no further adjustments. Note that the R channel has been fairly crushed, with a lot of clipping to black. The third screenshot shows a visualization of the R channel alone in the image, showing where the channel data has been blocked up completely to 0. I downloaded the blue island JPEG from your web site and looked at it; the R channel is even a bit more crushed, indicating you had a slightly more contrasty original working image before the conversion to sRGB did its damage.

So to recap, this situation involves a couple of different things. First is a monochromatic blue hue range that's very sensitive to contrast changes pushing the RGB colour channels into some hue shifts at certain points where the channels are hitting some ugly inflection points in the roll-off from highlights down to midtones. To avoid this first problem, I experimented with converting the image from RGB mode to Lab mode, and then making similar contrast adjustments to the L channel alone. (This requires Photoshop, I believe Lightroom has no support for Lab mode.) Doing this, I saw virtually none of the green banding effect, plus a retention of blue hues that were more true to the original colours in the DNG file. Lab mode is tailor made for this type of situation. Once done all the contrast adjustments in Lab mode, convert back to RGB mode and carry on with any other finishing work you want to do.

Note: If I continued to push the contrast quite extremely, even Lab mode edits were able to hit weird inflection points in the two colour channels of Lab, introducing some hue shifts. You're simply going to have to back off the global contrast pushes in this case. Go with a lighter and/or a more selective hand, dialing contrast in more selectively in areas rather than globally. Once the hue shifts are there, getting rid of them will be a lot harder than not triggering into them in the first place. But in Lab, doing a similar level of edits to approximate the contrast shown in your web site JPEG version of this shot, I saw no green banding.

Second issue: the conversion to sRGB is hammering the R channel of the image file. This introduces another pile of hue shifts by imbalancing the RGB colour channels due to the elimination of the R component from the upper sky and lower water. The 4th screen shot I've attached shows a softproofing view from a tool called Gamutvision, with a really useful gamut warning display.

The false colour rendering shows the areas of colours that are out of gamut when your blue island shot with my edits is converted from ProPhoto RGB to sRGB. There's simply a ton of dark blues that are out of the gamut range of sRGB; during the conversion to sRGB those pixels are going to be rendered with incorrect colours, which we know from looking at the histograms above is mainly because the R channel is getting clipped to 0 in those same regions.

The rendering of false colours in the gamut warning display corresponds to delta-E values in the horizontal range below, showing dE values from 0 to nearly 11. The rule of thumb is that dE values of 1 or less are basically unnoticeable colour changes; up to 3 may be noticeable but minor colour changes; and dE values above that are definitely more & more noticeable. (The gamut warning display in Photoshop shows out-of-gamut colours over a similar region as in this Gamutvision display, but Photoshop doesn't show by how much the colours are out of gamut -- just that they're in or out.)

So... the mere act of converting this image to sRGB triggers hue shift in a large area. The way to avoid this would be the same as the way to avoid any other out-of-gamut situation. If you really want the web JPEG file to look its best, then prior to the conversion to sRGB you'll need to custom edit the file to move the affected regions of the image away from the blue range that sRGB can't represent. You'd do this probably by increasing lightness and/or reducing saturation in those deep blues.

In my estimation, neither of these things has much to do with the 645Z. Certainly the second issue is just a standard matter of being able to capture and edit rich colours that fall outside the range of the sRGB colour space. Nothing to do with the camera, except by virtue of its ability to capture a rich colour range in the first place. Contrast edits simply push those particular colours further into a region outside what sRGB can handle.

The first issue of why this particular range of monochromatic blues triggers the green banding and so on is perhaps peripherally related to the 645Z, in the sense that the spectral response of the camera's sensor may be recording blues in a way that past cameras you've used do not. But that's not necessarily a "fault" of the camera, that's just the way it's recording these blues. Having become aware of it, there are ways you can edit for contrast and tone, without hitting the tripwire. The simplest of these involve edits in Lab mode which is designed to separate tone edits from colour edits.

Also, as a side note, the above confirms this file is a different situation from the other example DNG you posted. As I noted previously, the trigger issue with that first file was both raw image data green channels being heavily clipped straight out of the DNG. That's not an editing issue, that's an issue of over exposure followed by a challenge in how to do a channel reconstruction to compensate for the lost G channel raw data. This can be done in Photoshop, but not in Lightroom... or certainly not easily anyway.

Attached Images

       

[Royce Howland]

Rob: So back to your first example DNG. I know that your histogram in Lightroom is probably telling you this DNG file is not over-exposed. Certainly my Adobe Camera Raw histogram is telling me that when I open the DNG to convert the file for some tests. However, as I mentioned earlier in the thread, the RawDigger tool says the file definitely is over-exposed, and not by a small amount either. Both Green channels of the raw data (pre-de-mosaiced files have 2 G channels, and 1 each of R and B channels) are severely clipped in the area where you're having the problem with magenta and green shifts. Each G channel contains 1/4 of the raw pixel data recorded in the DNG, somewhere a little over 12 MP. Of those 12 MP in each G channel, RawDigger reports over 1 MP of them are clipped in the highlights. Neither Lightroom nor any other raw conversion tool will be able to do anything about that... there simply is no G channel data in the DNG file to convert.

I've attached an initial screenshot showing RawDigger's basic RGB render of the DNG file, with highlight clip warning turned on. The next screenshot shows just the 2nd of the 2 G channels, clipped in exactly the trouble region. The third screenshot shows the RawDigger 4-channel raw histogram, where both G channels clearly hit the clipping wall on the highlight end.

To try to fix this, I did a conversion of the DNG to 16-bit TIFF / ProPhoto RGB. Then I did a bunch of attempts at channel surgery. Basically, I made a selection of only the blue sky region of the image. Using that selection, I copied pieces of the R, G and B channels into their own layers, and blended them together in various ways. The results of these blending trials, I copied back into the G channel to replace the sky region with new data overwriting the area that previously was whacked out. The results looked different and in some cases better, so you might find this an approach that offers a possible solution with more work. Moving quickly, though, I wasn't finding one that I really liked.

On a lark, I tried something completely different. I went back to the original DNG in Adobe Camera Raw trying to see why it wasn't complaining of a clipped G channel when RawDigger showed the evidence of such a large amount of clipped G channel raw data. Just trying a bunch of scenarios, I attempted a range of different white point settings. Low & behold, choosing Tungsten produced a sky with a clean blue gradient, no presence of magenta / green banding or other side effects from that clipped hump in the G channel raw data. Probably due to skewing things heavily in favour of a much "bluer" blue sky. I quickly did a second raw conversion, grabbed the tungsten blue sky & copied it over top of the sky in the original converted file, applied a gradient masked curves layer and a quick hue layer to adjust the blue. I don't know what kind of blue you're looking for in this scene, perhaps a paler one. But adjusting the blue to look the way you need it to is far easier without having to deal with the nasty magenta / green banding caused by converting with a warmer colour temperature. The 4th screen shot shows after & before examples; clearly the blue in the after example still isn't right but it would be adjustable.

Why this whole situation is happening in the first place, I can't say. From your photography background, I will presume you well know how to use "expose to the right" technique to preserve highlights, read on-camera histograms, etc. All I can say for sure is what the data shows -- the DNG file's raw G channels are clipped, and that's directly where the rest of your challenges in the sky are coming from. Hopefully a mix of using a different colour temperature conversion for the sky and/or some channel surgery would give you some other technicques to fix it. But better to figure out why it's happening so you can avoid this kind of repair work...

[Kolor-Pikker]

Wow, Royce, bang up job on the analysis. I don't have the patience to dig this deep into technicalities.

Why this whole situation is happening in the first place, I can't say. From your photography background, I will presume you well know how to use "expose to the right" technique to preserve highlights, read on-camera histograms, etc. All I can say for sure is what the data shows -- the DNG file's raw G channels are clipped, and that's directly where the rest of your challenges in the sky are coming from. Hopefully a mix of using a different colour temperature conversion for the sky and/or some channel surgery would give you some other technicques to fix it. But better to figure out why it's happening so you can avoid this kind of repair work...

I recall one person wrote on another forum about a method called UniWB which aimed to intentionally skew the white balance during the shot to mimic the Raw capture by placing a heavy emphasis on green, I never used it myself, but if you're really paranoid about chroma clipping, it might be an interesting thing to check out. As for the the whole ETTR thing, OP's first shot with the clipped greens is pretty much exactly how I would have exposed that scene with my 5Dmk2, the shadows are all safely within the DR of the camera (if it had 11 stops that is) and the highlights are within the realm of "recoverability", except this isn't how you shoot the 645Z.

Edit: I've gone ahead and made a UniWB profile for the 645Z using DCRAW since I couldn't find one online: https://www.dropbox.com/s/ihj2rzuhrczjktz/UniWB_645Z.JPG?dl=0

To get it to work - download the above jpeg, put it on your SD card and navigate to the image in preview, press down to access the image menu (make sure you're not in histogram or detail view) and select "Save as Manual WB" and assign the new white balance to one of your three custom WB settings.

What this does is equalize the RGB channels so that what you see is a step closer to what the camera sensor sees, and since there are two green pixels for each red and blue, the image is going to have an obvious green skew. In theory, at this point if you see any of the color channels clip, then they are 100% lost. Just by switching between regular WB and this skewed one, you can see how highlight warnings that would be triggered previously may not be anymore.

[Gonzo]

That's a very interesting analysis Royce, thanks alot. I'll have to read it again, especially on how to prevent the issue.
This is the worst I could make it look in Iridient Developer. I'm not sure if I reached the point where the hues fall apart yet.
In the first image I used the contrast slider and Vibrance, and in the second an S-Curve in Lab mode along with a Chroma push.
Finally I set the colour profile to the embedded DNG metadata instead of using Iridient's default profile (third example).
Not sure what I was trying to achive, because I rarely do extrem pushes like these and can't trust my colour blind eyes.
But regarding hue changes the third one seems to be smoothest. I overdid the chroma (ab) push on purpose to bring out the uglies here too.

Attached Images

View Picture EXIF

File Date

 

View Picture EXIF

File Date

 

View Picture EXIF

File Date

 

[rob8888]

Hi Royce, Kolor-Picker, Gonzo and others.

Wow, Royce you went to a lot of work and care in thinking this through. Your knowledge about the mechanics of this is well beyond mine and I have learnt a lot.

I've just tried similar Canon 1DsMk3 and 5DMk2 raw frames of similar skies and pushed the heck out of them in Canon DPP - the OEM raw converter. I got clean predictable shift to white (even with contrast right up). The drop off into highlight burnout is elegant and clean even with some fairly hard pushing. Lovely.

So I ran the same files through Lightroom. Boom, nasty cyan blue banding fairly easily (though it's hard to gauge equiv inputs inthe two converters)!?

From quick testing I'm wondering if this could this be Adobe/LR specific issue? Could someone run the two 645z .DNG files through a different RAW converter you're used to and see if you can replicate this issue, or at least compare them?


Cheers
Rob

[Gonzo]

When I strip the image to naught contrast in LR it looks like crap in LR with vibrance at 100% (pic1)
I then went to Iridient Developer and used the same S curve in Luminosity mode on its Lighness channel and turned the standard contrast slider to -100 (pic2).

If you have a Mac (unfortunately Iridient Developer is Mac only) you could give Iridient a go. It's got a ton of features which can serve as a better toolset then anything else I've tried.
Yet I'm no expert, I've just played around with the features in context with this thread and learned quite a bit through

EDIT: The 8-bit sRGB version in the browser always like a bit crappy with gradients like these I guess. Or what's wrong with that? It looks disguisting, yet the TIFF looks just awesome!
Wondering how to get better 8-bit interpolation...

[Royce Howland]

My guess is it's not an "Adobe issue" as such, more likely the use of specific colour model math internal to many of the common tools in Lightroom and Photoshop. You can avoid the problem in Photoshop by making contrast adjustments in Lab mode instead of RGB mode, as I described, whereas in Lightroom there's no support for Lab so you're stuck. If other tools have the same issue, they're using the same kind of RGB mode edits under the hood as the Adobe tools. If some tools don't produce the magenta/green banding in the blue island photo, my bet is those tools are using Lab mode under the hood.

Other tools probably won't help with the other examples because those are caused by clipped data in the raw G channels, and the only way to handle them is reconstruction work. The easiest approach for that I feel is some form of surgery in Photoshop with layers, channels, etc.

[derekkite]

So essentially the range of blues is too wide to fit into the rgb color model, resulting in clipping or out of range values, especially if editing calculations take place in that color model. The lab model has more 'room' allowing a bit wider latitude. Converting from a wide to narrow color model is a well defined problem and most software does a pretty good job at it.

A short while ago the frustration of being in ignorance of what the letters and sliders actually did prompted me to read about lab vs rgb color models, which oddly enabled me to improve post processing results. Google lab color model and there are a bunch of articles that explain it.

[Royce Howland]

Derekkite, it's not so much that Lab mode has more colour range than RGB colour mode (i.e. wider vs. narrower gamut). It's more that Lab mode tone and colour transformations work on the image in a different way, cleanly separating lightness edits from colour edits for one thing. In RGB mode, editing tone impacts colour as a side effect; in Lab mode that isn't the case. That's why editing tone for the blue island photo using Lab mode can avoid hue shifts in the sky.

There are other colour modes as well, such as CMYK or Grayscale. And also other models of colour such as HSL that aren't made available as "modes" in Photoshop but which may be used under the hood in some of the filters or tools. Each colour mode / model has its strengths & weaknesses, which may make it more or less appropriate for certain kinds of image edits.

There is a colour range (gamut) issue at play with the blue island shot, but only at the very end with the conversion from the wide gamut ProPhoto RGB (or Adobe RGB) to narrow gamut sRGB. The deep saturated blues in the scene are definitely outside the gamut of sRGB, and so hue shifts will result unless the file is edited prior to the colour space conversion, to compensate for sRGB's limitations. But by that point, the file already had issues of magenta / green banding in the sky that came from the earlier tone edits. Those were the result of something else, not related to gamut size.

This stuff is a bit esoteric and most photographers don't need to know that much about it most of the time. But every so often a case like this comes up, and then some colour theory needs to be sorted out to understand why the problem is going on, and how to fix or avoid it...