[RayeR]

Hi,
it seems that I hit JPEG compression limits when trying to save some photos with clear deep blue sky (used polarizing filter). I must note that I shot in RAW and do some postpro in 16b TIFF and there's no banding visible in editor (no histogram holes). Neither if I save to 8b TIFF or PNG with lossless compression. But JPEG always run this hell regardless compression level I set. I tried many different programs to convert the tif to JPEG but I always got this poor result.
You can try on this sample: http://rayer.g6.cz/1tmp/jpegcrap/053-fixed-by-blur.png
And here are my results: Index of /1tmp/jpegcrap
none of those JPEGs is good as this original.
I read that other photographers also sometimes have this problem. Someone suggested to mix a monoch. gausian noise to sky. Yes, this helps avoid banding but doesn't look good too. And it's little bit mad to do this when I finally have a pentax cam that has quite a low noise to destroy image with more noise.
I'd like to know if there is some program that can save JPEG better - you can try.

If no, I can leave it in lossless format - but the TIFF of uncropped image is 3,9MB big and more complex images will be even bigger. So I tried lossy JPEG2000 format. It also gives good result: http://rayer.g6.cz/1tmp/jpegcrap/053-psp9-level05.jp2 even on high compression: http://rayer.g6.cz/1tmp/jpegcrap/053-psp9-level50.jp2 very high: http://rayer.g6.cz/1tmp/jpegcrap/053-psp9-level75.jp2
But the JPEG2000 is not such wide supported and there's issue with saving EXIF in into it. Is there somebody experienced with EXIFTool who can suggest me how to copy EXIF data from source TIFF/JPEG to JP2 ? It uses some different XML format of metadata...

---

Last edited by RayeR; 03-31-2014 at 02:12 AM.

[Steve.Ledger]

I used Faststone Image Viewer and saved your ping as an rgb jpeg. No banding.

[Na Horuk]

What focal length is the lens? With a polarizing filter, ultra-wide lenses can have odd dark patches in sky photos.
Which raw processor are you using?
Also, the problem might also be caused by colour space changes during the file processing/exporting. But a good raw editor should still work things out correctly. Colour space becomes more problematic when you upload the photo to a website.
Something like banding can also be created by post processing, if you increase saturation or contrasts in a certain way

[carrrlangas]

I think it´s due to how the JPEG algorithm works. Here´s a good article about it: Jeffrey Friedl's Blog

[stevebrot]

The banding has little to do with JPEG compression and a lot to do with the 8-bit/channel color limitations of the JPEG format. Normally it is not much of a problem, but with large regions of primary color and a shallow gradient (such as the sky), 256 shades just ain't enough. Gradients appear as discontinuities. As noted above, PP can make the problem worse.


Steve

[RayeR]

I used Faststone Image Viewer and saved your ping as an rgb jpeg. No banding.

Thanks a lot Steve Ledger!
I didn't know about Faststone Image Viewer. I can see it's quite cool piece of software, rich features but without unwanted bulls... in a few MBs (wonderful in new age of Vista/7/8 bloatware), no .NET, portable... and freeware

If I use standard YUV photometric preset on save it makes banding as other programs I tried but in RGB mode it's much better! Also CMYK looks well. I would say only a bit worse than the PNG. I wonder why Photoshop or PSP don't have such option for jpeg save. Probably the RGB model put more weight to colors than YUV where emphasis is on Y even if chroma is not subsampled so the compression then don't affect color information so hard...

>stevebrot
I don't have problem with original PNG image that I posted here (yes if you look carefully you may see something but its much better than all jpegs excluding jpeg2000 that I produced) and ALL OF THEM USE 8bit/channel, so it's not problem. Sure, if we could have 10b/channel JPEG it would be better...

>Na Horuk
I wouldn't discuss postproc in this case (LR->16b TIFF->JPG) as I saw good image during entire proces to save. But after I viewed saved jpeg it turned me very upset how it degraded the sky.

>carrrlangas
OK, maybe I could try also LR jpeg export, on the sunset example it seems that it can preserve smooth grad. I didn't tried yet beause I do further light postproc. and export to tiff16 from LR...

[Imageman]

Im uneasy about this, for one thing you don't say what purpose the jpegs are to be used for other than you want a smaller file, and you go on to say that the uncropped image is 3.9 megabytes as a TIFF.


This is a very small TIFF, my own TIFFs are anything from 80 megs up, 20 times larger than yours. I suspect theres very little information in the image for some reason. Could this be part of your issue.


Example recent image, simple sky hardly any detail, a plain cloudless blue sky and 3 contrails, TIFF size 106 megabytes, JPEG size 8.2 megabytes


Your TIFF is half the size of my JPEG and I have almost an empty frame.


Why is your TIFF so tiny. Why is there such a space issue that you need to convert this tiny TIFF to a JPEG. I would love TIFFs that small, TIFFs are huge lumbering things because they hold huge amounts of data.


My experience is to expect something similar to a 1 to 1 conversion of megapixel to megabyte for JPEG. My own camera is a 5 megapixel, and I get a JPEG of 8 megabytes. TIFFs are often 5 to 10 times bigger than the megapixel.


What camera are you using and what are your capture settings, I don't want to sound negative, but I simply don't understand whats going on.


Its almost as though your camera is a 1 megapixel model, or your camera is set to record a web quality image. Im sure that cant be right.


Im confused

[RayeR]

>Imageman
I have K-30 that has 16MP but for most of images I don't need to archive in full-res. I don't plan to print a billboard from it so I downsample usually to 3200x2133 or 2560x1707. I also don't want to waste big capacity so I use compression. But if compression works good you shouldn't see any visible degradation on monitor or small print.
In my case RAWs are about 15-25MB, and 48b uncompressed TIFFs (in midprocess) have 90MB. I wonder how you can get 80MB TIFF from just a single 5MP photo. If the image is not so complex, then it can fit (for 2560x1707/24bpp) in 3,8MB using LZW lossless compression.

Anyway the thread was not about processing or archiving photos. Let everybody decide ourself his tradeoff between size and required quality. The thread was about specific issue caused by JPEG compression on sky gradient. My attemps with different compression and different programs failed but Steve Ledger showed me that changing color model to RGB in compressor setting can make significant improvement - that I couldn't achieve even with smallest compression in YUV model. If it wouldn't solve it then JPEG2000 with smooth wavelet compression can be used. I think problem is solved for me...

[Imageman]

Your savagely downsampling and then using lossy compression on the resulting remains of the image, and then your post processing whatever is left.


That's your issue and problem right there.


Stop downsampling. Your throwing 99% of your image away, and you create these issues and problems yourself. The software cant cope with what your asking it to deliver. Everytime you throw substantial portions of your image away the software has to create pseudo artefacts to try to hold the image together, sure they look ok on a low resolution monitor and you think you have got away with it, but then when you try to do anything with the remains of your file it all unravels and you expose and magnify the artefacts hidden in the file.


You have a highly detailed image to begin with, and that means you have big files, those TIFF sizes you now quote are as I expected, dont crap your image out simply because you don't like the size of the file. Its that size it is for a reason.


Why do I have a 100 megabyte TIFF from a 5 megapixel camera?, simple, that's because I Upsample the image, to preserve smoothness in later post processing and minimise artefacts in post processing, I can always downsample later. Im doing the exact opposite of what your doing. I never have those issues that plague you.


My workflow works, as an example, that plain sky with 3 contrails was as follows. - my raw was 12.9 megabytes, my TIFF (upressed) was 106 megabytes, and I outputted 2 Jpegs, one 8.2 megabytes for storage and printing, and one 290 kilobytes for viewing. All post processing was done on the TIFF. Your destroying your file with compression before you post process, that breaks all the rules. Try it the other way round, preserve as much detail as you can right up to the end of post processing then compress. Not the other way round.


So begin again, take the original 15 Mb Raw, generate your 48 Mb TIFF, post process that image not a downsampled TIFF or downsampled anything, then after you have a finished image, save the TIFF as your archive and then save a Jpeg at highest quality, and a lower quality Jpeg if you need one. I expect a 10 to 12 Megabyte High res Jpeg from this workflow and a low res Jpeg of around 100 kb.


I don't expect you will see any of these artefacts.

[RayeR]

>Imageman
Your approach seems to me very extremistic one that recall me some audiophiles using a finger-thick silver litz reprocables to preserve best sound quality...
I'm just an amateur so I have different priorities. I also know a few profi photographers and they save all RAWs but nobody of them makes upscaled images.
I think that upscalling cannot add any extra information and I can do it anytime later. Also take in mind that megapixels are not RGB but there's bayer mask so effective resolution is less than if you have same count of true RGB pixels (like foveon - is it your case?). Then you have a lens in the optic path that makes it more or less blury, esp. on edges and cornes. So because of this reasons I don't hesitate to throw some redundant pixels that can be interpolated again (not 100% but very close)...
And regardless your ultrasized images JPEG compression will change them because it's lossy even on highest quality setting. This banding may also happen to you if look on 1:1 image on monitor or other artifacts on edges. Also you need to generate small images for web gallery where banding may also appear there. It's good to know how tune compression to produce best result for given size. That's all.

[Imageman]

Ok, I don't want an argument. You win. you can downsize and compress an image throwing away 90% of it and then post process and get a perfect result.


I stand corrected

[stevebrot]

>stevebrot
I don't have problem with original PNG image that I posted here (yes if you look carefully you may see something but its much better than all jpegs excluding jpeg2000 that I produced) and ALL OF THEM USE 8bit/channel, so it's not problem. Sure, if we could have 10b/channel JPEG it would be better...

Hmmm...your PNG appears to have better than 8-bit/channel color depth. My tools are ancient (PSP 7.04), but when I reduce your PNG to 8-bit/channel I get banding much like that in your bad jpegs. Ditto when I import your PNG into Photoshop Elements 6.0 and export as JPEG.


Steve

[Imageman]

Ok, ill have another go and see if I can persuade you.


converting a TIFF to Jpeg takes a smoothly rendered image and compresses it, throwing away detail and substituting detail and colours that can be grouped together, referenced where they are and then it keeps a rudimentary number of pixels in the file with references where similar ones exist. That's how lossy compression works.


Then it used this tone map to rebuild the image using the rudimentary pixels and their reference locations. In a very simplistic way, this is what Jpeg lossy compression does. The result is banding, you don't notice it in the jpeg, but its there.


And as soon as you try to post process this banded image, varying the brightness, or colour, or contrast, or saturation, or sharpness, the banding emerges.


If you downsample the source before Jpeg compression, you reduce the number of pixels the Jpeg engine can choose from to represent the detail and you increase the banding and it emerges sooner and more aggressively.


Traditional and current best practice is always always always post process on original files no downsampling is allowed.


This minimises any effects post processing has and renders the smoothest resulting images. No banding emerges when you post process because none exists in the image yet.


At the very end, after post processing is complete and you have your finished unbanded image, then you can downsample and save as a Jpeg. But beware, if you save as a low quality Jpeg you will create obvious banding due to this and this alone. Save as a high quality Jpeg not low.


The filesize you end up with is no larger than if you had downsampled first, and its no harder to do, so theres no problem doing it this better way. You just get a better result.


It wont cost you anything, it wont take longer, it wont be harder, it wont leave you with larger files, it is the recommended way, and all you have to do is try it.


I don't accept your argument that im advocating some unproven exotic method like your audio example, this isn't something that im proclaiming that you won't see the point to.


You claim that this method is ok for me but you have different requirements, well my requirements are images without bands in them, how are your requirements different to mine.


You are seeing images that have problems you cannot accept. You will either see an improvement if you do it my way or you wont. And if you see an improvement youll know I was right. If you don't youll know I wasn't right.


And my way isn't something ive come up with, its been the recommended standard way to process images for nearly 15 years. Your way is the unusual way. What they say is, if you downsample and save as a Jpeg and then post process, banding will emerge. Isnt that what your seeing.


So how about it, cant you spend 15 minutes trying the industry standard way for once.

---------- Post added 04-03-14 at 08:13 AM ----------

Ok, ill have another go and see if I can persuade you.


converting a TIFF to Jpeg takes a smoothly rendered image and compresses it, throwing away detail and substituting detail and colours that can be grouped together, referenced where they are and then it keeps a rudimentary number of pixels in the file with references where similar ones exist. That's how lossy compression works.


Then it used this tone map to rebuild the image using the rudimentary pixels and their reference locations. In a very simplistic way, this is what Jpeg lossy compression does. The result is banding, you don't notice it in the jpeg, but its there.


And as soon as you try to post process this banded image, varying the brightness, or colour, or contrast, or saturation, or sharpness, the banding emerges.


If you downsample the source before Jpeg compression, you reduce the number of pixels the Jpeg engine can choose from to represent the detail and you increase the banding and it emerges sooner and more aggressively.


Traditional and current best practice is always always always post process on original files no downsampling is allowed.


This minimises any effects post processing has and renders the smoothest resulting images. No banding emerges when you post process because none exists in the image yet.


At the very end, after post processing is complete and you have your finished unbanded image, then you can downsample and save as a Jpeg. But beware, if you save as a low quality Jpeg you will create obvious banding due to this and this alone. Save as a high quality Jpeg not low.


The filesize you end up with is no larger than if you had downsampled first, and its no harder to do, so theres no problem doing it this better way. You just get a better result.


It wont cost you anything, it wont take longer, it wont be harder, it wont leave you with larger files, it is the recommended way, and all you have to do is try it.


I don't accept your argument that im advocating some unproven exotic method like your audio example, this isn't something that im proclaiming that you won't see the point to.


You claim that this method is ok for me but you have different requirements, well my requirements are images without bands in them, how are your requirements different to mine.


You are seeing images that have problems you cannot accept. You will either see an improvement if you do it my way or you wont. And if you see an improvement youll know I was right. If you don't youll know I wasn't right.


And my way isn't something ive come up with, its been the recommended standard way to process images for nearly 15 years. Your way is the unusual way. What they say is, if you downsample and save as a Jpeg and then post process, banding will emerge. Isnt that what your seeing.


So how about it, cant you spend 15 minutes trying the industry standard way for once.

[RayeR]

Hmmm...your PNG appears to have better than 8-bit/channel color depth.

How did you determined this? I saved it as common 24bpp png and so I cannot reduce colors to 8-bit/channel. The count of unique colors is only 59179. My Jpeg files have similar color count.

[RayeR]

>Imageman
I'm not sure if we are understood. Of course I do all the postprocessing in full resolution and colors (in LR and PSP) and the last step I do is to downsample it, sharpen a bit and save as JPG. But regardless that the banding appeared in saved JPG (In this case I discovered it too late). Later I found that it depends on how much NR filter I use on sky area in LR. Seems that natural noise helps to prevent JPEG to reduce colors. You still suggesting that banding varies on used compression level but I wrote 10-times that it doesn't matter in this case. I tried set lowest compression and banding was still there. Steve Ledger gave valuable suggestion to use Faststone Image Viewer in RGB mode - that make much more difference than playing with compression level. Did you saw his attached image? Can you download the program and try YUV and RGB mode? It shows realtime preview so you'll see the difference! RGB mode has also drawback-it makes slightly more artifact around sharp edges like trees/bigger size - it's visible only when zooming more than 1:1 but banding is visible much more. And mentioned JPEG2000 don't have this drawback and compress better. Did you tried ever play with this format?