Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
04-09-2009, 03:39 PM   #31
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by Marc Sabatella Quote
Or do you mean, starting from TIFF or some other RGB format? That would seem patently impossible - the original RAW data is already gone.
Yes. DxO always writes RGB format. Be it TIFF of DNG.

Because DxO does lens distortion correction and lens softness correction it cannot maintain the original RAW data. Only vignetting correction and noise reduction can be done in the original RAW data.

So, I was looking for software which creates a Bayer pattern image from the full RGB data and writes this much more compact DNG format (1/4 of size).

This should be easy to do. All you need is simulate a mild antialias filter and pick a single color channel for each pixel.

Of course, a certain amount of information would be destroyed in doing so. But not that much because it is mostly "empty information" constructed from the original RAW data. So, it should be mostly reconstructable again.


Of course, I cannot really know because I couldn't find a converter allowing me to check it out... (Rumors are that an early version of LR actually could do it, though )

04-09-2009, 05:23 PM   #32
Veteran Member
Marc Sabatella's Avatar

Join Date: Sep 2006
Location: Denver, CO
Photos: Gallery
Posts: 10,685
QuoteOriginally posted by falconeye Quote
So, I was looking for software which creates a Bayer pattern image from the full RGB data and writes this much more compact DNG format (1/4 of size).
I think I see what you're getting at. You are talking abut creating a new "RAW" image (not *really* RAW, of course - more like twice-cooked! - but in a "RAW" format!) that doesn't reproduce the original RAW information, but *could* be used to regenerate the same RGB image that's represented by the TIFF.

No doubt it might be possible after a fashion. But I am not sure why you think this would be 1/4 the size of the TIFF. After all, what you are describing is just a another form of compression, really. If it were that easy, wouldn't someone have already implemented? Although perhaps you mean a lossy compession:

QuoteQuote:
This should be easy to do. All you need is simulate a mild antialias filter and pick a single color channel for each pixel. Of course, a certain amount of information would be destroyed in doing so. But not that much because it is mostly "empty information" constructed from the original RAW data. So, it should be mostly reconstructable again.
I suspect you'd be giving up more information than you would want if all you did was *throw away* two-thirds of the information in your TIFF (eg, the two unselected color channels). That's not "empty information" - it was interpolated quite carefully from the original Bayer pattern data. It would probably be possible to come up with a "reverse demosaicing" algorithm (a "mosaicing" algorithm, I guess!) that minimized the amount of data loss while still achieving a decent compression ratio. But I'm not sure you'd ever do nearly as well as the lossy compression employed by JPEG, which is pretty sophisticated and quite effective but works on a somewhat different principle.

Disclaimer: I am (was, anyhow) a programmer, but my programming experience is not in image processing. I know just enough to be suspicious that what you are describing wouldn't actually work as well as you might hope, but also enough to know that I don't really know :-)
04-10-2009, 04:50 AM   #33
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by Marc Sabatella Quote
Disclaimer: I am (was, anyhow) a programmer, but my programming experience is not in image processing. I know just enough to be suspicious that what you are describing wouldn't actually work as well as you might hope, but also enough to know that I don't really know :-)
We are in one boat here

But what are my options to get a smaller than 70MB file per K20D image? These 70MB are already compressed (lossless, LZW/ZIP).

- DNG with remosaicing (15 MB, what we have talked about, some loss, unclear how much exactly).
- JPG (8 Bit only, bad 8x8 block artifacts if you use levels afterwards!)
- JPEG 2000 (.jp2, not read by LR!)
- Microsoft HD Photo (not read by LR!)
- LZW TIFF at 70% the resolution (no!)

Did I miss anything? I really would prefer .jp2 if it were possible. The new JPEG XR may solve all this -- in a couple of years...

Now you understand why I asked for what I asked for.
04-10-2009, 08:37 AM   #34
Veteran Member
Marc Sabatella's Avatar

Join Date: Sep 2006
Location: Denver, CO
Photos: Gallery
Posts: 10,685
QuoteOriginally posted by falconeye Quote
But what are my options to get a smaller than 70MB file per K20D image? These 70MB are already compressed (lossless, LZW/ZIP).
I'd turn that around and ask another question: why do you need such a file? Why isn't having the processed RAW file good enough - why do you need a full resolution / losslessly compressed RGB version too? Usually, the only reason we need RGB versions of images would be to post online, to take to a third party printer, or to otherwise share with someone else. It's pretty rare that any of these applications would actually require a full resolution / losslessly compressed version of your images. Any images that are going to printed poster size, sure - but presumably you aren't doing that all that often, so it's shouldn't be a big issue to have 70MB versions of those few. And that's assuming 70MB really is the best you that can be done with TIFF lossless compression - have you tried all the different methods of TIFF compression - LZW versus packbits etc? Also, my understanding is that the PNG format offers one of the most efficient lossless compression schemes in the RGB world, so if your printer or client can work with PNG (not that common, but worth a shot), that might yield slightly better file sizes.

Still, I would assume tht the majority of your shots don't need to ever to saved in a full resolution / losslessly comrpessed RGB state. Assuming you've got a RAW processing program that remembers your RAW processing settings for each file (and if you don't, I'd say run, don't walk...), you would only need RGB versions of images you plan to share to share with others, and for most purpose, a reduced resolution JPEG should really be fine, shouldn't it?

QuoteQuote:
- JPG (8 Bit only, bad 8x8 block artifacts if you use levels afterwards!)
Why wouldn't you be doing levels after converting to JPEG when you are shooting RAW? Ideally, you would be doing most of what you need to the original RAW data, and only certain operations would require work post-conversion. For files that you plan to process this way, then yes you might want a full resolution output file since the RAW file itself would no longer contain all the data necessary to regenerate the output as necessary. But since you're done editing at that point, again, JPEG should really be fine "most" of the time.

04-10-2009, 10:37 AM   #35
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by Marc Sabatella Quote
I'd turn that around and ask another question: why do you need such a file? Why isn't having the processed RAW file good enough - why do you need a full resolution / losslessly compressed RGB version too?
Marc, you are absolutely correct.

But unfortunately, you have missed a bit in the thread of discussion above.

Canada_Rockies and myself use DxO as a preprocessor to LR. For some lenses. I for myself do it for the DA 18-250. The improvement in image quality is dramatic, almost unbelievable. All of a sudden, the DA 18-250 looks like a serious lens But this is a story for another day...

So, the DxO step cannot be skipped and the RAWs are gone. And DxO must be first in the workflow as well because it works on calibrated camera/lens profiles only.

So, while I entirely agree to your line of reasoning, it isn't applicable here.

Start to feel my pain?
04-10-2009, 12:24 PM   #36
Veteran Member
Marc Sabatella's Avatar

Join Date: Sep 2006
Location: Denver, CO
Photos: Gallery
Posts: 10,685
Yep, got it now! I guess if it were me I'd do as much as possible in DxO, use JPEG as my output file at a "comfortably low" level of compression, and not output anything from LR except as necessary (eg, reduced sized versins for screen viewing, full size versions for use with external print services that could then be deleted). But I'm probably working to more "casual" standards than you.
04-10-2009, 01:10 PM   #37
Veteran Member




Join Date: Nov 2006
Location: md-usa
Photos: Gallery | Albums
Posts: 2,580
I don't use NR in camera and if I want noise removal Neat Image works really well. I usually set it to only remove 1/2 of the weaker noise and it doesn't hurt the detail much.

04-10-2009, 04:06 PM   #38
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by Marc Sabatella Quote
Yep, got it now! I guess if it were me I'd do as much as possible in DxO, use JPEG as my output file at a "comfortably low" level of compression
Yes, this is an option. But I don't want to become an expert for two programs and so LR is my standard. In DxO, all parameters are flat except for correction of lens flaws. JPG isn't an option because it destroys the dynamic range (8 Bit only). I would have no problems with lossy JPEG2000 compression (maintains the DR). But LR cannot read it

So, for the time being, I except the paradox fact that my less capable lenses must show up as much larger files in LR...
04-10-2009, 07:18 PM   #39
Veteran Member
OregonJim's Avatar

Join Date: Mar 2009
Location: Willamette Valley, Oregon
Posts: 1,327
QuoteOriginally posted by falconeye Quote
\So, I was looking for software which creates a Bayer pattern image from the full RGB data and writes this much more compact DNG format (1/4 of size).

This should be easy to do. All you need is simulate a mild antialias filter and pick a single color channel for each pixel.
falconeye, I understand your logic. It's an interesting theory, but I think the re-mosaicing process would cause as much information loss as the JPEG compression algorithm (no hard data to back this up, just a gut feeling from playing around with similar ideas).

I think you would get less loss of detail by simply re-applying the Bayer pattern and stripping the unused color channels. Sure, LR will de-mosaic the image again when you import it, but a touch of Gaussian blur would likely nullify any artifacts.
04-12-2009, 05:14 AM   #40
Veteran Member
falconeye's Avatar

Join Date: Jan 2008
Location: Munich, Alps, Germany
Photos: Gallery
Posts: 6,871
QuoteOriginally posted by OregonJim Quote
It's an interesting theory, but I think the re-mosaicing process would cause as much information loss as the JPEG compression algorithm
IMHO, JPG is ruled out (as an archive format) because it compresses DR to 8 Bit (Dmax=2.4 only).
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
aperture, camera, iso, jpeg, ninja, noise, photography, photoshop, pixel, reduction, shot, shots
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Noise Reduction in K-x wed7 Pentax DSLR Discussion 4 09-26-2010 12:11 AM
Noise reduction? PHOTOCOP Digital Processing, Software, and Printing 17 04-10-2010 04:49 AM
K-X setup for Noise Reduction? newarts Pentax DSLR Discussion 4 02-09-2010 03:07 PM
Noise reduction richardsmith Digital Processing, Software, and Printing 4 10-26-2009 06:01 PM
Noise reduction on or off? mikeatnite Pentax DSLR Discussion 2 01-28-2007 04:10 AM



All times are GMT -7. The time now is 05:02 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top