Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
04-11-2020, 03:59 PM   #1
Pentaxian
BruceBanner's Avatar

Join Date: Dec 2015
Posts: 5,200
PixelShift to DNG/LibRaw LLC

So I was alerted to this little free program, I believe the intent is to assist cameras taking pixelshift shots (Pentax included) to being a 'proper' singe DNG file from which Raw Developers like Lightroom/Photoshop can properly handle and exploit the benefits.

I'm a little confused because a Pentax pixelshift image is already a DNG, mostly I take this shot to RawTherapee for some advanced Motion Correction before saving as a Tiff and then editing the Tiff file in LR. I really have limited experience in using Pixelshift in a more 'still life' situation where a trip to RawTherapee for motion correction is not required, but I thought taking the native pentax pixelshift DNG file to LR to work on worked anyway, but perhaps not? Would LR do its thing and just show only one of the 4 DNG frames and you're really only working and editing on that? Whereas at least with RawTherapee's process of ending with a Tiff you're actually deriving the benefits of the 4 frames (provided Motion Correction did not steal away too much information from the other frames)?

My only gripe with RawTherapee is the type of Tiff it provides, it's not a particularly great one in a sense that I feel I need to do some exposure correction before saving and having it to further work on in LR, that and the fact that many of my presets and LUTs prefer a DNG vs a Tiff to work with.

So i'm just wondering if anyone is using this small program and if I am correct in saying that taking a native pixelshift DNG file to LR actually is pointless before taking the DNG file elsewhere first to 'squish' the 4 frames properly so that LR/PS can actually derive the benefit?

Happy Easter!

BB

04-11-2020, 04:22 PM   #2
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2015
Location: Melbourne
Posts: 887
QuoteOriginally posted by BruceBanner Quote
My only gripe with RawTherapee is the type of Tiff it provides, it's not a particularly great one in a sense that I feel I need to do some exposure correction before saving and having it to further work on in LR, that and the fact that many of my presets and LUTs prefer a DNG vs a Tiff to work with.
I don't really want to keep blathering on about dcraw, but if you want some control over your TIFF, it may be worth experimenting with it.
I would also add that RawTherapee is more than likely using dcraw, but just not giving you access to all the options...and if you use dcrawps it also handles pixelshift and gives some control over motion correction.

Simply entering dcraw in a console/terminal window will give you all the options.

Cheers,
Terry
04-11-2020, 04:22 PM   #3
Pentaxian
photoptimist's Avatar

Join Date: Jul 2016
Photos: Albums
Posts: 4,462
Yes, the raw pixelshift image is a DNG, but the DNG "standard" is sufficiently complex that individual camera makers can easily create valid DNG files that other software makers' apps cannot properly handle or can't make best use of the data.

As you mentioned, in the case of pixelshift DNG, the single DNG file actually contains four DNG images. Apparently, many image processing apps don't know how to handle this and only access the first of the four DNGs. So if you want the added resolution and noise reduction of pixelshift, it has to first be processed by software that knows how to squish the four DNG images into one TIFF.

I've never used PixelShift2DNG, but it looks like it is able to unpack the pixelshift DNG into four individual DNG files for follow-on processing. That would let you pick which of the four images in a pixelshift set was the best one. You might also find that feeding the four unpacked DNGs in to some image stacking software can do a decent job of squishing although the results may not be quite as good as a dedicated squisher that knows what pixelshift is all about.

Happy Easter!
04-11-2020, 04:43 PM - 1 Like   #4
Pentaxian
BruceBanner's Avatar

Join Date: Dec 2015
Posts: 5,200
Original Poster
QuoteOriginally posted by tduell Quote
I don't really want to keep blathering on about dcraw, but if you want some control over your TIFF, it may be worth experimenting with it.
I would also add that RawTherapee is more than likely using dcraw, but just not giving you access to all the options...and if you use dcrawps it also handles pixelshift and gives some control over motion correction.

Simply entering dcraw in a console/terminal window will give you all the options.

Cheers,
Terry
I have not forgotten about dcraw, it's just I'm not proficient with commandlines and prefer a GUI, plus with RawTherapee and its excellent (can't be beaten) post Motion Correction capabilities (whereby you can select how much it corrects, see the results in a mask on the image etc) has simply prompted me to stick with this. But as I said... ending up with a Tiff is not always ideal for me.

QuoteOriginally posted by photoptimist Quote
Yes, the raw pixelshift image is a DNG, but the DNG "standard" is sufficiently complex that individual camera makers can easily create valid DNG files that other software makers' apps cannot properly handle or can't make best use of the data.

As you mentioned, in the case of pixelshift DNG, the single DNG file actually contains four DNG images. Apparently, many image processing apps don't know how to handle this and only access the first of the four DNGs. So if you want the added resolution and noise reduction of pixelshift, it has to first be processed by software that knows how to squish the four DNG images into one TIFF.

I've never used PixelShift2DNG, but it looks like it is able to unpack the pixelshift DNG into four individual DNG files for follow-on processing. That would let you pick which of the four images in a pixelshift set was the best one. You might also find that feeding the four unpacked DNGs in to some image stacking software can do a decent job of squishing although the results may not be quite as good as a dedicated squisher that knows what pixelshift is all about.

Happy Easter!
Ah, but I think this program is squishing the four DNG images into one DNG rather than Tiff? That would be preferable in some instances for my PP workflow.

I don't think that's what the program is doing, unpacking the four and letting you use one of the files, I think it's actually making a bonafide squishyness four DNG into one DNG for the use of deriving the benefit of pixelshift. If it simply allows you to strip away 3 (or bin them) and use the single DNG file then its not really pixelshifting anything and there is no benefit at all (except perhaps salvaging a shot whereby there was too much motion in the shot). If it's unpacking the Pentax DNG into 4 separate DNG files to use, then that also seems a little redundant a feature, perhaps we can take the 4 files to LR, use HDR merge but uncheck AutoAlign and Auto settings and just get a DNG file to work with that has the layers combined with the 1 pixel deviation (thus benefiting from the pixelshift concept)?

---------- Post added 04-12-20 at 10:08 AM ----------

Just had a quick test shot, handed it a 144mb DNG pixelshifted file, it detected pixelshift and spat out a single new DNG file of 75mb. I would say it's doing what I think, it's squishing the 4 DNG to making a single DNG with the pixelshift feature intact for deriving benefit in a Raw developer such as LR. Time to test now...

04-11-2020, 06:17 PM - 1 Like   #5
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2015
Location: Melbourne
Posts: 887
QuoteOriginally posted by BruceBanner Quote
I have not forgotten about dcraw, it's just I'm not proficient with commandlines and prefer a GUI
That's OK, the command line doesn't seem to be for everyone.

QuoteQuote:
...plus with RawTherapee and its excellent (can't be beaten) post Motion Correction capabilities (whereby you can select how much it corrects, see the results in a mask on the image etc)
You've just described the capabilities of dcrawps :-)

Cheers,
Terry
04-11-2020, 07:27 PM   #6
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,038
QuoteOriginally posted by BruceBanner Quote
to being a 'proper' singe DNG file from which Raw Developers like Lightroom/Photoshop can properly handle and exploit the benefits.
If that is what it does, then it is doing the development and merge and repackaging the resulting (TIFF?) as DNG*...i.e. no longer RAW.

As for your presets and LUTS...they all work against a rendered image, not the RAW data. They work better with RAW or TIFF because they have the full 14 bits of data.


Steve

* DNG is a packaging specification and is not limited to RAW capture data.

Last edited by stevebrot; 04-11-2020 at 07:44 PM.
04-11-2020, 07:52 PM   #7
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,038
QuoteOriginally posted by photoptimist Quote
As you mentioned, in the case of pixelshift DNG, the single DNG file actually contains four DNG images. Apparently, many image processing apps don't know how to handle this and only access the first of the four DNGs.
The reason why this happens is that Ricoh played loose* with the DNG specification such that PS DNG files are not well-formed and valid with three of the four nested files being invisible to DNG compliant tools.


Steve

* Would require a lot of keystrokes to fully explain.
04-11-2020, 08:18 PM - 3 Likes   #8
Pentaxian
BruceBanner's Avatar

Join Date: Dec 2015
Posts: 5,200
Original Poster
QuoteOriginally posted by stevebrot Quote
If that is what it does, then it is doing the development and merge...i.e. no longer RAW.

As for your presets and LUTS...they all work against a rendered image, not the RAW data. They work better with RAW or TIFF because they have the full 14 bits of data.


Steve
This is where I get a bit confused because my head thinks of DNG as being RAW still.

In essence how is a program that is merging 4 DNG files together in pixelshift fashion to spitting out a DNG vs RT that takes 4 DNG files and merges to spitting out a Tiff any better? In this instance isn't it more of a case of different containers for the RAW data? Is there any such thing as a pixelshifted RAW file? It seems to get one to work with in the first place means having 4 RAW files, to combine the 4 RAW files (either in camera like Pentax does) or through a program like RT immediately moves away from 'RAW' in the purest sense as its a merge be it DNG output (PS2DNG) or Tiff.

Would it be better to think of Pixelshift as a bracketed shot. Pentax simply bundles the RAW files together to making one RAW file (which you can split in DCU5 I think and RT at least?), as long as you stitch the images together (like Super Resolution) you gain the benefits of the process but have now begun to move away from RAW by doing so (like a HDR Merge of bracketed shots...)?

Anyway...

Here's my preliminary findings anyway that I thought I would share;

A pixelshifted file, starting as 144mb and using PS2DNG (PixelShift to DNG) to generate a single 75mb DNG file


A secondary non related non pixelshifted shot taken at the same time with as close to exact settings used for the pixelshifted shot as possible (i.e. it is not just one of the frames from the pixelshifted shot above but an actual separate shot taken).


I don't expect you to see much difference here in forum so you should be able to click the files and zoom in a bit more.

For treatment in PP I Autosynced the files in LR and used Auto Settings to bring exposures to being more balanced (I learned later however that the Auto Settings in this instance gave the files slightly different values, more on this later). I then also increased Sharpening to both files to being;

Amount: 100 (default is 40)
Radius: 1.5
Detail: 30
Masking: 0

The reason I do this additional sharpening has been in due to past experience of seeing lower noise brought into the image when increasing sharpening vs doing the same with a non pixelshifted shot.

Here is a screenshot of the two files compared at 1:1 magnification, the pixelshifted shot from PS2DNG on the right (file has a '(1)' in brackets);



I've circled red some areas that I think show an improvement on the PS2DNG file vs native. I think the purple textured brick and the red musical note brick show increased clarity and sharpness, I'm also seeing less noise in the shadows.

You might be able to inspect a bit better from these shots;

PS2DNG version;


native version;


Once I realised that the native dng file was given slightly different Auto Settings to the PS2DNG file I went about changing those values to match the exact same values as seen in the PS2DNG file. I didn't notice a substantial difference to make things any different on the above scenarios.

I then decided to push shadows to the max and compare how they looked, here's a screen grab (again PS2DNG file on the right);


I circled red again on the area that I feel the PS2DNG was doing better. Not only was the shadow area cleaner in noise, it seemed to have more detail, less backs (could see more).

I inspected another area of the image, this time some different colours;



Again I was seeing an improvement in the shadows, the left native image showing deeper blacks vs the PS2DNG showing lighter brighter shadows with less noise, increased perceived sharpness and details. I was also noticed the oof areas seemed much cleaner and I think this is where pixelshift can really excel. It's one thing to push sharp things sharper and have some noise come into the image, but often noise in bokeh is quite unpleasant. It can be fine, if it's a high quality noise such as authentic grain or well handled PP grain added, but typically smooth noiseless bokeh is generally welcomed and it seems again the PS2DNG does a better job in this regard.

Apologies for not uploading those parts in higher res for inspecting closer.


Summary

At this point in time the PS2DNG is seemingly doing something better than the native file can, how much better I'm not really sure, and if it's better than using RT or dcraw to generate a Tiff to play with is still not known to me at this point in time.

04-11-2020, 08:38 PM   #9
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,038
QuoteOriginally posted by BruceBanner Quote
At this point in time the PS2DNG is seemingly doing something better than the native file can, how much better I'm not really sure, and if it's better than using RT or dcraw to generate a Tiff to play with is still not known to me at this point in time.
RT uses dcraw as its converter and for PS is equivalent to dcrawps with a few bells and whistles. PS2DNG is made by the folk who brought us LibRaw, a competing open source RAW converter based on dcraw, but with tweaks. Whether PS2DNG works by merging prior to or after Bayer interpolation is the question. If prior, then the resulting DNG contains voltage, not pixel data. I will have to play with its output using both RawDigger and dcraw.


Steve
04-11-2020, 08:41 PM   #10
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 38,038
QuoteOriginally posted by BruceBanner Quote
This is where I get a bit confused because my head thinks of DNG as being RAW still.
Just remember that Lightroom can convert a TIFF on import to DNG as can the standalone DNG converter.

Edit: I did some looksee and found that my claim was inaccurate on both counts. Lightroom can package TIFF as DNG on export and the DNG Converter works only on RAW files.


Steve

Last edited by stevebrot; 04-15-2020 at 09:53 PM. Reason: Accuracy
04-11-2020, 08:43 PM   #11
Pentaxian
BruceBanner's Avatar

Join Date: Dec 2015
Posts: 5,200
Original Poster
QuoteOriginally posted by stevebrot Quote
RT uses dcraw as its converter and for PS is equivalent to dcrawps with a few bells and whistles. PS2DNG is made by the folk who brought us LibRaw, a competing open source RAW converter based on dcraw, but with tweaks. Whether PS2DNG works by merging prior to or after Bayer interpolation is the question. If prior, then the resulting DNG contains voltage, not pixel data. I will have to play with its output using both RawDigger and dcraw.


Steve
Well if u have time, please do play and get back to me

Are you also saying I can take the Tiff file spat out from RT, import into LR and have it be a DNG rather than Tiff for me to work on?
04-11-2020, 09:21 PM   #12
Pentaxian




Join Date: Feb 2015
Photos: Gallery | Albums
Posts: 7,914
I had tried the ps2dng program, saw no difference of its dng output compare to the original 4 image os file, both opened for processing in silkypix. I though ps2dng does the ps rgb merge into a single rgb dng by using camera parameters but letting no ps mode control evailable to user. Silkypix also does the same at reading the 4 frames, using camera parameter and not giving control to user. So i decided to uninstall ps2dng. Still i'd be curious to know what ps2dng actually does, still a bit of a mystery for me. I thought ps2dng is for Sony users who get 4 separate ps frames, and no real benefit for pentax.
04-11-2020, 11:08 PM - 1 Like   #13
Pentaxian
BruceBanner's Avatar

Join Date: Dec 2015
Posts: 5,200
Original Poster
QuoteOriginally posted by biz-engineer Quote
I had tried the ps2dng program, saw no difference of its dng output compare to the original 4 image os file, both opened for processing in silkypix. I though ps2dng does the ps rgb merge into a single rgb dng by using camera parameters but letting no ps mode control evailable to user. Silkypix also does the same at reading the 4 frames, using camera parameter and not giving control to user. So i decided to uninstall ps2dng. Still i'd be curious to know what ps2dng actually does, still a bit of a mystery for me. I thought ps2dng is for Sony users who get 4 separate ps frames, and no real benefit for pentax.
It's definitely doing something, and its definitely marketed towards Pentax users. If you cannot see the differences I speak of in the above examples I would suggest you do some more testing. The issue for me is two things;

1) Motion Correction. Relying on the cameras built in Motion Correction rather than using RawTherapee's better clean up job makes me feel the PS2DNG is only suitable for scenarios where movement has not come into play, such as still life work.

2) Is a PS2DNG any better or equal to using say silkypix or RawTherapee to process the Pixelshifted DNG file (regardless of motion)? I see some advantages in processing a pixelshift file as a DNG vs Tiff, but that might not be important to some.

Thus far I can absolutely confirm from ONE test that the 75mb PS2DNG file is superior to the single frame regular normal DNG file in terms of pixelshift advantages, but it might be a scale, and that DCU5/Silkypix/RT do it better.

Last edited by BruceBanner; 04-11-2020 at 11:14 PM.
04-11-2020, 11:12 PM   #14
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2015
Location: Melbourne
Posts: 887
QuoteOriginally posted by BruceBanner Quote
Are you also saying I can take the Tiff file spat out from RT, import into LR and have it be a DNG rather than Tiff for me to work on?
DNG is both a raw image format and a format that supports "non-raw", or partly processed, images, and conforms to the TIFF spec.
As I understand it a DNG file can be a container for TIFF images.
I don't really see any advantage in having LR generate a DNG from an imported TIFF, I think it would be essentially the same data, just packaged differently.

PS. I was going to have a look at PS2DNG and do some comparisons with dcrawps output, but now see that it is only available for Windows and MacOS, so that's put an end to that!

Cheers,
Terry

Last edited by tduell; 04-11-2020 at 11:26 PM. Reason: Additional info
04-11-2020, 11:56 PM   #15
Pentaxian
BruceBanner's Avatar

Join Date: Dec 2015
Posts: 5,200
Original Poster
QuoteOriginally posted by tduell Quote
DNG is both a raw image format and a format that supports "non-raw", or partly processed, images, and conforms to the TIFF spec.
As I understand it a DNG file can be a container for TIFF images.
I don't really see any advantage in having LR generate a DNG from an imported TIFF, I think it would be essentially the same data, just packaged differently.

PS. I was going to have a look at PS2DNG and do some comparisons with dcrawps output, but now see that it is only available for Windows and MacOS, so that's put an end to that!

Cheers,
Terry
It's to do with presets and LUTs, see here;

Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
benefit, correction, dcraw, dng, files, frame, frames, icon, image, images, lr, motion, pentax, photography, photoshop, pixel, pixelshift, pixelshift to dng/libraw, profile, rawtherapee, shift, shot, software, tiff, time
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Adobe DNG vs Pentax's DNG dflorez Digital Processing, Software, and Printing 8 12-07-2015 11:34 AM
Start the buisness, sole proprietorship or LLC ?? ChallengedOne General Talk 4 04-23-2011 09:24 PM
Linux -- Ubuntu Karmic 9.10, libraw 0.8.5, Raw file processing K7er Digital Processing, Software, and Printing 4 01-13-2010 08:35 PM
DNG in-camera vs DNG from Lightroom? rfortson Digital Processing, Software, and Printing 9 11-02-2008 11:06 PM



All times are GMT -7. The time now is 04:34 PM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top