Pixel Shift Corruption?

amoringello · 01-24-2016, 05:25 PM

Anyone else seeing pixel-shift images being corrupted after exporting from Pentax's Digital Camera Utility?

The following was shot on a tripod with 2-second timer, Manual Focus, using raw mode. (K3-ii)
The camera was not bumped. (and if it were, I would expect a lot worse results).

Any idea what is happening and has anyone else seen something like this? I have not heard of any such incidents before.

The first 100% crop is a section from the preview, the second is from the exported 16-bit TIFF.
(apparently 100% is DCU is not 100%,hmmm Anyway, you'll get the idea.)

This section and one other section of smaller area are affected.
The rest of the image is perfectly aligned and super detailed.

Last edited by amoringello; 01-24-2016 at 05:32 PM.

Scootatheschool1990 · 01-24-2016, 05:34 PM

I know nothing about this, but are you saying the k3ii has the ability to pixel shift?

amoringello · 01-24-2016, 05:45 PM

Originally posted by Scootatheschool1990

I know nothing about this, but are you saying the k3ii has the ability to pixel shift?

Yes, K3-ii has pixel shift.

Scootatheschool1990 · 01-24-2016, 05:53 PM

Originally posted by amoringello

Yes, K3-ii has pixel shift.

Wow. I'm behind the times. Thanks for bringing this to my attention. I wish I could help!

bar_foo · 01-24-2016, 06:15 PM - **1 Like**

That's a well-known side effect of absinthe.
Seriously, there's clearly a bug that has caused the software to misidentify the placement of that chunk of the pipe: the highly contrasted midline and the bottom have been confused by the software. Don't have a K-3 ii so no idea what to do about it--are there any options (such as anti-ghosting) you can adjust in the software?

Scootatheschool1990 · 01-25-2016, 12:10 AM

Originally posted by bar_foo

That's a well-known side effect of absinthe.
Seriously, there's clearly a bug that has caused the software to misidentify the placement of that chunk of the pipe: the highly contrasted midline and the bottom have been confused by the software. Don't have a K-3 ii so no idea what to do about it--are there any options (such as anti-ghosting) you can adjust in the software?

HA HA HA HA HA HA! Good one!

**Oakland Rob** · 01-25-2016, 02:12 PM

Have you tried processing it with something besides DCU?

Maybe try dcraw or ACR? or put up the raw image so others could try?

amoringello · 01-25-2016, 02:24 PM

ACR does not process pixel shift, or does not appear to do so.
It happily reads one image from the set, but does not process the entire set into a final, more detailed version. i.e. There is no difference between a pixel-shift and a non pixel-shift image when viewed side by side.

Does DCRAW do more than read a single image? Does it actually process pixel shift images? From what I hear other people say about it, it seems ambiguous as to whether they are actually seeing full and correct processing. (as opposed to the simple single-image).
Being so small, I do not see how DCRAW could do that, but maybe one of the libraries it includes has that capability?

Will DCRAW run on a Mac? (Some linux software can)
If DCRaw is an option, I might fire up a new Linux virtual machine and give it a try, but I'd hate to waste the hours of download, install and setup of the OS just to find it isn't any better than Lightroom. :-)

**microlight** · 01-25-2016, 02:32 PM

What happens if you develop the PSR RAW into a TIFF in-camera? Do you still get the same corrupted image?

amoringello · 01-25-2016, 03:35 PM

Originally posted by microlight

What happens if you develop the PSR RAW into a TIFF in-camera? Do you still get the same corrupted image?

OK, I feel stupid! I did not know you could in-camera process to TIFF. I knew you could to JPEG, of course. Wow,very cool!
Thanks!

The resulting TIF is not only in perfect condition, but it is even much sharper than the one processed by DCU!

Pro: no corruption, an even sharper resulting image
Con: it only generates an 8-bit TIFF

In this instance, I do a lot of post processing so having 16bit color is going to be a necessity.
At least I know for sure now that it is DCU and not that some gremlin bumped the camera between shots. :-)
I will probably contact Ricoh to send a bug report.

---------- Post added 01-25-16 at 05:39 PM ----------

Originally posted by bar_foo

That's a well-known side effect of absinthe.
Seriously, there's clearly a bug that has caused the software to misidentify the placement of that chunk of the pipe: the highly contrasted midline and the bottom have been confused by the software. Don't have a K-3 ii so no idea what to do about it--are there any options (such as anti-ghosting) you can adjust in the software?

Nice one! I would not think a lot of people would be familiar with what that was, given the small portion of the image.

**jbinpg** · 01-26-2016, 02:15 AM

The stock dcraw can not process a Pentax pixel-shifted image. However, user tomtor here on PF patched dcraw to do so and calls it dcrawps. His version can be found here in zip format:

https://github.com/tomtor/dcrawps

It can separate the 4-files of a pixel-shifted image *and* when processing the stack, it can also repair any ghosting artifacts by replacing the artifacts with bayer-interpolated pixels from the first image. The user is in control of how much replacement is required. Some examples of dcrawps usage can be found in the thread https://www.pentaxforums.com/forums/172-pentax-k-3/311865-pixel-shift-finicky-3.html#post3502137 Command line only at this time but known to work on linux and windows. It outputs 16-bit images.

Jack

**Oakland Rob** · 01-26-2016, 09:26 AM

Originally posted by amoringello

ACR does not process pixel shift, or does not appear to do so.
It happily reads one image from the set, but does not process the entire set into a final, more detailed version. i.e. There is no difference between a pixel-shift and a non pixel-shift image when viewed side by side.

Does DCRAW do more than read a single image? Does it actually process pixel shift images? From what I hear other people say about it, it seems ambiguous as to whether they are actually seeing full and correct processing. (as opposed to the simple single-image).
Being so small, I do not see how DCRAW could do that, but maybe one of the libraries it includes has that capability?

Will DCRAW run on a Mac? (Some linux software can)
If DCRaw is an option, I might fire up a new Linux virtual machine and give it a try, but I'd hate to waste the hours of download, install and setup of the OS just to find it isn't any better than Lightroom. :-)

ACR 9.3.1 DOES process pixel shift. It has on my Mac. It was a fairly recent update.

The regular dcraw will just process one frame, if you will, from the PS image.

The PS dcraw fork will process the PS images and merge them, and has more control over the process than ACR, and can also separate out the four images. I haven't used it that much, but it might be the better option where you've got movement in your PS images, which pretty much wrecks them.

If you go to the thread about finicky PS mentioned here towards the end I have a link with instructions on compiling the PS dcraw to run on a Mac. Regular dcraw is available through MacPorts.

amoringello · 01-26-2016, 10:22 AM

Originally posted by Oakland Rob

ACR 9.3.1 DOES process pixel shift. It has on my Mac. It was a fairly recent update.

The regular dcraw will just process one frame, if you will, from the PS image.

The PS dcraw fork will process the PS images and merge them, and has more control over the process than ACR, and can also separate out the four images. I haven't used it that much, but it might be the better option where you've got movement in your PS images, which pretty much wrecks them.

If you go to the thread about finicky PS mentioned here towards the end I have a link with instructions on compiling the PS dcraw to run on a Mac. Regular dcraw is available through MacPorts.

Thanks for the info for compiling on a Mac I will check that out. I hate to take the time to go through getting a Linux VM up and running.

What did you do to get ACR to handle PixelShift properly?
I have PS/LR CC with latest updates. Lightroom states ACR 9.3, but ACR directly reports version 9.3.1.
Both produce identical non-pixel-shift results. i.e. the images are much less detailed than true pixel-shift images.

---------- Post added 01-26-16 at 12:37 PM ----------

Originally posted by Oakland Rob

If you go to the thread about finicky PS mentioned here towards the end I have a link with instructions on compiling the PS dcraw to run on a Mac. Regular dcraw is available through MacPorts.

Ha! That was simple! I tried compiling earlier, but I didn't see where the NO_JASPER and the rest were fully handled (honestly did not look that hard, as I didn't expect it) :-)

Looks like there is some movement or differences between images? Not sure if this is enough to cause the behavior.

**Oakland Rob** · 01-27-2016, 11:01 AM

I can't really say why you're having trouble with ACR. Maybe you are getting correct results, but just don't like them. I tested with an image with movement, and see artifacts from each separate frame, so it's clearly using the different frames. I actually got better results at times with ACR than dcrawps, but maybe that's cuz I applied a lot of sharpening in ACR, which is what I'd do in DCU.

The first image is dcrawps with no settings; the second ACR. Obviously WB wasn't set in dcrawps so ignore that.

amoringello · 01-27-2016, 11:42 AM

Originally posted by Oakland Rob

I can't really say why you're having trouble with ACR. Maybe you are getting correct results, but just don't like them. I tested with an image with movement, and see artifacts from each separate frame, so it's clearly using the different frames. I actually got better results at times with ACR than dcrawps, but maybe that's cuz I applied a lot of sharpening in ACR, which is what I'd do in DCU.

The first image is dcrawps with no settings; the second ACR. Obviously WB wasn't set in dcrawps so ignore that.

hmmm, you could be right. Maybe it's me.
I'll try that and see if I get similar results with movement.

As it appears so far, the image is quite bland and non-sharp compared to what is exported from DCU and from an in-camera conversion. Honestly, the dcrawps exported version was also rather lack-luster in comparison. Looking very similar to ACR and thus visually identical to the straight single shot image.

So it might just be my expectations??? Just not sure why the discrepancy...
In-camera processing adds a lot of sharpening. Perhaps DCU does, also? I just cannot reproduce enough sharpening or clarity to get similar detail in ACR, Lightroom or Photoshop.

Anyway, I'll play around a bit more.

01-24-2016, 05:25 PM	#1
amoringello Veteran Member Join Date: May 2008 Location: Virginia, USA Photos: Gallery Posts: 1,562	Pixel Shift Corruption? Anyone else seeing pixel-shift images being corrupted after exporting from Pentax's Digital Camera Utility? The following was shot on a tripod with 2-second timer, Manual Focus, using raw mode. (K3-ii) The camera was not bumped. (and if it were, I would expect a lot worse results). Any idea what is happening and has anyone else seen something like this? I have not heard of any such incidents before. The first 100% crop is a section from the preview, the second is from the exported 16-bit TIFF. (apparently 100% is DCU is not 100%,hmmm Anyway, you'll get the idea.) This section and one other section of smaller area are affected. The rest of the image is perfectly aligned and super detailed. Last edited by amoringello; 01-24-2016 at 05:32 PM.

01-26-2016, 10:22 AM	#13
amoringello Veteran Member Join Date: May 2008 Location: Virginia, USA Photos: Gallery Posts: 1,562 Original Poster	Originally posted by Oakland Rob ACR 9.3.1 DOES process pixel shift. It has on my Mac. It was a fairly recent update. The regular dcraw will just process one frame, if you will, from the PS image. The PS dcraw fork will process the PS images and merge them, and has more control over the process than ACR, and can also separate out the four images. I haven't used it that much, but it might be the better option where you've got movement in your PS images, which pretty much wrecks them. If you go to the thread about finicky PS mentioned here towards the end I have a link with instructions on compiling the PS dcraw to run on a Mac. Regular dcraw is available through MacPorts. Thanks for the info for compiling on a Mac I will check that out. I hate to take the time to go through getting a Linux VM up and running. What did you do to get ACR to handle PixelShift properly? I have PS/LR CC with latest updates. Lightroom states ACR 9.3, but ACR directly reports version 9.3.1. Both produce identical non-pixel-shift results. i.e. the images are much less detailed than true pixel-shift images. ---------- Post added 01-26-16 at 12:37 PM ---------- Originally posted by Oakland Rob If you go to the thread about finicky PS mentioned here towards the end I have a link with instructions on compiling the PS dcraw to run on a Mac. Regular dcraw is available through MacPorts. Ha! That was simple! I tried compiling earlier, but I didn't see where the NO_JASPER and the rest were fully handled (honestly did not look that hard, as I didn't expect it) :-) Looks like there is some movement or differences between images? Not sure if this is enough to cause the behavior. Last edited by amoringello; 01-26-2016 at 10:41 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Pixel Shift is FINICKY!	Fenwoodian	Pentax K-3 & K-3 II	48	03-31-2016 07:41 AM
Pixel Shift	brophyart	Troubleshooting and Beginner Help	6	10-31-2015 02:53 PM
Pixel shift for landscapes	seachongo	Pentax DSLR Discussion	12	06-25-2015 03:25 PM
Pixel shift practical applications	Spodeworld	Pentax K-3 & K-3 II	20	04-25-2015 10:01 AM
Pixel shift and bracketing	Spodeworld	Pentax K-3 & K-3 II	9	04-23-2015 08:56 PM

01-24-2016, 05:34 PM	#2
Scootatheschool1990 Veteran Member Join Date: Jun 2012 Location: Boise Idaho Photos: Gallery \| Albums Posts: 466	I know nothing about this, but are you saying the k3ii has the ability to pixel shift?

01-24-2016, 05:45 PM	#3
amoringello Veteran Member Join Date: May 2008 Location: Virginia, USA Photos: Gallery Posts: 1,562 Original Poster	Originally posted by Scootatheschool1990 I know nothing about this, but are you saying the k3ii has the ability to pixel shift? Yes, K3-ii has pixel shift.

01-24-2016, 05:53 PM	#4
Scootatheschool1990 Veteran Member Join Date: Jun 2012 Location: Boise Idaho Photos: Gallery \| Albums Posts: 466	Originally posted by amoringello Yes, K3-ii has pixel shift. Wow. I'm behind the times. Thanks for bringing this to my attention. I wish I could help!

01-24-2016, 06:15 PM - 1 Like	#5
bar_foo Senior Member Join Date: Feb 2014 Location: Vancouver, BC Posts: 108	That's a well-known side effect of absinthe. Seriously, there's clearly a bug that has caused the software to misidentify the placement of that chunk of the pipe: the highly contrasted midline and the bottom have been confused by the software. Don't have a K-3 ii so no idea what to do about it--are there any options (such as anti-ghosting) you can adjust in the software?

01-25-2016, 12:10 AM	#6
Scootatheschool1990 Veteran Member Join Date: Jun 2012 Location: Boise Idaho Photos: Gallery \| Albums Posts: 466	Originally posted by bar_foo That's a well-known side effect of absinthe. Seriously, there's clearly a bug that has caused the software to misidentify the placement of that chunk of the pipe: the highly contrasted midline and the bottom have been confused by the software. Don't have a K-3 ii so no idea what to do about it--are there any options (such as anti-ghosting) you can adjust in the software? HA HA HA HA HA HA! Good one!

01-25-2016, 02:12 PM	#7
Oakland Rob Loyal Site Supporter Join Date: Apr 2015 Location: Bay Area California Photos: Gallery \| Albums Posts: 798	Have you tried processing it with something besides DCU? Maybe try dcraw or ACR? or put up the raw image so others could try?

01-25-2016, 02:24 PM	#8
amoringello Veteran Member Join Date: May 2008 Location: Virginia, USA Photos: Gallery Posts: 1,562 Original Poster	ACR does not process pixel shift, or does not appear to do so. It happily reads one image from the set, but does not process the entire set into a final, more detailed version. i.e. There is no difference between a pixel-shift and a non pixel-shift image when viewed side by side. Does DCRAW do more than read a single image? Does it actually process pixel shift images? From what I hear other people say about it, it seems ambiguous as to whether they are actually seeing full and correct processing. (as opposed to the simple single-image). Being so small, I do not see how DCRAW could do that, but maybe one of the libraries it includes has that capability? Will DCRAW run on a Mac? (Some linux software can) If DCRaw is an option, I might fire up a new Linux virtual machine and give it a try, but I'd hate to waste the hours of download, install and setup of the OS just to find it isn't any better than Lightroom. :-)

01-25-2016, 02:32 PM	#9
microlight Site Supporter Join Date: Sep 2011 Location: Hampshire, UK Posts: 2,129	What happens if you develop the PSR RAW into a TIFF in-camera? Do you still get the same corrupted image?

01-25-2016, 03:35 PM	#10
amoringello Veteran Member Join Date: May 2008 Location: Virginia, USA Photos: Gallery Posts: 1,562 Original Poster	Originally posted by microlight What happens if you develop the PSR RAW into a TIFF in-camera? Do you still get the same corrupted image? OK, I feel stupid! I did not know you could in-camera process to TIFF. I knew you could to JPEG, of course. Wow,very cool! Thanks! The resulting TIF is not only in perfect condition, but it is even much sharper than the one processed by DCU! Pro: no corruption, an even sharper resulting image Con: it only generates an 8-bit TIFF In this instance, I do a lot of post processing so having 16bit color is going to be a necessity. At least I know for sure now that it is DCU and not that some gremlin bumped the camera between shots. :-) I will probably contact Ricoh to send a bug report. ---------- Post added 01-25-16 at 05:39 PM ---------- Originally posted by bar_foo That's a well-known side effect of absinthe. Seriously, there's clearly a bug that has caused the software to misidentify the placement of that chunk of the pipe: the highly contrasted midline and the bottom have been confused by the software. Don't have a K-3 ii so no idea what to do about it--are there any options (such as anti-ghosting) you can adjust in the software? Nice one! I would not think a lot of people would be familiar with what that was, given the small portion of the image.

01-26-2016, 02:15 AM	#11
jbinpg Site Supporter Join Date: Mar 2008 Location: Prince George, BC Photos: Gallery \| Albums Posts: 3,546	The stock dcraw can not process a Pentax pixel-shifted image. However, user tomtor here on PF patched dcraw to do so and calls it dcrawps. His version can be found here in zip format: https://github.com/tomtor/dcrawps It can separate the 4-files of a pixel-shifted image and when processing the stack, it can also repair any ghosting artifacts by replacing the artifacts with bayer-interpolated pixels from the first image. The user is in control of how much replacement is required. Some examples of dcrawps usage can be found in the thread https://www.pentaxforums.com/forums/172-pentax-k-3/311865-pixel-shift-finicky-3.html#post3502137 Command line only at this time but known to work on linux and windows. It outputs 16-bit images. Jack

01-26-2016, 09:26 AM	#12
Oakland Rob Loyal Site Supporter Join Date: Apr 2015 Location: Bay Area California Photos: Gallery \| Albums Posts: 798	Originally posted by amoringello ACR does not process pixel shift, or does not appear to do so. It happily reads one image from the set, but does not process the entire set into a final, more detailed version. i.e. There is no difference between a pixel-shift and a non pixel-shift image when viewed side by side. Does DCRAW do more than read a single image? Does it actually process pixel shift images? From what I hear other people say about it, it seems ambiguous as to whether they are actually seeing full and correct processing. (as opposed to the simple single-image). Being so small, I do not see how DCRAW could do that, but maybe one of the libraries it includes has that capability? Will DCRAW run on a Mac? (Some linux software can) If DCRaw is an option, I might fire up a new Linux virtual machine and give it a try, but I'd hate to waste the hours of download, install and setup of the OS just to find it isn't any better than Lightroom. :-) ACR 9.3.1 DOES process pixel shift. It has on my Mac. It was a fairly recent update. The regular dcraw will just process one frame, if you will, from the PS image. The PS dcraw fork will process the PS images and merge them, and has more control over the process than ACR, and can also separate out the four images. I haven't used it that much, but it might be the better option where you've got movement in your PS images, which pretty much wrecks them. If you go to the thread about finicky PS mentioned here towards the end I have a link with instructions on compiling the PS dcraw to run on a Mac. Regular dcraw is available through MacPorts.

01-27-2016, 11:42 AM	#15
amoringello Veteran Member Join Date: May 2008 Location: Virginia, USA Photos: Gallery Posts: 1,562 Original Poster	Originally posted by Oakland Rob I can't really say why you're having trouble with ACR. Maybe you are getting correct results, but just don't like them. I tested with an image with movement, and see artifacts from each separate frame, so it's clearly using the different frames. I actually got better results at times with ACR than dcrawps, but maybe that's cuz I applied a lot of sharpening in ACR, which is what I'd do in DCU. The first image is dcrawps with no settings; the second ACR. Obviously WB wasn't set in dcrawps so ignore that. hmmm, you could be right. Maybe it's me. I'll try that and see if I get similar results with movement. As it appears so far, the image is quite bland and non-sharp compared to what is exported from DCU and from an in-camera conversion. Honestly, the dcrawps exported version was also rather lack-luster in comparison. Looking very similar to ACR and thus visually identical to the straight single shot image. So it might just be my expectations??? Just not sure why the discrepancy... In-camera processing adds a lot of sharpening. Perhaps DCU does, also? I just cannot reproduce enough sharpening or clarity to get similar detail in ACR, Lightroom or Photoshop. Anyway, I'll play around a bit more.