Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
01-16-2021, 01:44 AM   #1
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,173
Pixel shift motion processing technique

I wasn't sure if this was the right section to post, since I'll mention Panasonic pixel shift. If incorrect, pls move to non-Pentax section.

Here is the subject matter: I noticed that Panasonic S series are able to deal perfectly (or almost perfectly) with motion in their pixel shift captures. So I wondered if I could find some kind of white papers or information somewhere so that I could reproduce the Panasonic approach to my Pentax K1 pixel shift files, and doing so get the best of both world: the resolution of the K1 and the pixel shift motion correction of Panasonic, without spending money. So, I tried to do a search with google and other search engines hoping to get some patents or technical papers explaining the approach of Panasonic, but I couldn't find anything. Where could I find to advanced information about what Panasonic are using? If anyone knows? Any idea welcome. Thanks.

01-16-2021, 02:26 AM   #2
Site Supporter
Site Supporter
JensE's Avatar

Join Date: Dec 2015
Location: Leipzig
Photos: Gallery | Albums
Posts: 1,942
Are you aware of Rawtherapee's motion detection capabilities? I don't do a lot of Pixelshift processing, but it so far it has always produced what I wanted with a little tuning of parameters.
01-16-2021, 03:23 AM   #3
Forum Member
syyrmb's Avatar

Join Date: Sep 2020
Location: Qingdao
Posts: 94
RawTherapee has the best motion correction for Pentax pixelshift, heckflosse spent a lot of effort on the MC algorithm and it works very well. Dealing with moving cars, water, leaves, etc. isn't a problem to it.
01-16-2021, 03:51 AM   #4
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2015
Location: Melbourne
Posts: 1,506
dcrawps can also detect and manage motion in pixelshift.

Cheers,
Terry

01-16-2021, 06:08 AM   #5
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,173
Original Poster
With RT, motion correction is a tradeoff between total elimination of motion artifacts and removal of resolution that don't contain any moving parts, or preserve detail with partial removal of moving parts. I don't know if Panasonic are using the same method, would be interesting to know.
01-16-2021, 06:12 AM   #6
Loyal Site Supporter
Loyal Site Supporter




Join Date: Mar 2009
Location: Gladys, Virginia
Photos: Gallery
Posts: 27,603
QuoteOriginally posted by biz-engineer Quote
With RT, motion correction is a tradeoff between total elimination of motion artifacts and removal of resolution that don't contain any moving parts, or preserve detail with partial removal of moving parts. I don't know if Panasonic are using the same method, would be interesting to know.
RT can also do median, which results in some blurring of those areas. My guess is that Panasonic just does what Pentax does with motion correction, which is replace any areas with movement between frames with the first image. Anything more would be more like what Pentax does with "dynamic pixel shift" which is a bit different.
01-16-2021, 09:19 AM   #7
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,173
Original Poster
QuoteOriginally posted by Rondec Quote
My guess is that Panasonic just does what Pentax does with motion correction, which is replace any areas with movement between frames with the first image.
It's possible that Panasonic in-camera processing use more aggressive motion correction (larger mask radius), then Pentax does. The problem with Pentax MC is that it preserve sharpness quite well, but do a bad job at removing motion. I've played around with gimp, to generate a motion mask. What's limited selectivity between motion areas and non motion areas is pixel noise, aggressive motion reduction also removes details in areas that don't contain any moving parts. I was wondering if Panasonic use a special trick (using more than 4 exposures to reduce noise before computing a motion mask, for instance).

The basic problem of Pentax motion correction, which is also a limitation with RT, is that the noise level of a single exposure is higher than the noise level if the 4 PS composite. Therefore, noise differential prevents to detect motion areas very accurately, there is a tradeoff between preserving details and accuracy of motion detection.

My idea was to do two captures (or more) to knock out the noise: a 4 frames stack using composite average mode in RAW, followed by a 4 frames pixel shift with MC disabled, then difference of the 4 frames stack and 4 frame PS, so that the lower noise of the 4 x stack allows for more precise motion detection. Another idea was to capture two (or more) times 4 frames PS without touching the camera (using my IR remote), and then process each PS with upres to 144Mp TIF or RAW, then use Silkypix 10 (SP10) motion detection feature to eliminate parts than moved. SP10 has a feature that automatically detect if parts of images have moved between frames, and it's possible to export the result as RAW DNG... then post process that RAW like a single exposure RAW file.

01-16-2021, 10:28 AM   #8
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 42,007
QuoteOriginally posted by biz-engineer Quote
So I wondered if I could find some kind of white papers or information somewhere so that I could reproduce the Panasonic approach to my Pentax K1 pixel shift files, and doing so get the best of both world
They all work the same way...by not including areas that registered changes between frames by defaulting to the first frame.


Steve
01-16-2021, 10:44 AM   #9
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,173
Original Poster
QuoteOriginally posted by stevebrot Quote
They all work the same way...by not including areas that registered changes between frames by defaulting to the first frame.
Sure. A change (of pixel value) from frame to frame can come from shot noise, or change of light hitting that same pixel area, with no way to distinguish the difference between the two kinds of changes.
If the threshold of motion detect higher than the noise spread, pixels that are only affected by noise aren't replaced by values from frame #1, it misses pixels affected by motion.
If the threshold of motion detect is lower than the noise spread, motion is very well detected, but in that case the pixels value changes from shot noise are also being replaced by pixel values from frame #1, which defeat the purpose of pixel shift. If the noise spread follow a kind of normal distribution, there isn't any threshold at which motion is fully distinguished from noise.

Last edited by biz-engineer; 01-16-2021 at 10:53 AM.
01-16-2021, 11:25 AM   #10
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 42,007
QuoteOriginally posted by biz-engineer Quote
Sure. A change (of pixel value) from frame to frame can come from shot noise, or change of light hitting that same pixel area, with no way to distinguish the difference between the two kinds of changes.
At the single pixel level, no, but any algorithm that operates on that level would be flawed.


Steve
01-16-2021, 12:28 PM   #11
Site Supporter
Site Supporter
JensE's Avatar

Join Date: Dec 2015
Location: Leipzig
Photos: Gallery | Albums
Posts: 1,942
QuoteOriginally posted by biz-engineer Quote
Sure. A change (of pixel value) from frame to frame can come from shot noise, or change of light hitting that same pixel area, with no way to distinguish the difference between the two kinds of changes.
If the threshold of motion detect higher than the noise spread, pixels that are only affected by noise aren't replaced by values from frame #1, it misses pixels affected by motion.
If the threshold of motion detect is lower than the noise spread, motion is very well detected, but in that case the pixels value changes from shot noise are also being replaced by pixel values from frame #1, which defeat the purpose of pixel shift. If the noise spread follow a kind of normal distribution, there isn't any threshold at which motion is fully distinguished from noise.
It doesn't matter whether it's motion or the effect of changing light - there is no way to combine the PS frames in this case that would enhance the image for sure. For changes due to motion, they are normally spatially extended, so adjusting mask blur and sensitivity in Rawtherapee allows to mask moving parts fairly precisely even in the presence of noise - 'correctness', the amount of non-moving part still masked, will depend on the SNR.
01-16-2021, 01:02 PM   #12
Loyal Site Supporter
Loyal Site Supporter
MossyRocks's Avatar

Join Date: Nov 2017
Location: Minnesota
Photos: Gallery | Albums
Posts: 2,978
I've done a number of pixel shift shots with varying degrees of success but excluding operator issues it works pretty good. However out in the outdoors there is still movement that needs dealing with. I use rawtherapee and find I get best results using the custom motion correction option. In the custom motion correction I uncheck the fill in holes in the motion mask, equalize the brightness of the frames, and select median for moving parts. I find for me that usually produces the best results however if there was a good gust of wind or something else that made a mess of things I will often just fall back to using just one image.

If using a very good lens in its sweet spot on the right type of image the results are impressive and the noise reduction alone makes it worth it even if the resolution isn't improved with other lenses. However it has its place and isn't needed for every shot.
01-16-2021, 07:03 PM   #13
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
If i recall correctly, @heckflosse used any differences between the green pixels in the frames to determine motion.
01-17-2021, 04:18 AM   #14
Pentaxian




Join Date: Feb 2015
Photos: Gallery
Posts: 12,173
Original Poster
Installed again RT. Works pretty well. The big advantage is to be able to tweak sensitivity (to not include areas that actually don't contain any motion) and mask blur radius to soften transitions between motion and non motion areas. On the other hand, SPD10 isn't as good as motion correction (it does about the same as the native Pentax correction, and there is no way to tune it), but I find that SPD10 is more faithful for colors and quicker to use for lens CA corrections. If I learn more of RT so that can improve colors rendering, it'll be a very good option to have in my PP workflow. I've play around with a PS file containing small elements in motion (difficult to separate from noise), and the pixel shift output, after tweaking lens correction to best possible, removal of edge fringes and optimal contrast + sharpening, the output is amazing, I could print it to 24"x36" still showing very fine details.

---------- Post added 17-01-21 at 12:23 ----------

QuoteOriginally posted by jbinpg Quote
If i recall correctly, @heckflosse used any differences between the green pixels in the frames to determine motion.
That's a good idea, since there are twice as many green pixels covering the sensor area.
01-18-2021, 10:18 AM - 3 Likes   #15
Senior Member
heckflosse's Avatar

Join Date: Feb 2017
Posts: 171
QuoteOriginally posted by jbinpg Quote
If i recall correctly, @heckflosse used any differences between the green pixels in the frames to determine motion.
As I got pinged in this thread, let me comment

1) the main motion detection in RT is by comparing green pixels.
2) If there are not many greens in the raw, also comparing blue/red pixels can be used (though not as exactly as with green pixels)
3) for the regions with motion you can use every (not just the first) frame
4) for the regions with motion, using the median is intended for stuff like moving water, not for foliage.

For CA correction the raw CA correction in RT works really well. I use it always and never use the Lensfun CA correction. The raw CA correction in RT also is specialized to work with Pixelshift files (giving the same auto correction for all frames, which is important)

Finally: I'm glad, some folks are using RT for pixelshift :+1: It was a lot of work coding it.

Ingo
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
colors, correction, information, k1, lens, motion, output, panasonic, papers, photography, photoshop, pixel, pixel shift motion, pixels, rt, search, section, shift
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Panasonic SR-1 Pixel-Shift Works with Motion? Winder Canon, Nikon, Sony, and Other Camera Brands 33 04-23-2019 09:10 AM
Lloyd Chambers Prefers Pentax Pixel Shift To Sony's Pixel Shift Fenwoodian Pentax K-1 & K-1 II 2 07-16-2018 02:15 PM
Pixel shift motion correction in K-3II? microlight Pentax K-3 & K-3 II 23 08-02-2016 02:48 PM
Pixel Shift Motion Correction Quartermaster James Pentax DSLR Discussion 2 05-28-2016 04:21 PM
What's "pixel shift motion correction function"? Fenwoodian Pentax K-1 & K-1 II 36 02-27-2016 09:55 AM



All times are GMT -7. The time now is 08:51 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top