[Class A]

Ricoh has a couple of new articles on their K-1 special site explaining the development of the K-1 mark 2.

Thanks.

I note that -- unlike the pentaxklub article -- Ricoh is not (yet?) making any deeper claims about having made big strides in terms of AF improvements. The pentaxklub story of a new software that has been built from the ground up struck me as an exaggeration and so far Ricoh's own advertising seems to confirm that view.

[photoptimist]

So, if the image quality improves with motion, I would like to ask, "how much motion is optimum"? I have to believe that if no motion is bad, extreme motion (e.g. panning) is also bad. So, exactly how much motion is the perfect amount to fully take advantage of this wonderful new feature? For example, would someone with extremely shaky hands have too much shake? Or, would someone with rock sold hands have not enough shake?.

Excellent question with a complicated answer:

1. The optimum motion is the one used for old pixel shift which moves exactly in 1 pixel steps in a round-the-block motion in such as left-down-right, right-down-left, up-left-down, etc.

2. Motions of odd numbers of pixels (1,3,5,7, ..) provide the best resolution data.

3. Motions of an even number of pixels (2,4,6,8, ..) only reduce noise but don't improve resolution.

4. Straight motions parallel or perpendicular to the rows & columns of pixels provide less total resolution enhancement than do most angled motions or curving motions.

5. Large motions are worse than small ones because they reduce the total fraction of the frame that can be fully enhanced. (The greater the motion, the larger the strip of pixels around the edge of the frame with only 2 or 3 overlapping frames)

(A lot of how super-resolution works depends on luck with how the RGB patterns in the different frames land on top of each other. Fortunately, the chances of being unlucky across all the frames is so low than almost every dynamic pixel shift will show marked improvements when compared a single frame image. And even in the unluckiest cases, the result is still a 4-frame average which will reduce noise and improve color.)

[Fenwoodian]

Excellent question with a complicated answer:

1. The optimum motion is the one used for old pixel shift which moves exactly in 1 pixel steps in a round-the-block motion in such as left-down-right, right-down-left, up-left-down, etc.

2. Motions of odd numbers of pixels (1,3,5,7, ..) provide the best resolution data.

3. Motions of an even number of pixels (2,4,6,8, ..) only reduce noise but don't improve resolution.

4. Straight motions parallel or perpendicular to the rows & columns of pixels provide less total resolution enhancement than do most angled motions or curving motions.

5. Large motions are worse than small ones because they reduce the total fraction of the frame that can be fully enhanced. (The greater the motion, the larger the strip of pixels around the edge of the frame with only 2 or 3 overlapping frames)

(A lot of how super-resolution works depends on luck with how the RGB patterns in the different frames land on top of each other. Fortunately, the chances of being unlucky across all the frames is so low than almost every dynamic pixel shift will show marked improvements when compared a single frame image. And even in the unluckiest cases, the result is still a 4-frame average which will reduce noise and improve color.)

Interesting...

Paraphrasing - the optimum hand-held, dynamic pixel shift motion would be: an odd number of pixels shifted, angled (or curving) motions, and smaller overall motions. I don't think that many of us can do this type of complex motion consistantly. So, would it be fair to say that because one can't consistantly move optimally, then it's probably a good idea to shoot multiple shots of any dynamic pixel shift subject? By taking multiple shots, aren't you increasing your odds of achieving optimum movement and better super-res images?

I wonder if the "continuous" shooting setting is available when in the dynamic pixel shift mode?

[monochrome]

@ photoptimist Can’t the processor simply crop a few pixels off the edges and align the remaining slightly cropped section optimally?

[photoptimist]

Interesting...

Paraphrasing - the optimum hand-held, dynamic pixel shift motion would be: an odd number of pixels shifted, angled (or curving) motions, and smaller overall motions. I don't think that many of us can do this type of complex motion consistantly. So, would it be fair to say that because one can't consistantly move optimally, then it's probably a good idea to shoot multiple shots of any dynamic pixel shift subject? By taking multiple shots, aren't you increasing your odds of achieving optimum movement and better super-res images?

I wonder if the "continuous" shooting setting is available when in the dynamic pixel shift mode?

It's true that few of us will ever twitch the camera optimally. And yet chances are that every processed output pixel location will be on top of or overlapping with at least one red, one green, and one blue pixel from at least one or another of the four frames. People may be unlikely to replicate the full resolution boost of the optimum pixel shift motion, but those complex hand-held motions are likely to ensure a huge boost in resolution over a single frame.

And, yes, taking multiple shots helps.

---------- Post added 04-13-18 at 07:49 PM ----------

@ photoptimist Can’t the processor simply crop a few pixels off the edges and align the remaining slightly cropped section optimally?

Alas, no. The alignment is determined by the random motion of the photographers hand. The camera is measuring that alignment with the SR sensors but it's not fully controlling it as it does in traditional pixel shift. The camera has to use the data it got which may not be aligned as optimally as a traditional pixel shift image.


If we think about resolving some tiny little one pixel red dot in the the scene (imagine a tiny red berry in a very distant fruit tree) we need to have red-sensitive pixels see both the berry and the adjacent non-berry surroundings.

In a single-shot image, there's a 75% chance of missing the berry entirely because only 25% of the sensels on the sensor can see red light. If the light from that berry falls on a green or blue sensel, that sensel sees nothing (dark in the green or blue channel). And with the surrounding sensels seeing only green leaves, the final image would probably show a green or darkish-green spot among the leaves where the unseen berry lies.

In a standard pixel shift image, the camer moves the sensor so that red sensels are guaranteed to visit every bit of the image. The result is a beautiful 1-pixel red berry surrounded by green.

In the new hand held or dynamic pixel shift image process, the natural motion of the photographer moves the sensor between the frames but there's always some chance that red sensels never land on the part of the scene with the red berry. Roughly speaking there's about a 30% chance that none of the frames will catch the berry although the result is likely to show some hint of it in the green.

To summarize:
* the single shot image only has a 25% chance of resolving the red berry
* the traditional pixel shift image has a 100% of resolving the red berry
* the dynamic pixel shift image has about a 70% of resolving the red berry

Note: I'm playing a little fast and loose with the math here because there's a chance a little bit of berry overlaps with red sensel and the demosaiced image shows something in the green-yellow-orange spectrum depending on how much overlap there was. And with four frames in a dynamic pixel shift, the chance of some overlap in some frames is pretty high. But the point remains that if red sensels never land right on top of the red berry, the data will never show a nice saturated value in the red channel and the final dynamic pixel shift image will do a worse job resolving the red berry than traditional tripod pixel shift.

[Fenwoodian]

To summarize:
* the single shot image only has a 25% chance of resolving the red berry
* the traditional pixel shift image has a 100% of resolving the red berry
* the dynamic pixel shift image has about a 70% of resolving the red berry

Photoptimist, you should work for the Pentax marketing department. You just sold me on buying a K1ii camera. Moving up from 25% with my K1 handheld, to 70% with the K1ii handheld is certainly worth the extra money for the K1ii.

No need for me to read any K1ii reviews or see any K1ii images. I finally understand the technology, that's all that I need to jump in.

[Not a Number]

Maybe with the accelerator unit more of the edge pixels can be processed.

[jbinpg]

Since the sensor size is 24 x 35.9 (mm) for both K1 and K1 MrkII, why is the resolution different, even if only very small?

The K1 = 7360 x 4912 = 36,152,320 ~ 36.15 mpx

The K1 MkII = 7392 x 4950 = 36,590,400 ~ 36.59 mpx

In the MkII you miss = 0.44 / 4 = 0.11, that is the part of the red berry that had escaped you.

Maw, you may wish to check out this thread where file size is discussed:

iso with PEF vs DNG - PentaxForums.com

[Class A]

Excellent question with a complicated answer:

Your findings are insightful but I wonder whether they are based on an incorrect assumption.

Do we know that "Dynamic Pixel Shift" attempts any sensel value (as opposed to pixel value) alignment?

AFAIC, the most straightforward "Dynamic Pixel Shift" implementation would be the following:

1. While taking the shots, the SR system is active, but not in its usual mode in which it attempts to reduce camera-induced motion blur as much as possible. Instead, the SR system will seek to keep the alignment of the subsequent images within a certain range. In other words, it will not correct minute shake that is helpful to gather more information which helps the superresolution process. It will counteract drastic camera shake in an attempt to maximise the overlap between all images.
2. Each individual shot is then demosaiced individually in order to obtain full RGB estimates for each pixel position.
3. The individual images are than aligned. I would expect that alignment to be informed by the image data only, i.e., not taking any acceleration data into account. I have trouble imagining how additional acceleration data input could make the alignment more precise.

I do not know how "Dynamic Pixel Shift" works under the hood, but I'd be surprised if there were any attempt to align images at sensel data level. Unlike with true "Pixel Shift" there would be just too much variation regarding what one is left with and different demosaicing techniques would have to be used accordingly.

I think we can all agree that the term "Dynamic Pixel Shift" is unfortunate. I like the idea that it originates from an attempt to group the two "Pixel Shift" and "Dynamic Pixel Shift" functions within the same camera menu. I believe they should have found a better overarching term, like "Resolution Enhancement".

[MJKoski]

There is no problem with a tripod. Why use such weird feature when the primary issue is basically if subject is static or not? Limited R&D resources should have been used to simple and effective enhancements like real backlit buttons. Or OVF shutter piece. Come on, it is MK2! These would have been 3$ enhancements.

Now we have a mystery chip which seems to filter image in unknown way. Not sure yet, but there seems to be hints of Sony style spatial filtering which kicks in with moderate-to-high ISO values and long exposures. This messes up tiny texture details.

[biz-engineer]

There is no problem with a tripod. Why use such weird feature when the primary issue is basically if subject is static or not? Limited R&D resources should have been used to simple and effective enhancements like real backlit buttons. Or OVF shutter piece. Come on, it is MK2! These would have been 3$ enhancements. Now we have a mystery chip which seems to filter image in unknown way. Not sure yet, but there seems to be hints of Sony style spatial filtering which kicks in with moderate-to-high ISO values and long exposures. This messes up tiny texture details.

When Pentax engineers saw how you tested the K1 in the cold, they were afraid to modify the seals to add back-lighting on the K1 II.

[monochrome]

“The camera detects minute camera shake during handheld shooting and closely evaluates the motion, while also analyzing the four separate images before composing a single super-resolution image.”

Pretty straightforward explanation.

[photoptimist]

Your findings are insightful but I wonder whether they are based on an incorrect assumption.

Do we know that "Dynamic Pixel Shift" attempts any sensel value (as opposed to pixel value) alignment?

AFAIC, the most straightforward "Dynamic Pixel Shift" implementation would be the following:

1. While taking the shots, the SR system is active, but not in its usual mode in which it attempts to reduce camera-induced motion blur as much as possible. Instead, the SR system will seek to keep the alignment of the subsequent images within a certain range. In other words, it will not correct minute shake that is helpful to gather more information which helps the superresolution process. It will counteract drastic camera shake in an attempt to maximise the overlap between all images.
2. Each individual shot is then demosaiced individually in order to obtain full RGB estimates for each pixel position.
3. The individual images are than aligned. I would expect that alignment to be informed by the image data only, i.e., not taking any acceleration data into account. I have trouble imagining how additional acceleration data input could make the alignment more precise.

I do not know how "Dynamic Pixel Shift" works under the hood, but I'd be surprised if there were any attempt to align images at sensel data level. Unlike with true "Pixel Shift" there would be just too much variation regarding what one is left with and different demosaicing techniques would have to be used accordingly.

Although that sounds like a straight forward implementation, it's not how super-resolution works and it contradicts what Ricoh have said.

With super-resolution, extremely accurate knowledge of frame-to-frame alignment (as in knowing which sensels match across frames) is absolutely essential. And the better the estimate of the alignment (e.g., by using the SR system sensor data) the higher the potential resolution improvement.

Demosaicing an individual frame can't give full RGB estimates for each pixel position. The amount of red light or blue light in the scene that was covered by a green sensel is fundamentally unknown (see the tiny red berry example that I talked about a few posts ago). Demosaicing blurs the adjacent red and blue values across the green sensel but the result is a potentially bad estimate because maybe there's a red berry where the green sensel was looking. Neither super-resolution nor traditional pixel shift create the blurry demosaiced frames as a intermediate step because that's entirely counterproductive.

[normhead]

Adam, I watched the video that you posted on K1 MC. In it you say MC "possibly eliminates motion artifacts in a scene". That's an exaggeration too. Why pretend that it works better than it really does?

It does what it says it does, perhaps just not to your satisfaction but I have seen images taken with this feature.

Personally I just don't believe anyone's marketing department. I want to see how it works before I buy.

I've never tried the feature on my K-1. It's not really of interest. The Acceleration unit and AF of the K-1ii are. Honestly, I get such good results from the K-1 without Pixels Shift, I only use it when I'm bored.

[Kunzite]

There is no problem with a tripod. Why use such weird feature when the primary issue is basically if subject is static or not? Limited R&D resources should have been used to simple and effective enhancements like real backlit buttons. Or OVF shutter piece. Come on, it is MK2! These would have been 3$ enhancements.

Now we have a mystery chip which seems to filter image in unknown way. Not sure yet, but there seems to be hints of Sony style spatial filtering which kicks in with moderate-to-high ISO values and long exposures. This messes up tiny texture details.

Changing the camera's body is not a 3$ enhancement. You cannot support an idea just by claiming that it's cheap, you know?
IMHO it would be a very bad idea to give up on the upgrade just for backlit buttons and an OVF shutter.

Even without the accelerator, your image files are obtained in an unknown (to you) way. The accelerator works, the first reports are positive - and wonder where did you saw those hints.