Is the K20D sensor based on this patent?

[Olivier]

US patent 20050280727 describes a sensor that groups 3 or 4 detectors under a single micro-lens:



The variant that seems most interesting to me uses R, G, B and luminance (i.e. no color filter) photo detectors under the micro-lens. One of the main advantages listed in the patent description is the elimination of the anti-aliasing filter. A diffusion filter located between the micro-lens and the photo detectors makes sure they all see the same light color and intensity:



What would be the advantage of such a sensor? Let's first compare it with a traditional one that has a single photo detector under the micro-lens (assuming the same size micro-lens). The proposed sensor would have 4x more photo detectors, allowing it to record a complete set of RGBY values per pixel. It would have better resolution due to the lack of an AA filter, but the larger number of photo detector would seem detrimental to noise and dynamic range.

How bad would the noise be? Let's look at luminance noise first. The traditional sensor would capture roughly 1/3 of the incoming photons due to the color filter (ignoring other factors such as the photo detector's quantum efficiency for now). The proposed sensor would capture more light (roughly 1/2 of the incoming photons) thanks to the luminance photo detector not having a color filter. On the other hand it would have more readout noise because the photons have to be read out from 4 separate photo detectors. To evaluate luminance noise, let's assume that all 4 values are simply added (more sophisticated algorithms might be able to do better). This would cause some of the readout noise to cancel each other, with the total readout noise probably being around twice the amount for a single readout. So the proposed sensor would have twice the readout noise but a 50% stronger luminance signal, resulting in an relative noise increase of only 33%.

This does not sound too bad, especially since there would be no anti-aliasing filter. How much the AA filter costs in terms of resolution, and how much one gains by having per-pixel RGB values is hard to tell. However, based on what I gleaned in the Foveon vs. Bayer discussions in the Sigma forums it could vary from 1x to 4x depending on subject matter, with 2x being a reasonable "average" resolution gain for a true RGB sensor. In other words the proposed sensor could have a smaller number of micro-lenses (and therefore better noise characteristics) and still produce sharper images due to the lack of AA filter and to the additional color information.

How could this play out for the K20D? Assuming that this all pans out (i.e. comparable readout noise and micro-lens / filter / quantum efficiencies) Pentax might be able to build a sensor with e.g. 8M micro-lenses (32M photo detectors) to go against the D300 with its 12M Bayer photo detectors. Noise should be comparable (or maybe better for Pentax?) and the resolution should also be in favor of Pentax due to the lack of AA filter and to the per-pixel RGB values. Such a sensor would also be very interesting for B&W, as it would be only 1 stop worse than a true 8M B&W sensor. Image processing would require more horse power and storage due to the 4x increase in data values (due to the 32M photo detectors). However, to ease the pain I could imagine a "Bayer" mode where only one R, G or B photo detectors is sampled per micro-lens (i.e. per pixel). This would use a regular Bayer pipeline with only 8M values, providing higher processing speeds, but noise and dynamic range would of course be impacted.

What do you think?

Disclaimer: I don't really know what Pentax is up to. All I did was look for sensor-related patents from Pentax and dream for a while...

[jeffkrol]

You'd have 4 pixels per pixel so to speak. This is the concept like the Foveon (but that uses silicon depth/wavelength response so the sensors are "stacked" instead of a group of 3 (lum is missing , at least as a seperate pixel) or the Nikon patent:
Nikon's new full-color RGB sensor?: Digital Photography Review

The pros and cons of Bayer types vs these types can be researched.
The only real problem would be well size and the need for a pretty heavy processing pipeline.
EDIT: And we can all join the is it 14mp or just 3.5mp (3.5mp would have 14m(p) sensels) debates that plague the Sigma owners..

[JMS]

This sounds VERY interesting!

[Olivier]

Originally Posted by jeffkrol

You'd have 4 pixels per pixel so to speak. This is the concept like the Foveon (but that uses silicon depth/wavelength response so the sensors are "stacked" instead of a group of 3 (lum is missing , at least as a seperate pixel) or the Nikon patent:
Nikon's new full-color RGB sensor?: Digital Photography Review

The Nikon patent is also interesting. It has the advantage that all the photons are captured, as in the Foveon sensor. On the other hand I wonder how hard/expensive it would be to manufacture all these tiny dichroic mirrors. The proposed Pentax sensor does not seem to be much more difficult to manufacture than conventional sensors.

[clawhammer]

IIRC, all digital sensors today already have a micro-lens over each photosite.

[ricardobeat]

Originally Posted by Olivier

The Nikon patent is also interesting. It has the advantage that all the photons are captured, as in the Foveon sensor. On the other hand I wonder how hard/expensive it would be to manufacture all these tiny dichroic mirrors. The proposed Pentax sensor does not seem to be much more difficult to manufacture than conventional sensors.

I believe this is exactly the same way the Foveon sensor works, with dichroic mirrors separating light, or am I wrong? I thought this design was patented.

[mattdm]

Originally Posted by ricardobeat

I believe this is exactly the same way the Foveon sensor works, with dichroic mirrors separating light, or am I wrong? I thought this design was patented.

Foveon works by taking advantage of the way different wavelengths of light penetrate to different depths in silicon. The different-color photosites are literally on top of each other.

[Philippos]

Isn't this the Kodak patent?
I swear I saw something like this (too much like this) from Kodak, not that long ago!
If this is indeed a Pentax patent, I can imagine the rumours about it's legality starting very very soon.

[JanG]

Originally Posted by Olivier

The Nikon patent is also interesting. It has the advantage that all the photons are captured, as in the Foveon sensor. On the other hand I wonder how hard/expensive it would be to manufacture all these tiny dichroic mirrors. The proposed Pentax sensor does not seem to be much more difficult to manufacture than conventional sensors.

I remember a post on this site, in which someone claimed, Pentax was using a simple technology and blocking the other manufacturers by patents, so that they could do it, too, but have to pay a price for it.
That would fit perfectly for the Pentax vs. Nikon patents in this thread.

[Olivier]

Originally Posted by Philippos

Isn't this the Kodak patent?
I swear I saw something like this (too much like this) from Kodak, not that long ago!
If this is indeed a Pentax patent, I can imagine the rumours about it's legality starting very very soon.

That's an excellent question. The only difference between the Pentax patent (at least the embodiment I wrote about in the first post) and the upcoming Kodak sensors seems to be that the 4 photo detectors share a micro-lens. The diffusion filter also makes sure each photo detector sees the same light color and intensity, which prevents aliasing problems (i.e. moire patterns). The upcoming Kodak sensors, on the other hand, will have separate micro-lenses for each photo detector (i.e. pixel) and will still need an anti-aliasing filter on top of the sensor.

I wonder which of the Pentax and Kodak patents was submitted first. Does anybody know?

[Olivier]

Originally Posted by JanG

I remember a post on this site, in which someone claimed, Pentax was using a simple technology and blocking the other manufacturers by patents, so that they could do it, too, but have to pay a price for it.
That would fit perfectly for the Pentax vs. Nikon patents in this thread.

I can easily believe that. The Pentax patent seems quite broad and difficult to circumvent if you are after a sensor with per-pixel RGB values that is easy to manufacture.

[Olivier]

Originally Posted by Olivier

I wonder which of the Pentax and Kodak patents was submitted first. Does anybody know?

Answering my own question... Here is what I found:

The earliest Kodak patent of the new pattern I could find (but I did not search for very long) is 20070024931, which was filed on July 28, 2005. The Pentax patent is 20050280727 and was filed on June 16, 2005. This is a bit more than a month earlier. Moreover, the Pentax patent seems to be based on Japanese patent P2004-179642 from June 17, 2004. So to me it looks like Pentax was first.

However, the two patents are really quite different, so they should both be valid. In the Kodak patent every other detector is panchromatic (i.e. does not have a filter) while in the Pentax patent only one in 4 detectors does not have a filter. For reference, here are the Bayer pattern and the most likely new Kodak pattern:



Another key difference is that Pentax groups 4 detectors together under the same micro-lens to get RGBY values for the same pixel (and to allow the elimination of the anti-aliasing filter) while Kodak uses a single detector per pixel. In fact, while the Kodak pattern allows high sensitivity, anti-aliasing is likely to be quite a challenge. My guess is that we will see the Kodak pattern first in cell phones and similar devices where the sensor out-resolves the lens and sensitivity is key.