Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
10-12-2018, 05:16 AM   #1
Site Supporter
Site Supporter




Join Date: Dec 2012
Photos: Gallery | Albums
Posts: 2,787
Google Pixel 3 Computational Photography Advances

Even though it's DPReview, I found this article on what Google's been up to with regards to their Pixel smartphone's cameras very interesting.

A lot of what they're doing involves constantly buffering 10 or 15 images, prior to hitting the shutter button, so that they can do all kinds of multi-exposure blending with zero or minimal shutter lag. I have an original Pixel that's a few years old, and it defaults to essentially HDR mode. I'm often surprised by how good the images from the phone are.

But with the 3 they've upped it several levels, and now the default mode sounds a lot like Pentax' dynamic pixel shift. Just without the slower multi-exposure shutter/mirror/wait for computation part. In the phone it's seamless, in the background, indistinguishable from just hitting a button. They're even doing multi-image blended RAW now.

One of the thoughts I had about this is the converse of what you typically hear, which is that this technology makes smartphones closer to being on par with ILC cameras. I think this will eventually make it into ILCs, and enable APS-C cameras to have more like FF or even MF results. A full frame camera could out-resolve a MF. Everything they're doing with a 1/2.5" sensor could be done with much larger sensors with crazy good results. Pair that with a pro-level 70-200 or the new 50mm Pentax lens and you could have results that hard to duplicate with any camera available at any price today.

The limiting factors seem to be lack of processing power in the camera, larger sensors mean a lot more data to continually process, and if your camera is always on and processing lots of data your battery life is going to suffer. Another consideration for us DSLR folks is that much of this relies on continual buffering of lots of images, which pretty much means electronic shutter, which means mirrorless or an SLR in liveview. If you're always in Live View then why have the mirror at all?

10-12-2018, 06:10 AM   #2
Pentaxian
dsmithhfx's Avatar

Join Date: Sep 2015
Location: Toronto
Posts: 5,123
I feel put off by smartphone photography, mainly because of the form factor, which strikes me as absurdly outsized compared to the actual camera. As an analogy, would you rather chop vegetables with a swiss army knife. Well, if that's all you had...

I feel the camera should be a clip-on, brooch-sized lens + sensor, tethered wirelessly to the necessarily larger cpu (, battery, etc) carried elsewhere. Those google glass thingies were at least a step in the right direction.

I also intensely dislike the phone display as a "viewfinder", much as I hate back lcd displays on DSLRs and compact digicams. This is just wrong.

Having said all that, I admit to also being fascinated by the image quality the miniature cameras are capable of, and how idiosyncrasies of the technology affect photography as much as previous innovations.

As far as photographic quality -- which encompasses far more than just IQ -- I'd say it is possible to take good photographs in spite of the phone, rather than because of it, i.e. the phone itself will always be a rather large impediment.

If you're going to buy a phone for its camera -- don't. Instead, buy a cheaper phone (mine cost $50 new, with no contract, and it works great as a phone), and put the considerable sum you will save on a real camera.

Last edited by dsmithhfx; 10-12-2018 at 06:18 AM.
10-12-2018, 06:30 AM   #3
Site Supporter
Site Supporter




Join Date: Dec 2012
Photos: Gallery | Albums
Posts: 2,787
Original Poster
QuoteOriginally posted by dsmithhfx Quote
I feel put off by smartphone photography, mainly because of the form factor, which strikes me as absurdly outsized compared to the actual camera. As an analogy, would you rather chop vegetables with a swiss army knife. Well, if that's all you had...
Yes, the sensor and lens in the phone is tiny. But the processor, memory and associated electronics are what enables the tiny sensor and lens to punch well above their weight class. The processing power of a modern smartphone is orders of magnitude more powerful than a desktop of not that many years ago. Stuffing that into a smartphone sized package with a battery that lasts all day is a marvel.

QuoteQuote:
I feel the camera should be a clip-on, brooch-sized lens + sensor, tethered wirelessly to the necessarily larger cpu (, battery, etc) carried elsewhere. Those google glass thingies were at least a step in the right direction.
The camera/lens package would still need a battery. And a high-bandwidth always-on (or at least very frequently used) connection to the processing system. The power and bandwidth requirements would probably be the limiting factor to designing such a system. Seems much simpler and far less power intensive to have hard copper connections between sensor and processor.

Bluetooth is low power, but I think it maxes out at 2 Mb/sec, far, far too low for the continuous buffering of ten or fifteen 10+ Mpix pictures this kind of computational photography needs. Wifi could handle it, but at much higher power consumption.
10-12-2018, 07:19 AM   #4
Moderator
Loyal Site Supporter
Wheatfield's Avatar

Join Date: Apr 2008
Location: The wheatfields of Canada
Posts: 15,903
QuoteOriginally posted by ThorSanchez Quote
Even though it's DPReview, I found this article on what Google's been up to with regards to their Pixel smartphone's cameras very interesting.

A lot of what they're doing involves constantly buffering 10 or 15 images, prior to hitting the shutter button, so that they can do all kinds of multi-exposure blending with zero or minimal shutter lag. I have an original Pixel that's a few years old, and it defaults to essentially HDR mode. I'm often surprised by how good the images from the phone are.

But with the 3 they've upped it several levels, and now the default mode sounds a lot like Pentax' dynamic pixel shift. Just without the slower multi-exposure shutter/mirror/wait for computation part. In the phone it's seamless, in the background, indistinguishable from just hitting a button. They're even doing multi-image blended RAW now.

One of the thoughts I had about this is the converse of what you typically hear, which is that this technology makes smartphones closer to being on par with ILC cameras. I think this will eventually make it into ILCs, and enable APS-C cameras to have more like FF or even MF results. A full frame camera could out-resolve a MF. Everything they're doing with a 1/2.5" sensor could be done with much larger sensors with crazy good results. Pair that with a pro-level 70-200 or the new 50mm Pentax lens and you could have results that hard to duplicate with any camera available at any price today.

The limiting factors seem to be lack of processing power in the camera, larger sensors mean a lot more data to continually process, and if your camera is always on and processing lots of data your battery life is going to suffer. Another consideration for us DSLR folks is that much of this relies on continual buffering of lots of images, which pretty much means electronic shutter, which means mirrorless or an SLR in liveview. If you're always in Live View then why have the mirror at all?
Smartphones pretty much are the future of photography. ILC camera development is stagnant, and has been for several years. Meanwhile Smartphone cameras keep getting better and better. The other thing with smartphones is their ubiquitousness. Everyone has one, and for the most part, everyone upgrades every couple of years or so, and every couple of years, the camera in those devices has improved tremendously.
People have been forced into stand alone cameras and if they wanted something a bit better than a P&S they had to buy either a bridge camera or an ILC. Now we've seen the demise of the compact P&S, killed by the smartphone a few generations of phone ago. It won't be too many more generations before the bridge camera is also killed, as it is little more than a glorified P&S.
People want convenience above all else. Image quality is a very distant second (or possibly even further down the scale). Most images are now viewed on smartphone screens, so the need for real quality isn't there anyway.
The smartphone is convenient, and it's something that is with the user every moment of every day. The camera in it is, therefore, about as convenient as is possible. Compare this with a bridge camera, or gawd forbid, an ILC. Big, cumbersome, and they don't multitask.
The smartphone camera is getting to the point where it will do most of what users want, and for those special occasions where they need something better, they might have an ILC in the back of a closet. As the smartphone camera continues to improve, that ILC is going to see less and less use, and as more and more people see less and less use for ILCs, the sales of them will continue to drop.

10-12-2018, 07:49 AM   #5
Site Supporter
Site Supporter




Join Date: Dec 2012
Photos: Gallery | Albums
Posts: 2,787
Original Poster
QuoteOriginally posted by Wheatfield Quote
Smartphones pretty much are the future of photography. ILC camera development is stagnant, and has been for several years. Meanwhile Smartphone cameras keep getting better and better. The other thing with smartphones is their ubiquitousness. Everyone has one, and for the most part, everyone upgrades every couple of years or so, and every couple of years, the camera in those devices has improved tremendously.
People have been forced into stand alone cameras and if they wanted something a bit better than a P&S they had to buy either a bridge camera or an ILC. Now we've seen the demise of the compact P&S, killed by the smartphone a few generations of phone ago. It won't be too many more generations before the bridge camera is also killed, as it is little more than a glorified P&S.
People want convenience above all else. Image quality is a very distant second (or possibly even further down the scale). Most images are now viewed on smartphone screens, so the need for real quality isn't there anyway.
The smartphone is convenient, and it's something that is with the user every moment of every day. The camera in it is, therefore, about as convenient as is possible. Compare this with a bridge camera, or gawd forbid, an ILC. Big, cumbersome, and they don't multitask.
The smartphone camera is getting to the point where it will do most of what users want, and for those special occasions where they need something better, they might have an ILC in the back of a closet. As the smartphone camera continues to improve, that ILC is going to see less and less use, and as more and more people see less and less use for ILCs, the sales of them will continue to drop.
I don't disagree with much of what you're saying. For the majority of use cases and how photographs are viewed a Pixel 3 is a great camera. And smartphones will continue to get better. Most people won't be able to tell the phone from a GRIII... except where the phone's computational prowess will enable shots that the GRIII can't get, or would require a lot of off-camera post processing.

But I still think there's a huge opportunity to make a monster camera by combining a large sensor ILC with the technology in the phones. Imagine Pentax partnering with Google to make an APS-C or FF body that uses top-quality lenses, has great ergonomics, and has all the behind-the-scenes techological features of the Pixel. You could have stabilized on-the-fly, no lag HDR dynamic pixel shift images with a 150-450 lens, shot through a hybrid viewfinder, uploaded to the cloud almost instantly.
10-12-2018, 07:58 AM   #6
Pentaxian
normhead's Avatar

Join Date: Jun 2007
Location: Near Algonquin Park
Photos: Gallery | Albums
Posts: 40,442
I'm never impressed, until I see an image that couldn't be done with different equipment. Then the decision comes down to, "Do I want to take that type of Image."

No image to show me?

Nothing to see here, literally.
I can't even start the process.
10-12-2018, 08:04 AM   #7
Site Supporter
Site Supporter




Join Date: Dec 2012
Photos: Gallery | Albums
Posts: 2,787
Original Poster
QuoteOriginally posted by normhead Quote
I'm never impressed, until I see an image that couldn't be done with different equipment. Then the decision comes down to, "Do I want to take that type of Image."

No image to show me?

Nothing to see here, literally.
I can't even start the process.
This isn't about a pixel-peeping battle between a smartphone and a K-1.

It's a discussion about how computational advances will allow cameras of all types to achieve images that just aren't possible right now.

10-12-2018, 08:04 AM   #8
Pentaxian
dsmithhfx's Avatar

Join Date: Sep 2015
Location: Toronto
Posts: 5,123
QuoteOriginally posted by Wheatfield Quote
Smartphones pretty much are the future of photography.
For amateurs who are largely indifferent to the medium, except for taking selfies and what they had for lunch, and for tech enthusiasts to have something to prove, just not something I find interesting...


QuoteOriginally posted by Wheatfield Quote
ILC camera development is stagnant, and has been for several years.
Hardly.
10-12-2018, 08:54 AM   #9
Moderator
Loyal Site Supporter
Wheatfield's Avatar

Join Date: Apr 2008
Location: The wheatfields of Canada
Posts: 15,903
QuoteOriginally posted by dsmithhfx Quote
For amateurs who are largely indifferent to the medium, except for taking selfies and what they had for lunch, and for tech enthusiasts to have something to prove, just not something I find interesting...




Hardly.
What's come along in the last few years of ILC development that marks a massive improvement in cameras?
I'm not seeing anything.
Meanwhile, I have seen huge improvements in camera phones.
10-12-2018, 09:05 AM   #10
Pentaxian
dsmithhfx's Avatar

Join Date: Sep 2015
Location: Toronto
Posts: 5,123
QuoteOriginally posted by Wheatfield Quote
What's come along in the last few years of ILC development that marks a massive improvement in cameras?
I'm not seeing anything.
Meanwhile, I have seen huge improvements in camera phones.
Enjoy your phone.
10-12-2018, 09:46 AM   #11
Moderator
Loyal Site Supporter
Wheatfield's Avatar

Join Date: Apr 2008
Location: The wheatfields of Canada
Posts: 15,903
QuoteOriginally posted by dsmithhfx Quote
Enjoy your phone.
So you don't have anything to back up your statement?
10-12-2018, 10:09 AM   #12
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 42,007
QuoteOriginally posted by ThorSanchez Quote
It's a discussion about how computational advances will allow cameras of all types to achieve images that just aren't possible right now.
The way that I take the announcement in the original post is that the camera does continuous video into buffer and that "shutter" actuation is simply a slice from buffer. "Computational" merges might then be applied from adjacent slices in an effort to improve the image. Why that might be an advantage is hard to say without comparison to photos from the best of similar competing devices. At face value, the most obvious benefit of such an approach is extremely fast capture.

FWIW, "computational photography" is a euphemism for automated digital imaging with post-processing and image manipulation. In-camera focus stacking, pixel-shift, and HDR are reasonably good examples.

Can you suggest examples of images that "just aren't possible right now"? I am not asking for photos, just some text suggesting what the future might hold. (I read the article and was underwhelmed, except perhaps the bit at the end about bottom line being simpler photography.)


Steve

Last edited by stevebrot; 10-12-2018 at 10:18 AM.
10-12-2018, 10:40 AM   #13
Site Supporter
Site Supporter




Join Date: Dec 2012
Photos: Gallery | Albums
Posts: 2,787
Original Poster
QuoteOriginally posted by stevebrot Quote
The way that I take the announcement in the original post is that the camera does continuous video into buffer and that "shutter" actuation is simply a slice from buffer. "Computational" merges might then be applied from adjacent slices in an effort to improve the image. Why that might be an advantage is hard to say without comparison to photos from the best of similar competing devices. At face value, the most obvious benefit of such an approach is extremely fast capture.

FWIW, "computational photography" is a euphemism for automated digital imaging with post-processing and image manipulation. In-camera focus stacking, pixel-shift, and HDR are reasonably good examples.

Can you suggest examples of images that "just aren't possible right now"? I am not asking for photos, just some text suggesting what the future might hold. (I read the article and was underwhelmed.)


Steve
In the article it talks about using stacking/averaging/etc of 15 buffered, very low light photos to simulate a long exposure. You could use a similar technique with a large sensor ILC and a good lens to give you the light gathering capability of a f/0.35 lens but without the almost non-existent depth of field such a lens would give you wide open (if such a lens were even physically possible). Imagine using the 150-450 f/4.5-5.6, but having the equivalent light-gathering capability of an f/2.8 or wider?

You could do much better handheld dynamic pixel shift. You could make pixel shift resolution the default, and happen so fast that the processing is transparent. There could be a pixel shift RAW file with motion correction applied that's just the near-instantaneous result of pressing the shutter.

Of course all of this won't apply to all situations. You'd need a ton of processing to use some of this in fast action scenes... but eventually the processing throughput will be there. One day you might be able to use dynamic pixel shift resolution with a long lens in sports shooting.
10-12-2018, 11:53 AM   #14
Otis Memorial Pentaxian
stevebrot's Avatar

Join Date: Mar 2007
Location: Vancouver (USA)
Photos: Gallery | Albums
Posts: 42,007
QuoteOriginally posted by ThorSanchez Quote
In the article it talks about using stacking/averaging/etc of 15 buffered, very low light photos to simulate a long exposure.
I read the article. What is discussed does not break new ground except for the approach of buffering image frames (think of it as high resolution video at a frame rate of 60 in good light and somewhat less if dim) for routine use by extrapolation algorithms. That is impressive. I did not notice if they mentioned what FPS is manageable for regular shooting or with the special features turned on. I guess we will have to wait for a hands-on review.


Steve
10-12-2018, 12:15 PM   #15
Pentaxian
dsmithhfx's Avatar

Join Date: Sep 2015
Location: Toronto
Posts: 5,123
https://www.thephoblographer.com/2018/10/12/apple-eyefone-vs-google-pickle-x...ight-which-one
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
camera, cameras, google pixel, images, lot, lots, pixel, results, sensors, shutter
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Auto 110 only advances with winder attached notetoself066 Film SLRs and Compact Film Cameras 6 08-13-2018 02:21 PM
RICOH and Google Team Up to Allow RICOH THETA Spherical Images to be Posted on Google pid Photographic Industry and Professionals 18 01-30-2014 08:19 PM
Problem -- Pentax 67 film advances but no shutter? sleepyhead Pentax Medium Format 5 05-15-2010 09:54 PM
russia advances, wooooot Gooshin General Talk 20 06-26-2008 02:32 PM
Google google on the wall... procyon General Talk 4 04-04-2008 06:42 PM



All times are GMT -7. The time now is 10:33 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top