[ProfHankD]

But a 50% change in pixel count is in fact a 50% change in pixel count, which means 50% change in storage and processing needed. I'd much rather not have my camera that much slower. (And it will be. Not compared to some other camera, but compared to the same camera with less pixels.)

(Also my computer, no matter how fast my computer is, it would be handle smaller images faster.)

The in-camera processing speed is largely accomplished using parallel processing -- fewer pixels often get processed with less parallelism, so the speed change is often negligible. Processing more pixels in parallel will eat batteries faster.

The fix: On the K5, select JPEG Recorded Pixels in the Rec. Mode 1 menu, and select 10m, 6m, or 2m instead of 16m. The camera will still see all the pixels, but will (hopefully intelligently) rescale the image as requested before writing it out... so they'll be smaller, very low noise, files and process faster on your old PC. I'm sure any future camera will provide a similar menu choice.

Thing is, I'll bet you do not use the lower resolution modes, do you? Me neither.

[Anvh]

Like you said the camera sees still all the pixels and they will all get processed before getting compressed, so there is no speed gain in processing besides that the file is smaller so the write time to the SD card is faster.

[drougge]

You forget the processing also gets faster, that one actually goes faster then pixels count goes up.

I forget no such thing. I'm saying the same increase would give me more if the pixel count didn't increase. Proper buffer management would mostly help too, but Pentax has so for failed at that. (Pressing play should never give me an hour glass, it should just not show all pictures if they aren't ready. If I have just shot a single picture it should bloody well be able to show that one quickly, even if it's not written to the card yet.)

The in-camera processing speed is largely accomplished using parallel processing -- fewer pixels often get processed with less parallelism, so the speed change is often negligible. Processing more pixels in parallel will eat batteries faster.

The fix: On the K5, select JPEG Recorded Pixels in the Rec. Mode 1 menu, and select 10m, 6m, or 2m instead of 16m. The camera will still see all the pixels, but will (hopefully intelligently) rescale the image as requested before writing it out... so they'll be smaller, very low noise, files and process faster on your old PC. I'm sure any future camera will provide a similar menu choice.

Thing is, I'll bet you do not use the lower resolution modes, do you? Me neither.

Of course I don't use the lower resolution modes, just like I don't use the full resolution jpeg mode. Wanting to be able to mess with colour and exposure after the fact has nothing to do with resolution though, and I would use a RAW mode with 6 MP if it was provided.

If you have a PC where anything you want to do with your images feels instant I congratulate you, but my experience with my (not all that old) PC certainly doesn't suggest I would get that experience with a new one. (C2D 2200MHz, 4GB RAM.)

It's possible I'm actually OK with the in camera processing speed, it's hard to tell it apart from the write speed since the buffer management sucks.

[ProfHankD]

Of course I don't use the lower resolution modes, just like I don't use the full resolution jpeg mode. Wanting to be able to mess with colour and exposure after the fact has nothing to do with resolution though, and I would use a RAW mode with 6 MP if it was provided.

If you have a PC where anything you want to do with your images feels instant I congratulate you, but my experience with my (not all that old) PC certainly doesn't suggest I would get that experience with a new one. (C2D 2200MHz, 4GB RAM.)

It's possible I'm actually OK with the in camera processing speed, it's hard to tell it apart from the write speed since the buffer management sucks.

JPEG uses lossy "perceptually compressed" encoding, which makes it bad for reprocessing. That said, raw files are supposed to be unprocessed data as digitized off the sensor (even including some edge pixels that are not exposed but help to determine black point and noise characteristics). What you want is something not raw, but using lossless compression at the camera-rescaled resolution; perhaps PNG files? I agree that would be nice.

As for my PC being too slow, well, I'm the professor who built the world's first Linux cluster supercomputer in 1994. My group at the University of Kentucky operates two machine rooms full of cluster supercomputers for our research. That said, yup, they're too slow for some of the computational photography research I've been doing lately. I have individual algorithms that run for a month on a 1MP image. That's what research is all about -- pushing boundaries. It will only be a few years before an improved algorithm does the same stuff quickly at full resolution on a GPU-augmented cell phone... or inside your camera.

Actually, it's not the buffer management that sucks, it's the flash write speed and a smallish total RAM size. That's getting better too. Last year, DDR3 prices fell through the floor. In addition, the flooding bumped disk drive prices 2X-3X, which has given the SSD market quite a boost, so flash memory price/performance should be improving quickly soon. Be patient. Digital cameras are a very young technology that is still evolving. In the meantime, it's good research for me.

[jsherman999]

JPEG uses lossy "perceptually compressed" encoding, which makes it bad for reprocessing. That said, raw files are supposed to be unprocessed data as digitized off the sensor (even including some edge pixels that are not exposed but help to determine black point and noise characteristics). What you want is something not raw, but using lossless compression at the camera-rescaled resolution; perhaps PNG files? I agree that would be nice.

As for my PC being too slow, well, I'm the professor who built the world's first Linux cluster supercomputer in 1994. My group at the University of Kentucky operates two machine rooms full of cluster supercomputers for our research. That said, yup, they're too slow for some of the computational photography research I've been doing lately. I have individual algorithms that run for a month on a 1MP image. That's what research is all about -- pushing boundaries. It will only be a few years before an improved algorithm does the same stuff quickly at full resolution on a GPU-augmented cell phone... or inside your camera.

Actually, it's not the buffer management that sucks, it's the flash write speed and a smallish total RAM size. That's getting better too. Last year, DDR3 prices fell through the floor. In addition, the flooding bumped disk drive prices 2X-3X, which has given the SSD market quite a boost, so flash memory price/performance should be improving quickly soon. Be patient. Digital cameras are a very young technology that is still evolving. In the meantime, it's good research for me.

.

Nice to meet you, Dr. Dietz!

.

[drougge]

JPEG uses lossy "perceptually compressed" encoding, which makes it bad for reprocessing. That said, raw files are supposed to be unprocessed data as digitized off the sensor (even including some edge pixels that are not exposed but help to determine black point and noise characteristics). What you want is something not raw, but using lossless compression at the camera-rescaled resolution; perhaps PNG files? I agree that would be nice.

JPEG compression isn't that big a problem here, the bigger problem is rescaling the data to an 8 bit perceptual colour space. This happens with PNG too.

And what I want (as I may have said) is a lower resolution sensor. I'm just suggesting workarounds with rescaling. It should be possible to demosaic and then rescale the RAW data, but of course you start with something that's 3 times as large then. Probably why they don't offer it, you have to give up a lot of resolution for a small gain in file size. But maybe it would be possible to make it sparse (one colour component per pixel) again after rescaling.

(Sorry about using the word rescaling both about image dimensions and colour data, hopefully it's not confusing.)

Actually, it's not the buffer management that sucks, it's the flash write speed and a smallish total RAM size. That's getting better too. Last year, DDR3 prices fell through the floor. In addition, the flooding bumped disk drive prices 2X-3X, which has given the SSD market quite a boost, so flash memory price/performance should be improving quickly soon. Be patient. Digital cameras are a very young technology that is still evolving. In the meantime, it's good research for me.

Of course it's the buffer management that sucks. With proper buffer management (look at Linux for a mostly working example if you like) it could take an hour to write a single image to the card, and I could still take three images and press play and see them right away. There is certainly enough RAM in the camera for that.

Let's give a concrete example: Take a K-5 (or any other Pentax DSLR, I assume), set it to RAW DNG, enable instant review. Take a picture, note how long it takes before you see it on the LCD. Turn off instant review, take another image, press play, and see how much longer it takes before you see it. (Indeed, if you press play quickly you'll wait forever because the camera ignores buttons a little while after taking an image, but that's separate from the buffer management issues.)

[ProfHankD]

JPEG compression isn't that big a problem here, the bigger problem is rescaling the data to an 8 bit perceptual colour space. This happens with PNG too.

PNG supports 16 bits per color channel with lossless compression, although 16-bit coding is used mostly in scientific and cinema image processing. Pre-JPEG2000 JPEG (which is what cameras use) really destroys the data by setting gamma to something around 2.2, transformation to YUV colorspace, reduction of color data, and 8x8 block DCT-based frequency-domain compression. Here's a scary example from an Intro to Digital Photography short course I gave back in 2006:



Looks like a big problem to me. Of course, JPEGs of natural scenes rarely look so bad -- the whole point of JPEG is that you don't notice what's gone. However, when you postprocess, you make partially lost portions of the image data more visually significant, and that's why the results look bad.

Several years ago, I used a genetic algorithm to evolve a JPEG-to-raw conversion program for a particular Canon camera, and just last week there was a research presentation here at the University of Kentucky on the same topic using a different approach. In any case, the trick is that full resolution JPEGs actually have 3 color channels at each pixel, but the raw data only has one, and so recovery is less than perfect but better than you might expect. Much more is lost if the JPEG is at less than 1/4 the sensor pixel count -- which is where the 16-bit PNGs really shine.

[ProfHankD]

Let's give a concrete example: Take a K-5 (or any other Pentax DSLR, I assume), set it to RAW DNG, enable instant review. Take a picture, note how long it takes before you see it on the LCD. Turn off instant review, take another image, press play, and see how much longer it takes before you see it. (Indeed, if you press play quickly you'll wait forever because the camera ignores buttons a little while after taking an image, but that's separate from the buffer management issues.)

Yeah, that sucks, although it's still not exactly buffer management. It's more a matter of which tasks are being given priority... bad scheduling choices... combined with a very different processing pipeline for instant review vs. review. Instant review is usually straight from the sensor buffer data, whereas playback review usually reads the file back from the card (although admittedly on systems like Linux the system buffer would probably have still had a copy of the last image in memory). Use of FAT for the filesystem doesn't help either. Naturally, reading the file back from the card has to wait until the file has been completely written (not really, but somebody probably thought that). In some cameras, these types of bad choices are dictated by the use of custom hardware support for things like Bayer demosaicing and JPEG compression, but I doubt that's the case in the K-5.

In summary, I don't recall ever saying that all the software in cameras is well designed and well implemented. Let's just agree that Pentax should be ashamed of the little sequence that you described and leave it at that.

[jpzk]

This thread is getting old .... but I am riviving it!

With the pixel/MP war(s) still ongoing, and it looks like it will not end very soon (or ever?), one has to wonder about the quality of the images rendered by the "boosted" sensors nowadays.

What I mean is this: since I do not own a super-telephoto lens, longest focal being 300mm, I often need to crop some of my images up to 50% in order to be able to show a "decent size" image.
Anything beyond that will result in "pixellization" (is this the right term to use?) and therefore loss of details.

Would an APS-C sensor of 24mp allow me to crop that same image beyond 50% and still allow room for further cropping without compromising the details too much?
Would a FF-sensor camera at "only" 16mp give me better results?

Isn' it a fact that the size of the pixels/photosites is what really matters?

I was thinking about this when I read some of the recent posts (already several hundreds) regarding the supposedly upcoming K30 and the possibility (speculations) of a future "K3" with a 24mp sensor ...

Apart from buying a 500mm lens at a very high cost, would a 24mp APS-C sensor be the logical and less expensive alternative?

JP

P.S.: I shoot RAW, 100% of the time.

[ElJamoquio]

Generic remarks, not directly intended and perhaps not even germane...

If you have a perfect lens and a perfect sensor and a large amount of light, cropping (whether a smaller sensor or larger sensor artificially truncated down) makes a lot of sense.

If you have a real lens and a real sensor you get a decent improvement from using longer lenses and larger sensors for the same equivalent FOV.

Would more MP help you - well, perhaps. Honestly cropping your picture from 16 MP to 4 MP will have a negative effect for printing... for on-screen, not really. Going to a 24 MP sensor and cropping to 6 MP will be better of course, but still perhaps not quite what you'd want for printing.

I once put a Q on a DA* 300mm. I was not impressed.

[jpzk]

Generic remarks, not directly intended and perhaps not even germane...

If you have a perfect lens and a perfect sensor and a large amount of light, cropping (whether a smaller sensor or larger sensor artificially truncated down) makes a lot of sense.

If you have a real lens and a real sensor you get a decent improvement from using longer lenses and larger sensors for the same equivalent FOV.

Would more MP help you - well, perhaps. Honestly cropping your picture from 16 MP to 4 MP will have a negative effect for printing... for on-screen, not really. Going to a 24 MP sensor and cropping to 6 MP will be better of course, but still perhaps not quite what you'd want for printing.

I once put a Q on a DA* 300mm. I was not impressed.

Thanks for the reply.

I suppose my question was posted because of the few comments I have read in the recent past regarding the fact that more MP is not necessarily better, and that it is all about the "quality" of the sensor "sites".
Of course, a great lens will help, as you mention.

As for cropping, I get the idea. Thanks again.

JP

[Anvh]

PNG supports 16 bits per color channel with lossless compression, although 16-bit coding is used mostly in scientific and cinema image processing. Pre-JPEG2000 JPEG (which is what cameras use) really destroys the data by setting gamma to something around 2.2, transformation to YUV colorspace, reduction of color data, and 8x8 block DCT-based frequency-domain compression. Here's a scary example from an Intro to Digital Photography short course I gave back in 2006:



Looks like a big problem to me. Of course, JPEGs of natural scenes rarely look so bad -- the whole point of JPEG is that you don't notice what's gone. However, when you postprocess, you make partially lost portions of the image data more visually significant, and that's why the results look bad.

Several years ago, I used a genetic algorithm to evolve a JPEG-to-raw conversion program for a particular Canon camera, and just last week there was a research presentation here at the University of Kentucky on the same topic using a different approach. In any case, the trick is that full resolution JPEGs actually have 3 color channels at each pixel, but the raw data only has one, and so recovery is less than perfect but better than you might expect. Much more is lost if the JPEG is at less than 1/4 the sensor pixel count -- which is where the 16-bit PNGs really shine.

Only problem is a photo in 16bit PNG is just as large a RAW file so why would you limit yourself, RAW gives you many advantages over PNG without increasing the file size.

JPEG XR seems to be coming to camera's next year, that solves most of the problems you're talking about.

[Anvh]

About pixels, they are working about using the chips they are using for storage in SD card to make them into image sensor, that would mean a lot of pixels.

They are trying to let the pixels do more then just record red green and blue so overall the quality might actually go up.