Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
08-16-2007, 06:31 AM   #16
axl
Veteran Member




Join Date: Mar 2007
Location: Nove Zamky, Slovakia
Photos: Gallery | Albums
Posts: 7,181
QuoteOriginally posted by RiceHigh Quote
I'm a measurbator but I have never encountered the case that the WB setting with a PEF RAW file generated from my K100D could not be overriden. :-) At least this is true for the original Photo Lab. So, it would be most likely to be a software issue for your case afterall. Btw, which RAW convertor did you use?
I tend to agree here. Pentax suplied Photo laboratory uses value which was set in camera for the shot, but it will let you override it without any issues. So wheter you shoot with AWB or any preset, when you convert it with Sylkipix photo laboratory, you can select any other preset or manualy set temperature/select grey point, and results will be the same.

08-16-2007, 07:49 PM   #17
Veteran Member
WMBP's Avatar

Join Date: Dec 2006
Location: Dallas, Texas
Photos: Gallery
Posts: 2,496
The raw file (whether PEF or DNG) saved in the camera saves exactly what the camera "sees". If you shoot raw, you may for just about all practical purposes ignore the problem of white balance completely while you're shooting. This is one of the nice advantages to shooting raw. I leave the camera on auto WB and haven't thought about white balance in months.

Even if I did think about it, it has no effect on what's saved in the raw file. I am not certain but I think that the white balance setting does have an effect on the preview that the camera displays on the LCD. But I am never very interested in the colors of the preview -- mainly I'm looking at the histogram and/or the flashing indicators of blown highlights or darks.

If you don't save the raw data and instead ask your camera to convert to jpeg, well, the camera ends up throwing away a lot of info. The decision about what to throw away and what to keep is influenced by your white balance setting. That's why, if you save to jpeg in-camera and you screw up the white balance, there's only so much you to can to fix it in post-processing.

Will
08-16-2007, 09:25 PM   #18
Forum Member
Chuck Lee's Avatar

Join Date: Jan 2007
Location: Williamsburg, VA
Posts: 55
Let's test this theory with Photos

arbutusq,

As I said before I use SilkyPix Development Studio 3.0. Why? Because Pentax Photo Lab is powered by SilkyPix. I've used ACR and am not happy with the rendering intent. I've used Bibble Labs which is great but they were real slow getting the K10 PEF converter. I've used Pentax Photo Lab which is simply an embarassment for Pentax. I've used the freeware deals with unsatisfactory results. You can download a 30 day free trial of Silkypix to play around with it yourself here: SILKYPIX Developer Studio

In the meantime, I took 5 raw photos with my K100D of a Color Chart printed out on plain white paper under a desk lamp. I mounted the camera on a tripod and set the camera WB to 5 different settings. AWB, Daylight(DWB), Tungsten (TWB), Flash(FWB), and Manually Adjusted(MWB).

I copied each PEF twice and renamed the two duplicates *@2K and *@5K
I opened the raws up in SP and set all of the 2K's to 2000K, 0 Color Deflection and 0 Dark Adjustment. I did the same on all 5Ks.

Here's the resulting screenshot: Hope I don't get in trouble for this slightly larger image.



I can not see any difference in the color balance through each adjusted set which confirms what most members here are saying. RAW is RAW. Any other variation would have to be the result of the RAW conversion software and how it handles the PEF data.

I truly believe that the in camera WB settings are for preview on the K100D and for preview and in-camera jpeg development for the K10D.

Hope this helps,

Chuck
08-17-2007, 06:59 AM   #19
Veteran Member
jeffkrol's Avatar

Join Date: Sep 2006
Location: Wisconsin USA
Photos: Gallery
Posts: 8,434
Hmmm... not sure

If anyone bothered to read the dpreview thread you may not be so quick to assume RAW is RAW for all models...though the general consensus is it doesn't really make much difference..........................
Unequivocal evidence: Pentax SLR Talk Forum: Digital Photography Review
Or specifically, which implies adultered RAW due to WB for models other than the K10:
Again, note that the K10D appears ***not*** to do relative channel gain adjustments as do the former models.
Re: Similar with XT...: Pentax SLR Talk Forum: Digital Photography Review
I tried to revive this topicin a new thread at dpreview to see if the bitheads (ie my own funny way of saying programmers ect.) came up w/ any new conclusions but it was ignored. Think I'll just bump the old one.
ONE thing to note is that along w/ the RAW and WB discussion there is the discussion about high iso's and exposure. Seems, as in the Nikon groups at dpreview, many suggest NOT using high iso, just underexposing at base iso for the best results. My playing w/ my D seems to confirm that for me. If needed a 2-3 stop underexposure at, say iso 100-200 is better than a "good" exposure at iso 800-1000

08-17-2007, 07:42 AM   #20
Forum Member
Chuck Lee's Avatar

Join Date: Jan 2007
Location: Williamsburg, VA
Posts: 55
jeffkrol,

If I am interpreting correctly what this post is saying then color channel clipping occurs at certain WB settings which can result in lost color information. If you look at the screenshot I supplied in the previous post there may be some evidence there to support the hypothesis. Notice the histogram for the first image selected. I believe the red channel is shown clipped. Please, someone correct me if I'm wrong. The histogram for the 2K version shows no signs of red clipping. I imagine (this is subjective) that there is enough headroom per channel (about .7-.8 ev) to recover that red channel data during the post convert to 2K. What's interesting, is that the Tungsten WB sample is brighter than the Maunually Adjusted WB sample. Could it be that when using in-camera WB adjust the camera modifies it's raw output intensity to cover all channels and prevent channel clipping?

So my conclusion, and answer to arbutusq's question of "Which button should I push for best results?" would be to set the camera's WB manually for the lighting condition I'm shooting in. I sorta do this by practice anyway since Pentax AWB does not work very well for incandescent indoor lighting. Everything else falls under AWB for daylight, Flash for flash (unless I'm using an amber gel and then I use tungsten).

The reason of interest in this topic for me has more to do with noise at high ISO. I tested for quite some time to see if the in-camera WB settings would effect the level of noise generated at ISO 3200 in low light. Color Noise, especially red, is very noticeable and I was hoping to discover some WB trick to help mask it. The only thing I discovered was that no matter what the WB in the camera was set to I would get the same results in post. One cool thing I did discover was that you can mask the red noise by turning up the post color temperature to around 12K and convert to monochrome. I'm sure this is not new info to many of the knowledgable folks here at Pentaxforums.

This is a really good discussion and as always, a good learning experience.

Chuck
08-17-2007, 08:15 AM   #21
Veteran Member
WMBP's Avatar

Join Date: Dec 2006
Location: Dallas, Texas
Photos: Gallery
Posts: 2,496
I am not sure that looking at histograms tells us about the raw data. The histogram shows you the image as interpreted by your software (Lightroom or whatever). It doesn't tell you about the raw data itself. This is like trying to have a subtle argument about Biblical interpretation, where nobody's looking at the Hebrew or the Greek.

The in-camera white balance apparently setting gets stored with the raw file, I guess as EXIF info. I just took two photos of my kitchen, using two different white balance settings (flourescent light for one, cloudy day for another). Saved files in camera as PEF. Imported photos into Lightroom. Of course they look quite different at first -- and the histograms are quite different, too. But so are the white balance settings shown by Lightroom. One file is shown initially with a temp of 2750 and a tint of +5; the other is shown with a temp of 6150 and a tint of -16.

But if I change the settings for either file to equal the temp and tint of the other file -- they look identical. Histograms are identical, too.

I would think that the only way to know if this raw file's data = that raw file's data really, would be to crack open the files and view them in their raw, uninterpreted state and see if the numeric values were in fact the same or different. Photoshop, Lightroom, Aperture, Bibble -- all of these are interpreters. You don't see the raw data directly, ever. But the key point is that the raw data is still there, underlying your interpretations.

Of course, the raw data itself involves a great deal of interpretation. The camera takes the electronic responses of the sensors and does calculations that ultimately represent fine shades of light and dark or color. I suppose the question would be, does the white balance setting in the camera affect the camera's interpretation of the sensor data? I doubt it. The white balance setting seems to me a very high level setting, while the process of converting sensor responses to raw data undoubtedly occurs at a very low level. In any case, I don't think you'd be able to tell by looking at something as crude as a histogram or even a representation of the image on a computer screen.

Am I missing something?

Will
08-17-2007, 11:47 AM   #22
Veteran Member
jeffkrol's Avatar

Join Date: Sep 2006
Location: Wisconsin USA
Photos: Gallery
Posts: 8,434
QuoteOriginally posted by WMBP Quote
I am not sure that looking at histograms tells us about the raw data. The histogram shows you the image as interpreted by your software (Lightroom or whatever). It doesn't tell you about the raw data itself.
Right, but the Malykh software does just that. At least it looks at the RAW file saved PRIOR to applying any of the tags which an editor will do (believe you can shut some of these off in Bibble or some such editor) I'm assuming this is also non-gamma corrected as well which is also a tag. So image histogram should look identical w/ ANY WB setting. The thread seems to show that this is not always the case BUT I'm not sure that the images can be considered totally identical or that small variations don't show up. Afterall you have to take a series of shots w/ different WB in COMPLETELY identical conditions and then generate the RAW histograms (no editor can do the but the software by Malykh and modified by Gordon can)and compare statistically. Not an easy task and of course for real science needs to be repeated x number of times to account for shutter/diaphram variations ect.
A pure RAW file pre WB, gamma corrected ect. is really ugly. Colors are bad,dark and just NOT a real photo.
Found a good picture of a RAW file "as shot". Click on the 11mb TIFF after you go to the wedsite, if you have the bandwidth or don't mind waiting...
Raw File View

QuoteOriginally posted by WMBP Quote
The in-camera white balance apparently setting gets stored with the raw file, I guess as EXIF info. I just took two photos of my kitchen, using two different white balance settings (flourescent light for one, cloudy day for another). Saved files in camera as PEF. Imported photos into Lightroom. Of course they look quite different at first -- and the histograms are quite different, too. But so are the white balance settings shown by Lightroom. One file is shown initially with a temp of 2750 and a tint of +5; the other is shown with a temp of 6150 and a tint of -16.

But if I change the settings for either file to equal the temp and tint of the other file -- they look identical. Histograms are identical, too.

I would think that the only way to know if this raw file's data = that raw file's data really, would be to crack open the files and view them in their raw, uninterpreted state and see if the numeric values were in fact the same or different. Photoshop, Lightroom, Aperture, Bibble -- all of these are interpreters. You don't see the raw data directly, ever. But the key point is that the raw data is still there, underlying your interpretations.

Of course, the raw data itself involves a great deal of interpretation. The camera takes the electronic responses of the sensors and does calculations that ultimately represent fine shades of light and dark or color. I suppose the question would be, does the white balance setting in the camera affect the camera's interpretation of the sensor data? I doubt it. The white balance setting seems to me a very high level setting, while the process of converting sensor responses to raw data undoubtedly occurs at a very low level. In any case, I don't think you'd be able to tell by looking at something as crude as a histogram or even a representation of the image on a computer screen.

Am I missing something?

Will

Last edited by jeffkrol; 08-17-2007 at 11:53 AM.
08-17-2007, 12:08 PM   #23
Veteran Member
Ivan Glisin's Avatar

Join Date: Nov 2006
Location: Belgrade/Toronto
Posts: 656
RAW is RAW, tested!

QuoteOriginally posted by arbutusq Quote
It appears thatt WB is being applied to the RAW data before being saved as a PEF.
Fortunately that's not the case. I have developed a small C program to read RAW data straight from the binary file. Under incandescent light with incandescent WB setting image of a white wall appears white on the screen (in any RAW processing software).

However, directly accessing binary RAW file with my utility and reading 12-bit integers from the Bayer pattern shows that "R" photo sites have noticeably higher values than "G" and "B" photo sites, which is to be expected for incandescent light which is a "warm" light source.

This supports the theory that no WB adjustment is being made to the RAW file data.

Case closed.

08-17-2007, 01:04 PM   #24
Veteran Member
jeffkrol's Avatar

Join Date: Sep 2006
Location: Wisconsin USA
Photos: Gallery
Posts: 8,434
QuoteOriginally posted by Ivan Glisin Quote
Fortunately that's not the case. I have developed a small C program to read RAW data straight from the binary file. Under incandescent light with incandescent WB setting image of a white wall appears white on the screen (in any RAW processing software).

However, directly accessing binary RAW file with my utility and reading 12-bit integers from the Bayer pattern shows that "R" photo sites have noticeably higher values than "G" and "B" photo sites, which is to be expected for incandescent light which is a "warm" light source.

This supports the theory that no WB adjustment is being made to the RAW file data.

Case closed.
Care to debate graph 4: This is also from a RAW data "examiner" program.
Seems to show red shifts in RAW and more troublesome to me, really bizarre "manual wb" effects on raw data... RAW vs White balance: Pentax SLR Talk Forum: Digital Photography Review
08-17-2007, 03:29 PM   #25
Veteran Member
khardur's Avatar

Join Date: Feb 2007
Location: NW Massachusetts
Photos: Gallery
Posts: 1,496
a long-winded test. (sorry!)

I actually think that something IS altered in the RAW file depending on your white balance. I first took notice when I was taking some Infrared photos using a Hoya R72 filter.

Let me explain what happened first, then I'll post some test photos I just took a few minutes ago.
(and let it be known, I'm not a "measureba..." or whatever that is.. *shudder* - i.e. this is a somewhat unscientific test)

Shooting IR the other morning, halfway through I realize I had forgotten to take a manual WB reading. Now I thought, "okay, but I'm shooting in RAW, I should just be able to process my photos like normal and they should turn out exactly the same" Right?

WRONG.

Processing my IR pics requires me to set the white balance in my editing program beyond what the camera determined was proper. So I set my white balance, in this case using Photoshop and ACR - to 2200 and a -60 on the tint scale. The pictures taken without a WB taken didn't turn out the same as the ones taken with a WB reading. Even setting them all at my default starting point - I just couldn't produce the same tones, no matter what I did.

What it all means, I don't know. But I know I can't get the same results if I don't take a WB reading, and yet I adjust the WB in post processing to something within it's limits...

my first thought was "well maybe the camera can adjust the WB much farther than my post processing program limits me" ACR's color temperature adjusts from 2000-50000, PPL goes from 2500-10000.

But since PPL registers my manual white balance as 3350 or so - and I manually adjust the WB to 2200 or so in ACR - that conlusion doesn't make sense to me.


Anyway - here are two test pictures. One using a manual white balance, the other using "daylight" Both pictures were 1/25 sec at f/2.8, so I could get decent hand held times and do them quickly after changing the white balance.


This is the first set, processed in ACR. The first one I did a manual white balance in camera. For both pictures I set the white balance in ACR at 2200 and -60 on the tint, with no other adjustments to either one, other than resizing.
Attachment 4412
This is the second shot, with "daylight" as the white balance in camera, but processed exactly the same in ACR.
Attachment 4413



Now here are the same two photos, processed in PhotoLab at the same time, changing the WB in PhotoLab to 2500 and the "fine tunes" all the way toward "blue" and "green"
again, in-camera manual WB:
Attachment 4414

in-camera "daylight" setting:
Attachment 4415


Again - does the camera record different RAW data depending on the WB? I don't know for sure, I'm not a measuring sort, judging by the fact that I've got nearly 5K pictures and this is one of my first "test" series of shots.

All I know is, theoretically shooting RAW should allow me to take two pictures of the same scene in the same light with different white balance settings, change the white balance in post processing to the same setting, and come up with nearly identical colors.

It doesn't. But not that I'm mad about that - my (good) IR pictures come out wonderful, and whatever it changes when doing the manual WB is a good thing - because it obviously records things slightly differently, and allows me to come up with pictures like this:

(I had to soothe your eyes after those nasty test shots.)
Attachment 4421

Last edited by khardur; 01-29-2008 at 02:47 PM.
08-17-2007, 04:31 PM   #26
Veteran Member
WMBP's Avatar

Join Date: Dec 2006
Location: Dallas, Texas
Photos: Gallery
Posts: 2,496
QuoteOriginally posted by jeffkrol Quote
... A pure RAW file pre WB, gamma corrected ect. is really ugly. Colors are bad, dark and just NOT a real photo.
Found a good picture of a RAW file "as shot". ....
A pure raw file doesn't have colors at all, isn't dark or light, doesn't look like anything. It's merely a lot of data. I presume that it's more or less numeric in nature, like this:

8593766944855959403348454958
9483944855033078574584573339
9830493980554443939494494444
3933933333333330000000119485
8953434534536869449494494949
9449583305535354968686866686

As soon as you start looking at visual representations of raw images, you're dealing with conversion. Forget about white balance and how it might affect the look of the result. Take the very same raw file, and view it in a couple of different programs on the same computer, or in the same program on different computers, and you're likely to see different things. That's not surprising.

The only interesting question here, in my view, is one I certainly can't answer, namely: is the actual raw data affected by the white balance setting at the time of capture? I'm not asking if you can see a difference in two pictures on screen. If you can see something that looks like an image, you're way past the point of interest in this matter. I'm asking if the numeric data is the same or different, not just before gamma correction, but before any attempt to convert the numbers back to something visual.

Will
08-17-2007, 04:58 PM   #27
Veteran Member
jeffkrol's Avatar

Join Date: Sep 2006
Location: Wisconsin USA
Photos: Gallery
Posts: 8,434
QuoteOriginally posted by WMBP Quote
A pure raw file doesn't have colors at all, isn't dark or light, doesn't look like anything. It's merely a lot of data. I presume that it's more or less numeric in nature, like this:

8593766944855959403348454958
9483944855033078574584573339
9830493980554443939494494444
3933933333333330000000119485
8953434534536869449494494949
9449583305535354968686866686

As soon as you start looking at visual representations of raw images, you're dealing with conversion. Forget about white balance and how it might affect the look of the result. Take the very same raw file, and view it in a couple of different programs on the same computer, or in the same program on different computers, and you're likely to see different things. That's not surprising.

The only interesting question here, in my view, is one I certainly can't answer, namely: is the actual raw data affected by the white balance setting at the time of capture? I'm not asking if you can see a difference in two pictures on screen. If you can see something that looks like an image, you're way past the point of interest in this matter. I'm asking if the numeric data is the same or different, not just before gamma correction, but before any attempt to convert the numbers back to something visual.

Will
Actually what I've been trying to show is that the software mentioned in the links I've posted generate a statistical representation of a RAW files numerical values. As Gordon puts it:
................................................
The software works as follows:

1. Find the start of the raw data.

2. For all the rows and all the columns determine what colour channel the reading will be binned against from the Bayer array of the pattern such as:
RG
GB
repeated horizontally and vertically as for the Pentax cameras.

3. Uncompressed/Unpack/otherwise read the 12 bit data (again in the case of Pentax).

4. According to the value read, increase that bin number for that colour by one (all bins having been originally be initiallized to zero counts).

5. When all rows and columns have been read and binned, display all the bins (values) across the horizontal axis for each colour, with the colours being the logical or of the combinations of colours (ie additive colours) and the height proportional to the number if times the bin was increments.

That's a raw data colour histogram with no processing and that's what was produced here. If you read Java, the source code was provided with the program and one can see that this is what's done.
............................................................
which will show you if any transforms or manipulation have occurred due to WB settings... Maybe my RAW anology is a bit off (I mean we could get to the photon and charge capture level even before a numerical representation
or even analog voltage measurements prior to digitalization but I guess I was drawing a line somewhere). If we assume it's when the numbers are created is that at pre-debayering or post? I really have only a cursory understanding of all this but I believe enough to understand SOME basics
and yes some people do forget that the "picture" they see is not the RAW that is there BUT by not applying gamma correction and WB you can get a MUCH better picture of what a RAW file really is (post de-bayering and digitalization) but no it cannot be used to quantify wheter RAW has been tampered with BUT by inference you can possibly see that "something" in the raw has changed. So I understand what you are trying to say. Not sure you are understanding what I am trying to say though...
08-17-2007, 06:19 PM   #28
Veteran Member
jeffkrol's Avatar

Join Date: Sep 2006
Location: Wisconsin USA
Photos: Gallery
Posts: 8,434
QuoteOriginally posted by khardur Quote
I actually think that something IS altered in the RAW file depending on your white balance. ]
Nice IR there. And your info seems to fit the supposed pattern re: Manual WB.
The preset vs preset should not be that far off so if you do one in tungsten and 1 in daylight your "corrections" should have them coming out the same or at least closer than your manual vs preset. You may want to try that for fun...
BTW: What are you using to set the white balance and for what reason?
You may want to set a custom WB w/ a CC40M filter. That forces the color channel multipliers (gain I guess) to 1:1:1. May come in handy for IR, and then again maybe not. I've tried to see if my D files are modified using the software but apparently there is a problem w/ them and the program. Do believe Gordon "corrected" the programming for the D but will have to wade through that thread again. this is what a D file looks like w/ the RAW "viewer"
... something amiss. I know the program works for ds/dl series and K series BUT the D files are a bit of an oddball. 16bit w/ lots of zeros....
08-17-2007, 06:47 PM   #29
Veteran Member
Ivan Glisin's Avatar

Join Date: Nov 2006
Location: Belgrade/Toronto
Posts: 656
QuoteOriginally posted by jeffkrol Quote
Care to debate graph 4: This is also from a RAW data "examiner" program.
Very interesting!!! I did not try manual WB. Histograms for other modes are aligned with my results, however it seems that manual WB indeed manipulates raw data.

When I think about it, this in fact makes perfect sense: all preset WB modes are only approximations and can be even set by mistake. Engineers at Pentax have decided not to modify RAW data for preset WB modes and let software make adjustments later.

However, using manual WB assumes that photographer knows exactly what WB is and how to properly set white point manually. By doing manual WB step we are in fact precisely calibrating our cameras and I don't see a single reason why not applying calibration parameters to the source data immediately? (This logic is used in engineering and scientific instrumentation as well: you don't receive the "wrong" data and calibration parameters -- you get the right data after "embedded" calibration gets applied as close as possible to the signal source.)

Manual WB implementation can perhaps add an additional benefit of some sort of per channel dynamic range optimization to minimize channel clipping and/or noise per channel, especially in lighting condition that are "way off" compared to daylight.

And now here is one interesting thought: channel clipping suppression and per channel noise suppression should work better if sampling resolution is higher than output RAW resolution, say 22 bit sampling and 12 bit output. And looking at those numbers for some reason :-) I am beginning to think about the K10D PRIME engine: in theory, K10D should be able to produce better color under harsh lighting if manual WB is used. This would be in fact possible to test. (I am still DS*ist however, K10D*ist is required for the job.)
08-17-2007, 07:49 PM   #30
Veteran Member
khardur's Avatar

Join Date: Feb 2007
Location: NW Massachusetts
Photos: Gallery
Posts: 1,496
Jeffkrol: I've do a lot of experimenting - doing WB with the IR filter on shooting a grey card (actually an 18% gray cloth that doubles as a lens cleaning cloth) held in the sun. Sometimes I WB on trees as well, if I have enough of them in full sun to do a WB.

Hmm... so I definitely think you guys are onto something here regarding something being fine tuned during manual white balancing, because I get horrible clipping and bad colors doing IR if I don't do a white balance. Theoretically it shouldn't matter shooting RAW, but it does.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
camera, data, dslr, photography
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
RAW duplicate images - saving as RAW files jpzk Digital Processing, Software, and Printing 17 02-15-2010 07:01 PM
What's the easiest way to process RAW shots for a total RAW noob? Grimlock Digital Processing, Software, and Printing 6 09-03-2008 02:22 PM
Does the Photoshop RAW converter work with Pentax RAW Files? mconder Pentax DSLR Discussion 3 05-15-2008 12:14 PM
RAW, New Computer, PSE5 & 3.7 RAW Plug in. Ed in GA Digital Processing, Software, and Printing 6 11-20-2007 04:43 PM
Raw Therapee: free RAW converter/photo editor wheelgate Digital Processing, Software, and Printing 10 10-12-2007 06:44 PM



All times are GMT -7. The time now is 08:43 AM. | See also: NikonForums.com, part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top