I only have a cursory knowledge Dark frames and Bias frames but would like to know more details about them and how my K-1, or any DSLR would utilize or process these?

So, I learned about Light Frames, Dark Frames, Bias Frames, and Flat Frames from the DeepSkyStacker FAQ which I'll paraphrase below.

Light Frames
These are the bits you are photographing - they contain the image data - the "light" that you've deliberately set out to capture. You probably know this part already, but I thought I'd include it here for the sake of completeness.

---

Dark Frames
These are used to capture (and later subtract) the "dark noise" from the signal. This is a combination of Johnson-Nyquist noise, and various EMI from the rest of the camera.

Put simply, when the sensor is sitting there, due to Johnson-Nyquist noise, it is always generating some sort of signal. This type of noise is referred to as "Shot-Noise" - it looks like little spikes in an oscilloscope. The hotter the sensor, the more noise you get.

Overlaid on this is electrical noise from the camera - this is just the processor, the screen, the memory busses, and all sorts of other things. It is coupled across or transmitted directly through the camera wiring and you can't prevent it - you can only compensate for it!

You'll need 10-20 of these shots, every time you set up.

---

Bias Frames
Bias Frames are used to capture (and later subtract) the noise generated by the camera reading the sensor.

In addition to the noise above, the act of reading from the sensor also produces more noise - the idea of this frame is to capture that noise's effect on the final image so we can subtract it.

You'll need 10-20 of these shots, every time you set up.

---

Flat Frames
Flat Frames are used to correct for the lens itself - vignetting, dirt, etc. It will be again be subtracted from your light frames in your final image.

Again, you'll need 10-20 of these shots, every time you set up.

---

Dark Flat Frame
This one didn't seem to be covered very well on that page I linked to. As far as I know, it's just the same as a flat frame... but with the lens covered by something opaque. Note the settings used in creating the Light Flat Frame (above) and replicate that with the cap on.

I'm not convinced this is absolutely necessary as it would be covered by the dark frame... BUT... I may be wrong, especially since it is listed in numerous places as a thing that you should do.

I believe it may be used to further process the dark frame by attempting to remove artifacts caused by the optics under different exposure conditions.

[HR][HR]



To manually use these correction frames, you must combine each group of 10-20 frames into a master file by averaging them - giving you one master Dark Frame, Dark Flat, Bias, and Flat frame.

So... here's the annoying part... for EACH light frame, you must subtract your Dark, Dark Flat, Bias, and Flat frame.

If, like me, you have 10-100+ light frames, you can see why this might be tedious!

After doing that for EACH light frame, you can stack them together to get your final image.


Thankfully, you can use software to perform this autonomously. Most stacking applications should also allow you to combine and subtract your correction frames (listed above), and then stack the results. My personal favourite is deepsky stacker.

Now, as far as I know, DSLRs don't process these in-camera, it is handled at the image processing side.

Dark and bias frames aren't really used in the camera unless you have high ISO or long shutter noise reduction turned on. If you have them turned on the camera will take a picture of the back of the shutter at the same ISO and exposure length as the original picture. It will then subtract this dark image from the light image the camera took. The thinking is that the dark frame should only show systematic noise that is very similar to what would have been present in original shot so you can just subtract it. This however is not entirely true as there is truly random thermal and quantum noise that is also present in the dark frame that will also be subtracted. Since that noise is truly random and is in both the dark and the light frame subtracting it doesn't do much harm but the decrease in overall noise is less than the theoretical ideal. There will be cases where you are subtracting true random noise that causes you to hit 0 when it should have been a negative value, there would also be cases where you don't subtract something when you should have. Thus is the nature of randomness. Because of this you are hoping that the dark frame's systematic noise is the dominant thing being captured which is reasonable with one shot which is why there is a benefit to using the in camera noise reduction feature. However substantially better results can be had by capturing many dark and bias shots to drive out the true random noise and processing in a program on your computer.

The camera when doing its noise reduction does not take bias frames as those you really need a bunch get to the systematic noise as it usually gets lost in all the other noise.

The best way to use noise images is with a stacking program like Deep Sky Stacker (DSS), Sequator, Astro Pixel Processors (APP), etc. These program can consume a bunch of these shots and combine them properly to get a true picture of the systematic dark frame noise, and the systematic bias frame noise. I highly reccomend collecting and using flat frames as well with these programs as it correct vignetting in each shot giving you a better starting point to begin your edits after stacking.

Dark frames are very environment dependent, meaning you want to take them at the same temp, ISO, and exposure length as your light frames. Taking one provides minimal benefit but taking a pile will provide more. When the pile of them are combined and averaged the systematic noise show up as you drive the truly random noise down. This systematic noise is mostly the noise from the sensor and also the dark current noise. When I take these I usually do so at the end of the session as I am tearing down my equipment. I will typically end up with 30 to 40 images which doesn't in my mind seem to be an excessively small or large amount and would seem to be something that I would call about right. I will also take dark frames if there is a cloud that is going to be rolling by for a few frames until it leaves. If I take dark frames they are for the session and I don't keep a master library of them as that seems like more work than is needed.

Bias frames are not very environment dependent and can be captured at your convenience. What you are trying to do here is to capture the systematic noise from the read, AtoD converter, and signal amp portions of the camera system which on a modern camera should be very low which is why capturing them requires a large number of shots. With these I basically put the camera some place dark, set the shutter to the maximum speed, ISO to what I would shoot at, put the camera into the slow burst mode and walk away letting it take a huge number of these shots. By a huge number the general consensus that you really want at least 200 of these more is better but because noise goes down as 1/square root of the number of shots you really need lots more if going beyond 200 shots to see any real benefit, like 400 or even 800. Remember we are trying to capture the systematic noise for what should be very low noise components so you need to drive the other truly random noise floor below the systematic noise of those components which is the reason for the large number of shots. Typically I end up with something like 350 to 400 shots before I get board of taking these. I then have a set of masters for each ISO I shoot at saved on my computer ready to be used when I need them.

Related to these are flat frames which are frames that capture the vignetting of the lens/scope you are using at a given f-stop. Here f-stop is the important aspect as is having a clean sensor. Cover you lens/scope with a white t-shirt stretched over the front when it is focused to infinity. Point it at a reasonably bright uniformly lit area of the daytime sky (cloudy or hazy days work great for this) with the camera in Av mode with your aperture set to what you shoot astro images at with it and blast away. I usually take a buffers worth of shots at ISO 100 with a +.3 or +.7 EV compensation so that I can have a good average to work off of so that I'm not bringing noise into the system when correcting vignetting. Again since these are not session dependent you can shoot them at your continence and have a master set for each lens/f-stop combination you use.

While @cprobertson1 is correct in needing 10-20 frames of each type, the Dark and Bias frames can be reused, so you only need to shoot these one , or once every 6 months or so and they can be shot anywhere, anytime.

For Bias frames, cover the lens with a lens cap and set the shutter to the highest speed possible (1/8000) and fire off a bunch of frames, these become your bias frames and are used to subtract dark current from the frames.

Dark frames are typically shot at the same exposure time, ISO and temperature as your light frames (lens cap on like bias frames). I use one set of darks (about 30 of them) shot at ISO ISO 1600 for about 2-3 minutes, and reuse them for all my images.While it's suggested that darks be shot at the same temp as the lights, that's not critical, as most software will scale the darks. Also the Pentax cameras have very good noise profiles, so are not as thermally dependent on constant temps as other makes

Flats are a different animal entirely, These should be shot after the imaging session with the camera and lens/telescope in the same orientation and focus as you shot the light frames at. they should fit in the 30%-50% of the histogram.

Here's a good article about flat frames: How to Take Flat Frames for Astrophotography [The Easy Way]

Whoops! My bad - I thought they had to be done every time due to noise changes with temeprature vs ISO

I've not really bothered with the correction frames a lot yet - I tend to operate from my back yard where the light pollution is the biggest factor in quality, so I just live with the noise. I have done a few practice shots with them, but I'm saving the real effort for the next time I visit an actual dark site!

Thanks for that!

In my experience, the Sony sensors used in the more recent Pentax cameras are very low noise. As such, you can probably get away without using any dark frames in your processing since stacking will take care of much of the noise, assuming a sufficient number of subs. I only use bias and flat frames with PixInsight. Corrective masters would become more important the dimmer your target and thus the more stretching you must do to get a final image. You will need to experiment with your processing pipeline to see what works best for you.

I too often operate in that sort of environment (bright bortle 8 with a 9 about a mile away). A light pollution filter has been a godsend and one of the best purchases I've made. I have a 77mm one and a 49mm one with the 77 being used on my 300mm and smaller lenses and the 49mm one lives in the drop in filter holder on my 400mm. If you don't have one the hoya red intensifier is one that is well regarded on the cheap and what I use. However the NiSi natural night is suppose to be better so I'm thinking about getting one of those for the for the drop in on the 400.

Everything I have read has also stated that darks are basically session dependent given the sensor heat, exposure time, and ISO used all have effects on the systematic noise you are hoping to capture with darks. You can have a library of master darks but they would be specific to a given sensor temp, ISO and exposure length. I wouldn't think 1, 2, or even 5 degrees would make much difference but 10 or 20 would and that kind of difference isn't all that uncommon within a week where I am.

My preferred method is still to just take darks as I tear things down at the end of a session.

As you say a 5 degree difference won't have much of an effect on darks, but once you take a set of darks at various temps, you can reuse them later on. So you create some at 10, 20, 30, 40, 50, 60, etc... degrees in a year, and you can reuse them the next time you shoot in that temp. range.

Also, a lot of programs are moving towards bad pixel maps as opposed to dark frames.

You can also put the camera in a refrigerator or freezer and shoot dark frames then.

That's actually an interesting question you've raised regarding thermal noise.

So, we have a blackbox - roughly described as sensor<>databus<>ram<>CPU<>memory with other stuff in parallel to that like the screen and the button controls etc.

Each of these black boxes is made of a bunch of components, each of which generate electrical noise. One part of this noise is proportional to temperature - and this is usually listed as the thermal noise coefficient, and is available on a part-by-part basis in the datasheets.

While the electrical properties of the parts can be made to cancel out changes caused by temperature, the thermal noise always increases. Johnson-Nyquist noise is practiclaly a white noise - so irrespective of the part, it increases with temperature.

What we're really interested in is:
A) how much does it increase across the entire system
B) how much does the system filter it out

Because we're looking at a systems level, the error due to noise may be anywhere from parts-per-thousand to parts-per-billion.

The only way to know is to quantitatively measure it at various temperatures by putting it in a temperature-controlled oven and literally just testing it at a number of temperatures, and then measuring the noise produced in a dark frame.

I actually used to have a temperature-controlled oven for exactly this sort of thing... unfortunately, I sold it because I needed a better multimeter...

I suspect, since the Pentax cameras have a reputably good noise profile (I say "reputably" because I have never tested it myself!), that the thermal coefficient as it relates to noise, is fairly low. Thankfully, the errors aren't accumulative because of the way we average them - but at the same time, any increase in error will still translate to a change in image quality.

So from there, it would appear to depend on the exact circumstances of your shot. Your "pre-made" compensation frames will remove a fraction (likely the majority) of the noise under a given circumstance. Even if you remove, say, 90% of the noise instead of 99% - that's still an appreciable reduction.

It's like trying out two cars - one weighs 900kg and the other weighs 990kg. Chances are that you won't actually notice! It's not until you try to push them to their limits, say, on a race track, that the difference in weights becomes apparent.

SO - to conclude - I strongly suspect you may be right in saying that unless you are specifically pushing your equipment to its limits, that you could probably get away with compiling a set of premade dark frames at different temperatures, and if you want more accuracy, temperatures vs ISO.

If, on the other hand, you are pushing the limits of your equipment, then it may be worth taking your compensation frames with every shoot.

---

This part is just for fun, and to demonstrate that you can take your compensation too far...

Fun fact: because multilayer capacitors are made from sandwiched layers, their properties vary with their orientation relative to the gravitational field. Inductor values also change ever so slightly.

This means that the noise profile changes as your camera tracks along its equatorial mount - and if you wanted to really compensate for it, you'd need to take correction shots for every light frame!

Of course, unless your sensor is already down in the parts-per-million error region, adding a few ppm on either side of your measurement isn't going to matter all that much. To continue the car analogy - imagine a car weighing 900kg... and another weighing a gram or two more - unless you are at the absolute limits of what those cars are capable of, then you can't tell there even is a difference... but it get worse! The difference itself, even when observable, is still minuscule!

But it gets worse - we can also compensate for things like the current solar radiation, the current RF environment (the phone or computer you have near your camera adds some noise to it! Is it enough noise to matter? Probably not... but it's there!).

What about magnetic anomalies? Triboelectric effects caused by tectonic plate movement? Come to think about it... what about the movement of the tectonic plates? Turbulence in the atmosphere buffeting your setup... vibrations caused by earthquakes on the other side of the globe... your breathing... and the breathing of people on the otehr side of the planet!

The thing about compensating for everything around you is that you'll eventually reach the point where you've compensated for absolutely everything in the entire universe... allowing you to take an extremely,extraordinarily precise image of whatever you're pointed at... through the filter of the astronomical seeing!

You reach a point where your compensation decreases the error way below the error inherent in having a telescope inside an atmosphere. It's like manufacturing a nail to atom-scale precision and accuracy... only to use it as part of your first ever carpentry project - the errors in your carpentry far outweigh the errors in your nails!

My suggestion? Go to space, get away from that crazy atmosphere!

Fun fact - some space telescopes do compensate for fluctuations in solar radiation! Space is hard.

And there's the rabbit hole I was trying to avoid. LOL

Well this whole thread was basically born as a rabbit hole it was only a matter of time until we go past anything that would be useful to any normal person.

I don't do dark frames. I dither and use kappa-sigma rejection during stacking instead. I don't usually use LENR (Pentax calls it Slow Shutter Noise Reduction) anymore either for the same reason, although if I intend to take only one exposure or less than four subexposures, I will still use it because kappa-sigma rejection in my preferred stacker, Sequator, needs at least four subs.

Dithering also averages out the so called "color mottling" (which is real but is actually an artifact of demosaicing algorithms) and any FPN that may exist. However, I haven't actually seen any evidence of the latter yet in my subs, hence I also haven't bothered with bias frames. So far I have also gotten away without using flats with my lenses because I always stop them down. I do need to use flats with my telescope, however.

AlenK's gallery - AstroBin

Maybe I'll need to use darks, bias and flats for everything when I start pushing the processing hard enough to draw out the faint and esoteric stuff. But not yet.

PS. Using LENR (automatic dark frames subtracted for every exposure) or dark frames after the fact actually increases the random (shot) noise in the image compared to not using either. This is because the truly random component of the thermal signal and the random component of read noise are contained in every dark frame. When it comes to truly random noise, subtracting a dark frame does not subtract that noise: It always adds in quadrature (square root of the sum of the squares of the noises). Nothing "beats down" truly random noise other than getting more signal (i.e., more total exposure time on the object(s) of interest). And technically, the more signal the more noise as well but noise grows as the square root of the signal, so as the signal increases so too does the SNR, which is what we are actually concerned with.

Rabbit holes are fun! We're having a tea party down there - you should join us!

The real question is when it would benefit the average user to do their compensation frames as part of the shoot... vs using stock images you've prepared earlier.

I reckon it depends on how "important" you deem your shooting.

In my case, a lot of my shots are done from the garden, or a local field right outside the town.

For me, the decrease in image quality caused by light pollution is much greater than the decrease in quality caused by the in-camera noise and aberrations - and I should definitely consider making some stock generic frames as suggested above.

On the other hand, if I go somewhere special with half-decent conditions, I would consider it taking the compensation frames.

That's just in my case though - everybody has their own pain tolerances, and their own ideas on what constitutes the effort being "worth it"!

Interesting stuff - I'll need to look into that - I'm pretty sure I just have the generic processing on in the camera (but I'm unsure how much of that applies to the RAW files).

I never use LENR or High ISO NR in camera either. I started shooting with a Canon 20D about 15 years ago and hanging out with some amazing imagers from the Orange County Astronomers quickly broke me of that habit in favor of sets of calibration frames.

That said, I do use temp dependent darks when I shoot with my Cooled SBIG STF-8300C, since the camera is frequently ~20 degrees below ambient temp.