A few weeks ago I threatened to post this -- it's about time I made good on it!
I shoot in raw to preserve my post-processing options (in particular, it's nice to be able to correct exposure and white balance). But in general, I don't want to spend a lot of time post-processing my crummy pictures when only some relatively small percentage of them are keepers.
In using some other batch-enabled raw conversion tools for Linux (e.g. UFRaw, RawTherapee), I found that I couldn't settle on a single conversion profile to apply to all of my shots. Some of them needed more noise reduction, and some of them needed sharpening. But excess noise reduction and sharpening can really mess things up. What I found is that I can usually match my desired level of noise reduction and sharpening to the ISO value of the photo. But as far as I know, the Linux-compatible raw converters don't really allow for that, at least not easily. Maybe the fancy-pants commercial conversion programs like Lightroom (for Windows/Mac) and Bibble can do this, or maybe not. I haven't tried.
So that's why I wrote this script. It looks at the ISO value for each photo, then applies the appropriate level of noise reduction and sharpening. I can run it once in a directory full of images and walk away.
This script is intended to do "good enough" RAW to JPG conversion in the majority of cases. It's more than adequate as a tool to figure out which ones to subsequently tune by hand, and I think it gets the job done well enough that most of the resulting photos are usable as-is (on the web, at least).
Here's how it works:
First, for each PEF or DNG, it uses dcraw to convert the raw file to 16-bit-per-channel TIF. Noise reduction is applied at this stage.
Then it uses imagemagick to boost saturation a bit, apply a sigmoidal contrast enhancement curve (to emphasize the midtones) and a relatively conservative amount of local contrast enhancement (that's unsharp mask with a large radius). These adjustments are all applied to the 16-bit TIF.
Then it sharpens the image (via another unsharp mask pass, with a small radius) and converts it to 8-bit JPG.
Finally, the script copies all of the EXIF data from the original raw file to the new JPG.
It does ISO-dependent noise reduction ("NR") using dcraw's wavelet denoise function, and ISO-dependent sharpening using imagemagick's unsharp mask. In general, more noise reduction and less sharpening is desired at higher ISO values, and this script automates that.
You can choose your own NR and sharpening values at ISO values of 100, 200, 400, etc. up through 6400, and for actual ISO values in between those power-of-two selections, this script will interpolate between the adjacent NR and sharpening values. It's just linear interpolation, nothing fancy.
I find that this script works pretty well with photos from my Pentax K20D (I have dynamic range enhancement turned OFF). For other cameras and other camera settings, adjustments may be necessary -- particularly to the ISO-dependent noise reduction and sharpening profiles, but also to the (fixed) local contrast enhancement value, the saturation boost, and the gamma. All of these variables are set at the top of the script, so hack away!
Requirements:
dcraw (raw converter)
imagemagick (command-line image processing Swiss Army Knife)
exiv2 (exif data manipulation tool)
bash, of course, plus the customary GNU command-line utilities
I'm sure there are a million ways to improve this script, but for me, it has reached a point of diminishing returns where I don't see much benefit to adding more. I'll hand-process those cases. But if you want to add something, please feel free -- and post your improvements here!
I'll start with some ideas:
- Better noise reduction (e.g. more emphasis on reducing chroma-channel noise) using something like G'MIC
- Better handling of gamma and color profiles (I know next to nothing about this subject)
- Better error/exception handling
- Processing command-line options to override defaults
- Automatic exposure compensation when needed
- Windows and Mac versions
- etc. etc. etc.
Usage: run without any command-line options, and it will process all of the PEF and DNG files in the current directory. Alternatively, you can specify the files you want processed by giving their filenames. That's it!
Here ya go:
Code:
#!/bin/bash
#
# convertraw
# Convert from PEF/DNG to JPG
#
# Copyright 2009 Clarke A. Wixon
#
# This work is licensed under the Creative Commons
# Attribution-Noncommercial-Share Alike 3.0 Unported License.
# To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/
# or send a letter to Creative Commons, 171 Second Street, Suite 300,
# San Francisco, California, 94105, USA.
#
comment="Autoconversion script rev. 2009-06-26"
gamma="2.2"
contrast="5.0x35%"
lc="0x40.0+0.2+0.0"
saturation=105
jpgqual=85
isolevel=( 0 100 200 400 800 1600 3200 6400 )
nrlevel=( 25 25 50 100 200 400 800 1600 ) # change this to alter dcraw noise-reduction levels
sharplevel=( 0.7500 0.7500 0.5000 0.2500 0.0000 0.0000 0.0000 0.0000 ) # change this to alter imagemagick sharpening amounts
shopt -s nullglob # if there are no matching files, don't try to process the glob as an actual filename
if [ -z "$1" ]
then
infiles="*.PEF *.DNG"
else
infiles="$@"
fi
echo "Converting $infiles . . ."
for rawname in $infiles
do
# set filenames
if [[ "$rawname" =~ ".PEF" ]]
then
basename="${rawname%.PEF}"
else
basename="${rawname%.DNG}"
fi
tifname="${basename}.tif"
pefexif="${basename}.exv"
jpegname="${basename}-batch.jpg"
jpgexif="${jpegname%.jpg}.exv"
if [ -f $rawname ]
then
isoval=$( exiv2 print "$rawname" | grep "ISO speed" | awk '{print $4}' )
if (( $isoval >= 6400 ))
then
denoise=${nrlevel[7]} # max value
unsharp=${sharplevel[7]} # max value
else
denoise=${nrlevel[0]} # minimum default value; shouldn't be needed
unsharp=${sharplevel[7]} # minimum default value; shouldn't be needed
for index in 0 1 2 3 4 5 6
do
if (( ( $isoval >= ${isolevel[$index]} ) && ( $isoval < ${isolevel[$index + 1]} ) ))
then
interval=$(( ${isolevel[$index + 1]} - ${isolevel[$index]} ))
denoise=$(echo "scale=2; ${nrlevel[$index]} + ( ( ${nrlevel[$index + 1]} - ${nrlevel[$index]} ) * ( ( $isoval - ${isolevel[$index]} ) / $interval ) )" | bc | cut -d. -f1)
unsharp=$(echo "scale=4; ${sharplevel[$index]} + ( ( ${sharplevel[$index + 1]} - ${sharplevel[$index]} ) * ( ( $isoval - ${isolevel[$index]} ) / $interval ) )" | bc)
fi
done
fi
echo
echo "Converting $rawname from PEF to JPG (ISO "$isoval": NR "$denoise" sharpness "$unsharp")..."
dcraw -w -q 3 -n $denoise -4 -T -c -t 0 -v -H 0 -S 4095 "$rawname" > "$tifname"
echo "Processing..."
convert "$tifname" -verbose -depth 16 -gamma $gamma -modulate 100,"$saturation",100 -sigmoidal-contrast $contrast -unsharp $lc -unsharp 0x1.2x"$unsharp"x0.03 -depth 8 -strip -quality $jpgqual "$jpegname"
echo "Transferring EXIF and cleaning up..."
exiv2 extract -f "$rawname"
mv "$pefexif" "$jpgexif"
exiv2 insert -f -M"set Exif.Photo.UserComment charset=Ascii $comment (NR=$denoise gamma=$gamma sat=$saturation contrast=$contrast lc=$lc unsharp=$unsharp qual=$jpgqual)" "$jpegname"
rm "$jpgexif"
rm "$tifname"
echo "Done."
fi
done