Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
10-27-2018, 01:08 AM - 2 Likes   #1
Forum Member




Join Date: Jun 2015
Posts: 87
Memory Friendly Superresolution Script 8bit & 16bit (In python and hugin)

Hi all,

I browsed through the net for a superresolution script to play around with, however most scripts I found weren't really memory friendly, easily running out of memory with 24MP images and using large stacks.
So I made yet another my own version, using python and hugin. Can process jpg and tif, 8bit and 16bit.

System requirements:
Linux would probably a whole lot easier than using windows.
Hugin (open source panorama software)
Python3
- Python packages: pyvips

For a ~15 Image Stack of 8-bit 24MP Images:
RAM 4GB will be enough for 24MP images
~6-7GB HDD the tif files are huge
Takes ~12min. on my old i3-3225

There is actually no limit on stack-size concerning RAM when using the average method. Using median, at some point memory could run out (not soon).

How it works:
Step 1: Images are resized to 200% using Bilinear extrapolation (tried several, looked best to me), saved as tif (uncompressed to speed stuff up).
Step 2: Images are aligned using hugin, and cropped to an area all images cover. (this is done on the gpu)
Step 3: Average & Median values are calculated for each pixel (each channel RGB separately)
Step 4: The two output files are created and saved in the folder superresolution


How to use it:
Please see GitHub - Rudgas/Superresolution: Another take at the creation of superresolution images using python and hugin for installation instructions. I'll post about updates here in the thread.


(old) Results (Handheld High-Burst with a K3 and a A-50mm/f1.7) below:

Attached Images
View Picture EXIF
PENTAX K-3  Photo   

Last edited by manufocus; 11-03-2018 at 08:30 AM.
10-27-2018, 01:24 AM   #2
Senior Member




Join Date: Oct 2018
Posts: 242
QuoteOriginally posted by manufocus Quote
Step 3: Aligned images are sliced into 4 pieces to save on RAM
neat trick
10-27-2018, 05:50 AM   #3
Loyal Site Supportaxian
Loyal Site Supporter
jbondo's Avatar

Join Date: Sep 2013
Location: Texas
Photos: Albums
Posts: 503
Thank you very much for sharing and describing your work.
10-27-2018, 02:53 PM   #4
Forum Member




Join Date: Jun 2015
Posts: 87
Original Poster
^ And this is why I love pentaxforums :-)

10-27-2018, 08:31 PM   #5
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
I ran your program on my Ubuntu 18.10 laptop. Specs are here:

$ inxi
CPU: Dual Core Intel Core i7-5500U (-MT MCP-)
speed/min/max: 799/500/3000 MHz Kernel: 4.18.0-10-generic x86_64 Up: 44m
Mem: 1258.9/7890.0 MiB (16.0%) Storage: 931.51 GiB (38.8% used) Procs: 253
Shell: bash 4.4.19 inxi: 3.0.24

I decided to do a stress test using six full resolution K-1 16-bit tiff files. First run it bombed on def main(): All was fine after I added the shebang #!/usr/bin/python3.6 as the first line of the program. It went through the initial stages fine but then bombed as such:

Written aligned images to files with prefix "aligned_"
/usr/lib/python3/dist-packages/PIL/Image.py:2546: DecompressionBombWarning: Image size (143321043 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
DecompressionBombWarning)
Tiling: aligned_0003.tif
TIFFSetField: tempfile.tif: Unknown pseudo-tag 65538.
Traceback (most recent call last):
File "./Superresolution.py", line 196, in <module>
main()
File "./Superresolution.py", line 15, in main
tile()
File "./Superresolution.py", line 90, in tile
tile[:,:,:,1] = npimage[tdimh:,:tdimw,:]
ValueError: could not broadcast input array from shape (4889,7329,3) into shape (4888,7329,3)

OK. Perhaps it cannot handle the 16-bit tiffs. So I converted the same files to 8-bit tiffs and tried again. Same result. It seems there is an array error during the tiling process?

- Jack
10-28-2018, 04:18 AM   #6
Forum Member




Join Date: Jun 2015
Posts: 87
Original Poster
Thanks for trying the script!

QuoteOriginally posted by jbinpg Quote
All was fine after I added the shebang #!/usr/bin/python3.6 as the first line of the program.
I added a shebang line that should be compatible across all systems (#!/usr/bin/env python3) which will call the most recent python 3 version.

QuoteOriginally posted by jbinpg Quote
DecompressionBombWarning: Image size (143321043 pixels) exceeds limit of 89478485 pixels,
Should have probably mentioned this, this is a warning that the program is loading a compressed image (tif) into memory, however in memory images are huge because each pixel has to be stored individually for each color channel. This is a warning that the image that is loaded exceeds the expected size and that it might clog your RAM.

QuoteOriginally posted by jbinpg Quote
OK. Perhaps it cannot handle the 16-bit tiffs
Wasn't aware the K1 could do this, will try to implement it. As for now it is 8-bit only.

QuoteOriginally posted by jbinpg Quote
It seems there is an array error during the tiling process?
Yes! I had a rounding error for the tiles, I preallocated an array with the size of the tiles, however, if the enlarged image has an uneven number of pixels along one axis, tiles will not be the same size (from shape 4889 -> 4888). This is now fixed! (no pixels are lost, code is just not as neat as before)

---------- Post added 10-28-18 at 04:22 AM ----------

@jbinpg could you send me the stack of 16bit tif? Can't find a good source for them online and would like to implement the 16bit processing.
10-28-2018, 06:58 AM   #7
Forum Member




Join Date: Jun 2015
Posts: 87
Original Poster
Actually never mind about the images, found a decent stack.

10-28-2018, 03:59 PM   #8
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
Actually, I am not sure if the K-1 can produce 16-bit tiffs in camera. I always use either PEF or DNG. In this case, I produced 16-bit tiffs from PEFs using my raw editor of choice, darktable.

BTW, the warning about the tiff tag 65538 is due to old JPEG compression cruft left over from old Python tiff code. A PR to remove it was submitted for current Python 9 days ago according to my web search.

Yes, I noticed the bomb warning but since it was a warning I just disregarded it. It is a Python attempt to prevent DOS attempts by making the system deal with abnormally huge images and potentially expose memory overruns. One can set a max image size limit if you so wish to get around the warning.

Glad you found the rounding error and glad I could help in some small way. Please let us know when you have the next version up.

Jack
10-29-2018, 01:05 AM   #9
Forum Member




Join Date: Jun 2015
Posts: 87
Original Poster
Script is now updated with 16bit capabilities.

To process K1-sized images in 16bit, make sure you have around 6GB of available (free -h) memory.

A minor drawback with 16bit images, is that they have to be split into tiles before resizing wich only allows nearest-neighbor extrapolation rather than bilinear which is used for 8-bit. This is something I'm currently looking into, also I'm not sure yet about the theory of extrapolation methods for the creation of superresolution pictures.
10-29-2018, 07:50 PM   #10
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
Downloaded and installed the latest version. Immediately this was thrown:

Traceback (most recent call last):
File "./Superresolution.py", line 5, in <module>
from skimage import io, transform
ModuleNotFoundError: No module named 'skimage'

I couldn't open the included README file which is in .md format (a text file would be appreciated). After a bit of googling, I found that skimage is part of the perl module scikit-image. So I did a pip install of that and it dragged in a few more python modules. Tried again but same error message.

What am I missing?

Jack
10-29-2018, 09:15 PM   #11
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
Never mind. I finally got the python environments figured out and the necessary modules installed. For those running Ubuntu 18.10, the following installs the necessary python3.6 modules:

sudo apt install python3-skimage.

8-bit and 16-bit tiffs successfully super-rezzed. The 16-bit results were about 356M in size.

I hope somebody has Windows results for you.

However, once viewing the actual superres files, I realized that the four 16-bit tiled images that were merged for the final result have different luminosities and you can see a line between the four tiles. Shoot me a PM and I will send you a dropbox link so you can see for yourself.

Jack

Last edited by jbinpg; 10-29-2018 at 09:21 PM.
10-29-2018, 11:18 PM   #12
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2015
Location: Melbourne
Posts: 1,506
Hello,
I've been wanting to experiment with this but have also come unstuck with the skimage problem.
I'm running Fedora 28, and searched around and installed what seemed like the appropriate packages, but still no joy.

Package python3-scikit-image-0.13.0-5.fc28.x86_64 is already installed, skipping.
Package scikit-image-tools-0.13.0-5.fc28.noarch is already installed, skipping.

So, anyone have any idea what provides the skimage for Fedora?

Cheers,
Terry
10-30-2018, 03:08 AM   #13
Forum Member




Join Date: Jun 2015
Posts: 87
Original Poster
QuoteOriginally posted by jbinpg Quote
I couldn't open the included README file which is in .md format (a text file would be appreciated).
The format *.md is the markdown format which allows for a bit fancier formatting, but can still be viewed as plain text. If you take a look at GitHub - Rudgas/Superresolution: Another take at the creation of superresolution images using python and hugin the new and updated readme will be displayed below the files :-)
The file can be opened in any regular text-editor. (And is quite readable)

QuoteOriginally posted by jbinpg Quote
So I did a pip install of that and it dragged in a few more python modules. Tried again but same error message.
I see you solved it already and I included the debian install command in the readme file. Though I'm no expert on python package management, I'm suprised pip didn't work out. Could it be that you installed it for python2.7? You should be able to check by runnning
pip2 install --user scikit-image
pip3 install --user scikit-image
And see which one returns the already installed message.

QuoteOriginally posted by jbinpg Quote
However, once viewing the actual superres files, I realized that the four 16-bit tiled images that were merged for the final result have different luminosities and you can see a line between the four tiles. Shoot me a PM and I will send you a dropbox link so you can see for yourself.
I'm currently working on a different angle for 16bit images to allow bilinear extrapolation so stay tuned. Would be glad for the example images though!

QuoteOriginally posted by tduell Quote
I've been wanting to experiment with this but have also come unstuck with the skimage problem.
I'm running Fedora 28, and searched around and installed what seemed like the appropriate packages, but still no joy.
I'm sorry I can't help you with fedora specifically (I'm runnning Arch), but have you tried:
pip3 install --user scikit-image
?

-----

In other news, soon there will be a changelog and versioning so it is a bit easier to see which improvements have been made. Thanks for the feedback so far, keep it coming :-)
10-30-2018, 12:11 PM   #14
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
Terry - Ubuntu still has python 2.7 as the system default. The way they separate the tools is any install using just python will install 2.7 versions, whereas python3 installs 3.6. Perhaps Fedpra dpes something similar. I could only get things to work by installing 3.6 versions of skimage.

Jackt
10-30-2018, 02:42 PM   #15
Loyal Site Supporter
Loyal Site Supporter




Join Date: May 2015
Location: Melbourne
Posts: 1,506
QuoteOriginally posted by jbinpg Quote
Terry - Ubuntu still has python 2.7 as the system default. The way they separate the tools is any install using just python will install 2.7 versions, whereas python3 installs 3.6. Perhaps Fedpra dpes something similar. I could only get things to work by installing 3.6 versions of skimage.
Jackt
Thanks for that info.
I went searching and found that skimage is installed at /usr/lib64/python3.6/site-packages/skimage/, so now the mystery for me is why it isn't being found by the script.

Cheers,
Terry
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
array, error, folder, hugin, image, images, line, memory, photography, photoshop, pixels, process, python, script, size, step, superresolution, tiles
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Looking for photoshop script evaluators - various script (analysis and diagnostic) rechmbrs Digital Processing, Software, and Printing 0 08-12-2017 05:31 AM
A Practical Superresolution Tutorial in Adobe Photoshop interested_observer Photographic Technique 26 02-25-2015 04:18 PM
A short hugin tutorial sterretje Digital Processing, Software, and Printing 3 12-10-2013 06:40 AM
Hugin KrisK10D Digital Processing, Software, and Printing 7 03-30-2009 09:57 AM
Installing hugin. phoreal Digital Processing, Software, and Printing 3 10-01-2007 08:51 AM



All times are GMT -7. The time now is 11:01 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top