[normhead]

Is there anything to learn by a batch export versus a single export? Surely only an inherent issue (overheating) with an individual computer.?

I do it all the time, I usually work a days shoot to 5-30 files then do the titles and tags etc. then a batch at the end.

[GUB]

I do it all the time, I usually work a days shoot to 5-30 files then do the titles and tags etc. then a batch at the end.

Yeah but I mean for comparative purposes here.

[turbo_bird]

I bought a refurbished Dell desktop a couple years ago, thinking it would be great for photo editing based on the processor and ram, plus having an SSD. Turns out I was wrong on that. According to a computer guru friend of mine, it was overheating and throttling when doing batch exports or trying to render 4k timelapse videos, and the real bottleneck was the slow buss speed of the motherboard and the slow ram it used, both only running at 100mhz I think he said. He put together a new system for me using an AMD Ryzen with the same amount of ram and not a lot faster processor speed. It's way faster than my old computer, cost about 4 times as much, and takes up twice as much room on my desk, but it's amazing how fast it is. Rendering a timelapse from raw files before would have taken about 40min for 1000 frames, and now it's under 5 min. Photo editing was reasonably quick with ON1 on the old machine, but it's nearly instantaneous with the new one. At least ON1 lets me export in the background rather than tieing up everything.
Kristian

[biz-engineer]

That clock speed is a big yellow flag right there. What make/model laptop is it?The i7-8850 has a few variants, but mostly they run at 2.6GHz to 4.3GHz peak. The low-power frequency is 800MHz

DELL laptop, i7-8850U, peak clock speed can go up to 4GHz. Power management was set to the default "balanced", but the GPU was set to max power saving + medium power saving on the PCI bus + some mixed CPU cooling. CPU cooling can be manually set to passive or active. I figured "Passive" means the temperature is regulated via clock speed to save energy, and "Active" means temperature is lowered via fan, if the temperature continues to rise when the fan is at full speed the clock speed decreases. Changing the GPU power setting immediately made a big difference, I supposed it was a first bottleneck. Changing the CPU cooling mode avoids the CPU to slow down after processing a couple of images, but CPU clock still gets lowered to about 2.25GHz because the cooling air flow is insufficient to get rid of the heat. I didn't notice a speed difference by switching off the PCI bus power saving. The CPU now runs 100% and the clock is lowered a bit, I think that's the bottleneck , coming from the cooling. Batch export speed improved ~ 3x compared to before, but the most important is that working on raw images with sliders and local adjustments is now almost real time, that's the big improvement.

---------- Post added 06-01-22 at 08:15 ----------

Sorry to say this but this is not probably not a machine with the best specs for large batch image processing jobs on 100 full frame RAW files even if you fully max it out, put it in a very cold room, and leave it plugged in unless you just set up a job and leave it for a period. Even at 3 min/file the machine will complete the job easily overnight, for example.

Yes you are right. When I purchased the notebook, my priority was the compact small size for traveling and using it as a tablet. On paper the specifications were quite good at the time (2017), I looked at good battery life, but I didn't pay attention to the GPU spec, and notebook mother board designs are optimized for being power efficient, not highest performance.

---------- Post added 06-01-22 at 08:17 ----------

It is an i7-8850U processor, not a great choice but it should be capable of much better performance. The 800Mhz mode is used when saving power.

You are right.

---------- Post added 06-01-22 at 08:19 ----------

Your 3 minutes for 100 jpg files (Full size?) are very good

Arrhhh , no, it was 3 minutes for one file export, 30 minutes for 10 files...

---------- Post added 06-01-22 at 08:28 ----------

Your i7-8850U is a relatively low performing cpu by contemporary standards, only nominally about 3 times the speed of one of those old ones on passmark benchmark scores. I suspect that similar stipulations apply.

Normally, process can use the GPU to offload some of the work, but my GPU is probably not up to the task to help complex image processing. What makes me think that way is that Topaz Sharpen AI performs a "calibration" to decide how to load the machine, and in my case Topaz decides to use the CPU and not to use the GPU. So, not only my CPU isn't strong but it also take the full work load and the GPU does nothing. I've seen a GPU test done in 2021, my model of GPU gets 2% of the max performing GPU model in the list and the top 20 GPU (NVIDIA) deliver at least 60% of the max performance.

---------- Post added 06-01-22 at 08:31 ----------

In Darktable profiled denoise seems to slow things down a bit.

Exactly. The cleanliness of the output file impacts processing time. I processed /exported the same raw files with RT, the processing was much faster, it took less than a minute for 10 files, but the output image quality isn't as smooth and nice.

---------- Post added 06-01-22 at 08:41 ----------

Is there anything to learn by a batch export versus a single export?

Interesting, I haven't tried that. Also, I used Adobe DNG converter to compress some RAW files. The load time is 2 to 4 times quicker depending on what's inside the DNG file. In this process , I understand that the image processing software reads the DNG data and replicate a proxy of image data in memory, cache, and also in the file system before it can process the image through the UI. The amount of image proxy data determines real time processing speed. Export speed is different because it take the full amount of image data processed according to parameters set on the proxy image. So if I take a RAW file, go through Adobe DNG converter and modify setting such as changing embedded JPEG preview from full size to 8Mpixel (~4K) + data compression, the output DNG now loads two times faster in image processing software.

---------- Post added 06-01-22 at 08:46 ----------

According to a computer guru friend of mine, it was overheating and throttling when doing batch exports or trying to render 4k timelapse videos, and the real bottleneck was the slow buss speed of the motherboard and the slow ram it used, both only running at 100mhz I think he said.

Yes, the specifications is only part of the equation for the processing speed. There is also how fast is the bus to transfer data between HDD/SSD and RAM, how fast the data can transit between RAM and CPU, between CPU and GPU. The slowest component in the chain slows every else.

---------- Post added 06-01-22 at 08:51 ----------

He put together a new system for me using an AMD Ryzen with the same amount of ram and not a lot faster processor speed. It's way faster than my old computer, cost about 4 times as much, and takes up twice as much room on my desk, but it's amazing how fast it is.

It's probably got a much fast RAM access, faster data bus and much better cooling. I'm now thinking my next computer purchase will be a desktop.

---

Last edited by biz-engineer; 01-06-2022 at 12:47 AM.

[UncleVanya]

I have a laptop as well. But I’ve never attempted any batch conversions as it doesn’t match my way of working. I just never feel like the speed is that bad working interactively. Laptops as a rule have compromised performance at a given price point vs desktops.

Lenovo Flex 14 2-in-1 Convertible Laptop, 14 Inch FHD Touchscreen Display, AMD Ryzen 5 3500U Processor, 12GB DDR4 RAM, 256GB NVMe SSD, Windows 10, some AMD low power gpu is included. I have enabled support for it in DXO which drastically speeds up deep prime processing when I use it.

You corrected your export timing and said it was taking 3 mins plus… that’s slow. But if I understood with power management changes your note close to one minute? My cpu is very similar in performance based on tests I’ve read, however there are a few areas where my chip performs better including the GPU bus speed which is 4x faster. With DXO DeepPrime my results were close to 2 minutes until I forced GPU use (my GPU is semi supported by DXO) and it reduced processing time to closer to 20 seconds if I recall accurately. The laptop I have is tiny, thin and light. It’s not cutting edge but works reliably with A7RIII and KP raw files.

[Mooncatt]

Normally, process can use the GPU to offload some of the work, but my GPU is probably not up to the task to help complex image processing. What makes me think that way is that Topaz Sharpen AI performs a "calibration" to decide how to load the machine, and in my case Topaz decides to use the CPU and not to use the GPU. So, not only my CPU isn't strong but it also take the full work load and the GPU does nothing.

This is because your "GPU" really isn't. Your graphics processing is integrated in to the CPU, using your CPU to run the graphics calculations. The Intel 620 naming is only a designation of the specific graphics programming used by the CPU.

To see a performance uplift in something like Topaz's programs, you would need a dedicated graphics card. Most laptops don't have those, unless you buy a gaming laptop. It's also worth noting that those cards are not able to perform as well as their desktop counterparts within the same generation, but should still be substantially better than CPU processing.

[ProfessorBuzz]

DELL laptop, i7-8850U, peak clock speed can go up to 4GHz. Power management was set to the default "balanced", but the GPU was set to max power saving + medium power saving on the PCI bus + some mixed CPU cooling. CPU cooling can be manually set to passive or active. I figured "Passive" means the temperature is regulated via clock speed to save energy, and "Active" means temperature is lowered via fan, if the temperature continues to rise when the fan is at full speed the clock speed decreases. Changing the GPU power setting immediately made a big difference, I supposed it was a first bottleneck. Changing the CPU cooling mode avoids the CPU to slow down after processing a couple of images, but CPU clock still gets lowered to about 2.25GHz because the cooling air flow is insufficient to get rid of the heat. I didn't notice a speed difference by switching off the PCI bus power saving. The CPU now runs 100% and the clock is lowered a bit, I think that's the bottleneck , coming from the cooling. Batch export speed improved ~ 3x compared to before, but the most important is that working on raw images with sliders and local adjustments is now almost real time, that's the big improvement.

...
It's probably got a much fast RAM access, faster data bus and much better cooling. I'm now thinking my next computer purchase will be a desktop.

Glad to hear that solved it. 3x speed improvement is huge. Desktop is the way to go when doing lots of work on large files.

[NeoSpudly]

I would run a generic benchmark on the machine, something like : UserBenchmark: PC Speed Test Tool - Compare Your PC just to see if your hardware scores match up with the same hardware in other users scores. If they do you can kinda assume you don't have any hardware issues with the machine.

[biz-engineer]

This is because your "GPU" really isn't. Your graphics processing is integrated in to the CPU, using your CPU to run the graphics calculations. The Intel 620 naming is only a designation of the specific graphics programming used by the CPU.

You are right, Intel 620 is onboard the CPU, perhaps why power saving on the GPU slowed down everything.

---------- Post added 06-01-22 at 15:54 ----------

Glad to hear that solved it. 3x speed improvement is huge. Desktop is the way to go when doing lots of work on large files.

I could optimize the cooling, adding thermal conductive pad behind the laptop onto an aluminum plate. Also exporting one file at a time as soon as it is ready for export so that the CPU has time to cool again before another file needs to be exported.

[ProfessorBuzz]

I could optimize the cooling, adding thermal conductive pad behind the laptop onto an aluminum plate. Also exporting one file at a time as soon as it is ready for export so that the CPU has time to cool again before another file needs to be exported.

Or go simpler and get one of the $20 laptop cooler stands with 2 fans underneath the bottom.

[biz-engineer]

Is there anything to learn by a batch export versus a single export? Surely only an inherent issue (overheating) with an individual computer.?

Serialization. Or set CPU to power saving and let run the batch slowly overnight, to avoid laptop become hot like a boiler.

---------- Post added 07-01-22 at 07:10 ----------

Or go simpler and get one of the $20 laptop cooler stands with 2 fans underneath the bottom.

I've seen those and read user comments, I'm not sure about how effective these cooler stands are. Laptop design is the issue here, it's really designed to save space and not to get rid of heat long term, max CPU power only meant to run for a short time. At the back of the laptop behind CPU location, there is a sticker with the serial number which half melted after I ran a batch at full power for a half hour. Hopefully we get out of that Covid situation, and after that I'll get a desktop machine and keep the notebook for travel. In the meantime I'll process one file at a time.