[Igor123]

...
Backblaze sounds similar, may I ask you fine gents what kind of upload speeds you experience? I'm only getting about 5mb/s, and this is on data plans where I am not on unlimited (no NBN yet).

I like the idea of backing up more than just the RAW files and LR catalogs from a job, such as some personal (non image) files as well as perhaps more RAWs of the same job but that were initially culled and never made it into LR's library etc. But I fear my data plan and upload speeds must be taken into account, I doubt I can cope with a service like Crashplan or Backblaze currently.

I never thought about speed, but just tried a run and I get 10-15 mbyte/s upload (~100 mbit), and I'm in Sweden while their servers are in the US as I understand it, but I think that may be my ISP upload limit so perhaps not a backblaze limit. But since that just uses some disk and network and no cpu and could basically be "always on" if you wanted to, even while editing photos without interfering much with performance since it doesn't use a lot CPU, GPU or RAM, so I don't think the upload speed should be an issue anyhow?

Regarding what to backup, as I mentioned, with the commandline tool and b2 storage (and not the "whole computer backup"-plan) you can easily specify what to backup and not to further limit whats uploaded. The computer-backup has some build-in restrictions on temporary files, application files etc, so you can't restore a computer installation with OS and all from it, just the data. In my case I just run

b2 sync --dryRun --compareThreshold 1 [path to my photo storage]

to sync see what will be uploaded from the photo raw drive, no actual transfer, and then

b2 sync --compareThreshold 1 [path to my photo storage]

to actually to the sync, but there are many options and details that allow for customization and you can run this on a schedule in windows or linux at least (probably in macos too). The comparethreshold-part is just because linux uses different precision of file datetimes compared to windows, and I use both, so maybe not necessary in your case.

As for security as someone mentioned, you can either encrypt it yourself before uploading, or they also have support for you to provide/configure your own encryption key locally so that Backblaze themselves can't decrypt your data even if CIA tried to force them =).

Edit: About uploads/plan; since you define what to upload or not with b2, and it's all "syncing" and not uploading all the data every time, I don't see how you can get a full cloud backup using any less data no matter what solution you use? (Sure the whole-computer-plan from backblaze will backup more, but no need to use that). Or what you could do if willing to spend the time and effort you could try to compress raws using 7zip or so before uploading, but that won't gain much when dealing with raw files and also not worth it I think, since youre not doing that today? (nobody is probably..). But all in all, even if you just get 0.5 MB/s, leaving the computer on overnight would solve that, so the upload speed shouldn't be an issue regardless of plan I think. (if there's a cap on total upload in the plabn there might be issues, but that would go for any cloud backup regardless, and be more related to what you upload rather than to which cloud...)

---

Last edited by Igor123; 06-03-2019 at 02:49 PM.

[RGlasel]

just tried a run and I get 10-15 mbyte/s upload (~100 mbit), and I'm in Sweden while their servers are in the US as I understand it, but I think that may be my ISP upload limit so perhaps not a backblaze limit

Your upload speed is excellent, my home in Canada is in too old of a neighborhood to get fibre optic service and my "premium" upload speed is 1 mps (10 mps download). Backing up 100 GB to the cloud requires over 200 hours, 1 Terabyte is 90 days continuous uploading, assuming nothing slows down or times out. Once the first full backup is completed, it isn't so bad, but there is a very good chance my ISP will notice how much data I'm uploading and interrupt every so often, meaning that the first backup might never complete, or at best I won't be able to upload anything new for several months while I break up the first backup into manageable chunks. Cloud backup makes sense for my phone, anything else gets copied to an external harddrive.

As for server speed, the problem isn't with CPU or physical drive speeds, the problem is dividing a connection to the Internet backbone between hundreds or thousands of users who expect a response within a second or two of signing in and continuously updated progress reports. In your home, the computer you are using is probably the only device sending data to your backup device at any given time and the lamest backup device is going to give you vastly better service than the fastest server farm with layers of redundant routers and fibre connections. The difference in energy consumption between a local backup and cloud backup is mindboggling, no matter how many carbon credits Google or Facebook save up.

[fs999]

I wouldn't use SSD drives for backup, because of the write limitations...

I have opted for a NAS, after a disk overwrite (the vendor said the disk bay was hot swappable... It was not )

So I have now a WD my cloud EX2 ($300) and a WD my cloud EX2 Ultra ($600) NASes. One with 2x1 TB mirrored WD Red drives, which are full now, and the Ultra with 2x4 TB.

I put them in the cellar with a 1 Gb network cable connection. So now I can backup every evening, it takes a few minutes, and all my photos are accessible, for me alone, in the whole world, if I need it.

I am also planning to buy a surge protector so the NASes could safely be shut down.

[ChatMechant]

Google drive for clients and a 4tb Seagate NAS at home for a sort of cloud backup. Works very well so far(2+yrs), no monthly fees. I've also never in my life had a hard drive fail so am not too paranoid.

[RobA_Oz]

If your primary concern is loss of data through a home fire or other catastrophe, keeping a backup drive off-site is a much simpler solution than trying to backup your files on a cloud service. I used to do that with a drive I kept at work, but since I retired, I no longer have that option, so my solution is to keep a drive with a relative.

On the matter of SSDs versus HDDs, I recently tried to find a drive to replace the one in my daughter’s MacBook Pro, and was surprised to see that they are getting harder to obtain, even though they’re still available via eBay and the like. I opted for buying an SSD from a local store, which wasn’t that much more expensive than its HDD equivalent. As for rewrite limits, HDDs fail too, but using an SSD as a backup is probably fine, as long as you aren’t backing up every hour or so.

[gatorguy]

I wouldn't use SSD drives for backup, because of the write limitations...

I have opted for a NAS, after a disk overwrite (the vendor said the disk bay was hot swappable... It was not )

So I have now a WD my cloud EX2 ($300) and a WD my cloud EX2 Ultra ($600) NASes. One with 2x1 TB mirrored WD Red drives, which are full now, and the Ultra with 2x4 TB.

I put them in the cellar with a 1 Gb network cable connection. So now I can backup every evening, it takes a few minutes, and all my photos are accessible, for me alone, in the whole world, if I need it.

I am also planning to buy a surge protector so the NASes could safely be shut down.

Personally I think the concerns about write limitations is far overblown. Actual testing has shown the real life longevity is several times the stated ones.
https://www.ontrack.com/blog/2018/02/07/how-long-do-ssds-really-last/

[Igor123]

Personally I think the concerns about write limitations is far overblown. Actual testing has shown the real life longevity is several times the stated ones.
https://www.ontrack.com/blog/2018/02/07/how-long-do-ssds-really-last/

Agree, many quality SSDs are rated in number of "DWPD" - disk writes per day for the guarantee period of say five years, meaning you can write the entire disk space each day for several years without worrying, which is nowhere near the case for photo backups...but as for the price if using SSDs for backup that's another matter =)

---------- Post added 06-05-2019 at 12:12 AM ----------

Your upload speed is excellent, my home in Canada is in too old of a neighborhood to get fibre optic service and my "premium" upload speed is 1 mps (10 mps download). Backing up 100 GB to the cloud requires over 200 hours, 1 Terabyte is 90 days continuous uploading, assuming nothing slows down or times out. Once the first full backup is completed, it isn't so bad, but there is a very good chance my ISP will notice how much data I'm uploading and interrupt every so often, meaning that the first backup might never complete, or at best I won't be able to upload anything new for several months while I break up the first backup into manageable chunks. Cloud backup makes sense for my phone, anything else gets copied to an external harddrive.

As for server speed, the problem isn't with CPU or physical drive speeds, the problem is dividing a connection to the Internet backbone between hundreds or thousands of users who expect a response within a second or two of signing in and continuously updated progress reports. In your home, the computer you are using is probably the only device sending data to your backup device at any given time and the lamest backup device is going to give you vastly better service than the fastest server farm with layers of redundant routers and fibre connections. The difference in energy consumption between a local backup and cloud backup is mindboggling, no matter how many carbon credits Google or Facebook save up.

I see, I probably misunderstood your usecase then; but I agree that hosting the server at home isn't a great idea regardless so I agree about that!

About the initial upload, it might be overkill (and now I'm starting to sound like a backblaze fanboy, but I'm not, just a satisfied user that likes their quarterly disk failure reports =) ), but if you want to upload all your data over that connection you will have to bite the bullet and transfer the data somehow no matter which service provider you choose, and although expensive, if there's no other way to do it, they do offer a "fireball" solution where they rent you a 70TB box, shipped to you by "mail", and then you transfer you data to it at home and send it back, and they put in in their cloud and you can start doing incremental updates from there. But it's $550 for 30 days for that box so might be cheaper to move your server/computer to a friend or relative overnight somewhere where better connections without caps are available for the initial upload if that's a problem (but unless there is something in your contract about max upload I don't think there should be an issue even if you use all of it 24/7). Either way good luck in your search!

---

Last edited by Igor123; 06-04-2019 at 03:12 PM.

[leekil]

It sounds like cloud isn't really viable for you, with your slow data speeds? I mean, you can probably limit what is stored to Backblaze or whatever, but it really seems like the limiting factor for what you want is your ADSL speed. Don't you have *anywhere* you can store a backup drive offsite? Do you have a non-photography job, and an associated office to store a drive? Or maybe get a safe-deposit box?

[clackers]

I think the surest, lowest tech way are Week A, Week B USB hard drives that your data's synced to.

Each you bring home and attach to the computer in alternate weeks while the other leaves with you the next morning to spend 7 days at your workplace.

If you're real paranoid ("Won't they be together at home one night of the week?") you get a Week C hard drive. Having that Week C one at work fixes that, but of course it's now up to two weeks behind in its files list - unless you sync to both Week A and Week B first, then it's back to one week.

[fs999]

Personally I think the concerns about write limitations is far overblown. Actual testing has shown the real life longevity is several times the stated ones.
https://www.ontrack.com/blog/2018/02/07/how-long-do-ssds-really-last/

I had a bootable SSD drive and after one year I had unrecoverable read errors...

[monochrome]

Cloud storage for backup is only as good as the solvency of the provider and only as secure as conventional measures to prevent mischief. There is also the issue of upload caps by broadband providers. Despite having an AWS account that would allow for relatively cheap storage, I think I will pass.


Steve

Wasn’t Google Cloud Service down globally for most of the weekend recently? IIRC their repair tools were housed on the inaccessible Cloud Service, so they couldn’t fix the inaccessible Cloud Service.

Yikes!!!

[kaseki]

Wasn’t Google Cloud Service down globally for most of the weekend recently? IIRC their repair tools were housed on the inaccessible Cloud Service, so they couldn’t fix the inaccessible Cloud Service.

Yikes!!!

Please see Google may have taken this whole 'serverless' thing too far: Outage caused by bandwidth-killing config blunder ? The Register for today's version.

In any case, I recommend server quality drives (I use WD HDs with a Y in the part number) for storage either in a NAS or USB connected backup box. As clackers notes, one can support any level of paranoia by rotating drives and storing them in uncorrelated (disaster-wise) locations. You pick the disaster you consider the worst case. A local fire is different than a section of your state falling into the sea after an earthquake, which again is different than the Yosemite Caldera destroying the central USA. Of course, disasters that aren't personally survivable may not matter unless one is an international corporation.

[gatorguy]

Wasn’t Google Cloud Service down globally for most of the weekend recently? IIRC their repair tools were housed on the inaccessible Cloud Service, so they couldn’t fix the inaccessible Cloud Service.

Yikes!!!

Actually closer to 6 hours or so, maybe less, but who's counting?
There was another unrelated hours-long outage yesterday with Apple Cloud that's not been explained yet. AppStore, Apple Music etc unavailable to several regions for much of the morning.. It almost feels as tho we're expanding cloud services beyond what the delivery infrastructure can absorb.

Potentially unavailable cloud services is what keeps the data I store there as backup copies rather than primary. Great for secondary photo storage for instance and yes very VERY convenient to access copies of them from any computer or mobile device. The original's still stay ground-bound on my own storage media for obvious reasons.

[DeKay]

I backup every single photo to a HDD that is stored at work.
At the same time I also copy everything to a portable SSD that is with me at all times. This is more of a working copy & rejects get deleted.
Then the most important or “the cream” as I call them are copied to OneDrive for Business.
OneDrive makes it easy to transfer the files on and off my iPAD via FileBrowser for Business.

[stevebrot]

A local fire is different than a section of your state falling into the sea after an earthquake...

It was Ansel Adams, after all, that quite famously lost a substantial quantity of work* in a darkroom fire.


Steve

* Stated by some sources as being some 5,000 glass plates, many of which had never been printed.