These stats are wonderful and make me really appreciate the culture of the company. I've been considering becoming a customer because of these posts, since they reflect a lot of pride in the craft and care for the community.
But I'm stuck on one thing. Does Backblaze offer a solution for Linux backup? I've got an NFS server running that I use for home storage that I want to back up - but looks like Backblaze is only offering a Windows or Mac client.
Maybe the business version would work, since it claims to support NAS backup. But then the pricing seems lower than the personal edition (60$/computer/year = 5$/month < 6$/month) - unless that's implying that every computer that accesses the NAS is part of the fee?
So I guess: is there a reasonable Linux offering for home users from Backblaze? If not, what service do folks suggest?
Yev here from Backblaze -> the Computer Backup service does not offer an unlimited Linux backup service. We do have support for Linux with Backblaze B2 Cloud Storage and our integration partners (https://www.backblaze.com/b2/integrations.html?platform=linu... - I filtered the list by Linux for you). On the business side the NAS backup is also done via our parnterships and B2. On the consumer end, we haven't found a way to make unlimited backup sustainable with NAS/Linux since those devices/platforms typically have WAY more data than the average user.
I've been using restic with the Backblaze B2 backend for a home server backup, which seems to be as close as Backblaze will ever get to having a Linux client.
My rough numbers are 450GB stored monthly, 4GB downloaded monthly, 90k stored files, 385,000 individual transactions, which ends up costing about $2.25 in storage fees, $0.25 for transactions, and $0.10 for download bandwidth.
I've been very happily using Restic and B2 for a long time. It's cheaper than the unlimited service given I have ~600GB stored. Plus, it's a "real" backup: on-going series of snapshots, older files are not replaced when a new back up is made, I can restore from as many points in time that I want to store, and I don't have to worry about Backblaze deleting anything due to inactivity.
One underrated feature of Restic is tagging, which lets you identify a collection of snapshots. I can point multiple separate backups and devices to the same repository tracked with tags; thus, I de-dupe across all of them.
One thing that has been keeping from backing up more content to B2 has been finding an appropriate encryption strategy, and it looks like Restic manages encryption the remote backup repository automatically?
If I'm reading this correctly, Restic + B2 sounds like an absolutely godsend!
You can use backblaze B2 with duplicacy. I backup my machines to my NAS and the NAS itself to B2 using duplicacy. The free version is command line only (with text config files). It works really well and B2 prices are reasonable ! They have a billing estimator somewhere on their site.
I made a script to backup nested zfs volumes to B2 using duplicacy if anyone is interested : https://gist.github.com/icefo/07aab2789e5cfa71045343953aaf88... It makes a snapshot, backup that and handle unexpected network, power loss or backup that span longer than the cron interval gracefully.
If you're a Linux home user and you want a second copy of your data somewhere else in the event of a system failure / flood / fire I'd consider scripting something to B2 (Backblaze's object storage), AWS Glacier, or other archive priced cloud services.
If you're looking for a backup service to handle point in time recovery, differential deduplication, or other features of a backup service (vs. a second copy of your data archive) those also exist though I don't have a clear recommendation for home users.
On Backblaze's business pricing my understanding is they require minimum of 5 users so that's where you'll see the difference solved.
Yev here -> there's no minimum for our Groups feature (which Business Backup runs on) - so you can have Groups with 1 or 2 people in them, no minimum of 5.
I also use B2 even though I am on Mac. I bought Arq to backup to B2 and I will break even within the first year. I used to pay for the personal edition before realising that my new setup is far cheaper.
I have used restic and rclone with very crappy platforms (Google Drive and Microsoft OneDrive) for testing purposes and that worked fine enough so I can imagine it works even better on something like B2.
I also run Linux, and I use rclone[1] with Backblaze B2 (cloud storage). It works really well. I have it set on a nightly cron job locally. I also trigger it manually if I have something immediate (like photos transferred via adb from android that I want to ensure get backed up).
Excellent answers here. As can be surmised, the answer is for Linux, use b2.
But it just occurred to me, that for home use, if you're a special kind of masochist, you might expose your DAS to a vm running ReactOS and use the windows client?
I don't recommend it, and have no idea if it would work.. But would love to see a write-up if someone wanted to try it...
It's also a RAM hog (the client), slow restores and backups, restore problem stories online... Yes it's an option but has its own issues. And recent changes that make it unlimited except for (growing list of exceptions/exclusions).
I was on Crashplan for a long time, moved to RClone to Google and AWS and likely Duplicacy to those systems.
There was a post from BackBlaze a year (?) or so back where they commented on the Toshiba low failure rate with something along the lines of "they seem really reliable, but we buy in bulk and just don't have enough offers at low price for those, otherwise we would buy a lot of them".
Well I run a couple dozen Synology NAS in professionnal setup, as well as two in personnal setup (mine and my parents'), and ever since that post I made the experiment of having almost 50% of all drives be Toshibas, and I have to say they do seem to be much more reliable (on the scale of "why do every other drive from Seagate and WD keep dying first, and often their replacement dies first too").
It is still a scale of use where it's mostly anecdotical rather than verifiable data, so don't take this fun comment for more than that.
But I suspect a lot of people reading these posts are not interested for some large scale setup or anything like that but rather to know which drives to put in their home computer or NAS, and honestly I can highly recommend the Toshiba for that. They do tend to be a bit more expensive (around 10% more ? I buy them from ldlc.com and grosbill.com , french IT stores, no bulk buying or anything like that)
Of course no matter the brand never expect no failure and a Toshiba drive may just as much die in the first ten minutes so always plan for it.
If they need replacing less often, then presumably you’d be willing to spend more on them, so do I infer that they are sufficiently more expensive to wipe out the benefits of greater reliability?
Shout out to Seagate, every drive that my friends and I have bought from them have eventually failed, good to see that they fail in non-consumer use too, not just me! Stick to the Western Digitals.
Yev here from Backblaze -> The Western Digital drives that you've purchased will fail too. That's part of the whole point of these reports, all of the drives eventually fail out or reach a state where we have to replace them - it's not any one specific manufacturer. That's part of why having a backup is so important, even the SSDs in newer machines will eventually go wonky.
I appreciate your diplomatic language, but the time to failure does matter, and consumers don’t have the same cost structure that incentivizes replacing working drives the way you do.
FWIW I share GP’s experience with Seagate. I had quite a few of them, ranging in size from 500 gigs to 2TB. Every last one of them died relatively quickly, while most of my Hitachis, Toshibas, and WDs from that era still work.
Seagate earned a permanent boycott from this customer.
I feel like there might be a difference - do drives fail more if they're not continuously running? When I was working in a lab it was generally close to 70% failure rates for drives that were sitting in a box for more than a couple years.
Do you think the seagate barracuda deserves to take the “death star” title from HGST? I think HGST has redeemed themselves with sub 1% failure rates for years. Seagate managed to put out a 30% failure rate drive.
Then I stand corrected. Is there anyway to avoid drive failures or should we accept the fact that they will break eventually? Also, the data is interesting and appreciated.
As a teenager, I helped a friend revive a Seagate drive after it bricked due to faulty firmware. If I recall correctly, my friend had actually installed some firmware updates for the drive, but had not installed one recently enough to avoid the problem. We had to run wires to contacts on the board to allow us to run commands in a terminal (in Windows) on the only machine we could find with a serial port. When it worked, we felt like hackers from a movie!
I think those in external USB drives are the most likely to fail. It seems that they put their lower quality stuff or at least drives that didn't quite pass qc into those because they know the huge majority of those will have some data out on them and then never touched again.
Dissenting opinion: I've owned and used a lot of Seagate drives. I have had a total of one fail on me. Given that they are usually less expensive they are then competing drives, I think they're fine. At the rate of disk size growth I end up replacing the disks for size reasons before they fail anyway.
My experience is the opposite. 100% of the Seagate drives I have purchased for personal use (total: 4) experienced failures within the warranty period, and of the drives I had a hand in purchasing professionally at least half needed replacement within the warranty period (~64).
I think Seagate is fine if you are willing to overbuild and deal with the refurb process within the warranty period. I'm not, and will not personally buy a Seagate product again unless their reputation improves (similar to what happened with IBM/HGST post-deathstar). It's not worth the time or aggravation, to me.
I used to work for a long defunct storage manufacturer, reached the point where we skipped either even or odd (i can't remember) firmware versions on the seagate drives. RMA counts would always be higher on the even/odd number.
Call it paranoia but I see this "WD is amazing, Seagate is shit" sentiment everywhere, but the data doesn't back it up. Is this guerilla marketing by Western Digital?
Funny how everyone chimes in with their Seagate horror stories. My experience with them isn't much better.
But another fun story: I had a PSU blow up a couple years ago in a machine with three WDs and three HGST. All WDs were dead after that, the others worked flawlessly. Probably not a large enough sample size for any definite conclusions but at least it put a failure mode on my radar that wasn't there before.
I've ever only had one drive failure. It was a 3 TB Seagate. I remember it failing in like three years, so after the warranty had already expired.
My oldest drive is a 500 GB Western Digital from 2008 that's still operational today. I imagine its end is near, but I've thought about that for a couple of years now.
It looks like after stalling in 2017-2018, $/GB has dropped again - https://jcmit.net/diskprice.htm - but JCM doesn't have the large sample sizes Backblaze does.
Interesting how the checksumming process sounds very much like zfs's scrubbing process. One of the reasons I trust zfs with my large data volumes is because it proactively looks for problems and fixes them. (and most filesystems really can't look/check)
> By increasing the shard integrity check rate, we potentially moved failures that were going to be found in the future into Q3. While discovering potential problems earlier is a good thing, it is possible that the hard drive failures recorded in Q3 could then be artificially high as future failures were dragged forward into the quarter. Given that our Annualized Failure Rate calculation is based on Drive Days and Drive Failures, potentially moving up some number of failures into Q3 could cause an artificial spike in the Q3 Annualized Failure Rates. This is what we will be monitoring over the coming quarters.
Wouldn't survival analysis on interval-censored data handle this problem automatically? All of your observations of failure presumably are actually interval data, where all you know is that the drive failed sometime in between the last good check and the first bad check. Then it doesn't matter if some time periods have large intervals and others have small intervals, that just affects the precision of estimates.
Yev here -> well that chart hasn't been updated in a while. For starters we're just $0.01/GB for downloads (we dropped the price last year). Our performance is generally pretty good, and we're partnered with cloudflare (free egress) if you need more umph. But most of the time folks don't have any issues with just our regular service.
Readit News
But I'm stuck on one thing. Does Backblaze offer a solution for Linux backup? I've got an NFS server running that I use for home storage that I want to back up - but looks like Backblaze is only offering a Windows or Mac client.
Maybe the business version would work, since it claims to support NAS backup. But then the pricing seems lower than the personal edition (60$/computer/year = 5$/month < 6$/month) - unless that's implying that every computer that accesses the NAS is part of the fee?
So I guess: is there a reasonable Linux offering for home users from Backblaze? If not, what service do folks suggest?
My rough numbers are 450GB stored monthly, 4GB downloaded monthly, 90k stored files, 385,000 individual transactions, which ends up costing about $2.25 in storage fees, $0.25 for transactions, and $0.10 for download bandwidth.
I've been very happily using Restic and B2 for a long time. It's cheaper than the unlimited service given I have ~600GB stored. Plus, it's a "real" backup: on-going series of snapshots, older files are not replaced when a new back up is made, I can restore from as many points in time that I want to store, and I don't have to worry about Backblaze deleting anything due to inactivity.
One underrated feature of Restic is tagging, which lets you identify a collection of snapshots. I can point multiple separate backups and devices to the same repository tracked with tags; thus, I de-dupe across all of them.
If I'm reading this correctly, Restic + B2 sounds like an absolutely godsend!
Dead Comment
I made a script to backup nested zfs volumes to B2 using duplicacy if anyone is interested : https://gist.github.com/icefo/07aab2789e5cfa71045343953aaf88... It makes a snapshot, backup that and handle unexpected network, power loss or backup that span longer than the cron interval gracefully.
If you're looking for a backup service to handle point in time recovery, differential deduplication, or other features of a backup service (vs. a second copy of your data archive) those also exist though I don't have a clear recommendation for home users.
On Backblaze's business pricing my understanding is they require minimum of 5 users so that's where you'll see the difference solved.
Deleted Comment
I have used restic and rclone with very crappy platforms (Google Drive and Microsoft OneDrive) for testing purposes and that worked fine enough so I can imagine it works even better on something like B2.
[1]: https://rclone.org/b2/
But it just occurred to me, that for home use, if you're a special kind of masochist, you might expose your DAS to a vm running ReactOS and use the windows client?
I don't recommend it, and have no idea if it would work.. But would love to see a write-up if someone wanted to try it...
I was on Crashplan for a long time, moved to RClone to Google and AWS and likely Duplicacy to those systems.
O365 comes with 5tb of space that can be addressed by a diff backup tool like duplicati.
Only gotcha I can see is that it's 5x1tb
Well I run a couple dozen Synology NAS in professionnal setup, as well as two in personnal setup (mine and my parents'), and ever since that post I made the experiment of having almost 50% of all drives be Toshibas, and I have to say they do seem to be much more reliable (on the scale of "why do every other drive from Seagate and WD keep dying first, and often their replacement dies first too").
It is still a scale of use where it's mostly anecdotical rather than verifiable data, so don't take this fun comment for more than that. But I suspect a lot of people reading these posts are not interested for some large scale setup or anything like that but rather to know which drives to put in their home computer or NAS, and honestly I can highly recommend the Toshiba for that. They do tend to be a bit more expensive (around 10% more ? I buy them from ldlc.com and grosbill.com , french IT stores, no bulk buying or anything like that)
Of course no matter the brand never expect no failure and a Toshiba drive may just as much die in the first ten minutes so always plan for it.
I appreciate your diplomatic language, but the time to failure does matter, and consumers don’t have the same cost structure that incentivizes replacing working drives the way you do.
FWIW I share GP’s experience with Seagate. I had quite a few of them, ranging in size from 500 gigs to 2TB. Every last one of them died relatively quickly, while most of my Hitachis, Toshibas, and WDs from that era still work.
Seagate earned a permanent boycott from this customer.
I have 10 year old WD black and green drives still kicking. I still have a gen 1 or 2 intel ssd drive that’s still kicking.
I think Seagate is fine if you are willing to overbuild and deal with the refurb process within the warranty period. I'm not, and will not personally buy a Seagate product again unless their reputation improves (similar to what happened with IBM/HGST post-deathstar). It's not worth the time or aggravation, to me.
Seems like WD has similar rates.
But another fun story: I had a PSU blow up a couple years ago in a machine with three WDs and three HGST. All WDs were dead after that, the others worked flawlessly. Probably not a large enough sample size for any definite conclusions but at least it put a failure mode on my radar that wasn't there before.
My oldest drive is a 500 GB Western Digital from 2008 that's still operational today. I imagine its end is near, but I've thought about that for a couple of years now.
It looks like after stalling in 2017-2018, $/GB has dropped again - https://jcmit.net/diskprice.htm - but JCM doesn't have the large sample sizes Backblaze does.
Wouldn't survival analysis on interval-censored data handle this problem automatically? All of your observations of failure presumably are actually interval data, where all you know is that the drive failed sometime in between the last good check and the first bad check. Then it doesn't matter if some time periods have large intervals and others have small intervals, that just affects the precision of estimates.
https://github.com/gilbertchen/cloud-storage-comparison/blob...
Can anyone from backblaze say anything about their performance compared to other vendors?
The pricing is certainly ahead of others, so I would use if the performance is comparable to some of the leading group tested there.