Readit Newsjacobwilliamroy commented on Ask HN: What is the current state of the art in BIG (>5TB) cloud backups? · Posted by u/jacobwilliamroy
5 TB? I don't know how to count that low.
It's just bigger than any amount I've ever dealt with in my whole life (I'm 30) and also it's hard to find solutions to manage it without running into bandwidth caps and also things are running for a very long time on the client side: days or weeks.
jacobwilliamroy commented on Ask HN: What is the current state of the art in BIG (>5TB) cloud backups? · Posted by u/jacobwilliamroy
Like so many things, it depends-
How quickly do you need to be able to restore? Is it commercial or homelab?
The most cost-effective option by far would be to put a NAS device someplace offsite. You could use tailscale to connect to it remotely.
After that, depending on your access patterns, either a glacier-style s3 service (aws or backblaze/etc), or a rented bare-metal server with big disks some place inexpensive.
This is for a "warm" backup as opposed to a hot backup so restoration timeline would be like seven days.
jacobwilliamroy commented on Ask HN: Should I implement my own integrity checks on my fileserver? · Posted by u/jacobwilliamroy
SFTP protocol includes checksum hashing on each packet sent. So corruption from the network is very unlikely (as it is also encrypted).
On the write to disk side, you are probably best off using ZFS or btrfs as the filesystem. These contain the option for similar integrity checks / error correction on write.
What is your threat model? Are you concerned about adversarial changes to the data or just prevention of corruption? Either way an adversary would have to be deep in your system or mitm to get around the transfer protocol protections. And the transfer protocols used by SFTP should handle random network corruption.
I am only trying to ensure the data on the local system is the same as the data on the server. There is no adversary in the middle modifying data; so this is strictly about detecting corruption.
do you have friends in plural?
I DO
I HAVE 3
3 IS MORE THAN 1
Yes.
Do you make it like a fun game? Like when me and my friends in school would pass eachother coded notes and the cipher was an inside joke?
I'm genuinely curious: what was the pitch that you used to get others to start using signal?
Or to put it another way, why is state of the art important?