Readit News logoReadit News
xk3 commented on Electricity use of AI coding agents   simonpcouch.com/blog/2026... · Posted by u/linolevan
xk3 · 21 days ago
So less energy than a human brain uses...
xk3 commented on A Social Filesystem   overreacted.io/a-social-f... · Posted by u/icy
xk3 · 22 days ago
Makes me think of Syncthing... I wrote a wrapper for it that attempts to make it easier for people to use:

https://github.com/chapmanjacobd/syncweb-py

https://github.com/chapmanjacobd/syncweb-ui

Unfortunately, mesh storage systems are very different conceptually so it is difficult for people to think about permissions and access. You can bolt on something familiar but then it really limits the usefulness of mesh storage and you may as well just be using HTTP servers.

xk3 commented on Inside The Internet Archive's Infrastructure   hackernoon.com/the-long-n... · Posted by u/dvrp
stonogo · a month ago

    unless tape, and the infrastructure to support it, is dramatically cheaper than disk, 
This turns out to be the case, with the cost difference growing as the archive size scales. Once you hit petascale, it's not even close. However, most large-scale tape deployments also have disk involved, so it's usually not one or the other.

xk3 · 25 days ago
You might squirm at using refurbished or used media but those 3TB SAS ex-enterprise disks are often the same price or cheaper than tapes themselves (excluding tape drive costs!). Will magnetic storage last 30 years? Probably not but they don't instantly demagnetize either. Both tape and offline magnetic platters benefit from ideal storage conditions.
xk3 commented on Inside The Internet Archive's Infrastructure   hackernoon.com/the-long-n... · Posted by u/dvrp
textfiles · a month ago
It is on my desk to fix this soon.
xk3 · 25 days ago
Also, it would be good to regenerate the web seeds metadata (this doesn't change the info_hash section) when the mirrors (subdomain prefixes) change.

(like PHP code except it is binary data--it could be done on the fly)

xk3 commented on Linux boxes via SSH: suspended when disconected   shellbox.dev/... · Posted by u/messh
throwa356262 · 25 days ago
I suspect this service will be abused by all kind of people and will have to shut down.
xk3 · 25 days ago
or quickly subsidized by three letter agencies
xk3 commented on The rsync algorithm (1996) [pdf]   andrew.cmu.edu/course/15-... · Posted by u/vortex_ape
jraph · a month ago
And you'd probably use the snapshot feature of a filesystem like btrfs or zfs instead of hardlinks for deduplication :-)
xk3 · a month ago
Yes and something like btrfs-send or zfs-send is probably faster than fpsync
xk3 commented on The rsync algorithm (1996) [pdf]   andrew.cmu.edu/course/15-... · Posted by u/vortex_ape
ssl-3 · a month ago
The first time I got paid to use rsync was nearly 25 years ago. It provided for reasonably space-efficient, remote, versioned backups of a mail server, using hard links.

That mail server used maildir, which...for those who are not familiar: With maildir, each email message is a separate file on the disk. Thus, there were a lot of folders that had many thousands of files in them. Plus hardlinks for daily/weekly/whatever versions of each of those files.

At the time there were those who were very vocal about their opinion of using maildir in this kind of capacity, likening it to abuse of the filesystem. And if that was stupid, then my use of hard links certainly multiplied that stupidity.

Perhaps I was simply not very smart at that time.

But it was actually fun to fit that together, and it was kind of amazing to watch rsync perform this job both automatically and without complaint between a pair of particularly not-fast (256kbps?) DOCSIS connections from Roadrunner.

It worked fine. Whenever I needed to go back in time for some reason, the information was reliably present at the other end with adequate granularity -- with just a couple of cron jobs, rsync, and maybe a little bit of bash script to automate it all.

xk3 · a month ago
> there were a lot of folders that had many thousands of files in them

If you ever need to do something like this again, it's often faster to parallelize rsync. One tool that provides this is fpsync:

https://www.fpart.org/fpsync/

u/xk3

KarmaCake day2645March 29, 2019View Original