Very excited to see Fly restart development on Litestream after a 2ish year freeze!
I love Litestream and use it in every app I build now.
They advertise it as costing "pennies per day," but it's even less expensive than that. It obviously varies depending on how much storage you need, but I had a real app in production, and Litestream replication to S3 only cost me 2-3 cents ($0.02-$0.03) per month.[0]
The DX for deploying SQLite apps to Fly.io is rough. I'm a few hours into trying to get a production Rails app booting, but running into issues getting the database to initialize, migrate, and become writable. The root of my problem was the eager loading of a gem I wrote, but there were several layers of runners above it that made it hard to diagnose.
I wish they'd put a bit more effort into the DX here, but it probably doesn't make much sense from a biz PoV since big customers aren't going to be running these kinds workloads.
Curious if anybody here is deploying SQLite apps to production and what host they're using?
Every time I deploy something, it spins up 2 instances that are in some suspended state. I have to restart them like 3 times before they actually boot? And why can I never just pick one instance when launching an app.
Apps will randomly go into a suspended state, without explaining why. Contacting support says they ran out of capacity, but didn't automatically start them back when capacity was available?! That's the whole point of Apps (vs Machines), you keep my app running...
Fly is set up to be the best compute provider, but there are too many reliability and ergonomics issues.
Please stop updating flyctl every time i go to deploy an app
>The DX for deploying SQLite apps to Fly.io is rough. I'm a few hours into trying to get a production Rails app booting, but running into issues getting the database to initialize, migrate, and become writable. The root of my problem was the eager loading of a gem I wrote, but there were several layers of runners above it that made it hard to diagnose.
What's the Fly.io issue here? Aren't the issues you're describing in Rails not Fly.io?
I run several Go apps in production on Fly.io[0, 1, 2] and I've never had an issue with the Fly.io + SQLite part of it.
SQLite + Litestream makes things slightly more complicated, but again, I've never had issues with Fly.io specifically making that harder. I use a custom launch script in my Dockerfile that starts litestream with the -exec flag to start my app as a child process.[3]
I tried to go the litestream route on Fly.io, but there is too much that needs to be done to get it working. Specifically I was hoping scaling would be a lot easier, but master election kept breaking for me causing the whole app to not be able to come online. I just moved to Fly's managed postgres and called it a day.
Their managed postgres has gotten better, but its still a little sparse, so after about 6 months using it I am going to just take my DB to either Supabase or Planetscale.
Yeah, something that’s messed up that they don’t think is messed up is running `fly console` fires up another instance, which isn’t attached to the same volume, so you have to run `fly ssh console —pty`
It’s certainly not intuitive. It would be awesome if they sweat these details, but their deal is “here’s a bag of sharp knives”, which is good for some use cases.
I setup a fresh rails 8 app on Fly last year, using PG for the main data store but using SQLite for the ancillary solid stack dbs.
Only fuss I remember encountering was with fighting with rails migrating solid queue properly, but this seemed like a rails unit issue and don’t remember it being a Fly issue.
I’ve been contemplating migrating my pg primary to SQLite too. Anyways don’t have much else to offer other than an anecdote that I’m happily using fly with partial SQLite.
> But the market has spoken! Users prefer Litestream. And honestly, we get it: Litestream is easier to run and to reason about. So we’ve shifted our focus back to it.
That makes sense to me. LiteFS used FUSE, which meant figuring out how to run and mount a custom filesystem. Litestream is a single compiled Go binary that you point at the SQLite database file (and accompanying WAL file).
> In Litestream, we’re solving the problem a different way. Modern object stores like S3 and Tigris solve this problem for us: they now offer conditional write support. With conditional writes, we can implement a time-based lease. We get essentially the same constraint Consul gave us, but without having to think about it or set up a dependency.
Reading this blog post though, I couldn't see any reference to this. Is this supported in Litestream v0.5.0, or will it be for a future release?
I look forward to trying this out. Any benchmarks or demos on how long it actually takes to restore? I ended up cooking my own boring S3 backup because previously litestream took 20 minutes to restore something like 1000 rows. It felt extremely unoptimized. How long does restoration take today?
We have an in house application installed on a remote fleet with spotty internet access. Because of the spotty internet we have a lot of trouble setting up a reliable system for getting data home.
Could how does litestream handle backing up through a spotty connection and can we consolidate the backups into a central db an query against it?
Readit News
I love Litestream and use it in every app I build now.
They advertise it as costing "pennies per day," but it's even less expensive than that. It obviously varies depending on how much storage you need, but I had a real app in production, and Litestream replication to S3 only cost me 2-3 cents ($0.02-$0.03) per month.[0]
[0] https://mtlynch.io/litestream/#using-logpaste-in-production
I wish they'd put a bit more effort into the DX here, but it probably doesn't make much sense from a biz PoV since big customers aren't going to be running these kinds workloads.
Curious if anybody here is deploying SQLite apps to production and what host they're using?
Every time I deploy something, it spins up 2 instances that are in some suspended state. I have to restart them like 3 times before they actually boot? And why can I never just pick one instance when launching an app.
Apps will randomly go into a suspended state, without explaining why. Contacting support says they ran out of capacity, but didn't automatically start them back when capacity was available?! That's the whole point of Apps (vs Machines), you keep my app running...
Fly is set up to be the best compute provider, but there are too many reliability and ergonomics issues.
Please stop updating flyctl every time i go to deploy an app
What's the Fly.io issue here? Aren't the issues you're describing in Rails not Fly.io?
I run several Go apps in production on Fly.io[0, 1, 2] and I've never had an issue with the Fly.io + SQLite part of it.
SQLite + Litestream makes things slightly more complicated, but again, I've never had issues with Fly.io specifically making that harder. I use a custom launch script in my Dockerfile that starts litestream with the -exec flag to start my app as a child process.[3]
[0] https://github.com/mtlynch/logpaste
[1] https://github.com/mtlynch/picoshare
[2] https://github.com/mtlynch/screenjournal
[3] https://github.com/mtlynch/logpaste/blob/0.3.1/docker-entryp...
This why I prefer to take backup stuff in a side container, eg: https://github.com/atrakic/gin-sqlite/blob/main/compose.yml
As a side note, you might consider revisiting your dockerfiles and skip litestream build steps, eg. in your final stage just add line like this:
COPY --from=litestream/litestream /usr/local/bin/litestream /bin/litestream
Their managed postgres has gotten better, but its still a little sparse, so after about 6 months using it I am going to just take my DB to either Supabase or Planetscale.
It’s certainly not intuitive. It would be awesome if they sweat these details, but their deal is “here’s a bag of sharp knives”, which is good for some use cases.
Only fuss I remember encountering was with fighting with rails migrating solid queue properly, but this seemed like a rails unit issue and don’t remember it being a Fly issue.
I’ve been contemplating migrating my pg primary to SQLite too. Anyways don’t have much else to offer other than an anecdote that I’m happily using fly with partial SQLite.
> But the market has spoken! Users prefer Litestream. And honestly, we get it: Litestream is easier to run and to reason about. So we’ve shifted our focus back to it.
> In Litestream, we’re solving the problem a different way. Modern object stores like S3 and Tigris solve this problem for us: they now offer conditional write support. With conditional writes, we can implement a time-based lease. We get essentially the same constraint Consul gave us, but without having to think about it or set up a dependency.
Reading this blog post though, I couldn't see any reference to this. Is this supported in Litestream v0.5.0, or will it be for a future release?
https://fly.io/blog/litestream-revamped/https://news.ycombinator.com/item?id=44045292
So far I’ve stuck with the SFTP solution, since I don’t use any of the cloud object storage services that are hardcoded into the tool.[^2]
Big thanks to the developers.
[^1]: https://github.com/benbjohnson/litestream/pull/731
[^2]: https://litestream.io/guides/#replica-guides
Could how does litestream handle backing up through a spotty connection and can we consolidate the backups into a central db an query against it?