Somewhat OT and forgive me if I'm missing something on your github, I don't see how to set up a cluster locally - am I thinking about Neon wrong and it only runs on one node at a time with shared storage if you want to migrate which node runs a given database at a given moment?
Aurora Postgres is a fork of Postgres code, with modifications that allow it to leverage AWS' Aurora storage, and a few other things (replicas, etc.).
The goal is to remain 100% Postgres compatible. So the network protocol, SQL queries, and tools, all can be used without any change. There are (were?) some (read: very few) caveats/restrictions, though, because of the heritage of, and need to be compatible with, Amazon RDS Postgres.
Disclosure: Member of the founding-team of Aurora Postgres; left the team quite a while back, so my knowledge is quite outdated.
Edit: There's a lot of literature, and public information (articles, conference talks, etc.) published by the Aurora team over the years, which all allude to these facts, and you can use that material to build your confidence that it is in fact Postgres compatible.
For no particularly good reason, I was under the impression Postgres Aurora was just a Postgres compatibility interface layer on top of a common substrate shared with the MySQL Aurora implementation.
it's way better than vanilla RDS even without any clustering. it is far more stable under load / near capacity. the storage layer is amazing. people here who are underselling it relative to vanilla RDS simply lack operational experience under load and have a checklist-level understanding
There's some deep dives on the aws YouTube channel that talk about their storage layer if you want some detail.
AWS happily share quite a lot more under NDA as well, if that's an option for you.
The short answer is the proprietary changes are very well done in a way that provides no meaningful vendor lock-in.
I’m really glad to see this. Historically Aurora has been VERY slow to support new versions of Postgres (despite promising this would not be the case). But lately they have really stepped up their pace to the point where I would consider migrating some databases to Aurora. I’ma big fan of the feature that allows you to clone a large database quickly (in 30 minutes maybe). It’s extremely useful for troubleshooting issues where you need production data.
My understanding is the earlier versions of postgres required more expensive patching to support their proprietary storage layer.
Some of the common hooks have since been upstreamed (whether aws gets any credit or if it was part of other work I couldn't say), but in theory aws doesn't have to do as much work to support new versions as they used to.
We have a database with a few TB of data. We can create an aurora clone in around 10 mins give or take, but what we’ve found is that this the clone loads data in lazily as needed. Meaning the first time you hit a given table in a query it can be very slow. Afterwards it works well though
Ignoring the need for a Synchronous Replica, you may see Aurora Postgres extremely costly and not too great in performance compared to PostgreSQL. This has been neatly documented in this Article
What about RDS Proxy? I believe it's still only supports 13 which means if you use both the Proxy and Aurora you will have to wait? RDS Proxy does still mention only support for v13[1].
RDS Proxy is very specific with which versions it supports. If it says it doesn't support it, it doesn't support it.
Postgres 14 is supposed to have built-in connection pooling. If you're on it, do you still feel the need for RDS Proxy? We're using pgbouncer and can't decide if we should switch to RDS Proxy or upgrade to Postgres 14 and drop the external connection pooler.
I work in an enterprise software ecosystem and on current version (something we call the converged platform), we are getting our SaaS customers upgraded to Aurora from previous SQL DB. It’s just what our DevOps calls a “life and shift upgrade” (whatever the heck that means coz the issues that entails it gives us a hard time)
Lift and Shift — move an application from one environment to another with minimal architecture changes. The difficulty comes when you have built-in assumptions that can’t be replicated to the new environment. Some good examples would be clusters that rely on floating ips, ARP, interface bonding, SCSI, or multicast.
Over the years cloud providers have added these features but they weren’t always available and definitely not something you would reach for in a cloud native setup.
Readit News
https://cloud.google.com/blog/products/databases/introducing...
(I’m ceo of neon)
Deleted Comment
The goal is to remain 100% Postgres compatible. So the network protocol, SQL queries, and tools, all can be used without any change. There are (were?) some (read: very few) caveats/restrictions, though, because of the heritage of, and need to be compatible with, Amazon RDS Postgres.
Disclosure: Member of the founding-team of Aurora Postgres; left the team quite a while back, so my knowledge is quite outdated.
Edit: There's a lot of literature, and public information (articles, conference talks, etc.) published by the Aurora team over the years, which all allude to these facts, and you can use that material to build your confidence that it is in fact Postgres compatible.
For no particularly good reason, I was under the impression Postgres Aurora was just a Postgres compatibility interface layer on top of a common substrate shared with the MySQL Aurora implementation.
We use Aurora for all apps. It costs more, but needs little to no maintenance from our side
https://www.migops.com/blog/is-aurora-postgresql-really-fast...
[1] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...
Postgres 14 is supposed to have built-in connection pooling. If you're on it, do you still feel the need for RDS Proxy? We're using pgbouncer and can't decide if we should switch to RDS Proxy or upgrade to Postgres 14 and drop the external connection pooler.
Any references on this? I see a note somewhere about better connection handling, but nothing that would remove the need for connection pooling.
This is false. No PostgreSQL version has had a built-in connection pooler. There was a patch some years ago[0], but it didn't make it into PostgreSQL.
[0] https://www.postgresql.org/message-id/flat/ac873432-31cf-d5e...
Over the years cloud providers have added these features but they weren’t always available and definitely not something you would reach for in a cloud native setup.
Deleted Comment