dsandip (u/dsandip) - Readit News

dsandip commented on Streaming data in Postgres to 1M clients with GraphQL github.com/hasura/graphql... · Posted by u/tango12

dsandip · 3 years ago

FD: I work at Hasura.

Seeing some feedback on GraphQL - Hasura has had support for converting templated GraphQL into RESTish endpoints (with Open API Spec docs if needed). We are planning to do the same for this streaming API as well - does anyone have good examples of existing REST/RESTish endpoints that something similar?

dsandip commented on Streaming data in Postgres to 1M clients with GraphQL github.com/hasura/graphql... · Posted by u/tango12

maxpert · 3 years ago

While I love ingenuity of developers at Hasura (because I've been personally through these scaling challenges), I always get a gag reaction with GraphQL. I've honestly tried hard to digest it, and I can tell you at large scale where single DB won't cut it, you would either need to develop a large federation layer like [Netflix](https://netflixtechblog.com/how-netflix-scales-its-api-with-...), or just rip-it out. Streaming might elevate the problem a little, but I real problem still lurks under the hood. The fact that front-end community wants to fit everything under GraphQL bothers me, because every backend developer knows that a single tool/technology is usually not the best tool for solving all problems of your life. Remember the golden words, THERE IS NO SILVER BULLET!

dsandip · 3 years ago

FD: I work at hasura and work with users/customers.

Fair point about the DB scaling but not sure if everyone is going to run into this issue. Also, lots of solutions are emerging for this specific problem (with different trade-offs of course) like distributed databases (crunchy, YugaByte, Spanner, etc.). Most folks I work with get by with a reasonably sized DB and some read replicas.

Not a GraphQL problem though IMO.

dsandip commented on Show HN: America – Road Trip Simulator 4m3ric4.com... · Posted by u/0x389

dsandip · 3 years ago

As a recent transplant to the US, this looks like a fun way to get my road trip plans going :). Thank you for this!

dsandip commented on Is Amazon's cloud service too big to fail? fnlondon.com/articles/is-... · Posted by u/azureel

Artemis2 · 8 years ago

Terraform (https://www.terraform.io), which we use, is a neat way to abstract this configuration data. We keep the configuration files in git, and can do GitHub pull requests over our whole infrastructure and apply the configuration with confidence (to change the existing setup or re-create it from scratch). This works for multiple cloud providers and is great for security purposes (all changes are auditable, no configuration drift).

Heap has a great blog post on Terraform : https://heap.engineering/terraform-gotchas/

dsandip · 8 years ago

Essentially, a declarative configuration for infra is what you are getting at. You can take this further, using containers and orchestration tech, to abstract your application behind a declarative configuration making it infra agnostic (as it should be iMO). Obviously, not getting locked into any cloud provider services is a pre-requisite for this. Check out such an implementation here[1](full disclosure - I work here))

[1]: https://docs.hasura.io/0.14/ref/project-configuration-and-st...

dsandip commented on Is Amazon's cloud service too big to fail? fnlondon.com/articles/is-... · Posted by u/azureel

unkown-unknowns · 8 years ago

There are some open source implementations of parts of the APIs of cloud providers that might help someone a bit when trying to migrate. For example, Minio [1] [2] implements the AWS S3 v4 API.

[1]: https://news.ycombinator.com/item?id=12392081

[2]: https://minio.io/

dsandip · 8 years ago

We[1] have seen a fair number of requests for managed services as devs claim that "we don't have the time or skills to maintain these open source components" (quoting verbatim from a request on Intercom). I don't think this is about having open source substitutes in all the cases. Personally not a fan of how/where the build vs. buy debate is playing out here.

[1]: https://hasura.io

dsandip commented on The S3 outage or: How you can learn to stop worrying and love the multi-cloud blog.hasura.io/the-amazon... · Posted by u/anandr2013

jedberg · 9 years ago

Multi-cloud is a pipe dream. It's only slightly worse than hybrid cloud, which is also terrible.

Sure, you could use multiple clouds for a couple of small things, but it's all about the data.

To properly do multi-cloud (or hybrid cloud), you need to have the right data in the right place at the right time, or you need to have all your data in both places at all times.

The latter will cost you a fortune to maintain. Netflix does it across regions because their cost of operations, including keeping multiple copies of all the data, is lower than the cost of downtime, but that isn't likely to apply to most people.

The former can only work if you are really disciplined about keeping the right data in the right place, or if you have a data abstraction layer that is smart enough to put the right data in the right place at the right time (I'm not aware of the existence of such a product or that anyone is even working on such a thing).

The other problem with multi-cloud is that your software has to be built to the lowest common denominator. In other words, you can only use features that all your providers provide, again unless you go back to having software smart enough to do the right thing in the right place at the right time.

It's a nice thought but I think we're years away from viable multi-cloud.

dsandip · 9 years ago

Agree with most of the points you made. I remember the amount of effort my team in an ad-tech co. spent distributing load across multiple providers/regions to bring down latency and add redundancy. The larger point is that the ecosystem needs to enable this by product-ifying this (a data abstraction layer that is smart enough to put the right data in the right place at the right time).