nicksnyder (u/nicksnyder)

nicksnyder commented on What If We Could Rebuild Kafka from Scratch? morling.dev/blog/what-if-... · Posted by u/mpweiher

peanut-walrus · 8 months ago

So it's cheaper *on AWS*. Any cloud provider where cross-AZ traffic is not $$$, I can't imagine this architecture being cheaper.

Engineering solutions which only exist because AWS pricing is whack are...well, certainly a choice.

I can also think of lots of cases where whatever you're running is fine to just run in a single AZ since it's not critical.

nicksnyder · 8 months ago

The other clouds have fees like this too.

Even if this were to change, using object storage results in a lot of operational simplicity as well compared to managing a bunch of disks. You can easily and quickly scale to zero or scale up to handle bursts in traffic.

An architecture like this also makes it possible to achieve a truly active-active multi-region Kafka cluster that has real SLAs.

See: https://buf.build/blog/bufstream-multi-region

(disclosure: I work at Buf)

nicksnyder commented on Using gRPC for (local) inter-process communication (2021) mpi-hd.mpg.de/personalhom... · Posted by u/zardinality

palata · a year ago

I have been in a similar situation, and gRPC feels heavy. It comes with quite a few dependencies (nothing compared to npm or cargo systems routinely bringing hundreds of course, but enough to be annoying when you have to cross-compile them). Also at first it sounds like you will benefit from all the languages that protobuf supports, but in practice it's not that perfect: some python package may rely on the C++ implementation, and therefore you need to compile it for your specific platform. Some language implementations are just maintain by one person in their free time (a great person, but still), etc.

On the other hand, I really like the design of Cap'n Proto, and the library is more lightweight (and hence easier) to compile. But there, it is not clear on which language implementation you can rely other than C++. Also it feels like there are maintainers paid by Google for gRPC, and for Cap'n Proto it's not so clear: it feels like it's essentially Cloudflare employees improving Cap'n Proto for Cloudflare. So if it works perfectly for your use-case, that's great, but I wouldn't expect much support.

All that to say: my preferred choice for that would technically be Cap'n Proto, but I wouldn't dare making my company depend on it. Whereas nobody can fire me for depending on Google, I suppose.

nicksnyder · a year ago

If you are looking for a lightweight Protobuf based RPC framework, check out https://connectrpc.com/. It is part of the CNCF and is used and supported by multiple companies: https://buf.build/blog/connect-rpc-joins-cncf

gRPC ships with its own networking stack, which is one reason why those libs are heavy. Connect libraries leverage each ecosystem's native networking stack (e.g. net/http in Go, NSURLSession in Swift, etc.), which means any other libraries that work with the standard networking stack interop well with Connect.

nicksnyder commented on Ask HN: Who is hiring? (November 2024) · Posted by u/whoishiring

nicksnyder · a year ago

Buf | Software Engineer (Frontend or Infrastructure) | Remote (US or Canada) | Full time | https://buf.build/

Buf is building a Protobuf platform that provides end-to-end type safety and governance across all APIs and streaming data in Kafka. We’re deployed everywhere from small startups to the world’s largest enterprises, and we’re hiring to scale our growing business.

Tech stack: Go, TypeScript, Kubernetes

If you are interested to work at Buf, apply to one of our open roles here: https://buf.build/careers

nicksnyder commented on Ask HN: Who is hiring? (October 2024) · Posted by u/whoishiring

nicksnyder · a year ago

Buf | Software Engineer (Infrastructure, Go backend), Engineering Manager (Infrastructure) | Remote (US or Canada) | Full time | https://buf.build/

Buf is building a Protobuf platform that provides end-to-end type safety and governance across all APIs and streaming data in Kafka. We’re deployed everywhere from small startups to the world’s largest enterprises, and we’re hiring to scale our growing business.

Tech stack: Go, TypeScript, Kubernetes

If you are interested to help us build out our infrastructure platform or work on Bufstream (our drop-in replacement for Kafka that is 10x cheaper), apply here: https://buf.build/careers

nicksnyder commented on Ask HN: Who is hiring? (September 2024) · Posted by u/whoishiring

nicksnyder · a year ago

Buf | Software Engineer, Engineering Manager | Remote (US or Canada) | Full time | https://buf.build

Buf is building a new paradigm for APIs. Our open source and commercial software helps companies adopt Protobuf schemas throughout their stack, from gRPC APIs to streaming data in Kafka. We’re deployed everywhere from small startups to the world’s largest enterprises, and we’re hiring to scale our growing business.

Tech stack: Go, TypeScript, Kubernetes Open roles: https://buf.build/careers

nicksnyder commented on Ask HN: Who is hiring? (June 2021) · Posted by u/whoishiring

nicksnyder · 5 years ago

Sourcegraph (https://sourcegraph.com) | Software Engineer (frontend, backend, security), Product Manager, Director of Engineering, Director of Product | ALL REMOTE

Our mission at Sourcegraph is to make it so that everyone can code. Software has driven a lot of technological innovation, and yet today, only a small fraction of the world can contribute to this progress by writing code. We believe that enabling more people to code will create economic opportunity across the world and will drive better and faster technological progress that benefits everyone.

The problem developers face today is that the amount, complexity, and value of code is increasing quickly, and it is hard to keep up. We call this problem Big Code. Sourcegraph helps developers at our customers and in open source, find code, understand code, and automate changes to that code at scale in massive codebases.

It’s an exciting time to join because our business is growing rapidly (4x revenue growth last year with zero customer churn) and our $50m Series C from Sequoia has given us the opportunity to make these big ambitious bets while being fiscally responsible.

We are an equal opportunity workplace and our all-remote team is spread across the world. We embrace diversity and welcome people from all backgrounds and communities.

Transparency is a core value of ours and you can learn more about how we operate by reading our handbook: https://about.sourcegraph.com/handbook.

If you want to make software development more accessible to everyone, we would love to review your application to one of our many open roles! https://boards.greenhouse.io/sourcegraph91

Other useful links:

Our mission: https://about.sourcegraph.com/company/strategy

Our values: https://about.sourcegraph.com/company/values

Our customers: https://about.sourcegraph.com/customers/

Our code: https://github.com/sourcegraph/sourcegraph

Our blog: https://about.sourcegraph.com/blog

News about us: https://about.sourcegraph.com/news/

Series C funding: https://about.sourcegraph.com/blog/series-c-with-sequoia/

nicksnyder commented on Ask HN: Who is hiring? (February 2021) · Posted by u/whoishiring

nicksnyder · 5 years ago

Sourcegraph (https://sourcegraph.com) | Software Engineer, Security Engineer, Product Manager | ALL REMOTE

Sourcegraph is building Universal Code Search so developers can focus on solving problems instead of struggling to find and understand code.

We’re hiring for several teams:

* Code insights team (Frontend, Backend): The code insights team is looking for experienced, passionate engineers to build with us an entirely new product area of Sourcegraph. Code insights is the first feature in Sourcegraph that provides high-level information about code, building atop all Sourcegraph’s other features and even external data sources.

* Cloud team (Frontend, PM): The Cloud team is looking for a senior frontend engineer who knows how to build intuitive user experiences to make the power of Universal Code Search accessible to everyone on sourcegraph.com. You will have a lot of ownership to solve tough technical and UX problems that will drive massive adoption of our product by developers all around the world.

* Campaigns team (Full Stack/Frontend): The Campaigns team is looking for an experienced engineer to help us build a new product that will define how our industry accomplishes large scale code changes. Campaigns enable developers to make changes across many repositories and code hosts, using code.

* Search team (Backend, PM): We are looking for backend engineers who know how to design and build performance-sensitive distributed systems. You will make Universal Code Search more powerful and scale to our customers with the largest codebases in the world.

* Distribution team: We are looking for software engineers who are passionate about and skilled at making software easy to build, deploy, configure, monitor, and debug. You will have ownership to solve challenging problems that our customers face when they deploy and scale Sourcegraph on-premise in a variety of environments, and that we face when we deploy and scale Sourcegraph.com (the largest Sourcegraph installation in the world).

* Security team: We are looking for an engineer who will proactively improve the security of our codebase, our product, our cloud, and our customers’ on-premise deployments. You will be one of the early security team hires at Sourcegraph, where everyone contributes to our shared goals, from writing code to running vulnerability scans.

We are a completely open and transparent company and you can learn more about how we operate by reading our handbook: https://about.sourcegraph.com/handbook. Technologies that we use: Go, TypeScript, React, RxJS, GraphQL, Postgres, Docker, Kubernetes.

We are an equal opportunity workplace and our all-remote team is spread across the world. We embrace diversity and welcome people from all backgrounds and communities.

If you want to make software development more accessible to everyone, come join us!

Apply to one of our many open roles here: https://about.sourcegraph.com/company/careers

nicksnyder commented on Ask HN: Who is hiring? (January 2021) · Posted by u/whoishiring

nicksnyder · 5 years ago

Sourcegraph (https://sourcegraph.com) | Frontend Engineer, Backend Engineer, Security Engineer, Engineering Manager, Product Manager, and more | ALL REMOTE Sourcegraph is building Universal Code Search so developers can focus on solving problems instead of struggling to find and understand code. We just raised a $50 Series C from Sequoia (https://about.sourcegraph.com/blog/series-c-with-sequoia/) so we can accelerate our mission to enable everyone to code.

Universal Code Search helps developers find and discover code by providing the ability to quickly search across all code using regular expressions and advanced filters. Sourcegraph code intelligence helps developers navigate and understand code by providing hover tooltips, go-to-definition, and find-references, when viewing code on Sourcegraph or on a code host (via our browser extension). Sourcegraph campaigns enable developers to fix and evolve code at scale by automating search-and-replace and running arbitrary tools across massive codebases.

We are a completely transparent company and you can learn more about how we operate by reading our handbook: https://about.sourcegraph.com/handbook. Technologies that we use: Go, TypeScript, React, RxJS, GraphQL, Postgres, Docker, Kubernetes.

We are an equal opportunity workplace and our all-remote team is spread across the world. We embrace diversity and welcome people from all backgrounds and communities.

If you want to make software development more accessible to everyone, come join us!

Apply to one of our many open roles here: https://about.sourcegraph.com/company/careers