ThomasRooney (u/ThomasRooney)

ThomasRooney commented on The Pain That Is GitHub Actions feldera.com/blog/the-pain... · Posted by u/qianli_cs

ThomasRooney · 5 months ago

> A few days ago, someone compromised a popular GitHub Action. The response? "Just pin your dependencies to a hash." Except as comments also pointed out, almost no one does.

I'm surprised nobody has mentioned dependabot yet. It automates this, keeping action dependencies pinned by hash automatically whilst also bringing in stable upgrades.

ThomasRooney commented on Show HN: Jelly – A simpler shared inbox for small teams letsjelly.com/... · Posted by u/mlettini

ThomasRooney · 10 months ago

Would you mind explaining a bit more over why this has value over and above a google group in collaborative inbox mode?

Annecdotally, I think there's a lot of good problems for a new vendor to solve with a product in this category, but a collaborative inbox is really just the baseline of a solution. Personally, the main issue my team has with collaborative inboxes are not issues with handling who replys to each message, it's an issue of spam. Would love to have a vendor build a solution powerful enough to solve these specific problems:

  1. Filtering out automated beg-bounty outreach from any actual security issues by having some form of LLM responder: ideally having a bit of semi-automated back/forth (e.g. approved with a rich Slack button) to help determine if someone is serious or not (after two years of operating, I'm still at 100% of messages (over 1-2 messages per month per company) to security@example.com being spam; suspect over the mid-term it'll still be 98%+). 
  2. Filtering out spam where people are accidentally reaching out to the wrong company. 
  3. Filtering out spam where people are trying to sell us products we're not interested in. E.g. we attend conferences, for every actual conference email we get maybe 5 or 6 trying to sell us attendee email lists.

(would be happy to chat more, if you want to interview a potential customer; if you could really solve these above problems I'd pay you way more than your highest monthly rate on your pricing tier in a heartbeat, ideally scaling per email inbox rather than seat which would be likely be more lucrative for you, and more predictable for me)

ThomasRooney commented on Launch HN: Inkeep (YC W23) – Copilot for Support (think Cursor for help desks) · Posted by u/engomez

ThomasRooney · a year ago

We've been a customer for the past year (https://speakeasy.com/docs). I was honestly highly skeptical about putting a RAG powered search in front of our documentation site instead of what we were using (FlexSearch / Nextra). Have been delighted to be proved wrong.

The learning I've had is that whilst the majority of queries go through standard search patterns (i.e. users search for something that's covered by documentation), a subset of queries are not answerable by our documentation but only implied by it. I've direct experience that Inkeep is serving a large part of that user segment and reducing our support burden.

As a very recent/specific example from last week, we had a community user generating a terraform provider for an internal use-case. By putting error messages from our CLI tooling into Inkeep's "Ask AI" feature, they discovered a nuance in "x-speakeasy-match" (the error message implied it created a circular reference, but didn't spell that out) and self-served a solution.

Inkeep effectively turned our documentation into a guided tutorial on our product, specific to the customer. Pretty strong ROI.

ThomasRooney commented on Generating Code Without Generating Technical Debt? sourcery.ai/blog/chatgpt-... · Posted by u/fagnerbrack

ThomasRooney · 2 years ago

I’ve built conviction that code generation only gets useful in the long term when it is entirely deterministic, or filtered through humans. Otherwise it is almost always technical debt. Hence LLM code generation products are a cool toy, but no sensible teams will use them without an amazing “Day 2” workflow.

As an example, in my day job (https://speakeasyapi.dev), we sell code generation products using the OpenAPI specification to generate downstream artefacts (language SDKs, terraform providers, markdown documentation). The determinism makes it useful — API updates propagate continuously from server code, to specifications, then to the SDKs / providers / docs site. There are no breaking changes because the pipeline is deterministic and humans are in control of the API at the start. The code generation itself is just a means to an end : removing boilerplate effort and language differences by driving it from a source of truth (server api routes/types). Continuously generated, it is not debt.

We’ve put a lot of effort into trying to make an LLM agent useful in this context. However giving them control of generated code directly means it’s hard to keep the “no breaking changes”, and “consistency” restrictions that’s needed to make code generation useful.

The trick we’ve landed on to get utility out of an LLM in a code generation task, is to restrict it to manipulating a strictly typed interface document, such that it can only do non-breaking things to code (e.g. adjust comments / descriptions / examples) by making changes through this interface.

ThomasRooney commented on Show HN: OpenAPI to Terraform Provider Code Generation speakeasyapi.dev/... · Posted by u/ThomasRooney

rjst01 · 2 years ago

This sounds pretty cool, but I'd love to know a bit more about how you handle the impedance mismatch between OpenAPI and an IaaC provider. So far I've only dabbled in making small changes to existing providers but to my somewhat uninformed eyes it seems like a "draw the rest of the owl" situation.

Writing some terraform providers is very likely in our not-to-distant future, so I created an account and imported our spec. It failed validation and wouldn't let me proceed, and the errors provided weren't helpful. When I have a bit more time I'll throw it at another OpenAPI validator to try to work out where the problems are, but we're using this spec for code generation for several clients already and haven't had any issues. It's dynamically generated from our backend.

One more thing, I noticed you have logos for stripe, twilio and plaid on your site, above a "Learn how SDKs help" button, which 404's. If those are your customers that's pretty cool, but otherwise using other companies' logos is a bit shady.

ThomasRooney · 2 years ago

> This sounds pretty cool, but I'd love to know a bit more about how you handle the impedance mismatch between OpenAPI and an IaaC provider. So far I've only dabbled in making small changes to existing providers but to my somewhat uninformed eyes it seems like a "draw the rest of the owl" situation.

So the impedance mismatch is tackled in a few different ways:

CRUD:

The "operation" that interacts with the entity looks in an OpenAPI spec like "POST /entity", "GET /entity/{id}", "DELETE /entity". CRUD semantics aren't visible at this layer without making a bunch of heuristic guesses about the ways that people use (and mis-use) REST semantics.

Rather than guess, we add OpenAPI extensions to operations that guide these semantics. I.e. If `POST /entity` is used to create the entity, it is annotated with `x-speakeasy-entity-operation: MyEntity#create`. Similarly `#update`, `#read`, `#delete` for all other operations that interact with resources in terraform.

Entity Attributes:

The "entity" doesn't usually look the same across all CRUD request/responses. I.e. more attributes (e.g. `id`) are often returned in a response, that aren't in the request.

To tackle this, we annotate every JSON Schema in request/response with `x-speakeasy-entity: MyEntity` that applies to an interaction with an entity in terraform. Some versions of this might be bigger/smaller depending on API semantics. To build the terraform schema entry, we merge all of these together, applying inference logic to work out how each attribute is interacted with in the CRUD requests to determine the terraform properties.

E.g. if an attribute is returned in a CREATE API response, but isn't in a CREATE API Request, it's marked as `Computed`. If one attribute is in the UPDATE API Request, but not another, one is marked as `ForceNew` via a terraform plan extension (i.e. only modifiable with a full delete/create cycle) whereas the other is left alone.

The `type` / `format` that are visible in JSON Schemas make their way to runtime validations, e.g. ensuring that `format: date` are runtime validated in `YYYY-MM-DD` format.

In total there's around 35 different inference rules so far to cover how different bits of the JSON Schema / OpenAPI specification map into terraform state.

Hoisting: The "entity" that is been managed isn't always at the root level of a request/response body, but might be hidden at some deeper level.

E.g. 1: if a response body looked like `{data: {the-entity}}`, we annotate `the-entity` JSON Schema level and "hoist" that to the top level. Anything at a "higher" level is inlined into the resource.

E.g. 2: A parameter in a CREATE request will automatically be inlined into `the-entity`, even if not defined in the JSON Schema of a request body. E.g. `POST /{workspace_id}/entity` marked as `x-speakeasy-entity-operation: MyEntity#create` will not only have whatever request body is defined in the terraform type, but it will also gain an attribute `workspace_id` marked as `ForceNew` in the root of the state.

> Writing some terraform providers is very likely in our not-to-distant future, so I created an account and imported our spec. It failed validation and wouldn't let me proceed, and the errors provided weren't helpful. When I have a bit more time I'll throw it at another OpenAPI validator to try to work out where the problems are, but we're using this spec for code generation for several clients already and haven't had any issues. It's dynamically generated from our backend.

Thanks for letting us know. There's definitely gaps in our validation library: we operate on a relatively high level of strictness to minimize the complexity of code generation, but we're constantly trying to loosen it.

> One more thing, I noticed you have logos for stripe, twilio and plaid on your site, above a "Learn how SDKs help" button, which 404's. If those are your customers that's pretty cool, but otherwise using other companies' logos is a bit shady.

We'll remove them. It's meant to be an illustrator in how improving developer experience directly impacts revenue for a company, directly linking out to some sources. However definitely don't want us to appear shady. Some of our more recognizable customers are listed in the "Trusted By" section in the header.

ThomasRooney commented on Show HN: OpenAPI to Terraform Provider Code Generation speakeasyapi.dev/... · Posted by u/ThomasRooney

ThomasRooney · 2 years ago

Hey HN, after a fair few months of iteration, we're excited to share our latest offering in Speakeasy: auto-generation of Terraform providers using OpenAPI specifications.

The Problem: Building a Terraform Provider to expose an API via Infrastructure As Code (e.g. HCL, Pulumi, CDK) is expensive, error-prone, and highly repetitive.

However, if you don't have a mature terraform provider many products will not even be considered by mature organisations with a mandate to automate-all-the-things.

Our Solution:

1. Deep Integration with OpenAPI: Just point Speakeasy to your OpenAPI spec. Every change, every tweak, every evolution of your API is monitored, and we adapt in real-time.

2. Automated Terraform Provider Generation: Instead of manually coding a Terraform provider, Speakeasy synthesizes one for you, ensuring it remains in sync with your API's latest version.

3. Smart Schema Semantics: Based on CRUD operations, Speakeasy can smartly deduce and apply Terraform schema attributes like Computed, Optional, and Force Replace.

4. Continuous GitHub PRs: With each OpenAPI spec alteration, PRs are raised automatically against your Terraform provider repository, ensuring seamless and continuous integration.

Why Speakeasy for Terraform?

While several tools play around the fringes, none offer 100% automation via code synthesis. We've built and extensively tested our Terraform Provider Generation engine from OpenAPI, and have been in production with real customers for the last 6 months.

* It is possible to generate an OpenAPI specification for almost any server side framework (even things like ProtoBuf via REST Gateways) entirely automatically.

* Once your OpenAPI specification is automatically generated from your codebase, Speakeasy enables subsequent integration artifacts, like SDKs and Terraform Providers, to be automatically maintained with close-to-zero engineering effort.

* Once an API is exposed via a Terraform Provider, it becomes usable by the entire IaC ecosystem like Terraform, CDK, and Pulumi through the use of bridging tools.

* Speakeasy will also generate documentation, usage examples, support/guide you through the launch, and upgrade the provider automatically as the ecosystem matures.

Dive Deeper:

- Explore our product: https://www.speakeasyapi.dev

- Explore our CLI: https://github.com/speakeasy-api/speakeasy

- Explore our largest yet terraform provider: https://github.com/airbytehq/terraform-provider-airbyte

- Explore exposing a terraform provider via Pulumi: https://www.speakeasyapi.dev/post/pulumi-terraform-provider

- Explore a toy example: https://github.com/speakeasy-sdks/terraform-provider-hashicu...

A massive thanks to our early adopters and the vibrant Terraform community for guiding our journey.

HN, we’re eager for your insights. Whether it’s rigorous feedback, burning queries, or just wanting to geek out over Terraform and APIs, hit us up!