Readit News logoReadit News
NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
apwell23 · a month ago
I am not doubting 95% acceptance rate all. I've pure vibecoded many toy projects myself.

> in line with what they would have written,

point i am making is that they didn't know what they would've written. they had a rough overall idea but details were being accepted on the fly. They were trying out bunch of things and see what looks good based on a rough idea of what output should be.

In a real world project you are not both product owner and coder.

NathanKP · a month ago
To be clear I did not have a 95% acceptance rate. I'm saying that in the final published repo, 95% of the lines of code were written by AI, not by me. I discarded and refactored code along the way many times, but I did that by also using the AI. My end goal was to keep my hands off the code as much as possible and get better at describing exactly what I wanted from the AI.
NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
apwell23 · a month ago
> precise prompt

If there was such a thing you would just check in your prompts into your repo and CI would build your final application from prompts and deploy it.

So it follows that if you are accepting 95% of what random output is being given to you. you are either doing something really mundane and straightforward or you don't care much about the shape of the output ( not to be confused with quality) .

Like in this case you were also the Product Owner who had the final say about what's acceptable.

NathanKP · a month ago
> if you are accepting 95% of what random output is being given to you

I am not, and don't expect to be able to do that for many years yet. The models aren't that good yet.

I would estimate that I accepted perhaps 25% of the initial code output from the LLM. The other 75% of output I wasn't satisfied with I just unapplied and retried with a different prompt, or I refactored or mutated it using a followup prompt.

In the final project 95% of the committed lines of code in the published version were written by AI, however there was probably 4x as much discarded AI generated code along the way that was also written by AI. Often the first take wasn't good enough so I modified it or refactored it, also using AI. Over the course of using the project I got better at providing more precise prompts that generated good code the first time, however, I rarely accepted the first draft of code back from Kiro without making followup prompts.

A lot of people have a misguided thought that using AI means you just accept the first draft that AI returns. That's not the case. You absolutely should be reading the code, and iterating on it using followup prompts.

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
exclipy · a month ago
That's a compelling three file format.

Have you considered a fourth file for Implemented such that Spec = Implemented + Design?

It would serve both as a check that nothing is missing from Design, and can also be an index for where to find things in the code, what architecture / patterns exist that should be reused where possible.

And what about coding standards / style guide? Where does that go?

NathanKP · a month ago
That is interesting. So far we are just using the task list to keep track of the list of implemented tasks. In the long run I expect there will be an even more rigorous mapping between the actual requirements and the specific lines of code that implement the requirements. So there might be a fourth file one day!

Coding standards / style guide are both part of the "steering" files: https://kiro.dev/docs/steering/index

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
apwell23 · a month ago
> almost 95% AI coded

I think its because you didn't have hard expectations for the output. You were ok with anything that kind of looked ok.

NathanKP · a month ago
False. In order to maintain high quality I often rejected the first result and regenerated the code with a more precise prompt, rather than taking the first result. I also regularly used "refactor prompts" to ask Kiro to change the code to match my high expectations.

Just because you use AI does not mean that you need to be careless about quality, nor is AI an excuse to turn off your brain and just hit accept on the first result.

There is still a skill and craft to coding with AI, it's just that you will find yourself discarding, regenerating, and rebuilding things much faster than you did before.

In this project I deliberately avoided manual typing as much as possible, and instead found ways to prompt Kiro to get the results I wanted, and that's why 95% of it has been written by Kiro, rather than by hand. In the process, I got better at prompting, faster at it, and reached a much higher success rate at approving the initial pass. Early on I often regenerated a segment of code with more precise instructions three or four times, but this was also early in Kiro's development, with a dumber model, and with myself having less prompting skill.

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
NathanKP · a month ago
The original submission to HN stated that it was from Amazon / AWS in the title of the submission, however that has since been edited by a moderator to match the title of the blogpost, which does not mention Amazon / AWS.

To be clear, we have no intent to hide that Kiro is from Amazon / AWS, that's why you'll see Matt Garman, for example, posting about Kiro: https://www.linkedin.com/feed/update/urn:li:activity:7350558...

However, the long term goal is for Kiro to have it's own unique identity outside of AWS, backed by Amazon / AWS, but more friendly to folks who aren't all in on AWS. I'll admit that AWS hasn't been known in recent years for having the best new user or best developer experience. Kiro is making a fresh start from an outsider perspective of what's possible, not just what's the AWS tradition. So, for example, you can use Kiro without ever having an AWS account. That makes it somewhat unique, and we aim to keep it that way for now.

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
personjerry · a month ago
Why not deploy the game? I'd love to try it
NathanKP · a month ago
I have a personal deployment of the game, but it costs money to run the LLM so I'm not sharing that with all of Hacker News haha. I've got an appsec ticket open to host an "official AWS" version where AWS pays the LLM bill, but that might take a while longer to get approved. For now the best way to experiment is playing with it locally.

I'm also thinking of creating a fork of the project that is designed to run entirely locally using your GPU. I believe with current quantized models, and a decent GPU, you can have an adequate enough fully local experience with this game, even the dynamic image generation part.

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
adastra22 · a month ago
And then you have two separate specifications of your intent, with the ongoing problems that causes. It’s not the same thing.
NathanKP · a month ago
Yeah it would be nice if there was one way to specify the rules and intent, but you know how these things go: https://xkcd.com/927/

In all seriousness, I'm sure this will become more standardized over time, in the same way that MCP has standardized tool use.

I've long been interested in something that can gather lightweight rules files from all your subdirectories as well, like a grandparent rule file that inherits and absorbs the rules of children modules that you have imported. Something kind of like this: https://github.com/ash-project/usage_rules

I think over time there will be more and more sources and entities that desire to preemptively provide some lightweight instructive steering content to guide their own use. But in the meantime we just have to deal with the standard proliferation until someone creates something amazing enough to suck everyone else in.

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
ugh123 · a month ago
Love the game! would be interesting to see an example of prompts used to do this.
NathanKP · a month ago
Unfortunately, midway through the project I lost the file where I was keeping track of all the prompts I used as I built. I do have some of them, plan to publish a wrap up analysis of those at some point.

If you were referring to the prompts inside of the game, you might find those fun and interesting. This one in particular is the heart of the game: https://github.com/kirodotdev/spirit-of-kiro/blob/main/serve...

NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
erichocean · a month ago
How much are you using Kiro to improve itself? 100% of the time? 10% of the time? Never?
NathanKP · a month ago
It has grown over time as Kiro has developed. Many of the most recent features in Kiro were developed using Kiro specifications. We have a Twitch stream scheduled with some engineers from the Kiro team where we plan to take live Q&A about this in specific, how they are using Kiro to build Kiro, etc. I don't have the schedule setup yet, but we've got the channel setup here: https://www.twitch.tv/kirodotdev
NathanKP commented on Kiro: A new agentic IDE   kiro.dev/blog/introducing... · Posted by u/QuinnyPig
adastra22 · a month ago
“I really don’t want to do X”

“Kirk is actually quite good at this: you just have to do X”

“…”

NathanKP · a month ago
At the prompt: "I have extensively used Copilot, Continue, Cursor, Cline, Aider, Roo Code, and Claude Code. I do not want to move my files over again for Kiro [even if it's as simple as dragging and dropping files]. Do it for me"

Kiro will do it for you automatically.

u/NathanKP

KarmaCake day7718July 22, 2009
About
Nathan Peck, Senior Developer Advocate for Generative AI at AWS

Github: @nathanpeck | Email: peckn@amazon.com | Website: nathanpeck.com

[ my public key: https://keybase.io/nathanpeck; my proof: https://keybase.io/nathanpeck/sigs/B-e0NwWkpbMqqifxC4U92-iq4su6lSYxspt2QhfY2Ew ]

View Original