$8/100k tokens strikes me as potentially a TON if the idea is that we're going to be running this as part of the iterative local development cycle (or god forbid letting agents run it whenever they decide). As you mentioned, one of the issues with AI generated code is often that it writes too much and needs direction on shrinking down.
I could easily see hitting 10k+ LOC on routine tickets if this is being run on each checkpoint. I have some tickets that require moving some files around, am I being charged on LOC for those files? Deleted files? Newly created test files that have 1k+ lines?
It's $8/100K lines of code. Since we're using a mix of models across our main agent and sub-agents, this normalizes our cost.
> I could easily see hitting 10k+ LOC on routine tickets if this is being run on each checkpoint. I have some tickets that require moving some files around, am I being charged on LOC for those files? Deleted files? Newly created test files that have 1k+ lines?
We basically look at the files changed that need to be reviewed + the additional context that is required to make a decision for the review (which is cached internally, so you'd not be double-charged).
That said, we're of course open to revising the pricing based on feedback. But if it's helpful, when we ran the benchmarks on 165 pull requests [1], the cost was as follows:
- Autofix Bot: $21.24 - Claude Code: $48.86 - Cursor Bugbot: $40/mo (with a limit of 200 PRs per month)
We have several optimization ideas in mind, and we expect pricing to become more affordable in the future.
[1] https://github.com/ossf-cve-benchmark/ossf-cve-benchmark
In your explanation here, you mention running it per PR - does this mean running it once? Several times?