Readit News logoReadit News
Posted by u/Jet_Xu 10 months ago
Show HN: LlamaPReview – AI GitHub PR reviewer that learns your codebasegithub.com/marketplace/ll...
I built LlamaPReview to solve a common frustration: most AI code reviewers either require complex setup or don't truly understand project context.

Key differentiators:

1. One-click installation through GitHub Marketplace - no configuration needed 2. Analyzes your entire codebase first to understand: - Project structure - Coding patterns - Naming conventions - Architecture decisions 3. Completely free with no usage limits 4. Fully automated PR reviews with zero human intervention required

Technical implementation: - Built on top of llama-github (my open source project) - Focuses on deep code understanding rather than superficial linting - Provides context-aware suggestions with explanations

The goal is to handle routine reviews automatically so developers can focus on complex architectural decisions. Currently in production and processing real PRs.

Try it for free: https://github.com/marketplace/llamapreview/

Looking for feedback from the HN community, especially on: - What features would make this more useful for your workflow? - How do you currently handle code review automation? - What aspects of code understanding matter most to you?

Roritharr · 10 months ago
Where's the AI Running? Where are you sending the code? Are you keeping some of it?

I hate to be the compliance guy, but even from a startup perspective you'd at least want to mention what you promise to do here.

ljm · 10 months ago
I would want answers to all of these questions before touching an integration like this.

The underlying library it depends on is open source, but this app isn't. Presumably it's holding the codebase in state.

No website to speak of, just boilerplate text to satisfy Github's marketplace submission process.

Eugr · 10 months ago
Would be an instant no-go for any organization or individual that values their IP. Open Source - maybe.
KolmogorovComp · 10 months ago
> Where are you sending the code? Are you keeping some of it?

It does not really matter for FOSS projects. For those fearing licence laundering, don’t worry it will be done anyway for any public code.

neoecos · 10 months ago
The delay in answering this question makes me be more careful about this
snovymgodym · 10 months ago
then who pays for the capacity it runs on?
Jet_Xu · 10 months ago
Thanks for raising these important questions about data privacy and security. Let me clarify:

1. Code Processing: All code analysis happens in-memory during the PR review process. We don't permanently store any of your source code.

2. Data Retention: We only store the PR comments we generate, not the underlying code. This helps maintain a history of our suggestions while protecting your IP.

3. Privacy Focus: We take data privacy seriously and have successfully worked with both open-source and closed-source projects. We're always open to suggestions on how to further enhance our privacy measures.

If you have specific privacy requirements or suggestions, I'd be happy to discuss them.

refulgentis · 10 months ago
Feedback:

1. Description* reeks of AI slop; it extended a surface-level prompt into longer surface-level insights. *: description as in GitHub README

2. #1 creates a situation where I go through reading this long thing, and realize it has no answers to even the first-level questions that would be on anyones mind (what model? where is it run?). For this to become something I'll take the time to integrate into my core workflow and try, it has to be *much* more transparent.

3. Claims in the description are ~impossible.

3b. Up front, I feel your pain, there's a hard set of constraints to navigate here given A) marketing needs to be concise B) people play fast and loose with conciseness vs. accuracy C) you need to sounds as good as the people in B.

3c. That being said, we're crossing into year 3 of post-ChatGPT. People, especially in your target audience, will know when they're reading* that you're reframing "I give text to the LLM which can theoratically do $X" into features, and users expect features to be designed* and intentional. If they are, you should definitely highlight that to differentiate from people who just throw it into the LLM.

3d. Analyzes your entire repository context: impossible, literally, unless you're feeding it to Gemini only. I have about 20KLOC and its multiples of Llama context size.

3e. "Understands code relationships and dependencies" see 3c.

3f. "Contextual Analysis: Reviews code changes within the full repository context": see 3d.

3g. "Language Agnostic: Supports all major programming languages.": see 3c (is there actual work done to do this, or is this just "well, given I just send the text to the LLM, everything is supported"?)

4. nit: Should be "Show HN: LlamaPReview, AI Github PR Reviewer That Learns Your Codebase"

Jet_Xu · 10 months ago
Thank you for such detailed and thoughtful feedback! Really appreciate the time you took to analyze our claims and point out the areas needing more clarity.

You're absolutely right about the marketing copy - we should be more precise and transparent about what we actually do vs. what's aspirational.

Regarding "understanding code relationships and dependencies": We're building a knowledge graph of the entire repository that captures code relationships, function calls, and module dependencies. This graph is then used with GraphRAG to fetch relevant context for each PR, allowing the LLM to understand the broader impact of changes.

Important to note: We take privacy very seriously. All code analysis happens in-memory during PR reviews - we don't permanently store any source code or build persistent knowledge bases from customer code. The knowledge graph is generated and used on-the-fly for each review session.

This approach helps us work around context window limitations while providing meaningful insights. However, I should note that this feature is still under active development - we're continuously improving the graph construction and relevancy matching.

Would love to hear your thoughts on this approach. We're committed to building something genuinely useful for developers rather than just another LLM wrapper.

cutemonster · 10 months ago
> Analyzes your entire repository context: impossible

That might be sort of doable, by extracting all function signatures together with brief descriptions, and including that in the context, and maybe a graph showing how they call each other,

but none of the actual implementations. Except for the file(s) under review, which would be included in full.

Zondartul · 10 months ago
By "learns" do you mean "just shove the entire codebase into the context window", or does actual training-on-my-data take place?
Jet_Xu · 10 months ago
The "learning" process involves analyzing your codebase's context during PR reviews - we don't train on your data (we even will not save them but only calculate in memory). Instead, we use advanced context retrieval to understand:

- Project structure and architecture - Coding patterns and conventions - Dependencies and relationships between components

This allows us to provide more relevant and context-aware reviews while maintaining data privacy (some advanced features still is under developing)

sksxihve · 10 months ago
Are people really willing to commit code that was only reviewed by an AI? I personally wouldn't trust that for anything that is customer/revenue impacting. Obvious bugs and defects aren't all that hard to catch in normal code reviews but subtle race conditions/deadlocks/memory errors can be very tricky, do you have examples where it shows it can catch those?
brunooliv · 10 months ago
This assumes that human reviews also catch these which is DEFINITELY not the case either.

As long as you have good pipelines, linters, a careful suite of tests at different levels like unit, integration, e2e and if you can test things in an acceptable like environment then human code reviews offer very very little benefit…

sksxihve · 10 months ago
Your never going to catch 100% of issues, human or AI review, but I've found that in code reviews a lot of the benefit is when people ask questions about the code being reviewed and have a discussion on it.

Is the AI tool going to ask why something was implemented in a way that might not match the requirement specs? Is it even going to know what the requirements are for the code or is it going to rubber stamp a review because the code looks reasonable?

If you think human code reviews offer very very little benefit then you probably aren't doing them right.

t43562 · 10 months ago
PR reviewing isn't really about finding bugs because the tests should be doing that - but are the tests good enough? Is the approach sound and aligned with the architecture? And does anyone else understand it apart from the author?
Jet_Xu · 10 months ago
Great points about code review reliability. LlamaPReview is designed to be a complementary tool for senior developers, not a replacement for human review. Here's our approach:

1. It helps save senior developers' time by handling routine checks and providing initial insights 2. It analyzes the entire codebase context to provide more meaningful reviews 3. It's particularly useful for identifying patterns and relationships across the codebase

The goal is to make human reviewers more efficient, allowing them to focus on complex architectural decisions and critical business logic. We've seen positive results from both open-source and commercial projects using this approach.

nikolayasdf123 · 10 months ago
from your Privacy Policy, you straight up collecting users code. do you send it to someone else as well?

might make sense for open source. closed source is no go for this.

Jet_Xu · 10 months ago
Thanks for raising this important question. We will not store any code in our database. But we will leverage SaaS LLM API (e.g. GPT/Claude/Mistral) to help on the PR review - during this step, for sure we need to send code to these SaaS LLM for analyze. This is the main reason why we mentioned "collecting users code" in our privacy.
GavCo · 10 months ago
This reminds me of the PR Agent open source tool: https://github.com/Codium-ai/pr-agent

I've found the code walkthroughs very useful

Jet_Xu · 10 months ago
Thanks for mentioning PR Agent. While there are several tools in this space, LlamaPReview focuses on deep codebase understanding and context-aware reviews(advanced functions still under evolution). We'd love to hear about your experiences and what specific features you find most valuable in code review tools.
Squeeze2664 · 10 months ago
A name like llama-pr-review might help with searching for this thing. Preview being an actual word and all.
agilob · 10 months ago
Description says:

> Unlimited AI-powered PR reviews

FAQ says:

> A: Yes, we currently offer a free tier with usage limits. You can install and use LlamaPReview without binding any payment method.

Only "free tier" is available.