I do RAG for other types of structured data and this is fundamental to get relevant objects in your context.
My approach for code would be to create a graph structure with relationships between the different codepaths and expose a retrieval api through tools/function calling so that the LLM can query the codebase structure on top of doing semantic embedding similarity search and text similarity search.
You could also add a graph search for related elements for each element returned by the other search pipelines to increase the chance of having all the pieces of the puzzle in the context before using the LLM to solve the problem.
The other crucial thing to do would be to inspect dependencies (and their types, when possible) and maybe download documentation to offer tips that are accurate and not hallucinated.
Nowadays I get hallucinations for code generation as soon as things get hard, making LLM coding useful only for trivial code writing.
Analysing the code structure and dependencies would require plenty of work for each specific language, so it won't be a easy win like "just throwing RAG" - which is what the current players are doing to raise money - with mediocre results.
What was missing from the 1990 Web? The original Web did not effectively support the one-paragraph idea. A reader who encountered a Web page containing only one paragraph would have been startled and wondered if the page had been left unfinished.
I feel like this is both very right but also the million dollar question?
I don't think others necessarily quote-unquote "lost focus" on this problem, but it's not exactly easy to solve correctly, so in the meantime it's easier to create something with the next best approximation.
Typo of "codebase" as codebae: This is also the reason why the higher context window doesn’t matter. Even if you could feed your whole codebase into an LLM, you’d still face the same problem of missing the structural relationships of the codebae.
Readit News
I do RAG for other types of structured data and this is fundamental to get relevant objects in your context.
My approach for code would be to create a graph structure with relationships between the different codepaths and expose a retrieval api through tools/function calling so that the LLM can query the codebase structure on top of doing semantic embedding similarity search and text similarity search.
You could also add a graph search for related elements for each element returned by the other search pipelines to increase the chance of having all the pieces of the puzzle in the context before using the LLM to solve the problem.
The other crucial thing to do would be to inspect dependencies (and their types, when possible) and maybe download documentation to offer tips that are accurate and not hallucinated.
Nowadays I get hallucinations for code generation as soon as things get hard, making LLM coding useful only for trivial code writing.
Analysing the code structure and dependencies would require plenty of work for each specific language, so it won't be a easy win like "just throwing RAG" - which is what the current players are doing to raise money - with mediocre results.
> Half-baked ideas for a better future.
Maybe the article is intentionally supposed to read like that?
What was missing from the 1990 Web? The original Web did not effectively support the one-paragraph idea. A reader who encountered a Web page containing only one paragraph would have been startled and wondered if the page had been left unfinished.
I don't think others necessarily quote-unquote "lost focus" on this problem, but it's not exactly easy to solve correctly, so in the meantime it's easier to create something with the next best approximation.
https://stallman.org/articles/made-for-you.html
Dead Comment