Show HN: Transform your codebase into a single Markdown doc for feeding into AI

I use the following for feeding into AI

   find . -print -exec cat {} \; -exec echo \;

Which will return for each file (and subfolders) the filename and then the content of the file.

Then `| pbcopy` to copy to clipboard and paste it into ChatGPT or similar.

singpolyma3 · 6 months ago

I guess this only works for very small codebase?

OsrsNeedsf2P · 6 months ago

Correct, but it's the same as what OP shared.

You should use Aider/Cursor for proper indexing/intelligent codebase referencing

usagisushi · 6 months ago

Or, for a lazier approach:

    $ head -10000 *
    ==> package.json <==
    {
      "name": ...
      ...
    ==> tsconfig.json <==
    {
       "extends": ...
      ...

    $ head -10000 * | llm -s "generate a patch to switch this project to esm"

mohsen1 · 6 months ago

yek makes it a bit quicker and you can do all your piping with it:

https://github.com/bodo-run/yek

DrPhish · 6 months ago

That's very nice and compact. I do the same with a short bash script, but wrap each file in triple-backticks and attempt to put the correct language label on each eg:

Filename: demo.py

```python

   ...python code here...

```

genewitch · 6 months ago

Seconded because just having something autowrapped like that and putting the clipboard would save me time: release the snyder cut, er, bash script!

mbonnet · 6 months ago

Mind sharing the script?

Deleted Comment

How does this compare to / differ from https://github.com/yamadashy/repomix ?

tesserato · 6 months ago

Some advantages of CodeWeaver are that it is compiled, so it might be faster; you can grab a compatible executable from the releases section instead of using `go install` so, no dependencies. You can manually specify what to exclude via a comma-separated list of regular expressions so it might be more flexible. I never used Repomix so, those assumptions might not hold. On the other hand, remix seems to be awfully more complete, a full-fledged solution to convert source code to monolithic representations. I wrote CodeWeaver because I only needed something that worked and, occasionally, I could trust to keep sensitive data away from sketchy LLMs (And wasn't aware of other solutions).

apineda · 6 months ago

There is also https://github.com/regenrek/codefetch which I personally like

akoculu · 6 months ago

or https://github.com/azer/llmcat

imdsm · 6 months ago

I simply have a bash script called printall which takes in some args, and outputs markdown codeblocks with filenames and a tree. One of hundreds of scripts built up over the years.

ycombinatornews · 6 months ago

My question exactly. Repomix seems to be tested util for something like that.

ActVen · 6 months ago

Same question here. I have found repomix to get the job done really well.

SillyUsername · 6 months ago

or https://github.com/bodo-run/yek

Terretta · 6 months ago

CodeWeavers is a software company that focuses on Wine development and sells a proprietary version of Wine called CrossOver for running Windows applications on macOS, ChromeOS and Linux.

https://en.wikipedia.org/wiki/CodeWeavers

Trademark is active. It's an Ⓡ not just a ™, registered not just trademarked. To keep it, they have to demonstrate they defend it.

https://www.trademarkia.com/codeweavers-76546826

While this project drops the final "s", you don't get to launch an OS called "Window". The test is a fuzzy match based on likelihood of confusion.

jychang · 6 months ago

Yeah, I was thinking "what does the Wine guys have to do with this?"

This project is definitely going to get C&D'd.

williamcotton · 6 months ago

Do you think they would actually litigate? They seem like different products serving entirely different markets so I am not sure that the trademark infringement claim is very defensible. And how do they prove damages?

pstuart · 6 months ago

CodeLoom could work instead

crisbal_ · 6 months ago

beklein · 6 months ago

Tip: If you ever need to do this on a public GitHub repository you can use "gitingest".

This will open a website that creates a copy of all the file contents of the repo (code, docs, ...) It's a great tool to use when using new/obscure code with LLMs in my opinion.

The UX is so just easy and great, change the URL from <https://github.com/user_name/repo_name> to <https://gitingest.com/user_name/repo_name>

//edit: fixed URLs

mkagenius · 6 months ago

I copied the UX to my https://gitpodcast.com (creates podcast on a github repo, same replace `hub` with `podcast`)

pratyahava · 6 months ago

i am very impressed by gitpodcast, i just listened to one podcast and first of all i am pleased with the idea, the voices are also pleasant to listen to. thanks for sharing!

reddalo · 6 months ago

Unfortunate naming, given that CodeWeavers is already a company making a Windows "emulator" for Linux and macOS. [1]

[1] https://www.codeweavers.com/

Arch-TK · 6 months ago

CodeWeavers are actually making wine, not just some "emulator". They then distribute this along with some QOL tools as a commercial product called CrossOver.

lgas · 6 months ago

All names are taken. There's no need to point this out every time.

anamexis · 6 months ago

Not all names are registered trademarks for software.

Rexxar · 6 months ago

Some are more confusing than others.

gloosx · 6 months ago

Huewoblfan is not taken! Noiewoidc is free. XIONqlic – totally available, can mean a range of things! Ciohupoij – a bit of asian flavour but still a valid free name.

tecleandor · 6 months ago

As a note, CodeWeaver might be a confusing name, as CodeWeavers (the Wine development company) exists since 1996... ( https://en.wikipedia.org/wiki/CodeWeavers )

teekert · 6 months ago

My first though: Is this somehow using Wine?

It's not mentioned on the page but is it using [0] in the background? Edit -> It's a Go program so I guess not.

[0] https://github.com/microsoft/markitdown

pmx · 6 months ago

retropragma · 6 months ago

I really want a tool like this that can extract a function and its dependency graph (to a certain depth maybe, and/or exclude node_modules).

I wrote this library [1] and hope to add the fine-grained "reference resolution" utility to it at some point, which could make implementing such a tool a lot simpler.

[1]: https://github.com/aleclarson/ts-module-graph

therealmarv · 6 months ago

I use aider /copy-context command for that

https://aider.chat/docs/usage/copypaste.html

and with /paste you can apply the changes.

anotherpaulg · 6 months ago

Thanks for letting folks know about aider's /copy-context command.

To add some more detail, aider has a mode/UX that is optimized for "copy and paste" coding with LLM web chats. The "big brain" LLM in the web chat does the hard work, and a cheap/local LLM works with aider to apply edits to your local files.

There's a little demo video in the link above that should give you the gist.