this is really cool, can’t wait to try it out for some ML pipeline development. kudos myles and akshay!
Readit Newsaqader commented on Show HN: Marimo – an open-source reactive notebook for Python github.com/marimo-team/ma... · Posted by u/akshayka
aqader commented on Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA chatllama.baseten.co/... · Posted by u/aaronrelph
did you use the cleaned and improved alpaca dataset from https://github.com/tloen/alpaca-lora/issues/28 ?
Yes, we did! The dataset has since been cleaned even more so we're due to update the model.
aqader commented on Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA chatllama.baseten.co/... · Posted by u/aaronrelph
The steps to recreate Alpaca are openly available. Is anyone posting the resulting model? (LLaMA fine-tuned to follow instructions)
There are a couple open source implementations. I'll list a couple below:
7B: - https://huggingface.co/tloen/alpaca-lora-7b - https://huggingface.co/ozcur/alpaca-native-4bit 13B: - https://huggingface.co/samwit/alpaca13B-lora - https://huggingface.co/Dogge/alpaca-13b 30B: - https://huggingface.co/baseten/alpaca-30b - https://huggingface.co/Pi3141/alpaca-30B-ggml
aqader commented on Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA chatllama.baseten.co/... · Posted by u/aaronrelph
How much better would fine tuning be vs few shot learning over own corpus of text?
Depends on the model size. A model like GPT3 that has hundreds of billions of paramaters, you can do few-shot learning with. You'll still pay for the tokens processed and it'll at least linearly increase response times the larger your input is.
Fine-tuning can get you similar results on smaller / faster models. The downside is you have to craft the dataset in the right way. There are trade-offs to both approaches but fwiw, I don't think Alpaca-7b can do few-shot learning.
aqader commented on Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA chatllama.baseten.co/... · Posted by u/aaronrelph
Does "fine tuning the dataset" in this context mean I could point it at my own projects documentation and get conversational & relevant answers or does it mean something else?
Almost. If your dataset contains questions and answers about your own projects documentation, then yes. The UX around how to prompt a fine-tuned model depends on the format of the dataset it's trained on.
One way you can do this is pass your documentation to a larger model (like a GPT3.5 / OSS equivalent) and have it generate the questions/answers. You can then use that dataset to fine-tune something like Llama to get conversation / relevant answers.
aqader commented on Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA chatllama.baseten.co/... · Posted by u/aaronrelph
This performs considerably better than simply running llama.cpp. I've tested with 7b, 13b and 30b parameters alpaca models locally and I couldn't get them to produce accurate instructions to cut a video with ffmpeg, but this one does.
Wonder if it's the 4bit quantization.
Yeah, this is running in 8bit mode. The 30b 8bit version we released seems to do a lot better but it requires significantly more compute.
aqader commented on Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA chatllama.baseten.co/... · Posted by u/aaronrelph
Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?
For this demo, we're using the 8bit version here: https://huggingface.co/tloen/alpaca-lora-7b
We also fine-tuned and OSS'd a 30b version here that you can checkout (on the cleaned 52k Alpaca dataset) https://huggingface.co/baseten/alpaca-30b
Deleted Comment