Ask HN: What have you built with LLMs?

My "stack" is just Apple Shortcuts making HTTP POST API calls to OpenAI, which does stuff in MacOS via BetterTouchTool. I trigger each by hotkey or typing a few letter into Spotlight (with Alfred). One transcribes and summarizes whatever youtube URL is highlighted. One does grammar and style correction of whatever is highlighted (and replaces it). One simply replaces the Dictate key with OpenAI Whisper but otherwise works exactly the same as voice typing. It's just way more accurate. One replaces the magnifying glass key to have a voice conversation with ChatGPT (using Microsoft voice synthesis). The built in prompt keeps it's answers short and conversational. It's like asking Siri something, but much better. One simply reduces the highlighted text by ~50% by rewriting it shorter, for when I have typed too much. One gives the key points of whatever article is in the foreground tab, so I know what I'm about to read. One outputs purely code, for example I use my voice to say "javascript alert saying blah" and alert("blah"); will appear at my cursor. Of course, it's usually more complex boilerplate stuff, but it helps speed up my coding. Every time I find myself using an LLM repeatedly for something, I make it into a little Apple Shortcut to streamline it into my workflow, as if it were a built in MacOS feature.

clapslock · 2 years ago

Could you please share the prompt for the grammar and style correction shortcut? I've just started using it for the same purpose, but I haven't been able to find a prompt that yields consistent results. Sometimes, ChatGPT completely changes the style of my text.

geor9e · 2 years ago

I use role: system, temperature 0.7, prompt: Fix the spelling, grammar, punctuation, order, and sentence structure. It's does sometimes change the style too much, but not often enough to annoy me into fiddling with it.

dostick · 2 years ago

Have you tried Raycast? It has all the AI scripts you mentioned and many more. And many things done better, like showing diff before inserting grammar-corrected text.

rsanek · 2 years ago

Looks expensive at $20/month if you want GPT-4

mpalmer · 2 years ago

I have looking for a way to do "push to record audio" (instead of Mac's dictate) for ages, thanks for the push to look at Shortcuts!

Are you using the "Record Audio" action or something else? Ideally the shortcut would stop listening after a pause like the native Dictate feature does it. At a minimum Record Audio seems to require hitting spacebar to stop - not great but not terrible.

geor9e · 2 years ago

Yes, "Record audio". BetterTouchTool launches the shortcut on keydown, then clicks the Stop button on keyup.

isenhaard · 2 years ago

I really love that, super good ideas. I also generally love to create workflow optimizations. Will probably create some of your stuff for myself (I especially like the dictation replacement, could be super useful to me).

Wondering: How big is you monthly OpenAI bill when using all these tools? Only a few $$$, or is it higher?

geor9e · 2 years ago

Only a few dollars a month

tomcam · 2 years ago

You beast! They all sound awesome!

lemming · 2 years ago

These sound amazing, if you don’t mind sharing somehow, I’d love to see how these work. I’ve never used shortcuts, but I think you’ve inspired me to try.

geor9e · 2 years ago

I put couple screenshots here https://news.ycombinator.com/item?id=39283515 to show the API call part. The rest is just whatever you want it to feed into in Shortcuts. For launching a shortcut on keydown, and clicking out of it on keyup, I used BetterTouchTool like this https://i.imgur.com/sqJ7cOc.png

alizbazar · 2 years ago

Would love to learn more details about your setup! I use BetterTouchTool, too and wonder how to make use of it + shortcuts + the API

geor9e · 2 years ago

You might want BetterTouchTool too, since it adds things like cut and paste, to Apple Shortcuts All Actions list. I also use it as the initial trigger usually, to make a hotkey launch a Shortcut. Whisper looks like this https://i.imgur.com/ApAwf2E.png and ChatGPT looks like this https://i.imgur.com/g9f9ZDH.png .

dostick · 2 years ago

Nobody heard of Raycast?

nbbaier · 2 years ago

What are the settings and prompt you use for the youtube one?

I built an Interactive Resume AI chatbot where anyone can ask questions about my experience and skills: https://www.jon-olson.com/resume_ai/

The backend is a Python FastAPI that uses ChromaDB to store my resume and Q&A pairs, OpenAI, and Airtable to log requests and responses. The UI is Sveltekit.

I'm currently building a different tool and will apply some learnings to my Interactive Resume AI. Instead of Airtable, I am going to use LangSmith for observability.

I started writing and my Substack articles are also linked to via my website. I'm currently working on applying sentence window retrieval and that article will be out shortly. This is part of a #buildinpublic effort to help build my brand as well.

I've been unemployed since Sept as a Senior Software Engineer. The market is tough so I'm focusing on the above to help get employment or a contract.

kredar · 2 years ago

Nicely done Jon. I really like the UI - I wanted to have buttons as well but didn't find how to do it in Streamlit.

I also built Resume Chatbot but using slightly different stack: Python, Langchain, Faiss as vector store, MongoDB to store chat logs and Streamlit for UI. Here is a link: https://www.artkreimer.com/resume/ or you can try it on streamlit https://art-career-bot.streamlit.app/. Code is available here https://github.com/kredar/data_analytics/tree/master/career_.... Great thread and I got some ideas for my next project. Thanks a lot everyone.

FlyingWallrus · 2 years ago

Hey Jon - I'm Jon too - working on an AI startup in the recruiting space and will be hiring remotely. I like your resume ap and can definitely see utility in it. I'd be happy to connect and maybe see if there is a way we could work together! I'll find you on LinkedIn and send you an invite request.

dimaboyko42 · 2 years ago

Resume AI is cool, really nicely done, mate !

jonnycoder · 2 years ago

Thanks. This is the first real test apart from a couple dozen test users. I've received hundreds of prompts in the past 24 hours from Hacker News users, mainly from my suggested questions buttons.

The actual questions I got did not provide a response that is to my liking. Most of that is due in part because I'm using gpt3.5 since gpt4-turbo is a lot more expensive, and I can learn a lot more by using an inferior LLM.

For example, using an llm router to analyze the query and route to a specific helper function with a specific prompt would be helpful. Sometimes a user starts with a greeting but the response is a pre-written "Sorry an answer cannot be found". Questions are typically grouped into a category such as skills, experience, project, personal (ie: where are you located), preferences (ie: favorite language), and general interview questions (ie: why should I hire you). Questions in categories can be better answered by using a different prompt and/or RAG technique.

wayfareryouth · 2 years ago

I’m sorry it’s been tough. The job market for seniors and leads is still quite strong in Australia if you can move here

jonnycoder · 2 years ago

Thanks. I'm starting to realize that part of the problem is job search and matching.

I was contacted by a company recruiter for a small healthcare SaaS in California and had 3 interviews recently. When I looked up the job, only 7 people had applied in 2 weeks on LinkedIn. They are a very real company with very real people, but their job post is not getting seen (it's not a promoted post).

My next AI project will be to scrape LinkedIn jobs, analyze it for repost/promoted behavior, group it by consulting/headhunters vs company job post, eliminate duplicates, and filter based on my skillset and hard-no qualities (such as can't work if I live in California, must be in EST but I'm in PST timezone, requires Java experience, etc).

duckkg5 · 2 years ago

I don't like selling. I wanted a way to practice cold calling in a realistic way. I set up a phone number you can call and talk to an AI that simulates sales calls.

I ended up using it for more general purpose things because being able to have a hands-free phone call with an AI turned out to be pretty useful.

It's offline now, but here's the code with all the stack and deployment info: https://github.com/kevingduck/ChatGPT-phone/

Edit: forgot to mention this was all running off a $35 raspberry pi.

dsco · 2 years ago

So the AI tries to sell to you, or you try to sell to the AI? This sounds very intriguing but I can tell by your README that you're an engineer and not a sales guy - there are no distinct value propositions.

But it sounds damn creative as a project.

The AI answers the call and acts as a potential customer. They take on personas to simulate behaviors like difficult or reluctant customers. You then do your pitch, handle objections, etc. At the end you get a transcript that's 'graded' to show you where you could improve your sales approach.

And you're right, I'm not a sales guy. This project is for people like me who want a risk-free place to learn the basics of sales so that when I do talk to an actual human, I won't panic and freeze up like I always do.

satvikpendem · 2 years ago

Now you can turn this into an AI sales cold caller based on the data you could collect from how the AI reacts to your selling. That is to say, the entire system becomes a generative adversarial network.

Dead Comment

SpaceL10n · 2 years ago

Like exposure therapy for people afraid of sales. Very nice idea.

cushpush · 2 years ago

nice term this, exposure therapy"

Yes exactly!

alentred · 2 years ago

I like the idea very much! Using an LLM as a "sparring partner" for training in various areas. LLMs tend to hallucinate, so I find it harder to use them reliably in the context of decision making. Training however is a nice idea indeed: mistakes are not as critical, just as in real life any peer can make a mistake.

VoodooJuJu · 2 years ago

Very cool, sounds like a saleable product. I feel like there's already half a dozen landing pages with people trying to sell what you just made in the 18 hours since you've shared it here. That should however be a red flag to those same people, a demonstration in just how easily commoditized LLM products are.

jtolster · 2 years ago

Are you finding response time to be an issue? I can imagine some very long pauses might kill the flow of conversation.

It's not perfect, but it's tolerable, and not unlike some real-world calls where there's a slight delay. There are some "Hmm ..." and "well ..." scripted in as well to make it feels natural if there is a long response.

elicash · 2 years ago

To that point, I would love to hear an audio file of it in action since I see from GitHub the phone number is down.

qup · 2 years ago

That's cool. Thanks for sharing the source. What else has it been good at for you?

The cold call sales part can be replaced to suit any need. I had another version that was just a generic AI (no sales stuff). I found myself on walks frequently ringing up the chatbot ("Hey siri, call ChatGPT") and just asking it whatever is on my mind. "Tell me about Ghengis Khan" or "where's a good place to catch trout in north Georgia" or "how do I make baked ziti". Makes the walks go by super quickly.

Deleted Comment

craigdalton · 2 years ago

Would you be willing provide a live demo (via web interface) - as a preludebto providing a similar training bot as a consultant?

DrNosferatu · 2 years ago

Now do it for dating practice - great for nerds ;)

xtracto · 2 years ago

I helped "writing" a cookbook from my grandmother's recipes. For her 100th birthday, my dad rescued more than 250+ pages of recipes that my Grandma had collected over the years. Some were written in typing machine, others written by hand by her. So, my dad scanned (pictured) all the typed recipes, and "dictated" all the handwritten.

For the dictated recipes, I told him to dictate just "flat" the words and numbers. So that I had paragraphs of recipes.

For the scanned recipes, I used Google OCR (I found out it was the best one quality wise).

For both sets of recipes, I then used GPT4 to "format" the unformatted recipes into well formatted Markdown. It successfully fixed typos and bad OCR from Google.

We then pasted all that well formatted text into a big Google Docs, and added images. Using OpenAI image generation I generated images for each of the 250+ recipes. For some of them I had to manually curate it, given that some of the recipes are for typical Mexican food: For example there's a (delicious) recipe called "PibiPollo" that for the unitiated it may look like a stew, so I had to tell something like "large corn tamale with thick hard crust".

In the end, the book was pretty nice! We distributed digital copies within the family and everybody was amazed :) . I loved spending time doing that.

syntaxing · 2 years ago

This is absolutely awesome. I really want to do the same for my mom’s recipe before it’s too late. Though I wonder what would have happened if you went for GPT-V or LLaVa and the like. I have a hunch you might have been able to skip over the OCR part and straight from picture to markdown? Would be awesome if you can try and compare!

addandsubtract · 2 years ago

Would you mind sharing the cookbook or excerpts from it? I'd love to see it.

I cannot share the full book because a) I don't own the copyright and b) My dad (who ultimately owns it) still has plans to put sell it. And I feel it still requires some editing. But I can share a couple of sample pages:

https://drive.google.com/file/d/1OGE-zfNHHDnALbhgmf3lykBjcSg...

It is in Spanish though.

wonderfuly · 2 years ago

That's great!

abeisgreat · 2 years ago

I've done a handful of fun hardware + LLM projects...

* I built a real life Pokedex to recognize Pokemon [video] https://www.youtube.com/watch?v=wVcerPofkE0

* I used ChatGPT to filter nice comments and print them in my office [video] https://www.youtube.com/watch?v=AonMzGUN9gQ

* I built a general purpose chat assistant into an old intercom [video] https://www.youtube.com/watch?v=-zDdpeTdv84

Again, nothing terribly useful, but all fun.

63 · 2 years ago

Oh hey I just watched that pokedex video. It was so impressive! Deserves way more attention

moralestapia · 2 years ago

Indeed! Such a beautiful project!

aerosteelzero · 2 years ago

Great job on that Pokedex and the video entirely. So freakin cool!

bluecoconut · 2 years ago

We've made a lot of data tooling things based on LLMs, and are in the process of rebranding and launching our main product.

1. sketch (in notebook, ai for pandas) https://github.com/approximatelabs/sketch

2. datadm (open source, "chat with data", with support for the open source LLMs (https://github.com/approximatelabs/datadm)

3. Our main product: julyp. https://julyp.com/ (currently under very active rebrand and cleanup) -- but a "chat with data" style app, with a lot of specialized features. I'm also streaming me using it (and sometimes building it) every weekday on twitch to solve misc data problems (https://www.twitch.tv/bluecoconut)

For your next question, about the stack and deploy: We're using all sorts of different stacks and tooling. We made our own tooling at one point (https://github.com/approximatelabs/lambdaprompt/), but have more recently switched to just using the raw requests ourselves and writing out the logic ourselves in the product. For our main product, the code just lives in our next app, and deploys on vercel.

jefc1111 · 2 years ago

Having a play with datadm. It's really good and intuitive to use - good job! I'm getting errors now, but was having a lot of fun before.

lefstathiou · 2 years ago

This is cool. Thank you for sharing.

andher · 2 years ago

I've built several things! These include bots for code generation that you can tag onto issues, q&a on text etc.

The thing I'm working on now is AI mock interviewing. It's basically scratching my own itch, since I hate leetcode prep, and have found I can learn better through interaction. To paste a blurb from an earlier comment of mine:

I'm building https://comp.lol. It's AI powered mock coding interviews, FAANG style. Looking for alpha testers when I release, sign up if you wanna try it out or just wanna try some mock coding. If its slow to load, sorry, everything runs on free tiers right now.

I really dislike doing leetcode prep, and I can't intuitively understand the solutions by just reading them. I've found the best way for me to learn is to seriously try the problem (timed, interview like conditions), and be able to 'discuss' with the interviewer without just jumping to reading the solution. Been using and building this as an experiment to try prepping in a manner I like.

It's not a replacement for real mock interviews - I think those are still the best, but they're expensive and time consuming. I'm hoping to get 80% of the benefit in an easier package.

I just put a waitlist in case anyone wants to try it out and give me feedback when I get it out

Gonna apologize in advance about the copywriting. Was more messing around for my own amusement, will probably change later

Very cool, I signed up. I agree that practicing a coding interview is better under pressure. It's a much difference skill to solve a coding problem both under time pressure and pressure to speak your thoughts to entertain the interviewer. Only practice can help improve that skill.

Yeah, I agree, the scenario is totally different in an actual pressure situation, I've fumbled so many easy questions. I don't necessarily like leetcode style questions as the standard for the industry for interviewing, but its still a reality and, from what I'm noticing, becoming more difficult in terms of expectations.

Thanks for signing up, will send out an email once its ready to take for a spin!

hazard · 2 years ago

A Twitter filter to take back control of your social media feed from recommendation engines. Put in natural language instructions like "Only show tweets about machine learning, artificial intelligence, and large language models. Hide everything else" and it will filter out all the tweets that you tell it to.

Runs on a local LLM, because even using GPT3 costs would have added up quickly.

Currently requires CUDA and uses a 10.7B model but if anyone wants to try a smaller one and report results let me know on github and I can give some help.

https://github.com/thomasj02/AiFilter

eurekin · 2 years ago

That could actually be a universal ad-whacker for similarily stubborn sites (reddit)

beginning_end · 2 years ago

I've been thinking the same thing. It'll be interesting to see if we end up with prompt-injecting ads

jerpint · 2 years ago

I didn’t know you could interact with pages like that so easily with Chrome extensions