Most engineers focus on the algorithm or the model in the AI space. Doing so, they forget the most essential and time-consuming part: ensuring your project is practical and accessible to your end-user.
This post looks at two real-life use cases of how to build AI projects focused on the end-user.
Readit NewsAlyx1337 commented on Bringing the end-user into the AI picture taipy.io/posts/bringing-t... · Posted by u/Alyx1337
Yeah github.com/KoljaB is quite a collection of stuff! I agree.
It all seems your vision of JARVIS, which I share completely but haven't accomplished what you have, again excellent work and thank you for sharing, is very attainable. Probably combining your work along with KoljaB is very promising.
Thank you very much!
Wonderful hack, the overall response latency is the only thing that hurts the UX, if you can get the response time down would be epic. Nice work.
Thanks! There are ways to shave off the latency: hosting locally, using quantized/smaller models, streaming data instead of doing the tasks sequentially
Somewhat amusing to consider that the (in-character) Marvel Cinematic Universe JARVIS could have been an LLM!
And of course Ultron is an asshole, it was trained on input from Tony Stark!
Back in 2008/9 I wondered just what would be required to run JARVIS, something you could converse with naturally, would understand what you meant, and be able to take care of complex mechanical tasks. The Iron Man suits have always been mostly Do-What-I-Mean (DWIM) managed by JARVIS or other AI agents, and now all of that seems to be attainable.
It's going to be an interesting time discovering just how well a human and AI agent can work together. I could see a military personal spotter, keeping track of enemy combatants, managing larger awareness of the battlefield, etc. I wonder how much a soldier could safely offload?
Exactly my thought, I was like "Jarvis has got to be just a 2030 version of an LLM".
Yeah I actually considered making a spotter AI using computer vision in a game like ARMA 3 or Squad but kind of difficult. I made a spotter for ground vehicles on aerial imagery using YOLOv5 here: https://github.com/AlexandreSajus/Military-Vehicles-Image-Re...
There's a French defense company, Preligens, that actually does this currently
Here I was thinking about putting something like this in my home, and jokingly calling it Jarvis.
This will be a great starting point, shame you can't choose the models you want to talk to (ie use local models instead of OpenAI), but great nonetheless !
That was exactly my thought haha, I want Jarvis at home. You could easily modify my code to run a local LLM instead
We use this exact stack at work (OpenAI, ElevenLabs, Deepgram) for some exploratory use cases. The key issue we have now is latency with the LLM. Deepgram and Elevanlabs work brilliantly!
Great! What do you guys have in mind in terms of products using these tools. Yeah unfortunately it's hard to shave on latency.
I built this kind of thing for GPT-3 way back and then repurposed for 3.5 when I got API access to that. Though I used Whisper. I was hoping this would have wake word handling because that was what I struggled with but it appears that it just starts listening when you click a button or something.
Yeah I had the same issue so I used (stole) this answer on StackOverflow:
https://stackoverflow.com/questions/46734345/python-record-o...
Basically there's a library that records until it detects a silence
"Jarvis" is a trademark of Marvel, so that name will definitely not work. https://trademarks.justia.com/862/94/jarvis-86294162.html
Uh oh I hope I'm not in trouble