We are using over 50 billion LLM tokens for NLP/classification purposes per month. A mix of self hosted and cloud hosted models. But I have not attempted any fine tuning. Just prompt, (and perhaps more importantly) context “engineering”.
Readit Newsnkristoffersen commented on Ask HN: Has anyone deployed LLMs to production? · Posted by u/saaspirant
nkristoffersen commented on Show HN: FastGraphRAG – Better RAG using good old PageRank github.com/circlemind-ai/... · Posted by u/liukidar
What solutions are folks using to solve queries like "How many of these 1000 podcast transcripts have a positive view of Hillary Clinton"? Seems like you would need a way to map reduce and count? And some kind of agent assigner/router on top of it?
We do a lot of things with podcast and other audio media at https://listenalert.com
But in general we found the best course of action is simply label everything. Because our customers will want those answers and rag won’t really work at the scale of “all podcasts the last 6 months. What is the trend of sentiment Hillary Clinton and what about the top topics and entities mentioned nearby”. So we take a more “brute force” approach :-)
yes, definitely some skin deep similarities! We both transcribe podcasts, but what we do after that is very different ;-) (also, i'm a fan of his podcast).
There is also more generally Mention, Brand24, Meltwater, etc, in the media monitoring space. But all are generally weak in the audio media space.
When we stop looking at audio media like a newspaper article, things get more interesting.
I'm working on a podcast and audio media monitoring and research platform. Easily the most complex thing I've built. But still so much more to do to make it truly useful. Testing PR and public affairs use cases at the moment.
I used to work for a media monitoring company but was instantly struck by how old fashion everything was. And the total reliance on boolean searches meant only experts could find relevant information. This still appears to be the case for most players in the industry.
So I'm building a platform that finds what is important before you look for it. Novel entity linking, and sentiment analysis plus speaker tracking. It has come a long way from the proof of concept. Focused on audio media at the moment as it is the hardest to index compared to news articles in my opinion. And the hypothesis is audio media such as podcasts can contain so many juicy insights.
Next steps are converting pilot customers to paying customers, testing more markets (based in a tiny market now), raise a small pre-seed (bootstrapped at the moment), and quickly evolve the product based on feedback.
nkristoffersen commented on My MacBook Setup (the 2024 version) github.com/maoxiaoke/setu... · Posted by u/maoxiaoke
I only use a Mac for work through a company laptop but one thing I recently discovered is the world clock widget which is accessed through your clock.
You can show up to 4 different times besides your local time.
This is really handy to see the time in UTC and other timezones where some of your team mates might exist. This saves having to Google for timezone converters.
You can actually create multiple widgets of the same type. So as many timezones or cities you want to track you can just keep adding timezone widgets.
nkristoffersen commented on New embedding models and API updates openai.com/blog/new-embed... · Posted by u/Josely
3.5 has JSON mode as of the November release, but only in the November-dated model: gpt-3.5-turbo-1106.
I found it to reliably produce JSON correctly, but I've found 3.5 to be a poor performer at things like entity extraction and following directions compared to other fast models such as claude-instant (though that does not have function calling).
What kind of problems are you facing for entity extraction with 3.5? I am also currently working with 3.5 for entity extraction and entity linking. It is a fun pipeline but curious what issues you ran into?
You can't beat a mirror, it's just not possible especially in low light situations. Discplay screen based digital cameras will always be worse for many users. It's fine as an option but it's not killing the shutter just yet.
What makes you say that?
I can definitely argue that mirrorless is always better for most users! Thinner bodies, modern lens selection (so faster autofocus and wider aperture, etc), real-time image previews, better low-light preview (in both rear and viewfinder screens), video recording capabilities better match professional video cameras (by removing mirror complexity) which better matches the hybrid needs of the modern camera buyer.
nkristoffersen commented on Millions of active WebSockets with Node.js unetworkingab.medium.com/... · Posted by u/selvan
C1M is not that impressive anymore, these days one need to do C10M;)
Most of the work of accepting and holding concurrent TCP sockets done by the OS, not the language runtime. One can easily tune Linux kernel to 1M concurrent sockets.
The real issues: memory usage per concurrent socket (idle or active), and ability to do something useful with all these active connections, e.g. send pings every 30s, or broadcast a message to all of them.
I'm not sure NodeJS/C++ based system from this post will allow sending pings every 30s to 1M websockets, let alone to do something useful with them, beside some low-traffic or infrequent notifications (Of course one always need to perform realistic loadtests in order to answer these kinds of questions).
Erlang/Elixir/BEAM have a relatively large memory usage per active socket, but it allows doing something useful with them under an easy to use programming model (read: no callback hell).
isn't "callback hell" a bit old fashioned now? With promises and async/await etc.
nkristoffersen commented on Apple is struggling to build Mac Pro based on its own silicon tomshardware.com/news/app... · Posted by u/pulse7
Maybe it's a gross oversimplification but I thought that while the next iterations of the A and smaller M SoCs were designed in California, the bigger M chips are developed in Israel where the team is much younger.
I'm pretty sure its Austin Texas https://eu.statesman.com/story/news/2012/09/01/apple-buys-au...
It’s been a journey but getting close to launching our first version to pilot customers in August. We use an enormous amounts of AI tokens every month to extract data not possible with any traditional player in this media monitoring space. Benchmarking competitors, tracking impactful discussions, and receiving actionable brand insights.
If you are currently using one of the big media monitoring companies, I’d love to chat!
https://www.gossipinsights.com/en/top-companies/us/