Readit News logoReadit News
Hugsun commented on Structured Output with LangChain and Llamafile   blog.brakmic.com/structur... · Posted by u/brakmic
Hugsun · 2 months ago
The version of llama.cpp that Llamafile uses supports structured outputs. Don't waste your time with bloat like langchain.

Think about why langchain has dozens of adapters that are all targeting services that describe themselves as OAI compatible, Llamafile included.

I'd bet you could point some of them at Llamafile and get structured outputs.

Note that they can be made 100% reliable when done properly. They're not done properly in this article.

Hugsun commented on Structured Output with LangChain and Llamafile   blog.brakmic.com/structur... · Posted by u/brakmic
zingababba · 2 months ago
What should be used instead?
Hugsun · 2 months ago
I gave up after it didn't let me see the prompt that went into the LLM, without using their proprietary service. I'd recommend just using the API directly. They're very simple. There might be some simpler wrapper library if you want all the providers and can't be bothered to implement the support for each. Vercel's ai-sdk seems decent for JS.
Hugsun commented on Major sugar substitute found to impair brain blood vessel cell function   medicalxpress.com/news/20... · Posted by u/wglb
hedora · 3 months ago
Studies have shown artificial (and non-nutritional organic) sweeteners are much worse than sugar for decades.

For instance, they disrupt your metabolism, so equivalently sweet amounts of sweeteners cause more weight gain than sugar. (Due to increased hunger vs. eating nothing, decreased metabolism and decreased calorie burn.)

The study in the article isn’t surprising at all. Links between nutrisweet and migraine headaches have been well understood for a long time. It’s not surprising other similar chemicals have similar negative side effects.

There’s no valid reason to use artificial sweeteners (other than diabetes, but even then, gaining weight from the sweeteners is a problem if the diabetes is weight related.)

Hugsun · 3 months ago
I was under the impression that this is not the case. Aspartame has been studied a lot and not found to be harmful.
Hugsun commented on AGI is not multimodal   thegradient.pub/agi-is-no... · Posted by u/danielmorozoff
patrickscoleman · 3 months ago
It feels like some of the comments are responding to the title, not the contents of the article.

Maybe a more descriptive but longer title would be: AGI will work with multimodal inputs and outputs embedded in a physical environment rather than a frankenstein combination of single-modal models (what today is called multimodal) and throwing more computational resources at the problem (scale maximalism) will be improved with thoughtful theoretical approaches to data and training.

Hugsun · 3 months ago
I discovered that this is very common when posting a long article about LLM reasoning. Half the comments spoke of the exact things in the article as if they were original ideas.
Hugsun commented on Adventures in Symbolic Algebra with Model Context Protocol   stephendiehl.com/posts/co... · Posted by u/freediver
Hugsun · 3 months ago
I was very pleased to discover that Mistral's Le Chat has inbuilt support for python code execution and sympy is importable.

It will regularly use it and reliably when asked to.

Hugsun commented on Ollama violating llama.cpp license for over a year   github.com/ollama/ollama/... · Posted by u/Jabrov
Koshima · 3 months ago
I think it’s fair to push for clear attribution in these cases, but it’s also important to remember that the MIT license is intentionally permissive. It was designed to make sharing code easy without too many hoops. If Ollama is genuinely trying to be part of the open-source community, a little transparency and acknowledgment can avoid a lot of bad blood.
Hugsun · 3 months ago
Consensus seems to be forming around the fact that Ollama is not genuinely trying to be part of the open-source community.
Hugsun commented on Ollama's new engine for multimodal models   ollama.com/blog/multimoda... · Posted by u/LorenDB
andy_xor_andrew · 3 months ago
They are talking a lot about this new engine - I'd love to see details on how it's actually implemented. Given llama.cpp is a herculean feat, if you are going to claim to have some replacement for it, an example of how you did it would be good!

Based on this part:

> We set out to support a new engine that makes multimodal models first-class citizens, and getting Ollama’s partners to contribute more directly the community - the GGML tensor library.

And from clicking through a github link they had:

https://github.com/ollama/ollama/blob/main/model/models/gemm...

My takeaway is, the GGML library (the thing that is the backbone for llama.cpp) must expose some FFI (foreign function interface) that can be invoked from Go, so in the ollama Go code, they can write their own implementations of model behavior (like Gemma 3) that just calls into the GGML magic. I think I have that right? I would have expected a detail like that to be front and center in the blog post.

Hugsun · 3 months ago
Ollama are known for their lack of transparency, poor attribution and anti-user decisions.

I was surprised to see the amount of attribution in this post. They've been catching quite a bit of flack for this so they might be adjusting.

Hugsun commented on Gemini 2.5 Flash   developers.googleblog.com... · Posted by u/meetpateltech
neuah · 4 months ago
Exactly my experience as well. Started out loving it but it almost moves too fast - building in functionality that i might want eventually but isn't yet appropriate for where the project is in terms of testing, or is just in completely the wrong place in the architecture. I try to give very direct and specific prompts but it still has the tendency to overreach. Of course it's likely that with more use i will learn better how to rein it in.
Hugsun · 4 months ago
I've experienced this a lot as well. I also just yesterday had an interesting argument with claude.

It put an expensive API call inside a useEffect hook. I wanted the call elsewhere and it fought me on it pretty aggressively. Instead of removing the call, it started changing comments and function names to say that the call was just loading already fetched data from a cache (which was not true). I could not find a way to tell it to remove that API call from the useEffect hook, It just wrote more and more motivated excuses in the surrounding comments. It would have been very funny if it weren't so expensive.

Hugsun commented on AI 2027   ai-2027.com/... · Posted by u/Tenoke
Vegenoid · 5 months ago
I don't mean that the primary (or only) way that it interacts with a human can't be just text. Right now, the only way it interacts with anything is by generating a stream of tokens. To make any API calls, to use any tool, to make any query for knowledge, it is predicting tokens in the same way as it does when a human asks it a question. There may need to be other subsystems that the LLM subsystem interfaces with to make a more complete intelligence that can internally represent reality and fully utilize abstraction and relations.
Hugsun · 4 months ago
I have not yet found any compelling evidence that suggests that there are limits to the maximum intelligence of a next token predictor.

Models can be trained to generate tokens with many different meanings, including visual, auditory, textual, and locomotive. Those alone seem sufficient to emulate a human to me.

It would certainly be cool to integrate some subsystems like a symbolic reasoner or calculator or something, but the bitter lesson tells us that we'd be better off just waiting for advancements in computing power.

Hugsun commented on Qwen2.5-VL-32B: Smarter and Lighter   qwenlm.github.io/blog/qwe... · Posted by u/tosh
Alifatisk · 5 months ago
Ever since I switched to Qwen as my go to, it's been a bliss. They have a model for many (if not all) cases. No more daily quota! And you get to use their massive context window (1M tokens).
Hugsun · 5 months ago
How are you using them? Who is enforcing the daily quota?

u/Hugsun

KarmaCake day677June 9, 2021
About
https://arnaldur.be
View Original