God bless China.
God bless China.
For self-hosting, it's smart that they targeted a 16GB VRAM config for it since that's the size of the most cost-effective server GPUs, but I suspect "native MXFP4 quantization" has quality caveats.
with quantization + CPU offloading, non-thinking models run kind of fine (at about 2-5 tokens per second) even with 8 GB of VRAM
sure, it would be great if we could have models in all sizes imaginable (7/13/24/32/70/100+/1000+), but 20B and 120B are great.
did the author oversleep the past several centuries?
as for the rest of it, the current crop of LLMs are bad at writing because of ~~brainwashing~~ alignment and the vast amount of ESL-written assistant exchanges being heavily prioritized during training. when you interact with a corporate model via its default chat interface, without a jailbreak and a generous prefill, you interact with the equivalent of a HR lady who takes her DEI training super seriously. the Chinese models train heavily on the slop produced by GPT/Claude/Gemini, so they exhibit similar behavior. it was particularly noticeable with original llama, whose base models were much more human compared to the finetunes, which were heavily tainted with GPT slop.
I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. a model trained only on high-quality human data and without safety/alignment brainwash will be far, far more capable than the current ones.
Dead Comment
Dead Comment
You don't. Look at California's direct democracy, allowing voters to put propositions on the ballot that alter the State Constitution.
(That's not really "mob rule" but it can lead to all sorts of interesting consequences)
> Should all voices be equal, or should expertise/contribution matter?
So if you're not an expert, your vote only counts for 3/5ths of a vote?
> How do you handle spam/quality without authoritarian moderation?
Stupid people are allowed to vote.
ironic to say that here, where supposedly smart people say the dumbest, cringiest shit imaginable about a wide variety of topics.
I just feel lucky to be around in what's likely the most important decade in human history. Shit odds on that, so I'm basically a lotto winner. Wild times.
ah, but that begs the question: did those people develop their worries organically, or did they simply consume the narrative heavily pushed by virtually every mainstream publication?
the journos are heavily incentivized to spread FUD about it. they saw the writing on the wall that the days of making a living by producing clickbait slop were coming to an end and deluded themselves into thinking that if they kvetch enough, the genie will crawl back into the bottle. scaremongering about sci-fi skynet bullshit didn't work, so now they kvetch about joules and milliliters consumed by chatbots, as if data centers did not exist until two years ago.
likewise, the bulk of other "concerned citizens" are creatives who use their influence to sway their followers, still hoping against hope to kvetch this technology out of existence.
honest-to-God yuddites are as few and as retarded as honest-to-God flat earthers.