As for Datastar, all the signal and state stuff seems to me like a step in the wrong direction.
As for Datastar, all the signal and state stuff seems to me like a step in the wrong direction.
None of this could be justified without the war on drugs. Plus we'd get rid of the prime motivator for a lot of the world's crime.
Dynamic typing is why I eventually switched. Haskell scratches the same itches that Clojure did, but the compiler and type system are immensely helpful, and keep saving me from tripping over my own feet.
Assuming that everybody has the luxury to have additional office hours available is a bit far from reality in my opinion.
I mean, if you can offer office hours for an open source project you probably are already so popular that you are able to work on it fulltime, right? And if you're not that popular, you cannot offer office hours due to your daytime job; as you would have to decrease the rest of your remaining free time that you probably need to sleep and eat.
Are you certain? I'd think most open source development is done by paid developers, on company time.
And people will no longer need a duplicate set of libraries on their machines, all the way down to the gfx drivers. New software, and old stuff with 64-bit support, should have a lot less compatibility issues.
Deleted Comment
Deleted Comment
As much as I'd love true competition in the GPU/CPU space, it doesn't exist. AMD's cards simply cannot compete with Nvidia for GPGPU type scenarios, and even in its basic capacity, often have known heat/perf issues. Now that may be worth it for now to make a statement against the telemetry, but what if (less if and more when IMO) AMD then adds driver telemetry? And then intel?
These domains (Chip manufacture/GPU driver writing) are so advanced at this point that I don't see how competition could reasonably disrupt an incumbent over anything less than a samsung-grade failure (and even then probably not), and I'm concerned about the long-term wherein the producers realize this and through a combination of boiling the frog slowly and leaving consumers no other choices put themselves in the position to have a "pragmatic monopoly" of free reign over our machines. (I've always wondered what would happen from an antitrust sense, if it's "we're the only producer not because we WANT to but because we're the only ones who CAN")
We've certainly seen it happening with OSes, as well as some attempts from PC oems, I've always unfortunately thought it was just a matter of time until the more irreplaceable components got into the game too and I'd love some creative thoughts to actually stop the trend and not stand in its way, because I'm not sure we'll win that latter battle.
EDIT: as a child post points out, I completely forgot to mention drivers as well; as a strong argument to my "we don't have many options" thesis. AMD's linux support has been historically lacking next to NVIDIA which makes it a non starter in many cases.
As for drivers, AMD has pledged to open source their Vulkan and OpenCL implementations. While that release has been pending "legal review" forever now, alternative open source drivers are making great progress thanks to Vulkan's simpler driver model. While NVidia's generally had the "better" driver, both from adhering less strictly to the spec and having the manpower to routinely fix application bugs in their driver, that's all changed with Vulkan/DX12 being significantly closer to the hardware.
I'd say things are looking up for market balance.
I have a 16GB GPU as well, but have never run a local model so far. According to the table in the article, 9B and 8-bit -> 13 GB and 27B and 3-bit seem to fit inside the memory. Or is there more space required for context etc?
Inference engines like llama.cpp will offload model and context to system ram for you, at the cost of performance. A MoE like 35B-A3B might serve you better than the ones mentioned, even if it doesn't fit entirely on the GPU. I suggest testing all three. Perhaps even 122-A10B if you have plenty of system ram.
Q4 is a common baseline for simple tasks on local models. I like to step up to Q5/Q6 for anything involving tool use on the smallish models I can run (9B and 35B-A3B).
Larger models tolerate lower quants better than small ones, 27B might be usable at 3 bpw where 9B or 4B wouldn't. You can also quantize the context. On llama.cpp you'd set the flags -fa on, -ctk x and ctv y. -h to see valid parameters. K is more sensitive to quantization than V, don't bother lowering it past q8_0. KV quantization is allegedly broken for Qwen 3.5 right now, but I can't tell.