Should be simple
Should be simple
The focus here should be on the custom hardware they are producing and its performance, that is whats impressive. Imagine putting GLM-5 on this, that'd be insane.
This reminds me a lot of when I tried the Mercury coder model by Inceptionlabs, they are creating something called a dLLM which is like a diffusion based llm. The speed is still impressive when playing aroun with it sometimes. But this, this is something else, it's almost unbelievable. As soon as I hit the enter key, the response appears, it feels instant.
I am also curious about Taalas pricing.
> Taalas’ silicon Llama achieves 17K tokens/sec per user, nearly 10X faster than the current state of the art, while costing 20X less to build, and consuming 10X less power.
Do we have an idea of how much a unit / inference / api will cost?
Also, considering how fast people switch models to keep up with the pace. Is there really a potential market for hardware designed for one model only? What will they do when they want to upgrade to a better version? Throw the current hardware and buy another one? Shouldn't there be a more flexible way? Maybe only having to switch the chip on top like how people upgrade CPUs. I don't know, just thinking out loudly.
I don't think licensing fees to ARM are a problem per se, it's the "frictional costs" that it introduces. My understanding is that the Raspberry Pi developed their RP2040 because ARM had a reasonable package available for its low-end Cortex-M processors, which doesn't apply to their higher-end Cortex ones.
RISC-V doesn't even have these restrictions. Even extremely small outfits can play around with RISC-V designs. Most of them won't go very far, but with enough lottery tickets, one of them is bound to draw the winning numbers.
Sure there are considerable obstacles. I toy with microcontrollers. For hobbyists like me, ARM processors are still the most sensible choice. I'm not expecting any shift from that for at least five years. But who knows after that? Anything can happen.
I totally get the desire to switch to electric appliances for many reasons, but I am yet to meet an electric stove of any kind that I remotely enjoyed cooking on. Is this everyone's experience? Did I just not meet the right induction stove yet? Is there some sort of new technology on the horizon that will make electric stoves infinitely better?
Until you see a WWDC that doesn't announce a major release of macOS, expect more of the same.
To any involved Apple folks here, I feel your pain, and I'm sorry.
I sort of agree, but anyone typing out code at 120WPM isn't programming, they're writing. Programming is an intellectual activity, not a physical one so learning to touch type is worthwhile but might only gain you a few minutes a day of productivity over someone doing 40WPM in a home-grown hunt-and-peck style of typing?
The world the author is describing currently has LLM in it. Irrespective of the author liking it or not, it is here to stay. So to build a model of the world, you would still need to consult an LLM, understand how it can give plausible looking answers, learn how to effectively leverage the tool and make it part of your toolkit. It does not mean you stop reading manuals, books or blogs. It just means you include LLM also in those list of things.