This served us well for many years before migrating to use Kamal [3] for its improved remote management features.
[1] https://docs.servicestack.net/ssh-docker-compose-deploment
This served us well for many years before migrating to use Kamal [3] for its improved remote management features.
[1] https://docs.servicestack.net/ssh-docker-compose-deploment
This is the sort of question I don't trust AI with yet.
Rider is far better than VS for everything apart from Desktop UI Apps and perhaps Blazor WASM hot reloading, which is itself far behind the UX of JS/Vite hot reloading, so I avoid it and just use Blazor static rendering. Otherwise VS tooling is far behind Intellij/Rider for authoring Web dev assets, inc. TypeScript.
I switched to Rider/VS Code long before moving to Linux, which I'm happy to find works just as well in Linux. Not a fan of JetBrains built-in AI Integration (which IMO they've fumbled for years), but happy with Augment Code's Intellij Plugin which I use in both Rider and VS Code.
gpt-oss:20b = ~46 tok/s
More than 2x faster than my previous leading OSS models: mistral-small3.2:24b = ~22 tok/s
gemma3:27b = ~19.5 tok/s
Strangely getting nearly the opposite performance running on 1x 5070 Ti: mistral-small3.2:24b = ~39 tok/s
gpt-oss:20b = ~21 tok/s
Where gpt-oss is nearly 2x slow vs mistral-small 3.2. llama-server = ~181 tok/s
LM Studio = ~46 tok/s (default)
LM Studio Custom = ~158 tok/s (changed to offload to GPU and switch to CUDA llama.cpp engine)
and llama-server on my 3x A4000 GPU Server is getting 90 tok/s vs 46 tok/s on ollamaThankfully, this may just leave more room for other open source local inference engines.
Always had a bad feeling when they didn't give ggerganov/llama.cpp their deserved credit for making Ollama possible in the first place, if it were a true OSS project they would have, but now makes more sense through the lens of a VC-funded project looking to grab as much marketshare as possible to avoid raising awareness for alternatives in OSS projects they depend on.
Together with their new closed-source UI [1] it's time for me to switch back to llama.cpp's cli/server.
[1] https://www.reddit.com/r/LocalLLaMA/comments/1meeyee/ollamas...
gpt-oss:20b = ~46 tok/s
More than 2x faster than my previous leading OSS models: mistral-small3.2:24b = ~22 tok/s
gemma3:27b = ~19.5 tok/s
Strangely getting nearly the opposite performance running on 1x 5070 Ti: mistral-small3.2:24b = ~39 tok/s
gpt-oss:20b = ~21 tok/s
Where gpt-oss is nearly 2x slow vs mistral-small 3.2.so, the 20b model.
Can someone explain to me what I would need to do in terms of resources (GPU, I assume) if I want to run 20 concurrent processes, assuming I need 1k tokens/second throughput (on each, so 20 x 1k)
Also, is this model better/comparable for information extraction compared to gpt-4.1-nano, and would it be cheaper to host myself 20b?
Whilst my gen 1 MB Air has been too slow for anything, my 2013 MB Intel still looks and runs great which the kids still make good use of. My latest M2 MB is by far the best I've ever owned with great build quality, performance, battery life where it's the first time I can confidently travel without a power brick.
Whilst Apple's non-Desktop hardware is always best-of-class, I've become increasingly dissatisfied with the direction of macOS and Windows which IMO have both become power-user-hostile and have switched to a Linux desktop full-time. Everyone's been predicting the year of the Linux Desktop for 20+ years, but I believe we're at a turning point for Linux adoption with Windows 11 becoming an intolerable ad/spyware infested marketing platform and Apple's continued ignorance of developers and ambitions of turning its neglected macOS into a locked down appliance.
Hopefully Valve can continue their investments in Steam Deck and Arch Linux to accelerate the adoption, their contributions to Proton have already IMO unblocked the biggest barrier to adoption. Whilst currently a happy Fedora user I like the direction, taste, philosophy and community behind Omarchy from what I've seen after kicking the tires in a VM, will look into switching over after they bring out their ISO.