I was able to fit the model with decent speeds (30 tokens/seconds) and a 20k token context completely on the GPU.
For summarization, the performance of these models are decent enough. However unfortunately in my use case I felt using Gemini's Free Tier with it's multimodal capabilities and much better quality output made running local LLMs not really worth it as of right now, atleast for consumers.
https://archive.nytimes.com/www.nytimes.com/library/cyber/we...
I've used Moonlight + Nvidia Gamestream with ~40ms RTT and couldn't feel a difference in competitive shooters, so total latency must be pretty low.
Does it have something to do with the bandwidth requirements? (1 stream v/s potentially hundreds)
Instead I would assume, in order
- my config broke it
- OS update broke it
- the bios doesn’t properly handle any case that isn’t “preinstalled OEM windows”
I had a laptop that as far as I could tell, could only boot into windows’ default bootmgr.efi. I could turn off secure boot, and tamper with that efi to boot Linux, but it refused to acknowledge other boot loaders from within the bios. It wouldn’t surprise me in the slightest if secure boot isn’t properly handled. I’ve had too many issues with cheap computers having janky bioses.
1. Recursively lookup DNS, so domains will have to be blocked at the registrar level, since DNS is unencrypted, it can be blocked at ISP level as well.
2. Use a protocol alternative to DNS, a good mature example is GNS. It aims to replace DNS, with a built from group up, modernish protocol. Using a DHT and public-key cryptography.
3. There are "block chain" solutions to the whole domain problem, look at Handshake, ENS etc.
I don't think that's true. Android surfaces raw GNSS measurements including carrier phase (sub wavelength measurements) to do centimeter level positioning through the raw measurements API [0].
There's even an API to specify the phone antenna phase pattern to correct the carrier phase measurements (source: I implemented it [1]). For those that aren't familiar, the idea is that the antenna pattern on phones isn't perfectly symmetrical, and depending on the direction of the incoming signal, it may appear longer. Knowing the antenna pattern, you can correct for this.
[0] https://developer.android.com/reference/android/location/Gns...
[1] https://developer.android.com/reference/android/location/Gns...
And it doesn't expose ADR/carrier phase.