Can you think of a more efficient protocol between the modem and main cpu then the AT-based ril with constant status checks? Is there anyway to make the modem more efficient, since its cpu is just consuming power and not adding much to the functionality of the phone? Can it be a smarter gateway that only triggers a main cpu wakeup when there's something important to notify the user of, like an incoming call or sms?
> Can it be a smarter gateway that only triggers a main cpu wakeup when there's something important to notify the user of, like an incoming call or sms?
That's how it works.
> Is there anyway to make the modem more efficient, since its cpu is just consuming power and not adding much to the functionality of the phone?
Modem is sleeping most of the time. (at least with my driver) It's much more efficient than the main A64 SoC. Like 10mW sleep vs 100mW sleep.
> Can you think of a more efficient protocol between the modem and main cpu then the AT-based ril with constant status checks?
ModemManager/ofono are not using AT interface, I think. They are using QMI.
Anyway, you don't need constant status checks even with AT interface. You just enable status indications you want to receive and that's what you'll get without polling.
My first job was an ISP I co-founded. Our first dial-up system was a classic 120MHz Pentium w/128MB RAM and 4GB harddrive, that served as the terminal server for 16 phone lines and let users start SLIP/PPP or a shell connection. It was not unusual to have 16 users logged into it at once running applications on it.
It also was our main e-mail server and web server, hosting both our own website and several customer websites. We had a second one of the same specs for a few other things and backups, but that was our main workhorse.
[EDIT: They were running Linux 1.0 and 1.2, first with NCSA httpd, then Apache; when we first got them they ran Slackware, installed from floppies; I think we ended up installing Redhat at some point before we retired them; the modems were hanging off Cyclades serial cards; and they were literally hanging - for our first 16 lines we had US Robotics Sportster modems hanging on the wall. (I'm writing this in the hope someone will tell me to get off their lawn and tell me how they did more with less, btw.) ]
I bought my first PC in the late 80s. It was an 8086 by Amstrad, with a 4Mhz processor, 512kb of RAM and no HDD.
It was running MS DOS 3.30 Plus and I made my first programs in GW-Basic.
These modems are way powerful. This reinforces the need of open source firmware for them. They run their own OS and can stay on independent of the rest of the hardware consuming very little power. Imagine putting your computer to sleep while downloads continue and can survive geographical network migration.
A fully open source firmware for GPS/LTE modem is badly needed.
"They run their own OS and can stay on independent of the rest of the hardware consuming very little power."
It's worse - there's a second, independent computer inside your phone that you have no control over: the SIM card.
The SIM is a standalone computer with its own processor and memory which your carrier can communicate with and upload programs to run on it without your knowledge ("OTA updates", etc.)
Seeing the ShadySIM project mentioned on that page reminded me of this DEFCON talk, which talks about some of the capabilities of the SIM OS (some of them run Java!) and what power they grant you over phones if you have control over the underlying cell network (legitimately or otherwise).
And you see some of what you're talking about in closed systems too, where the system integrator can control both sides.
The Wii, and PlayStations since the PS3 have had system control processors that can perform downloads, updates, and general system maintenance on a low power processors even while the majority of the system is off. In the Wii that processor was also responsible for the majority of the network stack even when accessed from the main processor. On the PS4 that processor even ran a relatively complete FreeBSD.
So a modem on a pine phone can survive the HN hug of death (and the number one post no less)? Am I missing something here? Because there's no way I believe that!
Since we're here, I did a quick grep through today's access log. It appears that your link got me 573 visitors to that article and 56 people clicking through to the other article linked therein.
My mind is blown that anyone writes software for stacks that max out at few hundred requests per second.
Everything I've seen comparing "high level" languages says programmers work around the same efficiency whatever the language. So why isn't everything done in Go, C#, and Java where 100,000 requests/second is trivial?
Even if you don't have much load, isn't the possibiliy of a cat DDoS'ing you by sleeping on someone's F5 key at least slightly concerning?
I swear half these companies that claim their site is "being DDoS'ed!" it's just somebody just running HTTrack to archive an article they like.
A static site is really easy to host and HN's hug of death isn't actually that intense; the site itself only gets like six million views a day, a fraction of those click on every link, and no syndicated page has much reach.
People badly overestimate what it takes to serve a website nowadays. I don't know whether it's just because we're all used to 10+ second website loads as our browser groans under the weight of all the trackers, or if we all have too much experience with web pages that run multiple 10ms+ unoptimized queries against databases that pull way too much data back, etc. and so we all think web pages are some sort of challenge, but webpages themselves really aren't that hard. It's all the stuff you're trying to put in to them that is the challenge.
I hit SHIFT-CTRL-r to watch the page reload under the network debugger so I could get a snapshot of the total size, which looks to be about 100KB, just eyeballing the sums, so at 10MB/s it should be able to serve around 100 requests/second, which is enough for most websites, even ones getting hugged.
I recently migrated a personal blog running on a hugo static site to digital ocean with automated github deployment, SSL certificates, CDN, and a custom domain. It took all of 15 minutes including updating the DNS on my custom domain registrar.
I speak as a software engineer whose blog has hit the HN frontpage multiple times, and who has optimized my blog to be static & lightweight, just like the author. In my experience, a front-page HN post (in the top 3) leads to 30-40 page hits per second. The page at https://nns.ee/blog/2021/04/01/modem-blog.html weights 44.4 kB transferred (HTML, CSS, images, fonts, etc, all compressed over HTTPS). So this is 11-15 Mbit/s of peak bandwidth, below the maximum throughput the author measured on the Pine phone modem (20.7 Mbit/s).
And the bottleneck is going to be purely network. Not CPU. Not disk. Just serving a small set of files cached in RAM by the pagecache.
It's above the max, no? The author measured 20Mbps between the phones main OS and the rest of the world, but 10Mbps between the phone and the modem (over the adb bridge).
My website also got multiple times in the HN frontpage and there was never any lags even on a very cheap VPS. The websites that get the HN hug of death are often the one requiring database access (e.g. wordpress).
But also, serving static content is more or less a matter of network connection speed. You can cache everything in RAM and write data to a socket very quickly. HTTPS makes things a little wonky because you can’t just sendfile() files directly into a socket but the overhead is still pretty minimal.
Slightly misleading title. This blog post is not hosted over LTE. Instead, the web server just happens to run inside the Pinephone modem.
Internet --> USB --> Pinephone CPU --> (ADB forwarding) --> Pinephone modem with webserver.
That's cool, but I am slightly disappointed. I was hoping to find out that some ISPs allowed to forward ports over LTE.
Here, I don't really see the advantage over doing it on the main CPU, since it will have to forward packets anyway. And you can use the same reasoning with whatever the pinephone is connected to.
Sorry if it's a bit misleading! Wasn't my intention.
However, there's nothing stopping me from getting a data SIM from my provider (who does allow forwarding ports over LTE). I just currently don't have any spare SIM cards at hand. But that is the plan.
Although the latency will likely skyrocket, I would be very interested in measuring the energy efficiency. The main CPU could even be completely shut down.
You can even imagine calling the main CPU as backup if there are too many incoming requests, though it would be better to use a (likely available) ethernet-over-USB instead of ADB forwarding, and that scenario is unlikely if just for a static blog.
Anyway, if you do so, please make sure to publish a new blog post, post it here, and plot the power consumption during the hug. That would be some interesting data!
You can get carrier grade connections that would allow this kind of routing- it is not part of a standard consumer package. This type of capability is managed via the "Access Point Name" feature in phones.
I had been looking to do this exact thing for so long. It didn't occur to me that it was possible to unlock the AT commands for the pinephone. I just wanted to turn it into a modem. It's a pretty solid price for that piece of tech alone, since dev kit LTE modem boards are >$100 in all the places I could find.
The Quectel modems are fun parts. We are using one in a project as a kind of theft protection. I couldn't belive at first "this is a whole computer with Linux on it".
I really wonder why modem manufacturers feel the need to encapsulate everything so much. Is it for regulatory reasons, to protect their secret sauce, or for our convenience? On the one hand it makes simple things really easy with some AT commands, on the other hand it makes debugging quite frustrating because it is a black box.
Regulatory reasons. There are a lot of things that FCC regs say that devices that transmit either must or must not do, which made more sense back when everything was analog. Now that the device is a full computer and can be programmed to do whatever you want, the way the industry has decided to best fulfill all those requirements is to make the computer do exactly and only what is required and then lock it down into a black box.
If you just unlocked everything, it would probably be better at what it does, but it would also not be possible to sell to consumers.
Do those regulations explicitly say that the RF device is not allowed to be programmable by the user? And isn't the user legally forbidden from substantially modifying the RF behaviour in any case?
> I really wonder why modem manufacturers feel the need to encapsulate everything so much.
The alternative would be using an RTOS of some sort, implementing a full network stack (with > 100Mb throughput) and a USB device, as the bare minimum. Then you also have I2S audio, GNSS, etc.
It probably makes more sense using an entire OS at some point.
BTW, this is typical Qualcomm, that makes the chipsets for these modems.
I've been playing with the ESP32's a lot more recently (which go the RTOS route), and in reading the code for network stacks for these devices, it did occur to me how simple they were in terms of features. I wouldn't say that one needs the entirety of the Linux kernel to be able to implement all of those things, but it does build off of a more well used stack and reuses a lot of well-tested components.
So from this perspective, it's honestly more secure probably, too, as opposed to trusting a hardware company to hire embedded software engineers to reinvent an entire stack from scratch. The latter of which are in rare supply and we all know that reinventing things is a process that often leads to vulnerabilities.
Locking down the communication to just AT commands then makes sense for similar reasons.
Why not offload all that to the driver running on the main application processor? The risk of having a second, black-box running Linux is that it’ll get outdated and/or compromised and used as a persistence mechanism for malware.
I wonder if Justine's Redbean[0] webserver, which got recently posted on HN[1], would support ARM in such use case. Its executable size is 460KB, it even allows Lua scripting, and benchmarks 1 million pages per second on desktop PC.
While you'll probably run out of memory first, there's the enormous overhead of running qemu's translation, too. Using QEMU to emulate x86 on my POWER9 system, I get... roughly the performance of a 2001-2004 pentium processor, and the host cpu is running at 3.8GHz and comparable speed wise to a 2018ish intel processor.
In spite of the author's insistence, I don't think Cosmopolitan is really usable on anything but x86, the QEMU thing is more a cute proof-of-concept. For very simple programs I guess...
Readit News
Beautiful seeing this on top of HN. :)
That message is from me. Nice to see someone using the access for something cool.
One of the reasons I decided to add it there was to give some visibility to this "feature" of EG-25G FW to people interested enough to look at dmesg.
That's how it works.
> Is there anyway to make the modem more efficient, since its cpu is just consuming power and not adding much to the functionality of the phone?
Modem is sleeping most of the time. (at least with my driver) It's much more efficient than the main A64 SoC. Like 10mW sleep vs 100mW sleep.
> Can you think of a more efficient protocol between the modem and main cpu then the AT-based ril with constant status checks?
ModemManager/ofono are not using AT interface, I think. They are using QMI.
Anyway, you don't need constant status checks even with AT interface. You just enable status indications you want to receive and that's what you'll get without polling.
And you can setup wakeup on call/sms too.
The world is a small place indeed.
You can run DNS, a VPN, or a small web server even on a VPS with 32MB of RAM so those are pretty cool specs.
[0] https://xnux.eu/devices/feature/modem-pp.html
It also was our main e-mail server and web server, hosting both our own website and several customer websites. We had a second one of the same specs for a few other things and backups, but that was our main workhorse.
[EDIT: They were running Linux 1.0 and 1.2, first with NCSA httpd, then Apache; when we first got them they ran Slackware, installed from floppies; I think we ended up installing Redhat at some point before we retired them; the modems were hanging off Cyclades serial cards; and they were literally hanging - for our first 16 lines we had US Robotics Sportster modems hanging on the wall. (I'm writing this in the hope someone will tell me to get off their lawn and tell me how they did more with less, btw.) ]
My P1 would stutter while playing MP3s :-))
A fully open source firmware for GPS/LTE modem is badly needed.
It's worse - there's a second, independent computer inside your phone that you have no control over: the SIM card.
The SIM is a standalone computer with its own processor and memory which your carrier can communicate with and upload programs to run on it without your knowledge ("OTA updates", etc.)
https://osmocom.org/projects/cellular-infrastructure/wiki/SI...
ftp://www.3gpp.org/tsg_sa/WG3_Security/TSGS3_30_Povoa/Docs/PDF/S3-030534.pdf
https://www.youtube.com/watch?v=31D94QOo2gY
The Wii, and PlayStations since the PS3 have had system control processors that can perform downloads, updates, and general system maintenance on a low power processors even while the majority of the system is off. In the Wii that processor was also responsible for the majority of the network stack even when accessed from the main processor. On the PS4 that processor even ran a relatively complete FreeBSD.
Democratizing these techniques would be killer.
[1]: <https://xyrillian.de/thoughts/posts/latency-matters-aftermat...>
Since we're here, I did a quick grep through today's access log. It appears that your link got me 573 visitors to that article and 56 people clicking through to the other article linked therein.
My mind is blown that anyone writes software for stacks that max out at few hundred requests per second.
Everything I've seen comparing "high level" languages says programmers work around the same efficiency whatever the language. So why isn't everything done in Go, C#, and Java where 100,000 requests/second is trivial?
Even if you don't have much load, isn't the possibiliy of a cat DDoS'ing you by sleeping on someone's F5 key at least slightly concerning?
I swear half these companies that claim their site is "being DDoS'ed!" it's just somebody just running HTTrack to archive an article they like.
I hit SHIFT-CTRL-r to watch the page reload under the network debugger so I could get a snapshot of the total size, which looks to be about 100KB, just eyeballing the sums, so at 10MB/s it should be able to serve around 100 requests/second, which is enough for most websites, even ones getting hugged.
Deleted Comment
I speak as a software engineer whose blog has hit the HN frontpage multiple times, and who has optimized my blog to be static & lightweight, just like the author. In my experience, a front-page HN post (in the top 3) leads to 30-40 page hits per second. The page at https://nns.ee/blog/2021/04/01/modem-blog.html weights 44.4 kB transferred (HTML, CSS, images, fonts, etc, all compressed over HTTPS). So this is 11-15 Mbit/s of peak bandwidth, below the maximum throughput the author measured on the Pine phone modem (20.7 Mbit/s).
And the bottleneck is going to be purely network. Not CPU. Not disk. Just serving a small set of files cached in RAM by the pagecache.
No idea if he's caching there, though.
But also, serving static content is more or less a matter of network connection speed. You can cache everything in RAM and write data to a socket very quickly. HTTPS makes things a little wonky because you can’t just sendfile() files directly into a socket but the overhead is still pretty minimal.
Deleted Comment
You pretty much never need that, but many/most sites do it somewhere anyway.
A static site running on a modem should be fine, but there is nothing saying they cant/arent using a CDN too, which would be even better.
It was just used to catch attention.
But with a CDN in front it might likely work out.
Internet --> USB --> Pinephone CPU --> (ADB forwarding) --> Pinephone modem with webserver.
That's cool, but I am slightly disappointed. I was hoping to find out that some ISPs allowed to forward ports over LTE.
Here, I don't really see the advantage over doing it on the main CPU, since it will have to forward packets anyway. And you can use the same reasoning with whatever the pinephone is connected to.
However, there's nothing stopping me from getting a data SIM from my provider (who does allow forwarding ports over LTE). I just currently don't have any spare SIM cards at hand. But that is the plan.
Although the latency will likely skyrocket, I would be very interested in measuring the energy efficiency. The main CPU could even be completely shut down.
You can even imagine calling the main CPU as backup if there are too many incoming requests, though it would be better to use a (likely available) ethernet-over-USB instead of ADB forwarding, and that scenario is unlikely if just for a static blog.
Anyway, if you do so, please make sure to publish a new blog post, post it here, and plot the power consumption during the hug. That would be some interesting data!
Thank you, sincerely!
I really wonder why modem manufacturers feel the need to encapsulate everything so much. Is it for regulatory reasons, to protect their secret sauce, or for our convenience? On the one hand it makes simple things really easy with some AT commands, on the other hand it makes debugging quite frustrating because it is a black box.
If you just unlocked everything, it would probably be better at what it does, but it would also not be possible to sell to consumers.
The alternative would be using an RTOS of some sort, implementing a full network stack (with > 100Mb throughput) and a USB device, as the bare minimum. Then you also have I2S audio, GNSS, etc.
It probably makes more sense using an entire OS at some point.
BTW, this is typical Qualcomm, that makes the chipsets for these modems.
So from this perspective, it's honestly more secure probably, too, as opposed to trusting a hardware company to hire embedded software engineers to reinvent an entire stack from scratch. The latter of which are in rare supply and we all know that reinventing things is a process that often leads to vulnerabilities.
Locking down the communication to just AT commands then makes sense for similar reasons.
Rest: no. Some modem manufacturers have SDKs to get your application on the device as a feature.
I wonder if Justine's Redbean[0] webserver, which got recently posted on HN[1], would support ARM in such use case. Its executable size is 460KB, it even allows Lua scripting, and benchmarks 1 million pages per second on desktop PC.
[0] https://justine.lol/redbean/index.html [1] https://news.ycombinator.com/item?id=26271117
[0] https://www.fefe.de/gatling/
In spite of the author's insistence, I don't think Cosmopolitan is really usable on anything but x86, the QEMU thing is more a cute proof-of-concept. For very simple programs I guess...
Running QEMU on a modem would be another hack for a day :)