It would be neat if there was a standard interface for device drivers, similar to what POSIX does for user-space. That way, interesting kernels like this one can just comply with that and the enormous amounts of Linux modules can be ported to comply with that standard so that they can also be loaded by redox, blueos,etc..
But the complication I suppose is data-structures being accessed by drivers that reside in the core kernel and other assumptions that come with linking against a monolith program like the Linux kernel. It would be momentous to simply get Linux drivers to comply with a kernel-agnostic ABI.
IMO, the reason we have a sea of open source drivers available now is because of the lack of a standard driver interface.
If one existed, companies would not be compelled to release their drivers' source, and would just release closed source drivers. As it stands, kernel drivers must be open source because the kernel API/ABI changes, and drivers must be recompiled against new kernel releases. It's infeasible to release a compiled driver .ko and have it work with new kernel releases.
Similarly, companies will not be incentivized to mainline their drivers for hardware outside of hobbists' interests. We're blessed with a plethora of drivers for enterprise, cloud and industry hardware that would otherwise have never been released beyond vendors' customers' deployments.
What would happen to the Linux driver ecosystem is what happened with Nintendo, Sony, Apple and FreeBSD. You get closed source drivers siloed away in proprietary systems that will never be released. The deployed drivers will come with restrictions on use and distribution, as well, so it wouldn't be like you could pluck out compatible drivers to use elsewhere.
> It's infeasible to release a compiled driver .ko and have it work with new kernel releases.
It's not. At Ksplice we had a build farm and whenever a supported distro would release a new kernel, we would generate new .ko files for that kernel, usually within 24 hours. It was a lot of work, and very much specific to the Ksplice product. These days, between docker and DKMS, and limiting yourself to supporting a specific device, you'd have a much easier time of building a build farm to release a compiled .ko, if you were a hardware manufacturer that wanted to support that.
> It would be neat if there was a standard interface for device drivers, similar to what POSIX does for user-space.
Plan 9 does this but instead of a binary interface that exposes machine details it hides them behind 9P, a simple RPC file tree protocol. This same protocol is also served by user space programs so it's universal. The benefit of all this is the system is small and very light weight. Its an OS a single human can grok from kernel to user space.
Since 9p abstracts everything it's kernel and language agnostic. A Plan 9 kernel can be written in Rust and serve the same 9P tree to a Plan 9 written in C, Go, Zig, D, etc. The user space drivers and services can also be written in any language as well as the programs accessing them. 9P is machine agnostic so a Plan 9 network can be made of disparate machine architectures letting you mix x86, Arm, Power, Mips, Risc-V, etc. Stupid simple cross compiling is an out of the box feature, just change the objtype env variable.
I can export those devices/services to other systems using 9P, cifs, nfs, and so on. I can export the sound device of a Plan 9 machine using cifs to a Windows box and a Windows program could open that file and play sound by writing 16bit stereo audio to it.
Your data structures are then pretty strait forward, e.g audio(3): "Audio data is a sequence of stereo samples, left sample first. Each sample is a 16 bit little-endian two's complement integer; the default sampling rate is 44.1 kHz." http://man.postnix.pw/9front/3/audio
It's a fantastic concept which frees all the services and hardware from the confines of a old school POSIX/Unix machine. Since Plan 9 in NOT Unix you also don't have to worry about crusty old POSIX. It has its own C dialect that is mostly C99 compliant and a very nice C library that beats smelly old ANSI C. I highly recommend learning how it works and giving it a go. Its not for everyone but man, I really dig how its just a patch-bay of networked 9P stuff. Wiring up a network of machines and hardware is ezpz.
I was always fascinated by plan 9 ever since finding out about it and only ever having heard of the movie referenced in Seinfeld. Way back then I didn’t think it would keep seeing use and development, I’m glad I’m wrong :)
Yes, we really really need to do this. It would be what LLVM was for compiler frontends (and thus languages) starting 15-20 years ago: a Cambrian explosion enabler.
I regularly see articles pop in here about OS development happening in China but I find it very hard to find resource in English about what’s actually happening.
Could anyone give an overview of what Huawei and Vivo are doing? I understand it’s mostly RTOS to use on phone. How does it compare to QNX and Linux? Is it as ambitious as Fuchsia?
Apparently they are shipping. It’s weird that we have reached a point where there seem to be two worlds not talking to each other much.
I think there might be more of this coming. The era where US was leading everything and expecting everyone to be a good boy who report everything is long gone due to the current state of affairs in the tech world.
I'm not Chinese but I can only support such efforts that make everyone less reliable on main actors. That said they even share their work so it's not like they are going full mute.
China's emergence was inevitable - they have the numbers. Last one I heard was 200 million people in STEM careers alone. That's more than the entire US workforce.
I expect technological development to explode and my advice is for anyone interested in it to learn Mandarin. Including myself.
> development happening in China but I find it very hard to find resource in English about what’s actually happening.
For years I've had this issue with pretty much everything happening in China, from business to politics to culture. For me personally, getting a window into China has been the number one game changer with LLMs. It's easier than ever to find and digest Chinese sources.
I feel like this a problem in general for topics outside “the West” or even just the Anglosphere. There is a tantalising amount of information that is siloed away in other languages. I was reading a Wikipedia article on one of the campaigns waged by the Ottomans in Europe and the English version was threadbare (and poorly written) in comparison to the Hungarian Wikipedia equivalent which was three times longer and had more images, maps and diagrams. It also cited a wealth of sources that were, again, not in English. This is a natural result of the fact that the ones “closest” to the event in question will generally be the ones most qualified and ready to report upon it.
> I regularly see articles pop in here about OS development happening in China but I find it very hard to find resource in English
It's challenging for open source communities in the west to collaborate with their counterparts in China primarily because of language, but also the collaborations can't really happen in the public places that we're all used to. Western social platforms are blocked in China, and Chinese social platforms are not appealing to the west for one reason or another. Even places like Github are frequently blocked on partially inaccessible in China.
So there really just isn't any good place for people to meet and collaborate, and learn from each other.
It's even harder to get an accurate picture of what Chinese companies are actually doing. Like the wiki page for Huawei's OS Next is pretty incredible. By which I mean, rather actually unbelievable. https://en.m.wikipedia.org/wiki/HarmonyOS_NEXT
They created a brand new microkernel with Linux ABI and driver support in containers? Or... did they just slightly fork Linux and pretend they invented it?
Having used HarmonyOS, it feels totally like Android. Even the back buttom behavior and the app lifecycle feels the same.
Some might argue that this is intentional, but to me, this more likely shows that HarmonyOS is just a hard fork of Android without sources released and, likely, with their own virtual machine implementation (ARK instead of ART).
Seeing Illumos lx-branded zones and WSL1, I think it is plausible that this actually does that. It is less research-level hard then simply requiring a lot of person-hours to slavishly reproduce an underdefined interface.
I think this comment is emblematic of a broader western trend underestimating recent Chinese technological development. It is understandable, given how backward they were for so long. The last five years though, things have really gotten crazy over there in terms of investment and progress.
My understanding is that Harmony OS is a full fledge OS at this point. We will start hearing more about it when devs will need to support it for their mobile apps that ship to LATAM and Africa.
From their website (translated): Blue River Operating System 2 is the industry's first operating system written in the Rust language from the kernel to the system framework. A series of security features of the Rust language can detect security vulnerabilities caused by improper memory use during the compilation stage, making it inherently more secure from the source.
Harvey is Plan 9 built with a GCC/Clang tool chain instead of the Plan 9 tools. AFAIK both projects are abandoned (edit: R9 might still be alive) and most devs have moved on to 9front which is a modern fork of Plan 9.
Readit News
But the complication I suppose is data-structures being accessed by drivers that reside in the core kernel and other assumptions that come with linking against a monolith program like the Linux kernel. It would be momentous to simply get Linux drivers to comply with a kernel-agnostic ABI.
If one existed, companies would not be compelled to release their drivers' source, and would just release closed source drivers. As it stands, kernel drivers must be open source because the kernel API/ABI changes, and drivers must be recompiled against new kernel releases. It's infeasible to release a compiled driver .ko and have it work with new kernel releases.
Similarly, companies will not be incentivized to mainline their drivers for hardware outside of hobbists' interests. We're blessed with a plethora of drivers for enterprise, cloud and industry hardware that would otherwise have never been released beyond vendors' customers' deployments.
What would happen to the Linux driver ecosystem is what happened with Nintendo, Sony, Apple and FreeBSD. You get closed source drivers siloed away in proprietary systems that will never be released. The deployed drivers will come with restrictions on use and distribution, as well, so it wouldn't be like you could pluck out compatible drivers to use elsewhere.
It's not. At Ksplice we had a build farm and whenever a supported distro would release a new kernel, we would generate new .ko files for that kernel, usually within 24 hours. It was a lot of work, and very much specific to the Ksplice product. These days, between docker and DKMS, and limiting yourself to supporting a specific device, you'd have a much easier time of building a build farm to release a compiled .ko, if you were a hardware manufacturer that wanted to support that.
Plan 9 does this but instead of a binary interface that exposes machine details it hides them behind 9P, a simple RPC file tree protocol. This same protocol is also served by user space programs so it's universal. The benefit of all this is the system is small and very light weight. Its an OS a single human can grok from kernel to user space.
Since 9p abstracts everything it's kernel and language agnostic. A Plan 9 kernel can be written in Rust and serve the same 9P tree to a Plan 9 written in C, Go, Zig, D, etc. The user space drivers and services can also be written in any language as well as the programs accessing them. 9P is machine agnostic so a Plan 9 network can be made of disparate machine architectures letting you mix x86, Arm, Power, Mips, Risc-V, etc. Stupid simple cross compiling is an out of the box feature, just change the objtype env variable.
I can export those devices/services to other systems using 9P, cifs, nfs, and so on. I can export the sound device of a Plan 9 machine using cifs to a Windows box and a Windows program could open that file and play sound by writing 16bit stereo audio to it.
Your data structures are then pretty strait forward, e.g audio(3): "Audio data is a sequence of stereo samples, left sample first. Each sample is a 16 bit little-endian two's complement integer; the default sampling rate is 44.1 kHz." http://man.postnix.pw/9front/3/audio
It's a fantastic concept which frees all the services and hardware from the confines of a old school POSIX/Unix machine. Since Plan 9 in NOT Unix you also don't have to worry about crusty old POSIX. It has its own C dialect that is mostly C99 compliant and a very nice C library that beats smelly old ANSI C. I highly recommend learning how it works and giving it a go. Its not for everyone but man, I really dig how its just a patch-bay of networked 9P stuff. Wiring up a network of machines and hardware is ezpz.
Here one famous one from early 1980s, there others to read about, for anyone into compiler tooling history,
https://en.wikipedia.org/wiki/Amsterdam_Compiler_Kit
Most other OSes have had driver ABIs throughout all their existence.
Could anyone give an overview of what Huawei and Vivo are doing? I understand it’s mostly RTOS to use on phone. How does it compare to QNX and Linux? Is it as ambitious as Fuchsia?
Apparently they are shipping. It’s weird that we have reached a point where there seem to be two worlds not talking to each other much.
I'm not Chinese but I can only support such efforts that make everyone less reliable on main actors. That said they even share their work so it's not like they are going full mute.
I expect technological development to explode and my advice is for anyone interested in it to learn Mandarin. Including myself.
For years I've had this issue with pretty much everything happening in China, from business to politics to culture. For me personally, getting a window into China has been the number one game changer with LLMs. It's easier than ever to find and digest Chinese sources.
I feel that it is quite obvious the next century will have China leading the pack, and I'd really like to be able to prepare for that.
China is mindblowingly huge. There has to be A LOT happening at any one time.
It's challenging for open source communities in the west to collaborate with their counterparts in China primarily because of language, but also the collaborations can't really happen in the public places that we're all used to. Western social platforms are blocked in China, and Chinese social platforms are not appealing to the west for one reason or another. Even places like Github are frequently blocked on partially inaccessible in China.
So there really just isn't any good place for people to meet and collaborate, and learn from each other.
They created a brand new microkernel with Linux ABI and driver support in containers? Or... did they just slightly fork Linux and pretend they invented it?
Some might argue that this is intentional, but to me, this more likely shows that HarmonyOS is just a hard fork of Android without sources released and, likely, with their own virtual machine implementation (ARK instead of ART).
Deleted Comment
https://www.usenix.org/conference/osdi24/presentation/chen-h...
Harvey is Plan 9 built with a GCC/Clang tool chain instead of the Plan 9 tools. AFAIK both projects are abandoned (edit: R9 might still be alive) and most devs have moved on to 9front which is a modern fork of Plan 9.
https://github.com/vivoblueos/book/blob/main/src/SUMMARY.md
All of these repos are days old.
Dead Comment