Thanks! As @Tiberium mentioned, Nim 2.0 defaults to using ARC (Automatic Reference Counting), so no runtime GC. The Nim compiler is quite smart about when to copy/move and when to destruct, with some hints like lent/sink for when you want to have a bit more control.
Keep in mind that I also need to use ptr (as opposed to ref) types in a lot of cases when working at a low level, so there might be a need for some manual memory management as well.
> What where the most troublesome parts of the project?
Task switching. It's a very intricate process, you really have to understand how to structure everything (especially the task stack) so that you can switch everything from underneath the CPU as if nothing has happened (the CPU is obliviuous to which task is running, including the kernel itself). Add to that switching between user mode and kernel mode (during interrupts and system calls) and it becomes even more challenging.
> Also, any tips if anyone want to write an OS from scratch, aswell?
As @deaddod said, you need to read a lot. Two invaluable resources for me were the Intel Software Development Manuals (SDM), and the osdev wiki. The SDM can be daunting, but it's surprisingly very readable. The osdev wiki has great content, but it can be a hit or miss. I complement them with various blog posts and code on github to really understand a certain topic.
That being said, the most important aspect of this process is to have tons of curiousity and to be passionate about low-level systems programming. I love to learn how things really work at the lowest level. Once you learn it, you'll discover that there's no magic, and that you can write something yourself to make it work.
Basically, OSDev is one of the few realms you can't really take shortcuts in. It's kind of like learning Rust, in that you'll have a lot of foundational work until you get some real payoff. Unlike rust, however, there isn't just some cliff where you start getting it; it's a constant uphill trudge.
Learning the boot process of your target architecture, adding core functionality (process scheduling, filesystem/VFS support, IO management, etc), adding driver support, supporting your video device (just getting a basic framebuffer, and then adding each piece after that), supporting USB, supporting CRT and POSIX (if you choose to do so), etc are all herculean tasks of their own.
That being said, it's a super incremental process, so you'll get to watch it grow and expand at each step.
Reading up on the FreeBSD and Linux kernels are good starts. As well as reviewing other hobby OSes such as Serenity, TouruOS, Haiku, etc. And the OSDev wiki is invaluable.
Also, accepting you probably aren't going to build the next big OS or trying to compete with the big dogs is something you'll have to humble yourself with.
I don't use Nim, but it's an interesting language. I've read someone else complaining about having to make changes to an old project every time he went to recompile it. I'm wondering how true this is, so in other words, I'm wondering about the frequency and severity of breaking changes in the language.
I haven't faced such issue. I think the only issue I faced was when I upgraded to 2.0, they made `--threads:on` the default, so I had to turn it off, but that's about it.
Doesn't happen a lot, but it does happen from time to time. Of course only if you actually update your Nim version, so it's not like interpreted languages like Python where stuff stops working if a new version comes out and your package manager upgrades it.
I did consider Swift for a brief moment, but was put off by the fact that targeting bare-metal was almost non-existent. It was more work than I would like to put into bootstrapping the dev environment.
No, it isn't. The type must explicitly be a nullable type or pointer to be nullable. All other values must have a valid initialization. Anything not marked as a pointer or ref is by default managed by its scope. This includes dynamic types like seqs and strings, which are pointers to heap memory but managed by the stack scope and deallocated upon leaving scope.
To be honest, I haven't found this to be an issue (yet). I try to keep most of my types value types (cannot be null), which the compiler can pass by reference under the hood if it detects that it's too big, without compromising memory safety.
I use the Options module which has a none/some check. None is the absence of a value. You can test for this quite easily and I see it as a feature, not a bug.
Thanks! Actually writing has helped me so many times in improving the design and implementation. It forces me to question my assumptions, and ask myself: would the reader be able to understand why I made such decision? I have to justify everything I do, which helped me remove unnecessary complexity and focus on the more important aspects.
Nice, I love to see stuff like this. I've been an on-again, off-again Nim "ecosystem guy" for several years. It's great to see this delightful little project is still chugging along.
To either of you, whenever you are doing something new from scratch, it can be useful to consider the granularity of provided abstractions & services which khaledh seems to be doing. I see fusion only has like 8 syscalls presently. It's not in Nim, but along these lines ("how much" individual calls do), you might want to consider an approach like this: https://github.com/c-blake/batch to amortize costs of crossing expensive call boundaries.
As others mentioned, Nim is a statically typed programming language that compiles down to C, C++, and JavaScript. It has great C interop, which makes systems programming easy. As for why Nim, here's an excerpt from my accompanying site[0]:
> Why Nim? It's one of the few languages that allow low-level systems programming with deterministic memory management (garbage collector is optional) with destructors and move semantics. It's also statically typed, which provides greater type safety. It also supports inline assembly, which is a must for OS development. Other options include C, C++, Rust, and Zig. They're great languages, but I chose Nim for its simplicity, elegance, and performance.
As for the overall design goals of Fusion, I have high ambitions, which I list on the same page I referenced. I don't want to build another Unix-like OS; I'd like to experiment with fundamental issues in OS design, such as using a single-address space and capability-based security for protection. Another aspect I'm trying to explore is how processes/tasks are modeled, which I believe should be modeled as state machines with statically-typed channels to communicate between each other (this is not new, it's been done in Singularity OS[1]). There's rudimentary support in the kernel for channels and using them from user space, but it's still early.
Seeing more projects in Nim makes me happy. I'm a (mostly) Python and JavaScript programmer who is interested in the benefits of also knowing a modern, fast, statically-typed language. Among a candidate list of Go, Rust, Zig, or Nim, I like Nim the most. It feels the most "Pythonic" in the sense of very little syntax clutter when I'm reading code. I also love, love, love using a REPL to prototype new code, and INim does it well. The biggest problem with Nim currently is its small community size, which makes the universe of available and maintained software libraries smaller than in the other language communities. It's a chicken-or-egg problem, but can be solved by more devs (including me!) being "the change you want to see in the world".
To be honest, I haven't found the community size to be an issue. The Nim forum[0] has a vibrant community, and is the place I go to for help, and the response is usually quick and on point. The language is also evolving in a careful manner, with Araq at the helm I think it's going to be even better in the long term.
As for the ecosystem, yes, it's not as big as Python or Rust, but surprisingly the standard library has most of what people need. I rarely look for 3rd party packages to do something.
That being said, I acknowledge that Nim is on the lesser known languages of the spectrum, but that doesn't take away from its merits as a very promising language that does what it's supposed to do very well.
One thing I think the community should focus on more is IDE support. The VSCode extension is good, but has some rough edges. I also prefer JetBrains IDEs, and the official Nim plugin is very lacking to say the least. I have another side project to create a JetBrains plugin for Nim[1], but I haven't gone far with it yet.
My world: I need to use OpenCV. The existing OpenCV bindings (nim-opencv) haven't been touched in years because the author left* the Nim community. (And that really stinks! It was created by dom96, who also created Nimble, Jester, and a ton of other useful stuff in the Nim world.) ... So... I created my own OpenCV bindings and published it (https://nimble.directory/pkg/mvb). But they're minimal because I'm just one dude and haven't had the time to complete the bindings (either manually, or ideally, using an automated binding generator tool). I will.. eventually.. I hope! Meanwhile, OpenCV bindings for Rust and Go are robust and well-maintained.
Now I'm playing around more with Nostr (and the Lightning Network)... Nostr libraries for Nim are not as complete or well-maintained as those in Rust, Go, or even Python, etc.
I'm not letting that stop me from using Nim for my projects... I love Nim! But it does mean I have more work to do (and code to maintain). I can make that choice because I'm my own boss and run my own company. But I could see others not making the same choice for rational reasons.
* And dom96 left, unfortunately, because of harassment and abuse, which is another possible reason why Nim isn't as well adopted as Go, Rust, etc. If people want to see Nim succeed more, they also need to focus on improving community safety, too.
https://news.ycombinator.com/item?id=38999296
Maybe it's just because I'm getting older, but I struggle to read Nim when I'm forced to use two spaces for indentation. I can barely make out each block of code, and I don't like to rely on my IDE to make a language readable.
I use it at work for an important CLI tool that backs a number of our systems. My style is almost exactly like Python. Four-space tabs, snake_case, my own rules for indenting parameters and keywords. You do not need to use the official Nim style, you can absolutely write it like Python, Ada, however you like.
Readit News
Q: has the GC of Nim caused any challenges?
(And if not, would you attribute that to Nim unique GC that does NOT “stop-the-world”?)
https://nim-lang.org/1.4.0/gc.html
Keep in mind that I also need to use ptr (as opposed to ref) types in a lot of cases when working at a low level, so there might be a need for some manual memory management as well.
Task switching. It's a very intricate process, you really have to understand how to structure everything (especially the task stack) so that you can switch everything from underneath the CPU as if nothing has happened (the CPU is obliviuous to which task is running, including the kernel itself). Add to that switching between user mode and kernel mode (during interrupts and system calls) and it becomes even more challenging.
> Also, any tips if anyone want to write an OS from scratch, aswell?
As @deaddod said, you need to read a lot. Two invaluable resources for me were the Intel Software Development Manuals (SDM), and the osdev wiki. The SDM can be daunting, but it's surprisingly very readable. The osdev wiki has great content, but it can be a hit or miss. I complement them with various blog posts and code on github to really understand a certain topic.
That being said, the most important aspect of this process is to have tons of curiousity and to be passionate about low-level systems programming. I love to learn how things really work at the lowest level. Once you learn it, you'll discover that there's no magic, and that you can write something yourself to make it work.
[0] https://www.intel.com/content/www/us/en/developer/articles/t...
[1] https://wiki.osdev.org
Basically, OSDev is one of the few realms you can't really take shortcuts in. It's kind of like learning Rust, in that you'll have a lot of foundational work until you get some real payoff. Unlike rust, however, there isn't just some cliff where you start getting it; it's a constant uphill trudge.
Learning the boot process of your target architecture, adding core functionality (process scheduling, filesystem/VFS support, IO management, etc), adding driver support, supporting your video device (just getting a basic framebuffer, and then adding each piece after that), supporting USB, supporting CRT and POSIX (if you choose to do so), etc are all herculean tasks of their own.
That being said, it's a super incremental process, so you'll get to watch it grow and expand at each step.
Reading up on the FreeBSD and Linux kernels are good starts. As well as reviewing other hobby OSes such as Serenity, TouruOS, Haiku, etc. And the OSDev wiki is invaluable.
Also, accepting you probably aren't going to build the next big OS or trying to compete with the big dogs is something you'll have to humble yourself with.
Deleted Comment
(since they seem very similar at this point, yet Swift is more battle tested)
Deleted Comment
I like languages that disallow null by default (e.g. Rust, OCaml etc) because it seems to be a huge source of errors.
Deleted Comment
https://0xc0ffee.netlify.app/osdev/01-intro.html
Some day I want to write an RTOS in Nim. I enjoy writing embedded programs in Nim and it’d be fun to make an RTOS.
I'm hoping these questions aren't too basic, I have no context whatsoever for understanding this so hope someone can explain.
> Why Nim? It's one of the few languages that allow low-level systems programming with deterministic memory management (garbage collector is optional) with destructors and move semantics. It's also statically typed, which provides greater type safety. It also supports inline assembly, which is a must for OS development. Other options include C, C++, Rust, and Zig. They're great languages, but I chose Nim for its simplicity, elegance, and performance.
As for the overall design goals of Fusion, I have high ambitions, which I list on the same page I referenced. I don't want to build another Unix-like OS; I'd like to experiment with fundamental issues in OS design, such as using a single-address space and capability-based security for protection. Another aspect I'm trying to explore is how processes/tasks are modeled, which I believe should be modeled as state machines with statically-typed channels to communicate between each other (this is not new, it's been done in Singularity OS[1]). There's rudimentary support in the kernel for channels and using them from user space, but it's still early.
[0] https://0xc0ffee.netlify.app/osdev/01-intro.html
[1] https://en.wikipedia.org/wiki/Singularity_(operating_system)
As for the ecosystem, yes, it's not as big as Python or Rust, but surprisingly the standard library has most of what people need. I rarely look for 3rd party packages to do something.
That being said, I acknowledge that Nim is on the lesser known languages of the spectrum, but that doesn't take away from its merits as a very promising language that does what it's supposed to do very well.
One thing I think the community should focus on more is IDE support. The VSCode extension is good, but has some rough edges. I also prefer JetBrains IDEs, and the official Nim plugin is very lacking to say the least. I have another side project to create a JetBrains plugin for Nim[1], but I haven't gone far with it yet.
[0] https://forum.nim-lang.org
[1] https://github.com/khaledh/nimjet
Now I'm playing around more with Nostr (and the Lightning Network)... Nostr libraries for Nim are not as complete or well-maintained as those in Rust, Go, or even Python, etc.
I'm not letting that stop me from using Nim for my projects... I love Nim! But it does mean I have more work to do (and code to maintain). I can make that choice because I'm my own boss and run my own company. But I could see others not making the same choice for rational reasons.
* And dom96 left, unfortunately, because of harassment and abuse, which is another possible reason why Nim isn't as well adopted as Go, Rust, etc. If people want to see Nim succeed more, they also need to focus on improving community safety, too. https://news.ycombinator.com/item?id=38999296
Deleted Comment
Deleted Comment
Deleted Comment