I use Linux, FreeBSD, NetBSD, and OpenBSD all for fun, learning, and profit(the first two).
At the very least it is nice to make acquaintance with at least one BSD because it will probably expand your knowledge on Linux in ways you wont be able to anticipate.
For example, FreeBSD got me into kernel development, full system debugging, network stack development, driver development, and understanding how the whole kit fits together. Those skills transferred back and forth with reasonable fidelity to Linux, and for me jumping into Linux development cold would have been too big a leap.. especially in confidence and developing a mental model.
For my personal infrastructure, I tend to use FreeBSD because in many ways it is simpler and less surprising, especially when accounting for the passage of time. ifconfig is still ifconfig, and it works great. rc.d is all I need for my own stuff. I like the systematic effects of things like tunables and sysctl for managing hardware and kernel configuration. The man pages are forever useful to new and old users. The kernel APIs and userland APIs are extremely stable akin to commercial operating systems and unlike Linux.
There are warts. There are community frictions. The desktop story and some developer experiences will be perpetually behind Linux due to the size of the contributor base and user base. The job market for BSD is very limited compared to Linux. But I don't think it's an all or nothing affair, and ideally in a high stakes operation you would dual stack for availability and zero-day mitigation (Verisign once gave a great talk on this).
> tend to use FreeBSD because in many ways it is simpler and less surprising, especially when accounting for the passage of time. ifconfig is still ifconfig, and it works great. rc.d is all I need for my own stuff.
That sounds very appealing to me. I have to keep a small number of servers running, but its not my main focus and I would like to spend as little time on it as possible.
I have started using Alpine Linux for servers (not for my desktop, yet) because it is light and simple. Maybe BSD will be the next step.
The UI generally quite stable and well documented, which is awesome.
They also have things like `rpm` in ports that you can install. Why? Because you can enable linux binary compatibility[0] and run linux binaries on it (this implements the linux kernel interface, it's not a VM/emulator). It's also backward compatible with its own binaries back to FreeBSD 4 (circa 2000).
Though you may not need that as the ports/packages collection is pretty comprehensive.
It also comes with some nifty tools built-in for isolation (similar to but predating cgroups/containers) as "jails". It also has a hypervisor built in (bhyve) for virtualization if you do need to run any linux VMs or anything for any reason.
The way I usually sum up the difference to people is that FreeBSD is designed while Linux is grown. FreeBSD feels much more like a cohesive whole than Linux.
Really, the only reason I'm not running it everywhere is that the industry has kind of settled on linux-style containers for... absolutely everything, and the current solution for that on FreeBSD is basically "run linux in a VM".
I've run the same freebsd system for the last 15 years just hosting a small site for some of my friends from back in university.
I've migrated it through system upgrades and security fixes, but nothing else needed to change. usual uptime is about 3 years between major release updates.
I completely agree! If you're looking to deeply understand how Linux works under the hood, I highly recommend trying out Linux From Scratch. It gave me invaluable insight into the system, especially when I first explored it 20 years ago. Building everything from the ground up—without relying on prepackaged distros or libc—was a game changer.
I ran through that about 10 years ago. Not difficult for a reasonably seasoned Unixy person to run through but it quite frankly scared the shit out of me on the basis of how much stuff requires patching and hacking. Nothing fits together nicely. It gave me a whole new appreciation of macOS and FreeBSD.
I really really enjoyed using FreeBSD. It isn't a distro but a full OS that you can and easily fully rebuild from source. The people are great, the tech is good.
Yeah, you know, a lot of us talk about FreeBSD being great because you aren't surprised by design choices—it mostly just makes sense.
But I don't think we talk enough about the joy of not being surprised by updates. I'm about to do an upgrade from 13.2 to 14.1 this weekend and I am very confident that I won't have to worry about configuring a new audio subsystem, changing all my startup scripts to work with a new service manager paradigm, or half my programs suddenly being slow because they're some new quasi-container package format.
I worked at a large service provider and had to fix bugs and improve performance in TCP/IP and Intel Ethernet drivers. I ended up really enjoying it so it became a hobby and consulting gig as well.
I also still use FreeBSD on my NAS. But after many years, the desktop experience was pretty sad and made me switch to Windows + Linux for my hardware tinkering. On one side, the lack of manpower shows in many places, unfortunately. I'm talking modern WiFi, GPU support, or power-save mechanisms. On the other side, many Open Source projects only support Linux and getting their projects to compile + run on FreeBSD was a pain, too.
I mean, in addition to what kev009 mentioned, FreeBSD has so many great things to offer: For example, a full-featured "ifconfig" instead of ip + ethtool + iwconfig. Or consistent file-system snapshots since like forever on UFS (and ZFS, of course). I never understood how people in a commercial setup could run filesystem-level backups on a machine without that, like on Linux with ext4. It's just asking for trouble.
So, I'm happy to see this thread about FreeBSD here! Maybe we can make the Open Source scene a bit more diverse again with regards to operating systems…
It's an engineering tradeoff and market adaptation.
If you are a typical SaaS provider the complexity may be beyond your capabilities and budgets.. allegory to a local delivery business choosing to build a long term relationship with a single vehicle manufacturer and dealer.
If you are a high stakes service provider, you need to start thinking about how to get out from being controlled by a single vendor and market flux and plot your own destiny.. allegory to a national shipping carrier sourcing vehicles from multiple manufacturers and developing long term relationships with them to refine the platform.
Because tradeoffs and diminishing returns are a thing? For example having one backup is better than having no backups. Backups spread across earth, moon and mars would be better but probaboy not worth the cost.
It’s like driving a car. Most people can drive without needing to understand the internals, and it’ll still get them from point A to B. But just like some people enjoy diving deep into car mechanics, others enjoy understanding the intricacies of software.
For me, that deeper knowledge is an advantage. It helps me quickly evaluate tradeoffs between databases, debug at the OS level, or dismiss a library still relying on select if I expect heavy load. This insight saves time and increases efficiency.
> The largest failure was with btrfs — after a reboot, a 50 TB filesystem (in mirror, for backups) simply stopped working. No more mounting possible. Data was lost, but I had further backups. The client was informed and understood the situation. Within a few days, the server was rebuilt from scratch on FreeBSD with ZFS — since then, I haven’t lost a single bit.
As someone who admins a lot of btrfs, it seems very unlikely that this was unrecoverable. btrfs gets itself into scary situations, but also gets itself out again with a little effort.
In this instance “I solve problems” meant “I blow away the problem and start fresh”. Always easier! Glad the client was so understanding.
The first time I tried it out about 4 years old, I bricked it within a few days!! It was on a new (to me) Linux distro or maybe an existing one but I heard it was cool and the snapshots sounded neat.
I stayed away for a while but have it again on a Garuda install. I never completely give up on a technology, I hope they get it together.
As someone who has used it in my day job since 2014, I find it around 5% believable. I've had nasty performance issues on old kernels, but never a single instance of unrecoverable data loss, and I've run it in plenty of pathological cases. Experiences differ
> It's the single most unreliable untrustworthy filesystem I've used in the 21st century.
I think the “experiences differ widely” point makes sense with this comment too. Synology uses btrfs on the NAS systems it sells (there’s probably some option to choose another filesystem, but this is the default, AFAIK). If it were to be “the most unreliable untrustworthy filesystem” for many others too, Synology would’ve (or should’ve) chosen something else.
FWIW, this wasn't always the case. I recall that BTRFS reliability was much different, say, 10–15 years ago. The post touched those ancient times as well, so that isn't that much of a stretch.
Around that time, SLES made btrfs their default filesystem. It caused so many problems for users that they reversed that decision almost immediately.
I was pleased with my home lab btrfs, had a 12TB raid1, and the PSU rail connected to the backplane sometimes would go down under load. Many scary errors but never lost anything. Took me 2 months to debug and replace the PSU
Why do people use btrfs and similar filesystems for production use? They are by no means dumpster fires. But the internet is littered with stories of "X happened, then I realized Y & that I wasn't getting my data back"
Btrfs has some nice features - e.g. compression and snapshots, which i didn't knew i'd even like before using them. Not only they have saved me a few times from bad updates ("saved" in the sense that i was able to pretty much instantly revert, it saved time, i wouldn't lose anything even without btrfs), but they also help with things like "i'm going to run this script to process 29837894293 files - and the script might have some bugs in it, so i want to be sure i wont lose anything" (i.e. make snapshot, run script, check results, compare snapshot with current state to ensure nothing is lost, delete snapshot). Snapshots are also useful for diffing FS state, e.g. before and after installing some program.
As for the stories, AFAICT often the reason is that the user didn't know they could get their data back - or they are stories from many years ago when btrfs was buggy, but AFAIK those issues have been long solved (i think some specific case with some RAID setup still has issues but this is hearsay and AFAICT from the same hearsay, that setup isn't really necessary with btrfs in the first place).
Using btrfs is more complicated than using ext or something similar, especially since most tools that deal with files/filesystems are made only with ext-like features in mind - to the point where sometimes i wonder what the point is and i'm considering switching to ext3 or ext4, but then i remember snapshots and i'm like, nah :-P.
I like the idea and would like to learn more, but it looks like "migrated stuff without testing ahead of time and it turned out faster for some reason". Was it the memory allocations? Was it the disk latency? The hypervisor? Could it be replicated by other means? It was a fun read, but the reasoning/understanding was missing. I hope people investigate deeper before making changes like that.
If you look for benchmarks comparing databases on Linux/BSDs you'll find lots of nuance in practice and results going both ways depending on configuration and what's being tested.
No, after 20 years of use and comparative testing of similar setups. Frankly, I have always placed little importance on benchmarks as I consider them extremely specific. I am interested in real-world use cases.
The goal of the talk and the article is not to urge people to migrate all their setups, but simply to share my experience and the results achieved. To encourage the use of BSDs for their own purposes as well. It’s not to say that they are the best solution; there is no universal solution to all problems, but having a range of choices can only be positive.
All a man needs is food in his stomach and a place to rest at the end of the day. Everything else is vanity
What proportion of global GDP is dedicated to fulfilling our basic material needs?
It is mostly unnecessary. Inspite of the huge productivity gains made since the seventies, the current generation of young Americans are poorer than their parents and grandparents were at their age.
So what does all the IT optimization bring? Just more wealth for the owners and redundancies for their employees, including Joe Bloggs here.
It is time people in IT got to understand this. In the long term their activities are not going to improve their wealth. They are one of the few professions whose job is to optimize themselves out of a living, unless they own the means of the production their are optimizing, which they don't.
> So what does all the IT optimization bring? Just more wealth for the owners [...] It is time people in IT got to understand this
I understand it alright, but I'm trapped. Closer to 50 than to 40, I've got a family to run. I could be interested in another profession, but our daily lives & savings would tank if I stopped working, for learning another profession. Also, there's no other profession that I could realistically learn that would let me take nearly the same amount of money home every month. If someone lives alone, they could adjust their standard of living (-> downwards, of course); how do you do that for a family?
Furthermore, there is no switchover between "soulless software job for $$$" and "inspiring software job for $". There are only soulless jobs, only the $ varies. Work sucks absolutely everywhere; the only variable is compensation -- at best we can get one that "sucks less".
When I was a teenager, I could have never dreamt that programming would devolve into such a cruel daily grind for me. Mid-life crisis does change how we look at things, doesn't it. We want more meaning to our work (society has extremely decoupled livelihood from meaning), but there's just no way out. Responsibilities, real or imaginary, keep us trapped. I'd love to reboot my professional life, but the risks are extreme.
FWIW, I still appreciate interesting tasks at work; diving into the details lets me forget, at least for a while, how meaningless it all is.
The current generation of Americans are absolutely rich enough to just get food and have shelter. The ones that struggle are the ones that want to live in popular cities precisely because of the available improvements beyond shelter and food.
The houses of the 50s were shit tier and spread around the entire US. You can go buy them today for cheap in the 98% of locations people don’t want to live in.
He only works for himself only if he gets a steady income stream so long as the systems he manages stay up, which I'm sure he doesn't.
There is a reason for the "you will own nothing and you will be happy" ideology being promoted by "PTB", ie subscriptions for ink cartridges, heated seats and advanced suspensions in newer cars.
Corporations now want a continuous income stream from the services provide by physical products they have "sold", but they don't want their employees and subcontractors from earning some of that income stream.
Some IT administrators have been know to schedule regular "downtimes" on perfectly performing systems just to ensure their users and bosses don't take their service for granted.
I switched from FreeBSD to Linux, mainly because of the bad Java support and the simple fact that Linux became way more popular, which added to the difference in software availability.
FreeBSD has pretty good Java support? Sure if there is a new LTS release it takes a couple of months before it is ported to FreeBSD but that's about it?
For a long time they didn't have any. I recall from mid 90s version 4+? They were still trying to port.. or was it early 2000s lol either way it's ancient history
I've recently discovered systemd-nspawn which is an alternative to LXC, builtin and integrated into systemd. Much lighter than full VMs and it's quite similar to Solaris Zones and FreeBSD jails. One way to use it is to extract an OCI (Docker) image to a path, that way you can reuse the container tooling provided by Docker, Podman et al.
I've barely touched the BSDs and it's been a few years since I last used Solaris so I can't make much of a comparison as a user myself.
I've become a fairly loyal OpenBSD user in the last 3-4 years. The base OpenBSD load includes a substantial amount of network capabilities, and is cleanly implemented. It's almost too cleanly implemented, to the point of making me feel sort of guilty when I start to clutter up an install with a bunch of packages...
If my needs for storage were more complicated, I would probably use FreeBSD ZFS, but UFS suffices for my rather modest needs.
I use OpenBSD for desktop, web and mail services. There are some limitations, but none that are serious enough to warrant dealing with running another BSD, or Linux distribution.
I would differ on a small point. For SOHO usage, I think that docker compose is perfectly viable and often simplifies backup, migration and moving to a new server. Just my own take on this. A lot of apps really only need one instance with a good backup strategy and not hot failover instances and can handle an hour of down time once a year or two as needed, which I rarely experience.
As mentioned in the article, it also serves as a decent set of instructions, assuming the actual dockerfile(s) for the services and dependencies are broadly available. You can swap out the compose instance of PostgreSQL for your dedicated server with a new account/db, relatively easily. Similar for other non-app centered services (redis, rabbitmq, etc). You can go all in, or partly in and in any case it does serve as self-documenting to a large degree.
Readit News
At the very least it is nice to make acquaintance with at least one BSD because it will probably expand your knowledge on Linux in ways you wont be able to anticipate.
For example, FreeBSD got me into kernel development, full system debugging, network stack development, driver development, and understanding how the whole kit fits together. Those skills transferred back and forth with reasonable fidelity to Linux, and for me jumping into Linux development cold would have been too big a leap.. especially in confidence and developing a mental model.
For my personal infrastructure, I tend to use FreeBSD because in many ways it is simpler and less surprising, especially when accounting for the passage of time. ifconfig is still ifconfig, and it works great. rc.d is all I need for my own stuff. I like the systematic effects of things like tunables and sysctl for managing hardware and kernel configuration. The man pages are forever useful to new and old users. The kernel APIs and userland APIs are extremely stable akin to commercial operating systems and unlike Linux.
There are warts. There are community frictions. The desktop story and some developer experiences will be perpetually behind Linux due to the size of the contributor base and user base. The job market for BSD is very limited compared to Linux. But I don't think it's an all or nothing affair, and ideally in a high stakes operation you would dual stack for availability and zero-day mitigation (Verisign once gave a great talk on this).
That sounds very appealing to me. I have to keep a small number of servers running, but its not my main focus and I would like to spend as little time on it as possible.
I have started using Alpine Linux for servers (not for my desktop, yet) because it is light and simple. Maybe BSD will be the next step.
They also have things like `rpm` in ports that you can install. Why? Because you can enable linux binary compatibility[0] and run linux binaries on it (this implements the linux kernel interface, it's not a VM/emulator). It's also backward compatible with its own binaries back to FreeBSD 4 (circa 2000).
Though you may not need that as the ports/packages collection is pretty comprehensive.
It also comes with some nifty tools built-in for isolation (similar to but predating cgroups/containers) as "jails". It also has a hypervisor built in (bhyve) for virtualization if you do need to run any linux VMs or anything for any reason.
The way I usually sum up the difference to people is that FreeBSD is designed while Linux is grown. FreeBSD feels much more like a cohesive whole than Linux.
Really, the only reason I'm not running it everywhere is that the industry has kind of settled on linux-style containers for... absolutely everything, and the current solution for that on FreeBSD is basically "run linux in a VM".
[0] https://docs.freebsd.org/en/books/handbook/linuxemu/
I've migrated it through system upgrades and security fixes, but nothing else needed to change. usual uptime is about 3 years between major release updates.
freebsd is an awesome server platform.
Check it out: https://www.linuxfromscratch.org/
The CON is coming up, https://freebsdfoundation.org/news-and-events/event-calendar...
Fall 2024 FreeBSD Summit November 7-8, 2024 San Jose, CA
But I don't think we talk enough about the joy of not being surprised by updates. I'm about to do an upgrade from 13.2 to 14.1 this weekend and I am very confident that I won't have to worry about configuring a new audio subsystem, changing all my startup scripts to work with a new service manager paradigm, or half my programs suddenly being slow because they're some new quasi-container package format.
I mean, in addition to what kev009 mentioned, FreeBSD has so many great things to offer: For example, a full-featured "ifconfig" instead of ip + ethtool + iwconfig. Or consistent file-system snapshots since like forever on UFS (and ZFS, of course). I never understood how people in a commercial setup could run filesystem-level backups on a machine without that, like on Linux with ext4. It's just asking for trouble.
So, I'm happy to see this thread about FreeBSD here! Maybe we can make the Open Source scene a bit more diverse again with regards to operating systems…
If you are a typical SaaS provider the complexity may be beyond your capabilities and budgets.. allegory to a local delivery business choosing to build a long term relationship with a single vehicle manufacturer and dealer.
If you are a high stakes service provider, you need to start thinking about how to get out from being controlled by a single vendor and market flux and plot your own destiny.. allegory to a national shipping carrier sourcing vehicles from multiple manufacturers and developing long term relationships with them to refine the platform.
It really just fragments my knowledge to be honest.
Say "I gotta get things done".
Get me to a terminal. You've got Mac OS command line flags, GNU, BSD. Great.
Then it's some kind of asinine config to interact with some piece of software, all to achieve "generally the same thing", just a different way/flavor.
I really don't see the benefits.
For me, that deeper knowledge is an advantage. It helps me quickly evaluate tradeoffs between databases, debug at the OS level, or dismiss a library still relying on select if I expect heavy load. This insight saves time and increases efficiency.
As someone who admins a lot of btrfs, it seems very unlikely that this was unrecoverable. btrfs gets itself into scary situations, but also gets itself out again with a little effort.
In this instance “I solve problems” meant “I blow away the problem and start fresh”. Always easier! Glad the client was so understanding.
As someone who used it all day every day in my day job for 4 years, I find it 100% believable.
I am not saying you're wrong: I'm saying, experiences differ widely, and your patterns of use are not be universal.
It's the single most unreliable untrustworthy filesystem I've used in the 21st century.
I stayed away for a while but have it again on a Garuda install. I never completely give up on a technology, I hope they get it together.
I think the “experiences differ widely” point makes sense with this comment too. Synology uses btrfs on the NAS systems it sells (there’s probably some option to choose another filesystem, but this is the default, AFAIK). If it were to be “the most unreliable untrustworthy filesystem” for many others too, Synology would’ve (or should’ve) chosen something else.
Around that time, SLES made btrfs their default filesystem. It caused so many problems for users that they reversed that decision almost immediately.
As for the stories, AFAICT often the reason is that the user didn't know they could get their data back - or they are stories from many years ago when btrfs was buggy, but AFAIK those issues have been long solved (i think some specific case with some RAID setup still has issues but this is hearsay and AFAICT from the same hearsay, that setup isn't really necessary with btrfs in the first place).
Using btrfs is more complicated than using ext or something similar, especially since most tools that deal with files/filesystems are made only with ext-like features in mind - to the point where sometimes i wonder what the point is and i'm considering switching to ext3 or ext4, but then i remember snapshots and i'm like, nah :-P.
If you look for benchmarks comparing databases on Linux/BSDs you'll find lots of nuance in practice and results going both ways depending on configuration and what's being tested.
The goal of the talk and the article is not to urge people to migrate all their setups, but simply to share my experience and the results achieved. To encourage the use of BSDs for their own purposes as well. It’s not to say that they are the best solution; there is no universal solution to all problems, but having a range of choices can only be positive.
Bloke is not acquainted with Keynesian economics.
https://www.youtube.com/watch?v=9OhIdDNtSv0
https://www.youtube.com/watch?v=NO_tTnpof_o
All a man needs is food in his stomach and a place to rest at the end of the day. Everything else is vanity
What proportion of global GDP is dedicated to fulfilling our basic material needs?
It is mostly unnecessary. Inspite of the huge productivity gains made since the seventies, the current generation of young Americans are poorer than their parents and grandparents were at their age.
So what does all the IT optimization bring? Just more wealth for the owners and redundancies for their employees, including Joe Bloggs here.
It is time people in IT got to understand this. In the long term their activities are not going to improve their wealth. They are one of the few professions whose job is to optimize themselves out of a living, unless they own the means of the production their are optimizing, which they don't.
It is their employers that do.
I understand it alright, but I'm trapped. Closer to 50 than to 40, I've got a family to run. I could be interested in another profession, but our daily lives & savings would tank if I stopped working, for learning another profession. Also, there's no other profession that I could realistically learn that would let me take nearly the same amount of money home every month. If someone lives alone, they could adjust their standard of living (-> downwards, of course); how do you do that for a family?
Furthermore, there is no switchover between "soulless software job for $$$" and "inspiring software job for $". There are only soulless jobs, only the $ varies. Work sucks absolutely everywhere; the only variable is compensation -- at best we can get one that "sucks less".
When I was a teenager, I could have never dreamt that programming would devolve into such a cruel daily grind for me. Mid-life crisis does change how we look at things, doesn't it. We want more meaning to our work (society has extremely decoupled livelihood from meaning), but there's just no way out. Responsibilities, real or imaginary, keep us trapped. I'd love to reboot my professional life, but the risks are extreme.
FWIW, I still appreciate interesting tasks at work; diving into the details lets me forget, at least for a while, how meaningless it all is.
The houses of the 50s were shit tier and spread around the entire US. You can go buy them today for cheap in the 98% of locations people don’t want to live in.
Sounds like he understood it just fine. He owns the means of production.
There is a reason for the "you will own nothing and you will be happy" ideology being promoted by "PTB", ie subscriptions for ink cartridges, heated seats and advanced suspensions in newer cars.
Corporations now want a continuous income stream from the services provide by physical products they have "sold", but they don't want their employees and subcontractors from earning some of that income stream.
Some IT administrators have been know to schedule regular "downtimes" on perfectly performing systems just to ensure their users and bosses don't take their service for granted.
I also recall a problem with mmap() but I’m not sure if it was related to Java or something else
I've barely touched the BSDs and it's been a few years since I last used Solaris so I can't make much of a comparison as a user myself.
If my needs for storage were more complicated, I would probably use FreeBSD ZFS, but UFS suffices for my rather modest needs.
I use OpenBSD for desktop, web and mail services. There are some limitations, but none that are serious enough to warrant dealing with running another BSD, or Linux distribution.
As mentioned in the article, it also serves as a decent set of instructions, assuming the actual dockerfile(s) for the services and dependencies are broadly available. You can swap out the compose instance of PostgreSQL for your dedicated server with a new account/db, relatively easily. Similar for other non-app centered services (redis, rabbitmq, etc). You can go all in, or partly in and in any case it does serve as self-documenting to a large degree.