This is a good place as any to ask: I'm an experienced Elixir dev so I know my way around Supervisors, GenServers, Registries, etc. but I feel there is more to OTP, the myriad of erlang modules for fault tolerance, which are not immediately visible unless you spend a lot of time reading the erlang docs.
Is there a good book that is focused on advanced OTP? On the internals of the BEAM and the hairier parts of the Erlang ecosystem? I can only find basic to intermediate introductory material. Perhaps I should just spend time reading the erlang website back-to-front...
---
1: I'm taking a small break from writing a parallel, distributed website crawler with homegrown HTTP client. Quite fun, very fast, to replace my first, rough prototype written in Rust. The hard part is accepting that most HTTP servers on the open Internet are hopelessly broken in a million ways.
> most HTTP servers on the open Internet are hopelessly broken in a million ways.
20 years ago, picking up OCaml, as an initial project I decided to write a password cracker to run against my WiFi router. The webserver on my WRT-54G was sending back errors, but Yahoo's webserver (more popular than Google at the time) was perfectly happy replying correctly to my HTTP client's requests.
I had to break out Ethereal (since renamed Wireshark) to debug why Yahoo was fine with my HTTP client but my WRT-54G wasn't. It turns out I didn't realize that OCaml character escapes that began with zero were still decimal, not octal (unlike C). So, I was sending a line terminator of vertical-tab shift-in instead of carriage-return line-feed. I guess kudos to whomever wrote Yahoo's webserver to make proper sense of such wonky requests, though I think it was far too liberal in what it accepted.
> I guess kudos to whomever wrote Yahoo's webserver to make proper sense of such wonky requests, though I think it was far too liberal in what it accepted.
Depending on exactly when, you may have been hitting Yahoo's custom http server originally written by David Filo or yapache, a patched version of Apache 1.3.
Being taught, or at least mentioned, in schools with programming courses.
Resources that target young people or beginner (we already have some, but more, and more widespread).
Advertising outside the Elixir circle (Chris McCord asked the other day what podcasts would be a good fit for that).
Python has a major edge in France, being defined as the reference language (teached at high school and engineering schools), and as such it sets the frame.
For now, when I mention Elixir around me, although there is interest, the feedback is that it seems like a great language for connoisseurs and people with special needs (e.g. very high concurrency), which most people have no need for.
I personally think it's a great language overall, including from a maintenance point of view, and I try to advertise it as such.
Source: a dad with a 15-yo son who is totally into programming, and quickly saw that Python is everywhere together with Javascript (from a YouTube / blog posts / discord point of view). He doesn't understand why I "bother" with Elixir :-)
As someone who's enough of an early adopter to have been running Rust in production since ~2016, and who takes a good look at Elixir every few years, there are two major things holding me back:
1. Lack of strong typing. Dynamic typing is great on 10,000 line projects with two developers. Once you reach 50,000 lines and have a little developer turnover, it stops being fun. And dynamic typing also usually means far less IDE support. I'm spoiled; I want auto-complete.
2. Elixir is amazing for UIs like Discord, where your users interact in real time. If each of your users lives in their own little world, Elixir's greatest strengths have less chance to shine.
Right now, TypeScript is a no-brainer, because React is the dominant choice for UIs, and using the same language on both client and server is a win. And frankly, TypeScript's lovely, even if the ecosystem is a constantly-churning trash fire. So for a variety of commercial projects, TypeScript is a responsible choice. Typed Python, C#, Java, Kotlin, Swift or Rust all have compelling use cases, too.
But given stronger typing and an project that needed Elixir's strong support for distributed systems, I'd definitely consider Elixir.
Strong typing would not be an unwelcome addition to the Erlang space but compared to most dynamically typed languages I find Erlang/Elixer to be infinitely more usable. If you aren't using guards and specific type pattern matching you arguably are doing it wrong. And those things give you much of the type info you need while developing. No compile time help but it's a lot better than most other dynamic languages. I've built very large projects in Erlang and didn't find typing a real issue.
I'm still new to Elixir, and see Erlang from over here in Elixir land. Recently on Twitter there were debates on static vs. dynamic typing, and the static typists said that it made software more reliable. Which made me think about all the literature I see about dynamically-typed Erlang has been making the most reliable software around for 30 years. I don't know how large Ericsson's telephone switching software is but I think it's more that 50,000 lines.
> The power of a type system, the expressiveness of functional programming, and the reliability of the highly concurrent, fault tolerant Erlang runtime, with a familiar and modern syntax.
Haven't used it, but I'd like to hear about others experience.
Honestly, I think a lot of Erlang's issues are superficial (some people find its syntax to be ugly and a bit warty). The other thing it seems to suffer from is the chicken and the egg of it's not popular because it's not popular. It is also a poor fit for certain domains, such as mobile apps.
- Ericsson (they designed Erlang) is using it for its mobile products/networks. Approx 40% of all mobile traffic worldwide goes through Ericsson's infra.
- WhatsApp used/uses Erlang - again millions (billions?) of users
- Bet365 / Klarna use Erlang - millions of users
- from OSS, RabbitMQ, is written in Erlang
I could go on... Erlang has great adoption, but it's probably smaller than, e.g. Python, due to Erlang's specific applications.
I thought Armstrong wrote that Ericsson replaced Erlang with C++ or some such thing, because they didn’t want to maintain a language and wanted something more off-the-shelf.
I’m fairly new to elixir, but the two things I liked the least were having to use asdf to get a working version of it that didn’t fight with the language server, and really the language server not being particularly smart or fleshed out. It’s not catastrophic, but there are many other languages that are much smoother. Particularly the auto-formatting leaves a lot to be desired. Thankfully these really aren’t existential problems because the language itself is so pleasant to use.
Here is the list of things that I feel are still holding Elixir back from wider adoption:
1. Not enough companies sharing their experience - most companies want to lead and not follow. There has been a lot of success with Elixir but too often I've heard companies not wanting to indicate they're using Elixir out of concern that it will tip off their competition (I've heard this from two companies now, big ones) or they just don't have a culture of sharing. The most impactful case study on Elixir was one of the first, the Bleacher Report case study https://www.erlang-solutions.com/case-studies/bleacher-repor.... They did an excellent job of articulating the direct business value of transitioning from Rails to Phoenix. We need more of these stories shared
2. Lack of talent - this is a chicken or egg problem but one thing that comes up quite often. If there were more companies actively hiring in Elixir there would be more devs learning Elixir but companies are reluctant to hire for Elixir because there aren't enough devs. Most of the courses/guides are aimed at transitioning Senior level talent. At DockYard through our Academy https://academy.dockyard.com/ we decided to tackle this from the opposite end and commit to producing high quality junior Elixir engineers.
3. Higher price point for talent - the recent Stack Overflow surveys regularly rank Elixir at the top for cost. Personally I think this stat is being influenced by the fact that Elixir has a higher % of more Senior level talent so the average salary will be skewed higher. But that nuance doesn't come through in the cost rankings. If true then as the ecosystem's talent pool starts to normalize we should see this cost average out.
4. Lack of off-the-shelf solutions - Objectively Elixir is faster to build and less costly to maintain than nearly every other tech stack I've ever used. However, because it is so easy to build something we have developed a culture of "just make it from scratch" too often. When newbies ask "how do I do X" questions like auth or something else in another tech stack that they can simply add a node module or a Ruby gem in Elixir most times we lack those pre-built solutions. So the actual build cost is higher because we can't simply drop that solution in and accelerate the dev process.
I've been selling Elixir services for almost a decade at this point and I'm still very bullish on Elixir. We, along with many others, have been trying to close these gaps. It's happening but the most impact anyone reading this can make that is using Elixir is to share your experiences. Write blog posts. Publish case studies. Indicate the reduction in cost, faster times to market, etc... these are the data points that are going to be most compelling and important in the coming decade.
I maintain erlang libraries. I contributed to OTP. I am from Elixir.
I cannot agree. Erlang is a pita to with. From the lackluster test libraries, the lack of module aliasing, the atrocious UX of parse transform, and so much more.
Erlang bring nearly nothing that Elixir cannot do and the UX of elixir is by far better.
While I agree with the sentiment, there are two superpowers in the mix for me (pun intended):
1. Erlang / BEAM itself as the runtim with lots of powerful capabilities (what you probably are talking about) - you need to know this stuff at some point to fully leverage the technology
2. Elixir the language (+ excellent modern tooling) ontop, which really helps to significantly speed up productivity/average code quality - I don't like to over-reductionist approach here, DX is also an important thing to consider instead of the raw foundation
Elixir brings so much to the BEAM ecosystem with the improved UX from the tooling and libraries. If you want a better language on the BEAM go with LFE in my opinion.
Maintenance patch apparently, good work team. Looks like they fixed a few shell issues as well but afaict `ctrl-d` to end session hasn't made the cut again uff.
Readit News
Is there a good book that is focused on advanced OTP? On the internals of the BEAM and the hairier parts of the Erlang ecosystem? I can only find basic to intermediate introductory material. Perhaps I should just spend time reading the erlang website back-to-front...
---
1: I'm taking a small break from writing a parallel, distributed website crawler with homegrown HTTP client. Quite fun, very fast, to replace my first, rough prototype written in Rust. The hard part is accepting that most HTTP servers on the open Internet are hopelessly broken in a million ways.
20 years ago, picking up OCaml, as an initial project I decided to write a password cracker to run against my WiFi router. The webserver on my WRT-54G was sending back errors, but Yahoo's webserver (more popular than Google at the time) was perfectly happy replying correctly to my HTTP client's requests.
I had to break out Ethereal (since renamed Wireshark) to debug why Yahoo was fine with my HTTP client but my WRT-54G wasn't. It turns out I didn't realize that OCaml character escapes that began with zero were still decimal, not octal (unlike C). So, I was sending a line terminator of vertical-tab shift-in instead of carriage-return line-feed. I guess kudos to whomever wrote Yahoo's webserver to make proper sense of such wonky requests, though I think it was far too liberal in what it accepted.
Depending on exactly when, you may have been hitting Yahoo's custom http server originally written by David Filo or yapache, a patched version of Apache 1.3.
1) The BEAM book - https://blog.stenmans.org/theBeamBook/
2) Erlang and OTP in Action by Merritt, Logan and Carlson
* In Erlang/OTP 27 we're getting triple-quoted strings, so this patch adds a warning for "accidental" triple-quoted strings.
* In Erlang/OTP 27 exact comparison of 0.0 and -0.0 will change so there is a warning for that as well to help detect issues earlier.
* FIPS support in crypto module for OpenSSL 3.x
Resources that target young people or beginner (we already have some, but more, and more widespread).
Advertising outside the Elixir circle (Chris McCord asked the other day what podcasts would be a good fit for that).
Python has a major edge in France, being defined as the reference language (teached at high school and engineering schools), and as such it sets the frame.
For now, when I mention Elixir around me, although there is interest, the feedback is that it seems like a great language for connoisseurs and people with special needs (e.g. very high concurrency), which most people have no need for.
I personally think it's a great language overall, including from a maintenance point of view, and I try to advertise it as such.
Source: a dad with a 15-yo son who is totally into programming, and quickly saw that Python is everywhere together with Javascript (from a YouTube / blog posts / discord point of view). He doesn't understand why I "bother" with Elixir :-)
1. Lack of strong typing. Dynamic typing is great on 10,000 line projects with two developers. Once you reach 50,000 lines and have a little developer turnover, it stops being fun. And dynamic typing also usually means far less IDE support. I'm spoiled; I want auto-complete.
2. Elixir is amazing for UIs like Discord, where your users interact in real time. If each of your users lives in their own little world, Elixir's greatest strengths have less chance to shine.
Right now, TypeScript is a no-brainer, because React is the dominant choice for UIs, and using the same language on both client and server is a win. And frankly, TypeScript's lovely, even if the ecosystem is a constantly-churning trash fire. So for a variety of commercial projects, TypeScript is a responsible choice. Typed Python, C#, Java, Kotlin, Swift or Rust all have compelling use cases, too.
But given stronger typing and an project that needed Elixir's strong support for distributed systems, I'd definitely consider Elixir.
> The power of a type system, the expressiveness of functional programming, and the reliability of the highly concurrent, fault tolerant Erlang runtime, with a familiar and modern syntax.
Haven't used it, but I'd like to hear about others experience.
https://elixir-lang.org/blog/2023/09/20/strong-arrows-gradua...
- Ericsson (they designed Erlang) is using it for its mobile products/networks. Approx 40% of all mobile traffic worldwide goes through Ericsson's infra.
- WhatsApp used/uses Erlang - again millions (billions?) of users
- Bet365 / Klarna use Erlang - millions of users
- from OSS, RabbitMQ, is written in Erlang
I could go on... Erlang has great adoption, but it's probably smaller than, e.g. Python, due to Erlang's specific applications.
in the programming community as a whole erlang/elixir are not very common.
it's important to ack this distinction becuase we still have a hill to climb with spreading awareness for OTP.
1. Not enough companies sharing their experience - most companies want to lead and not follow. There has been a lot of success with Elixir but too often I've heard companies not wanting to indicate they're using Elixir out of concern that it will tip off their competition (I've heard this from two companies now, big ones) or they just don't have a culture of sharing. The most impactful case study on Elixir was one of the first, the Bleacher Report case study https://www.erlang-solutions.com/case-studies/bleacher-repor.... They did an excellent job of articulating the direct business value of transitioning from Rails to Phoenix. We need more of these stories shared
2. Lack of talent - this is a chicken or egg problem but one thing that comes up quite often. If there were more companies actively hiring in Elixir there would be more devs learning Elixir but companies are reluctant to hire for Elixir because there aren't enough devs. Most of the courses/guides are aimed at transitioning Senior level talent. At DockYard through our Academy https://academy.dockyard.com/ we decided to tackle this from the opposite end and commit to producing high quality junior Elixir engineers.
3. Higher price point for talent - the recent Stack Overflow surveys regularly rank Elixir at the top for cost. Personally I think this stat is being influenced by the fact that Elixir has a higher % of more Senior level talent so the average salary will be skewed higher. But that nuance doesn't come through in the cost rankings. If true then as the ecosystem's talent pool starts to normalize we should see this cost average out.
4. Lack of off-the-shelf solutions - Objectively Elixir is faster to build and less costly to maintain than nearly every other tech stack I've ever used. However, because it is so easy to build something we have developed a culture of "just make it from scratch" too often. When newbies ask "how do I do X" questions like auth or something else in another tech stack that they can simply add a node module or a Ruby gem in Elixir most times we lack those pre-built solutions. So the actual build cost is higher because we can't simply drop that solution in and accelerate the dev process.
I've been selling Elixir services for almost a decade at this point and I'm still very bullish on Elixir. We, along with many others, have been trying to close these gaps. It's happening but the most impact anyone reading this can make that is using Elixir is to share your experiences. Write blog posts. Publish case studies. Indicate the reduction in cost, faster times to market, etc... these are the data points that are going to be most compelling and important in the coming decade.
Elixir is sort of a gateway drug to Erlang, sooner or later you want the real thing.
I cannot agree. Erlang is a pita to with. From the lackluster test libraries, the lack of module aliasing, the atrocious UX of parse transform, and so much more.
Erlang bring nearly nothing that Elixir cannot do and the UX of elixir is by far better.
And I am not talking of syntax here.
1. Erlang / BEAM itself as the runtim with lots of powerful capabilities (what you probably are talking about) - you need to know this stuff at some point to fully leverage the technology
2. Elixir the language (+ excellent modern tooling) ontop, which really helps to significantly speed up productivity/average code quality - I don't like to over-reductionist approach here, DX is also an important thing to consider instead of the raw foundation
Without it, to run RabbitMQ in a Windows containers, you have to run the enormous Server Core images.
A niche issue, I know, but a constant thorn in my side!
[0] https://github.com/erlang/docker-erlang-otp/issues/360