Readit News logoReadit News
mononcqc commented on A Pipeline Made of Airbags   ferd.ca/a-pipeline-made-o... · Posted by u/packetlost
thih9 · a year ago
Would a middle ground be possible? E.g. by default use stateless containers, but for certain stacks or popular app frameworks support automated stateful deploys?
mononcqc · a year ago
Two years after writing A Pipeline Made of Airbags, I ended up prototyping a minimal way to do hot code loading from kubernetes instances by using generic images and using a sidecar to load pre-built software releases from a manifest in a way that worked both for cold restarts and for hot code loading: https://ferd.ca/my-favorite-erlang-container.html

It's more or less as close to a middle-ground as I could imagine at the time.

mononcqc commented on Erlang's not about lightweight processes and message passing   github.com/stevana/armstr... · Posted by u/stevan
jerf · 3 years ago
There is a very common pattern in the world where people conflate goals with results. Of course, when I say that, it's obvious that the two things aren't the same, but the unexamined assumption that they are the same sneaks in anyhow when you aren't looking, a cognitive shortcut hard to catch yourself making and harder yet to get in front of and deal with.

In the light of this statement, the answer to what I think is the thesis question of that entire piece:

"This begs the question: why aren't language and library designers stealing the structure behind Erlang's behaviours, rather than copying the ideas of lightweight processes and message passing?"

Is that while Erlang has a lot of good goals, the results of how they got there are simply not the state of the art. Or, to put it another way, language designers are not copying Erlang, and they are correct to not copy Erlang.

I respect Erlang a lot. They were a good 10-15 years ahead of their time. However, you will note that if you add 10-15 years to the creation date of Erlang, you still end up in the past. If Erlang were to come out today, fresh, nobody had seen it before, into an otherwise identical programming language environment, I would say it makes several mistakes.

One I've written about before is that Erlang is a non-Algol language for no reason: https://news.ycombinator.com/item?id=7277957 (Note the library mentioned in that post, suture, is now mature, and I use it all the time. It works for what I need.) But in the context of this post, that's less critical.

The other major mistake I'd say it made if it came out in 2023 is that it is a totalizing environment. By that I mean that it has this built in implicit assumption that it is responsible for all the reliability in the system, and you don't get Erlang's features very conveniently if you don't use it as the complete control backplane for your entire system. You run an Erlang cluster, and it bundles all the message passing, restarting, reliability, cluster management, software deploy, and everything into one system.

But for the world we live in today, that's a price not worth paying. We don't need the Erlang message bus to be the only message bus. The Erlang message bus is, frankly, not very good, and it's actively terrible if you want to use it for one non-Erlang process to communicate to another. We don't need the Erlang message bus. We have a dozen message busses, some in the cloud, some commercial, some open source, some that double as retention (Kafka), all of which scale better, none of which tie you to Erlang's funky data types.

And then, within the matrix of those message busses, we don't need Erlang's restart capability. We have an abundance of ways to restart processes, from systemd, to kubernetes, to any number of other ways.

We don't need Erlang clusters for redundancy any more. You just run multiple copies of a service against the message bus, on multiple systems for redundancy.

We don't need Erlang's behaviors. We have interfaces, traits, object orientation, and even just pushing that entire problem up to the OS process level, or writing a cloud function, and any number of ways of achieving the same goal.

Erlang's software deploy is interesting, but we have a lot of options for it. The whole attempt to do live updates is interesting, but it also imposed a lot of constraints that systems that don't have that need, which is the vast majority of them, don't need or want. This is perhaps the space where the state of the art isn't that far ahead of Erlang. It's still a mess, despite all the churn in this space. But even so, with all the options available, you can probably find something better for your system than the Erlang way of upgrading software, even if it isn't necessarily much easier.

The cognitive hazard that Erlang presents the community in 2023 is that it has some very good writing on the topic of reliability and its other goals, and then, naturally, one segues into the discussion of how Erlang solved the problem. And it was a very interesting solution for the time. I used Erlang for many, many years back when it was effectively the only solution to these problems.

But it isn't the only solution anymore. The space has exploded with options. Unsurprisingly, the ones that a highly innovative pioneer tried out first are not the best, or the only. They chose well. Let me again emphasize my respect for the project. But it's not on the cutting edge anymore.

Granted, the diversity of options does mean the real world has gotten quite chaotic, where you may have three message busses attaching systems implemented in a dozen different languages, but that's something you can take up with Conway's Law. Erlang couldn't work with Conway's Law without totally converting your entire company to it, which just isn't going to happen.

The reason why language designers aren't rushing to copy Erlang is that what was excellent and amazing in 2000 (and, again let me underline, I mean that very seriously, it was a cutting edge platform built with a lot of vision and moxie) is, in 2023, mediocre. Erlang is a mediocre language (Elixir is at least "good", Erlang is mediocre), attached to a mediocre message bus, with a type system that doesn't even reach mediocre, with a mediocre totalizing approach to system design where there's a very significant impedence mismatch between it and the rest of the world, with an at-par-at-best VM (I won't call that mediocre, but where it used to be head-and-shoulders above everything else in certain ways, it is now merely competitive), with mediocre standard libraries, and a mediocre product fit to its own stated goals. It just isn't the best any more.

The state of the art right now is super chaotic. I can hardly get two systems deployed on the same infrastructure any more, because there's always some reason something in that list has changed. But when the chaos settles and best practices emerge, something that I'd say is at least a good 5 years away, the result will clearly have Erlang inspiration in it for sure... but it won't look a lot like Erlang on the surface.

What is worth copying has largely been copied. It doesn't look exactly like Erlang, but this turns out to be a good thing.

mononcqc · 3 years ago
I'm not quite sure how Erlang's world is totalizing. It has ways to ship things in a very integrated manner, but I have shipped and operated Erlang software that was containerized the same as everything else, in the same K8s cluster as the rest, with the same controls as everything else, with similar feature flags and telemetry as the rest, using the same Kafka streams with the same gRPC (or Thrift or Avro) messages as the rest, invisibly different from other applications in the cluster to the operator in how they were run aside from generating stack traces that look different when part of it crashes.

That it _also_ ships with other ways of doing things in no way constrains or limits your decisions, and most modern Erlang (or Elixir) applications I have maintained ran the same way.

You still get message passing (to internal processes), supervision (with shared-nothing and/or immutability mechanisms that are essential to useful supervision and fault isolation), the ability to restart within the host, but also from systemd or whatever else.

None of these mechanisms are mutually exclusive so long as you build your application from the modern world rather than grabbing a book from 10-15 years ago explaining how to do things 10-15 years ago.

And you don't _need_ any of what Erlang provides, the same way you don't _need_ containers (or k8s), the same way you don't _need_ OpenTelemetry, the same way you don't _need_ an absolutely powerful type system (as Go will demonstrate). But they are nice, and they are useful, and they can be a bad fit to some problems as well.

Live deploys are one example of this. Most people never actually used the feature. Those who need it found ways (and I wrote one that fits in somewhat nicely with modern kubernetes deployments in https://ferd.ca/my-favorite-erlang-container.html) but in no way has anyone been forced to do it. In fact, the most common pattern is people wanting to eventually use that mechanism and finding out they had not structured their app properly to do it and needing to give it a facelift. Because it was never necessary nor totalizing.

Erlang isn't the only solution anymore, that's true, and it's one of the things that makes its adoption less of an obvious thing in many corners of the industry. But none of the new solutions in the 2023 reality are also mutually exclusive to Erlang. They're all available to Erlang as well, and to Elixir.

And while the type system is underpowered (and there are ongoing area of research there -- I think at least 3-4 competing type systems are being developed and experimented with right now), that the syntax remains what it is, I still strongly believe that what people copied from Erlang were the easy bits that provide the less benefit.

There is still nothing to this day, whether in Rust or Go or Java or Python or whatever, that lets you decompose and structure a system for its components to have the type of isolation they have, a clarity of dependency in blast radius and faults, nor the ability to introspect things at runtime interactively in production that Erlang (and by extension, languages like Elixir or Gleam) provide.

I've used them, I worked in them, and it doesn't compare on that front. Regardless of if Erlang is worth deploying your software in production for, the approach it has becomes as illuminating as the stacks that try and push concepts such as lack of side-effects and purity and what they let you transform in how you think about problems and their solutions.

That part hasn't been copied, and it's still relevant to this day in structuring robust systems.

mononcqc commented on AI converts between hundreds of programming languages   codelanguageconverter.com... · Posted by u/yodon
mononcqc · 3 years ago
I tried it with Erlang. Their "Sample Code 2" does not generate working code (lowercase variable assignments), and their "Sample Code 3" does not even generate valid syntax, flat out leaving bits of typescript untranslated.
mononcqc commented on Erlang/OTP 25.0 Release   erlang.org/news/157... · Posted by u/nifoc
_randyr · 3 years ago
> the 'with' construct allows a list of conditional patterns and then a general 'do' block, whereas the Erlang 'maybe' allows mixed types of expressions that can either be conditional patterns or any normal expression weaved in together at the top level.

This seems slightly incorrect to me. You can write expressions in Elixir's with macro too, by simply swapping the arrow for an equals sign. For example, this is perfectly valid Elixir code:

    with {:ok, x} <- {:ok, "Example"},
         IO.puts(x),
         len = String.length(x) do
      IO.puts(len)
    end
Did you mean something else?

mononcqc · 3 years ago
See https://news.ycombinator.com/item?id=31425298 for a response, since this is a duplicate. TL:DR; I had never seen it and had no idea it was possible because I don't recall seeing any documentation or post ever mentioning it! Ignorance on my part.
mononcqc commented on Erlang/OTP 25.0 Release   erlang.org/news/157... · Posted by u/nifoc
OkayPhysicist · 3 years ago
The with statement in Elixir already allows for abitrary expressions between the with and the do. I'm not sure what I'm missing here.
mononcqc · 3 years ago
You're right. After all these years (and even writing a book that had Elixir snippets in it) I had never seen a single example showing it was possible and did not know it could do it.

Well there you go, I guess the pattern is equivalent but incidental.

mononcqc commented on Erlang/OTP 25.0 Release   erlang.org/news/157... · Posted by u/nifoc
rkangel · 3 years ago
It is very much that. The Erlang team have not been too proud to steal good ideas from Elixir. Elixir has been a good source of fresh thinking for the BEAM ecosystem which has helped both the Erlang and Elixir side.
mononcqc · 3 years ago
The Erlang 'maybe' expression expands on what 'with' allows in Elixir, mostly because the 'with' construct allows a list of conditional patterns and then a general 'do' block, whereas the Erlang 'maybe' allows mixed types of expressions that can either be conditional patterns or any normal expression weaved in together at the top level.

It is therefore a bit more general than Elixir's 'with', and it would be interesting to see if the improvement could feed back into Elixir as well!

The initial inspiration for the 'maybe' expression was the monadic approach (Ok(T) | Error(T)) return types seen in Haskell and Rust, and the first EEP was closer to these by trying to mandate the usage of 'ok | {ok, T}' matches with implicit unwrapping.

For pragmatic reasons, we then changed the design to be closer to a general pattern matching, which forced the usage of 'else' clauses for safety reasons (which the EEP describes), and led us closer to Elixir's design, which I felt was inherently more risky in the first drafts (and therefore I now feel the Erlang design is riskier as well, albeit while being more idiomatic).

So while I did get inspiration from Elixir, and particularly its usage of the 'else' clause for safety reasons, it would possibly be reductionist to say that "the good ideas were stolen from Elixir." The good ideas were stolen from Elixir, but also from Rust, Haskell, OCaml, and various custom libraries, which have done a lot of interesting work in value-based error handling that shouldn't be papered over.

I still think these type-based approaches represent a significantly positive inspiration that we could ideally move closer to, if it were possible to magically transform existing code to match the stricter, cleaner, more composable patterns that they offer.

In the end I'm hoping the 'maybe' expression still provides significantly nicer experiences in setting up business logic conditions in everyday code for Erlang user, and it is of course impossible to deny that I got some of the form and design caveats from the work done in the Elixir language already :)

Also as a last caveat: I am not a member of the Erlang/OTP team. The design however was completed and refined with their participation (and they drove the final implementation whereas I did the proof of concept with Peer Stritzinger and wrote the initial EEP), but the stance expressed in my post here is mine and not the one of folks at Ericsson.

mononcqc commented on Errors are constructed, not discovered   ferd.ca/errors-are-constr... · Posted by u/Tomte
blueflow · 3 years ago
Nothing you wrote is about errors in itself, you exclusively talk about the social interpretations/construction of it. These are things that every adult needs to be able to improvise and express on-the-fly, nothing 'static' that could be fruitfully discussed independently of specific occurrences.

Late edit: So, when interpreted in my context, its utter crap! Your article is shit! You explicitly invited your readers to disregard cross-checking against your own intentions[1], saying they "don't matter". Its basically a free card for me to project my prejudices onto you, without you having anything to conter it. You are a fragile individual who does not think things through.

[1] https://ferd.ca/inclusiveness-in-language-for-outsiders-look..., Section "Death of the Author"

mononcqc · 3 years ago
So this disagreement is an interesting example of error being a construction in itself: I will posit that you are wrong or not interpreting my point of view properly, and you will say that I am wrong or possibly not explaining things properly.

There is an objectively quantifiable disagreement. But its nature (and even whether it is desirable or not) are possibly camped in subjective terms. Of course you could argue that I am objectively wrong — though trying to prove that with my own writings is risky since we’ve established I’m not a trustworthy source — but that in itself does not resolve the overall disagreement from existing.

This sort of situation can also happen in software where an ambiguous specification yields two distinct compliant implementations that nevertheless do not work together.

mononcqc commented on Errors are constructed, not discovered   ferd.ca/errors-are-constr... · Posted by u/Tomte
c3534l · 3 years ago
It seems like you're conflating blame with bugs.
mononcqc · 3 years ago
Well if all you have is bugs but you never look into how they get there and how they happen, you’re not going to improve your process or tools much.

Also blame and accountability and responsibility are all subtly different.

mononcqc commented on Errors are constructed, not discovered   ferd.ca/errors-are-constr... · Posted by u/Tomte
c3534l · 3 years ago
This doesn't seem mysterious or confusing to me. What happened wasn't the launch of a new random power outage feature - it was a problem that needs to be fixed.
mononcqc · 3 years ago
Fixing the bug does not necessarily fix anything else it caused though, and none of the questions in my post are answered by “fix it”.

Here’s another one: should the incident actually have an impact on anyone of the companies employees yearly reviews? Either positive or negative? Why?

mononcqc commented on Errors are constructed, not discovered   ferd.ca/errors-are-constr... · Posted by u/Tomte
blueflow · 3 years ago
This story makes me see a pattern in this author's writing: While looking at the world, they see that something is off, that something is not as one would to be expected, accurately points it out the dissonance, and then draws really weird conclusions that do not seem conclusive to me.

Yes, errors are not objective, but they aren't constructed, either. Our expectations are constructed (docs and standards likewise), and the mismatch to it (the error) is something objective that can be discovered.

Really odd framing...

mononcqc · 3 years ago
To illustrate the point, let's imagine that 5 years ago, a driver in the linux kernel was written in such a way that interplay with a new disk drive could corrupt data when power failed. The disk on its own (and with other OSes) is fine, and the linux kernel in all other cases are fine.

An open-source database is being used and operated as a service by a vendor, which a SaaS company relies on to provide a feature that your organization uses to manage data on behalf of users.

We now have a chain that includes: users <- customer organisation <- SaaS vendor <- DB as a service vendor <- OSS DB maintainers <- Linux maintainers <- Driver writers <- Hardware vendors.

There is suddenly a power outage at the DB as a service vendor (because of an unmaintained powerline falling over) and their UPCs appear not to be functional for yet unknown reason (cost cutting or supply chain issues during covid time may receive some blame). Your users lose their data regardless.

What is the bug? Who is at fault? Is it the engineer? The team who wrote the code? The QA folks? The organization that hired them? Who should fix the issue? Who should be on charge with repairing data corruption? Whose backups should be trusted most? The least? Have you been lenient in your usage of a SaaS vendor? Has the SaaS vendor been lenient in the services they use? Which actors can be considered liable from a legal standpoint? Which actors can be considered liable from an ethical or moral standpoint? Are your customers the one who made a bad decision contracting you? Can there be more than one party responsible? Do any of these answers changed based on whether the power loss is caused by an act of god or bad maintenance? Based on which jurisdiction you're in? How do you define honest mistakes? Negligence? A bug is a bug because the software did not meet the expectations that were set. Were the expectations reasonable? Who should have managed them?

Events happen. The meaning we attach to them is of course based on expectations and standards and the environment and context, but the way we build our explanation, the ways we attach blame and accountability varies. You can sometimes decide to assign accountability to individuals, sometimes to systems, sometimes both. Sometimes only some or sometimes neither.

So sure, you can point at the actual technical lines of code and say "these aren't doing what they should", but if you do this in a vacuum without also wondering who decided what these lines should be doing and what pressures were at play when they were written, are you necessarily learning a lot about how events unfold and how they might unfold in the future?

A systemic perspective will yield different reactions than one based on personal engineer responsibility, which will be different from one that looks at it from an insurer's point of view, which will be different from one which looks at it from an education point of view, etc. So the lens you take to look at the events surrounding the error and the interpretation you make of it are absolutely crucial to the corrections and learnings that follow.

u/mononcqc

KarmaCake day2658March 17, 2009
About
Blog: https://ferd.ca books: https://learnyousomeerlang.com, https://erlang-in-anger.com, https://propertesting.com
View Original