It feels like problem with these aren't the ideas, rather the "let's just" approach/expectation in front of them.
For instance "Let's just add an API." I think the approach to an API as "just" a feature to your product will be about as successful as saying "let's just add a UI". To implement a successful UI one needs to be thoughtful, thorough, and bring in people who specialize in it. Why should any other interface for your product be any different? It's not that it's a bad, or good, idea, rather one that shouldn't "just" be done.
At a previous employer we had a rule. The only person allowed to say, "just", was the developer responsible for actually making it work. Another developer who said it, just volunteered themselves.
> “Precisely. It’s what I call a ‘Lullaby Word.’ Like ‘should,’ it lulls your mind into a false sense of security. A better translation of ‘just’ in Jeff’s sentence would have been, ‘have a lot of trouble to.'”
Ok so if you have a problem and the developer thinks that the solution to it is to add an API, all they have to do to get their way is to say "let's just add an API!", although they would of course be the one that had to make it work.
Yes I like this rule, how many things I could have solved at the worst run place I ever worked at if it had been implemented there. Although I guess the management might not have liked that essentially I could choose what got done and what I got to work on just by running my big mouth.
facetiousness aside, I'm trying to point out that most often when people say "let's just" it is because they are advocating a course of action and that rule won't work unless the course they are advocating gets chosen. If they say "let's just make an api" and you say no we're doing a UI and you will work on it because you used the word "Just" I guess that's a way to lose your tech talent relatively quickly.
I am totally going to advocate for this at my workplace. I myself am guilty of it - not saying "just", but questioning people's estimates based on my limited understanding of the research they had to do to come up with the estimates in the first place.
There’s a fundamental difference between being able to recognize the best option and being able to implement the option.
These sorts of rules are really about people feeling devalued or disliking being volunteered or told what to do (often by people they consider less knowledgable). They aren’t really about effectively distributing work.
“Just” gets a bad wrap. There’s a sort of hidden assumption here that “you can just” is equivalent to “you can easily”. It sometimes means that but more
generally it means something more like “…will be easiest” which can be true even when the action itself is hard or a lot of work.
Punishing people with more work doesn’t make sense in a well run organization. Work is continuous and more work only affects the backlog, not the actual developer’s life or experience.
Exactly. There's no "just" when you're adding an API. APIs require a lot of work and complexity to do right:
APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all.
APIs require authentication and authorization. This means setting up OAuth2 or at least having a secure API token generation, storage, and validation.
APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't.
APIs need to be performant or you database may be crushed under the load. This may involve caching which then adds the complexity of cache invalidation and other servers/processes to support caching.
APIs need to be rate-limited or sloppy clients will hammer your API.
APIs require thorough documentation or they're useless. You may even have to add SDKs (libraries for different programming languages) to make it easier for people to use the API.
APIs need good error messages or users who are getting started will not know why their calls are failing.
I disagree, those are all technical issues that are typically blocked by socio-technical and organizational problems.
Retrofitting an API is a context specific problem, with few things that generalize but the above aren't the hard parts, just the complex ones.
I am going to point you to a Public Policy book here, look at page 32 and their concept of 'wicked problems', which applies to tech but doesn't suffer the problems with the consulting industry co-opting as much as in the tech world.
Obviously most companies won't accept the Amazon style API edicts that require Externalizable interfaces and force a product mindset and an outside in view.
Obviously an API is still a "Cognitively complex problem" as defined by the above.
But what makes most API projects fail is the fact that aspects arise that result in ‘wickedness’ A.K.A intractability.
API gateways, WAFs, RESTful libs with exponential backoff with jitter....etc... all exist and work well and almost any system that you have a single DB you are concerned about killing is possible to just add a anti-corruption layer if you don't have too much code debt, complicated centralized orchestration etc....
But these general, but difficult tasks like adding an API almost never fail due to tech reasons, but due to politics, poor communication, focusing on tech and not users, unrealistic timelines, and turf battles.
You are correct that "APIs require thorough documentation or they're useless" but more importantly they need enforceable standards on contract ownership, communication.
The direction of control, stability, audience, and a dozen other factors effect the tradeoffs there as to what is appropriate.
Typically that control is dictated purely by politics and not focused on outcomes.
That is often what pushes these complex problems into failed initiatives, and obviously this is far more complicated. Often orgs can't deal with the very real uncertainty in these efforts and waste lots of project time on high effort, low value tasks like producing gantt charts that are so beyond the planning horizon that they can only result in bad outcomes at best.
The point being is that unless you are on the frontier of knowledge and technical capabilities, it is almost never the tech that causes these efforts to fail.
lol, I can already hear the responses of plenty of managers/management I worked with to these points.
> APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all.
That's fine. It's a first iteration we can change it later if people complain. Lets get something out there and iterate.
> APIs require authentication and authorization.
Lets not worry about authorization/permissions. It's just one key or whatever account they use to log in with now.
> APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't.
You're saying you don't know how to do that?
> APIs need to be rate-limited or sloppy clients will hammer your API.
Either "Lets worry about that later when it happens" or "Here is the first github link from a google result for 'api rate limit open source free'. Lets use that"
> APIs require thorough documentation or they're useless.
Either "We need to have the API first, docs, clis, etc could come later after we have gauged the usage or had asks for them. We can handhold the customers asking for them for now." or "here is a github project that autogenerates docs, SKDs and clis from an OpenAPI spec"
> APIs need good error messages or users who are getting started will not know why their calls are failing.
You're saying you don't know how to do that?
"It doesn't have to be perfect. We have customers who are asking for it and to win them we need to implement something then we can work with them to improve it. Otherwise we will lose them"™
And there's an implied risk that once you have added an API for customers to exfiltrate data or integrate with their own system (or, worse, IMHO: a competitor's tool) then they'll never return to your site again
I'm cognizant that if it truly is a make-or-break for the deal, there may not be any choice, but along with all the risks you cited is an underlying obsolescence one
Every single one of these is true of a website, but we have far more websites than APIs.
If there's one thing APIs suffer from more, it's "social cost to make changes". A concurrent vN+1 largely resolves that though, unless your API consuming ecosystem is large enough to be worth investing vastly more resources into.
Most professional advice is like relationship advice, people extrapolate something that went wrong for them into general cases but that rarely directly applies to someone who is not you in that very similar situation. And the advice that seems to work consistently gets so generic that is almost useless. Something like "be thoughtful, try to do the right thing and reflect back to see how it went" doesn't write blog posts or sells books!
I am pretty sure presales as a career exists solely because of the word 'just'. We have to make sure they wont get spooked when we start lifting the rest of the iceberg out of the water.
Yes exactly. The article would be much better positioned as What it actually takes to make systems ideas "just work". Then it isn't about whether it can or can't but rather how it succeeds or fails. Having details about what it takes then stops people from saying "just" because they don't know what they're talking about.
But I pretty much agree 100% about DSLs, it's an unnecessary/cute complication. The only people who should be allowed to make them should be ones who have made a successful programming language and updated it to deal with all their mistakes or a 2nd version/2nd language, and still got many things wrong.
Yes, and a very well designed API can be the foundation of and moat for a successful business. Stripe is one of the best examples. Even as hundreds of copycats pop up, their excellent API design and the trust devs have in them to nail the next API design, means they can continue to expand into other financial products with ease.
It'd have been delightfully ironic had either of these Steves concluded their essays with a named methodology to "just" apply whenever faced with these "let's just" situations but alas...
(4) Anomaly detection is not inherently a problem of distributed systems like the others, but someone facing the problems they've been burned with might think they need it. Intellectually it's tough. The first algorithm I saw that felt halfway smart was https://scikit-learn.org/1.5/modules/outlier_detection.html#... which is sometimes a miracle and I had good luck using it on text with the CNN-based embeddings we had in 2018 but none at all w/ SBERT.
I've written two DSLs (one with a team) and I'd consider them both successful. They solved the problem and no one cursed them out. I think the most important factor is they were both small.
They were very similar. I even reused the code. One was writing rules to validate giant forms, the other was writing rules to for decisions based on form responses.
Ok, just ranting on DSLs. Good DSLs take someone from can't to can. A DSL that's meant to save time is way less likely to be useful because it's very likely to not save you time.
In both of my DSLs, it's that we needed to get complex domain behavior into the program. So you either need to teach a programmer the domain, partner a programmer with a domain expert, or teach a domain expert how to program.
Putting the power in the hands of the domain expert is attractive when there's a lot of work to be done. It frees up programmers to do other things and tightens the feedback loop. If it's a deep domain, it's not like you want to send your programmer to school to learn how to do this. If it's shallow, you can probably have someone cheaper do it.
A DSL comes with a lot of cognitive overhead. If the other option is learning a full programming language, this becomes more reasonable.
A time saving DSL is where someone already knows how to write code, they just want to write less of it. This is generally not as good because the savings are marginal. Then when some programmer wants to change something, they have to learn/remember this whole DSL instead of more straightforward code.
Actually, this makes a simpler rule of thumb. A DSL for programmers is less likely to be a good idea than a DSL for non-programmers.
DSL are just great, have never failed for me. I have done it several times, the last one just the past year, for programming complex ASICs. I‘ve seen uncountable times working like a charm.
For example, bash, SQL DSLs may be immediately useful by protecting against shell,sql injection: shutil.run(sh"command {arg}") may translate to subprocess.run(["command", os.fspath(arg)])
No shell--no shell injection. The assumption is that it enables sh"a | b > {c}" syntax (otherwise just call subprocess.run directly). Implementing it in pure Python by hand would be more verbose, less readable, more error-prone).
I think the theory of domain-specific languages is very valuable, even if there's rarely a need for a full implementation of one.
As I see it, a DSL is just the end-state of a programmer creating abstractions and reusable components to ultimately solve the real problem. The nouns and verbs granted by a programming interface constrain how one thinks, so a flexible and intuitive vocabulary and grammar can make the "real program" powerful and easy to maintain. Conversely, a rigid and irregular interface makes the "real program" a brittle maintenance nightmare.
jOOQ is a disaster and I would not recommend it to anyone.
You write some SQL queries, test them in datagrip or whatnot, then spend the next several hours figuring out how to convert them to the DSL. This problem is compounded when you use "exotic" SQL features like json expressions. Debugging is "print the generated sql, copy it into datagrip/whatnot, tune the query, then figure out how to retrofit that back into the DSL".
It's a huge waste of time.
The primary selling point of jOOQ is "type safe queries". That became irrelevant when IntelliJ started validating SQL in strings in your code against the real data. The workflow of editing SQL and testing it directly against the database is just better.
This is a very specific and popular subset of the DSL point: Let's just invent a language L that is better than horrible standard language X but translates to X. Imagine the vast cubicle farms of X programmers who will throw off their chains and flock to our better language L!
In many scenarios (including JOOQ and all ORMs), X is SQL. I should know, I spent years working on a Java-based ORM. So believe me when I say: ORMs are terrible. To use SQL effectively, you have to understand how databases work at the physical level -- what's a B-tree lookup, what's a scan, how these combine, etc. etc. You can often rely on the optimizer to do a good job, but must also be able to figure out the physical picture when the optimizer (or DBA) got things wrong. You're using an ORM? To lift a phrase from another don't-do-this context: congratulations, you now have two problems. You now have to get the ORM to generate the SQL to do what really needs to be done.
And then there are the generalizations of point made above: There are lots of tools that work with SQL. Lots of programmers who know SQL. Lots of careers that depend on SQL. Nobody gives a shit about your ORM just because it saves you the trouble of the easiest part of the data management problem.
Yep, abstracting away SQL is a common and very costly mistake. The article is about more general system design, otherwise I would have expected to see that in the list.
I've never seen a good DSL beside something like regular expressions, and even there, I hear, a lot of people are upset by the language.
Examples of popular DSLs that I would characterize as bad if not outright failures:
* HCL (Terraform configuration language). It was obvious from the very beginning that very common problems haven't been addressed in the language, like provisioning a variable number of similar appliances. The attempts to add the functionality later were clumsy and didn't solve the problem fully.
* E4X (A JavaScript DSL for working with XML). In simple cases allowed for more concise expression of operations on XML, but very quickly could become an impenetrable wall of punctuation. This is very similar to Microsoft's Linq in that it gave no indication to the authors of how computationally complex the underlying code would be. Eventually, any code using this DSL would rewrite it in a less terse, but more easy to analyze way.
* XUL (Firefox' UI language for extending the browser's chrome). It worked OK if what you wanted to do was Firefox extensions, but Firefox also wanted to sell this as a technology for enterprise to base their in-house applications on Firefox, and it was very lacking in that domain. It would require a lot of trickery and round-about ways of getting simple things done.
* Common Lisp's string formatting language (as well as many others in this domain). Similar to above: works OK for small problems, but doesn't scale. Some formatting problems require some very weird solutions, or don't really have a solution at all (I absolutely hate it when I see code that calls format recursively).
All in all. The most typical problem I see with this approach is that it's temporary and doesn't scale well. I.e. it will very soon run into the problems it doesn't have a good solution for. Large programs in DSL languages are often a nightmare to deal with.
I'm always baffled by hate for DSLs until I realize that what people are criticizing aren't DSLs, but DSLs you have to write from scratch. If you host your DSL on Lisp, then all you have to write is your domain logic, not the base language. Most of the work is already done, and your language is useful from day one. I don't understand why people insist on creating new languages from scratch just to watch them die on the vine, when these langs could have been hosted DSLs on Lisp and actually get used.
Not just Lisp, but any language that has strong support for either literal in-language data expressions like JSON or YAML, or meta-language support like Ruby, Elixir, JSX/TSX (or both!).
Every time you write a React JSX expression, terraform file, config.yaml, etc., you're using a DSL.
I once wrote a JSON DSL in Ruby that I used for a template-based C# code generator. This enabled a .NET reporting web app to create arbitrarily shaped reports from arbitrary rdmbs tables, saving our team thousands of hours. Another team would upload report data to a SQL Server instance, write a JSON file in the DSL, check it against a tiny schema validator website, submit it, and their reports would soon be live. One of the most productive decisions I ever made.
This is generally a terrible way to work. Making a bunch of custom syntax even in the same language is just adding more stuff to memorize for no gain.
Even in C using the "goes to operator" of while(i --> 0) or using special operator overloading like the C++ STL >> and << operators for concatenation is just making people memorize nonsense so someone writing can be clever.
People don't give presentations with riddles and limericks either. It can be clever as a puzzle but when things need to get done, it is just indulging someone showing off their cleverness at the expense of everyone who has to deal with it.
Exactly. Good DSL are typically (although not always) embedded in another. When that is the case, they tend to be a perfect abstraction (if decently implemented)
Eh, you can host DSLs in Kotlin and C# these days, you don’t even have to sell your engineering team on Lisp. The biggest challenge is to explain how an embedded DSL differs from being just a library (interop outside of the eDSL to the host language is still hard).
I was gonna say something like "DSLs work great when they're small, purposeful and easy to test", I guess yours kind of helps when they're not what I'd suggest :)
Counterexample: regex. In terms of how successful DSLs are, I think, something like Perl's regular expression is, at least, in the top ten. Most regex users don't care about there being an IDE for it, I don't think there's a lot of value for regex autocomplete, even if such thing existed.
Counterpoint: k8s is a bad orchestration system with bad scaling properties compared to the state machine versions (Borg, Tupperware/Twine, and presumably others). I say this as someone who has both managed k8s at scale and was a core engineer of one of the proprietary schedulers.
It is, and from experience it is also a good example how control loops are harder than you think. Few people understand it, and it is the source of much underlying trouble.
(3.b) "Being clever rather than over-provisioning" is not generally thought of as a good idea. People would be rather apprehensive if you told them "I'm don't something really clever so we can under- or exactly-provision". I mean, sure, it may indeed work, but that's not the same thing.
(5) Hybrid parallelism - also, many people think it's a bad idea because it makes your software system more complex. Again, it may be very useful sometimes, but it's not like many people would go "yes, that's just what I'm missing right now, let's do parallelism with different hardware and different parts of the workflow and everything will run something kind of different and it'll all work great like a symphony of different instruments".
You don't get the luxury of offline migrations or single-master writes in the high-volume payments space. You simply don't have the option. The money must continually flow.
I've yet to see a DSL work great. Every single time, I'm asking "why isn't this just Python (or some other lang)" especially the times when it's some jacked up variant of Python.
There are many successful examples of all of these. Using “almost” as an escape hatch doesn’t work here.
This is just pessimism and weary cynicism. I get it, I’ve felt that way too, and sometimes it’s hard to talk an eager engineer out of a bad idea. But for me, this vibe is toxic.
A lot of those "successful" examples have teams of battle-scarred engineers dealing with all the failures of those ideas. Control loops running away to infinity or max/min bounds, a cache that can't recover from a distributed failure, corrupted live-migrated state, bursts causing overload at inconvenient times, spurious anomaly-detection alerts informing you of all the world's holidays, etc.
Underneath all of those ideas is a tangle of complexity that almost everyone underestimates.
I know I'm not alone in this, but after doing this for more than 20 years I can't shake the idea that we are doing it wrong -- meaning programming. Is it really this nit-picky, brittle, and hard?
The brittleness is what gets me. In physical mechanical and even analog electrical systems there are tolerances. Things can almost-work and still work for varying degrees of work. Software on the other hand is unbelievably brittle to the point that after 50 years of software engineering we still really can't re-use or properly modularize code. We are still stuck in "throw it away and do it over" and constantly reinventing wheels because there is no way to make wheels fit. The nature of digital means there are no tolerances. The concept isn't even valid. Things fit 100% or 0%.
We keep inventing languages. They don't help. We keep inventing frameworks. They don't help. We keep trying "design patterns" and "methodologies." They don't help. If anything all the stuff we invent makes the problem worse. Now we have an ecosystem with 20 different languages, 50 different runtimes, and 30 variants of 5 OSes. Complexity goes up, costs go up, reusability never happens, etc.
I remember for a while seeing things like the JVM and CLR (the VM for C# and friends) as a way out-- get away from the brittle C API and fully compiled static native code and into a runtime environment that allowed really solid error handling and introspection. But that paradigm never caught on for whatever reason, probably because it wasn't free enough. WASM is maybe promising.
I think that's fair: the success stories are very rarely "yay, we did it!" but much more often "this single change was the sole focus of a team/multiple teams for X months, and the launch/release/fix was considered a culmination of a significant investment of resources".
Basically agree. This is a list of systems ideas which are harder than one might initially think, and should be approached seriously, never casually.
That slant-rhymes with "sound good but almost never work" but in detail is completely different. When treated as difficult problems, and committed to accordingly, having them work and work well is a normal result, eminently achievable.
As afterthoughts, or when naïvely thought to be easy, then yeah, they frequently go poorly.
I don't read it that way. I read it as engineers engaging in pre-optimization for no business benefit. It's utterly rampant in the industry because it's fun to design and build a redundant auto-scaling spaceship vs. just over-provisioning your server by 200% for a tenth (or less!) of the cost and having backups ready to deploy in a few hours.
Sometimes these ideas make sense - after you need them. Not designed-in at the early product stage. Very few products go on to need the scale, availability, or complexity most of these implementations try to solve.
Mmmm danger! Then let’s stop doing anything that is difficult?
If that is bot the message, what is that? “These things are hard, often don’t work, but GO FOR IT”?
I pretty much read: “try to avoid” which is bad advice in my opinion. Like “documenting SW properly while doing development is hard, and often goea wrong” so what?!
Pessimism and weary cynicism can be very valuable in many tech environments though. It keeps you stable, working on tried and tested things, and safe. There's a lot of situations where that's super valuable
I had to learn that attitude in order to operate at one employer. I had to unlearn it to survive at the next, where engineers were better and routinely pulled off things I had dismissed as unrealistic.
The value of that approach is very situational…though I will acknowledge that the majority of places probably warrant at least some of that.
So many people here trying to thread the needle looking for subtle decision functions for exceptions. It's pretty simple, really: these ideas are awesome when I do them, and never work as intended when that idiot before me did them.
I would add "Domain Driven Design" - locking your business design in place by trying to make your application match your business structure is a recipe for disaster. If you have a small or stagnant business you probably won't notice any issues. If your business is successful and/or grows, you're going to immediately regret trying to build domains with horrific descriptive names tied to your already obsolete business practices. Instead, design around functionality layers (how we've been doing it for decades, tried and true), and as much as possible keep business logic in config, rows in databases, and user workflows, which makes them extremely flexible.
You'll regret both options. You mentioned the pitfalls of "domain driven design" (outdated language, little code/systems reuse for new endeavors).
However, a highly abstract design with all the business logic in config, workflows, etc will only makes your system extremely flexible as long every one up and down the organization is fairly aware of the abstractions, the config, and uncountable permutations they can take for your business logic to emerge.
Those permutations quickly explode into a labyrinth of unknown/unexpected behaviors what people will rely on. It also makes the cost of onboarding new developers, changing the development team insurmountable. Your organization will be speaking 2 different languages. Most seemingly straightforward "feature asks" that break your abstraction either become a massive system re-design/re-architect or a "let's just hack this abstraction so it's a safer smaller change for now". The former will always be really hard unless you have excellent engineers who have full understanding of the entire system and its behavior and code base along with and excellent engineering practices and processes, and still will take you months or years to pull off. The latter is the more likely to happen and it's why all those "highly abstract, functionality layers, config driven, business logic emerging) projects start perfect and flexible and end up as a "what the fuck is even this".
After a system is implemented, that emergent business logic becomes the language everyone will speak in. Having your organization speaking 2 or 3 completely irreconcilable languages is very painful and unless you have multiple folks up, down and sideways in the organization that can fluently translate between the 2, you'll be in a world of pain and wish you had some closer representation of your domain
Along with "make impossible states impossible to represent". If you're designing your types to make a state unrepresentable, you'd better be absolutely sure the state really is impossible for the lifetime of the design.
I don't understand, you're not allowed to change the types? Suppose you disallow any number other than 1, 2, and 3, and you model this with some enum with three members or something. Then you find out that actually a prospective client would love to also work with with the number 4, then you just add 4 to the enum in your next release, no?
If its not impossible, you should handle it though.
I think what that quote is against is the common middle ground where states are expected to be 'impossible' and thus not handled and cause bugs when they are found to be not actually.
Either deciding that they are possible or must be impossible is usually better and which one to go with depends on the specifics
I don't understand the load-responsive control loop one. That's a basic and fundamental component in countless systems. The centrifugal governor on a 1800's steam engine or 1900's victrola record player is a load-responsive control loop. All of electronics is a mesh of load-responsive control loops. The automatic transmission in your car...
The usual issue is the addition of control loops without much understanding of the signals (CPU utilization is a fun one), and the addition of control loops without the consideration of other control loops. For example, you might find that your cross region load balancer gets into a fight with your in-process load shedding, because the load balancer's signals do not account for load shedding (or the way they account for the load shedding is inaccurate). Other issues might be the addition of control loops to optimize service local outcomes, to the detriment of global outcomes.
My general take is that you want relatively few control loops, in positions of high leverage.
It’s not totally clear, but it could be talking about CPU load in particular, which has some problems as described in https://arxiv.org/abs/2312.10172.
I've always used connection backlog as the metric for load and it's worked pretty well. Most web servers have it as a number you can expose as a metric. It's not perfect but it's at least a true measure of when servers are behind.
There is a pattern to all of these problems, notably that they are all orthogonal concerns that add constraints to the sequential data-munging programming model that programmers are familiar with. Whenever you add constraints, you add things that future programmers need to think about, for all future development they do on the system. It is very easy to get into a situation where the system is overconstrained and it's impossible to make forward progress without relaxing some of the constraints. Even if it's not impossible, it's going to be slow, as developers need to consider how their new feature interacts with the API/security/synchronization/latency/other-platforms/native-code that the system has already committed to supporting.
That's also why it's possible to support all of these attributes. If you make say transparent data synchronization a core value prop of the platform, then all future development supports that first, and you evolve your feature set based on what's possible with that constraint. That feature set might not be exactly what your users want, but it's what you support. Your product appeals to the customers for whom that is their #1 purchase decision.
I've worked on several DSLs, a P2P cache, and a project employing hybrid parallelism. All of them worked. All great fun to create. With one exception these projects were good investments (The P2P cache wasn't necessary so never really paid off). My point is that it's definitely wrong to say these things almost never work. They are complicated but that complexity brings functionality which is hard to achieve in another way. The lesson from the P2P cache example was to be sure you actually need that functionality.
Readit News
For instance "Let's just add an API." I think the approach to an API as "just" a feature to your product will be about as successful as saying "let's just add a UI". To implement a successful UI one needs to be thoughtful, thorough, and bring in people who specialize in it. Why should any other interface for your product be any different? It's not that it's a bad, or good, idea, rather one that shouldn't "just" be done.
It was a rule that worked well for us.
> “Precisely. It’s what I call a ‘Lullaby Word.’ Like ‘should,’ it lulls your mind into a false sense of security. A better translation of ‘just’ in Jeff’s sentence would have been, ‘have a lot of trouble to.'”
Yes I like this rule, how many things I could have solved at the worst run place I ever worked at if it had been implemented there. Although I guess the management might not have liked that essentially I could choose what got done and what I got to work on just by running my big mouth.
facetiousness aside, I'm trying to point out that most often when people say "let's just" it is because they are advocating a course of action and that rule won't work unless the course they are advocating gets chosen. If they say "let's just make an api" and you say no we're doing a UI and you will work on it because you used the word "Just" I guess that's a way to lose your tech talent relatively quickly.
These sorts of rules are really about people feeling devalued or disliking being volunteered or told what to do (often by people they consider less knowledgable). They aren’t really about effectively distributing work.
“Just” gets a bad wrap. There’s a sort of hidden assumption here that “you can just” is equivalent to “you can easily”. It sometimes means that but more generally it means something more like “…will be easiest” which can be true even when the action itself is hard or a lot of work.
APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all.
APIs require authentication and authorization. This means setting up OAuth2 or at least having a secure API token generation, storage, and validation.
APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't.
APIs need to be performant or you database may be crushed under the load. This may involve caching which then adds the complexity of cache invalidation and other servers/processes to support caching.
APIs need to be rate-limited or sloppy clients will hammer your API.
APIs require thorough documentation or they're useless. You may even have to add SDKs (libraries for different programming languages) to make it easier for people to use the API.
APIs need good error messages or users who are getting started will not know why their calls are failing.
Retrofitting an API is a context specific problem, with few things that generalize but the above aren't the hard parts, just the complex ones.
I am going to point you to a Public Policy book here, look at page 32 and their concept of 'wicked problems', which applies to tech but doesn't suffer the problems with the consulting industry co-opting as much as in the tech world.
https://library.oapen.org/bitstream/id/f3358d25-56c0-47f2-90...
Obviously most companies won't accept the Amazon style API edicts that require Externalizable interfaces and force a product mindset and an outside in view.
Obviously an API is still a "Cognitively complex problem" as defined by the above.
But what makes most API projects fail is the fact that aspects arise that result in ‘wickedness’ A.K.A intractability.
API gateways, WAFs, RESTful libs with exponential backoff with jitter....etc... all exist and work well and almost any system that you have a single DB you are concerned about killing is possible to just add a anti-corruption layer if you don't have too much code debt, complicated centralized orchestration etc....
But these general, but difficult tasks like adding an API almost never fail due to tech reasons, but due to politics, poor communication, focusing on tech and not users, unrealistic timelines, and turf battles.
You are correct that "APIs require thorough documentation or they're useless" but more importantly they need enforceable standards on contract ownership, communication.
The direction of control, stability, audience, and a dozen other factors effect the tradeoffs there as to what is appropriate.
Typically that control is dictated purely by politics and not focused on outcomes.
That is often what pushes these complex problems into failed initiatives, and obviously this is far more complicated. Often orgs can't deal with the very real uncertainty in these efforts and waste lots of project time on high effort, low value tasks like producing gantt charts that are so beyond the planning horizon that they can only result in bad outcomes at best.
The point being is that unless you are on the frontier of knowledge and technical capabilities, it is almost never the tech that causes these efforts to fail.
> APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all.
That's fine. It's a first iteration we can change it later if people complain. Lets get something out there and iterate.
> APIs require authentication and authorization.
Lets not worry about authorization/permissions. It's just one key or whatever account they use to log in with now.
> APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't.
You're saying you don't know how to do that?
> APIs need to be rate-limited or sloppy clients will hammer your API.
Either "Lets worry about that later when it happens" or "Here is the first github link from a google result for 'api rate limit open source free'. Lets use that"
> APIs require thorough documentation or they're useless.
Either "We need to have the API first, docs, clis, etc could come later after we have gauged the usage or had asks for them. We can handhold the customers asking for them for now." or "here is a github project that autogenerates docs, SKDs and clis from an OpenAPI spec"
> APIs need good error messages or users who are getting started will not know why their calls are failing.
You're saying you don't know how to do that?
"It doesn't have to be perfect. We have customers who are asking for it and to win them we need to implement something then we can work with them to improve it. Otherwise we will lose them"™
Deleted Comment
I'm cognizant that if it truly is a make-or-break for the deal, there may not be any choice, but along with all the risks you cited is an underlying obsolescence one
If there's one thing APIs suffer from more, it's "social cost to make changes". A concurrent vN+1 largely resolves that though, unless your API consuming ecosystem is large enough to be worth investing vastly more resources into.
But I pretty much agree 100% about DSLs, it's an unnecessary/cute complication. The only people who should be allowed to make them should be ones who have made a successful programming language and updated it to deal with all their mistakes or a 2nd version/2nd language, and still got many things wrong.
Often I see developers creating new APIs ad-hoc all the time instead of curating and enhancing the one they already have.
https://steve-yegge.blogspot.com/2009/04/have-you-ever-legal...
It'd have been delightfully ironic had either of these Steves concluded their essays with a named methodology to "just" apply whenever faced with these "let's just" situations but alas...
Deleted Comment
(2) Elastic Load Balancer is a control loop responsive to workloads, that kind of thing is a commodity
(3) Under-provisioning is rampant in most industries; see https://erikbern.com/2018/03/27/waiting-time-load-factor-and... and https://www.amazon.com/Goal-Process-Ongoing-Improvement/dp/0...
(4) Anomaly detection is not inherently a problem of distributed systems like the others, but someone facing the problems they've been burned with might think they need it. Intellectually it's tough. The first algorithm I saw that felt halfway smart was https://scikit-learn.org/1.5/modules/outlier_detection.html#... which is sometimes a miracle and I had good luck using it on text with the CNN-based embeddings we had in 2018 but none at all w/ SBERT.
They were very similar. I even reused the code. One was writing rules to validate giant forms, the other was writing rules to for decisions based on form responses.
Ok, just ranting on DSLs. Good DSLs take someone from can't to can. A DSL that's meant to save time is way less likely to be useful because it's very likely to not save you time.
In both of my DSLs, it's that we needed to get complex domain behavior into the program. So you either need to teach a programmer the domain, partner a programmer with a domain expert, or teach a domain expert how to program.
Putting the power in the hands of the domain expert is attractive when there's a lot of work to be done. It frees up programmers to do other things and tightens the feedback loop. If it's a deep domain, it's not like you want to send your programmer to school to learn how to do this. If it's shallow, you can probably have someone cheaper do it.
A DSL comes with a lot of cognitive overhead. If the other option is learning a full programming language, this becomes more reasonable.
A time saving DSL is where someone already knows how to write code, they just want to write less of it. This is generally not as good because the savings are marginal. Then when some programmer wants to change something, they have to learn/remember this whole DSL instead of more straightforward code.
Actually, this makes a simpler rule of thumb. A DSL for programmers is less likely to be a good idea than a DSL for non-programmers.
I cannot understand why seems it was bad for him…
For example, bash, SQL DSLs may be immediately useful by protecting against shell,sql injection: shutil.run(sh"command {arg}") may translate to subprocess.run(["command", os.fspath(arg)])
No shell--no shell injection. The assumption is that it enables sh"a | b > {c}" syntax (otherwise just call subprocess.run directly). Implementing it in pure Python by hand would be more verbose, less readable, more error-prone).
As I see it, a DSL is just the end-state of a programmer creating abstractions and reusable components to ultimately solve the real problem. The nouns and verbs granted by a programming interface constrain how one thinks, so a flexible and intuitive vocabulary and grammar can make the "real program" powerful and easy to maintain. Conversely, a rigid and irregular interface makes the "real program" a brittle maintenance nightmare.
Nail on the head time - somewhere else in the thread is jooq which is (yet another) SQL DSL where you end up with from(table).where(student=bob)
This is a perfect example of why the programmer should (just?) learn SQL instead of the DSL - and your comment nails it
You write some SQL queries, test them in datagrip or whatnot, then spend the next several hours figuring out how to convert them to the DSL. This problem is compounded when you use "exotic" SQL features like json expressions. Debugging is "print the generated sql, copy it into datagrip/whatnot, tune the query, then figure out how to retrofit that back into the DSL".
It's a huge waste of time.
The primary selling point of jOOQ is "type safe queries". That became irrelevant when IntelliJ started validating SQL in strings in your code against the real data. The workflow of editing SQL and testing it directly against the database is just better.
jOOQ reinforces the OP's point about DSLs.
In many scenarios (including JOOQ and all ORMs), X is SQL. I should know, I spent years working on a Java-based ORM. So believe me when I say: ORMs are terrible. To use SQL effectively, you have to understand how databases work at the physical level -- what's a B-tree lookup, what's a scan, how these combine, etc. etc. You can often rely on the optimizer to do a good job, but must also be able to figure out the physical picture when the optimizer (or DBA) got things wrong. You're using an ORM? To lift a phrase from another don't-do-this context: congratulations, you now have two problems. You now have to get the ORM to generate the SQL to do what really needs to be done.
And then there are the generalizations of point made above: There are lots of tools that work with SQL. Lots of programmers who know SQL. Lots of careers that depend on SQL. Nobody gives a shit about your ORM just because it saves you the trouble of the easiest part of the data management problem.
Examples of popular DSLs that I would characterize as bad if not outright failures:
* HCL (Terraform configuration language). It was obvious from the very beginning that very common problems haven't been addressed in the language, like provisioning a variable number of similar appliances. The attempts to add the functionality later were clumsy and didn't solve the problem fully.
* E4X (A JavaScript DSL for working with XML). In simple cases allowed for more concise expression of operations on XML, but very quickly could become an impenetrable wall of punctuation. This is very similar to Microsoft's Linq in that it gave no indication to the authors of how computationally complex the underlying code would be. Eventually, any code using this DSL would rewrite it in a less terse, but more easy to analyze way.
* XUL (Firefox' UI language for extending the browser's chrome). It worked OK if what you wanted to do was Firefox extensions, but Firefox also wanted to sell this as a technology for enterprise to base their in-house applications on Firefox, and it was very lacking in that domain. It would require a lot of trickery and round-about ways of getting simple things done.
* Common Lisp's string formatting language (as well as many others in this domain). Similar to above: works OK for small problems, but doesn't scale. Some formatting problems require some very weird solutions, or don't really have a solution at all (I absolutely hate it when I see code that calls format recursively).
All in all. The most typical problem I see with this approach is that it's temporary and doesn't scale well. I.e. it will very soon run into the problems it doesn't have a good solution for. Large programs in DSL languages are often a nightmare to deal with.
Every time you write a React JSX expression, terraform file, config.yaml, etc., you're using a DSL.
I once wrote a JSON DSL in Ruby that I used for a template-based C# code generator. This enabled a .NET reporting web app to create arbitrarily shaped reports from arbitrary rdmbs tables, saving our team thousands of hours. Another team would upload report data to a SQL Server instance, write a JSON file in the DSL, check it against a tiny schema validator website, submit it, and their reports would soon be live. One of the most productive decisions I ever made.
Even in C using the "goes to operator" of while(i --> 0) or using special operator overloading like the C++ STL >> and << operators for concatenation is just making people memorize nonsense so someone writing can be clever.
People don't give presentations with riddles and limericks either. It can be clever as a puzzle but when things need to get done, it is just indulging someone showing off their cleverness at the expense of everyone who has to deal with it.
I'll take a stab at fleshing this out: DSLs work great when they have an IDE with autocomplete and a quick (or instant) feedback loop.
Deleted Comment
And yet Terraform/Tofu continues to poison people's brains. It boggles(!) the mind
(5) Hybrid parallelism - also, many people think it's a bad idea because it makes your software system more complex. Again, it may be very useful sometimes, but it's not like many people would go "yes, that's just what I'm missing right now, let's do parallelism with different hardware and different parts of the workflow and everything will run something kind of different and it'll all work great like a symphony of different instruments".
You don't get the luxury of offline migrations or single-master writes in the high-volume payments space. You simply don't have the option. The money must continually flow.
This is just pessimism and weary cynicism. I get it, I’ve felt that way too, and sometimes it’s hard to talk an eager engineer out of a bad idea. But for me, this vibe is toxic.
Underneath all of those ideas is a tangle of complexity that almost everyone underestimates.
The brittleness is what gets me. In physical mechanical and even analog electrical systems there are tolerances. Things can almost-work and still work for varying degrees of work. Software on the other hand is unbelievably brittle to the point that after 50 years of software engineering we still really can't re-use or properly modularize code. We are still stuck in "throw it away and do it over" and constantly reinventing wheels because there is no way to make wheels fit. The nature of digital means there are no tolerances. The concept isn't even valid. Things fit 100% or 0%.
We keep inventing languages. They don't help. We keep inventing frameworks. They don't help. We keep trying "design patterns" and "methodologies." They don't help. If anything all the stuff we invent makes the problem worse. Now we have an ecosystem with 20 different languages, 50 different runtimes, and 30 variants of 5 OSes. Complexity goes up, costs go up, reusability never happens, etc.
I remember for a while seeing things like the JVM and CLR (the VM for C# and friends) as a way out-- get away from the brittle C API and fully compiled static native code and into a runtime environment that allowed really solid error handling and introspection. But that paradigm never caught on for whatever reason, probably because it wasn't free enough. WASM is maybe promising.
That slant-rhymes with "sound good but almost never work" but in detail is completely different. When treated as difficult problems, and committed to accordingly, having them work and work well is a normal result, eminently achievable.
As afterthoughts, or when naïvely thought to be easy, then yeah, they frequently go poorly.
I don't read it that way. I read it as engineers engaging in pre-optimization for no business benefit. It's utterly rampant in the industry because it's fun to design and build a redundant auto-scaling spaceship vs. just over-provisioning your server by 200% for a tenth (or less!) of the cost and having backups ready to deploy in a few hours.
Sometimes these ideas make sense - after you need them. Not designed-in at the early product stage. Very few products go on to need the scale, availability, or complexity most of these implementations try to solve.
If that is bot the message, what is that? “These things are hard, often don’t work, but GO FOR IT”?
I pretty much read: “try to avoid” which is bad advice in my opinion. Like “documenting SW properly while doing development is hard, and often goea wrong” so what?!
The value of that approach is very situational…though I will acknowledge that the majority of places probably warrant at least some of that.
However, a highly abstract design with all the business logic in config, workflows, etc will only makes your system extremely flexible as long every one up and down the organization is fairly aware of the abstractions, the config, and uncountable permutations they can take for your business logic to emerge.
Those permutations quickly explode into a labyrinth of unknown/unexpected behaviors what people will rely on. It also makes the cost of onboarding new developers, changing the development team insurmountable. Your organization will be speaking 2 different languages. Most seemingly straightforward "feature asks" that break your abstraction either become a massive system re-design/re-architect or a "let's just hack this abstraction so it's a safer smaller change for now". The former will always be really hard unless you have excellent engineers who have full understanding of the entire system and its behavior and code base along with and excellent engineering practices and processes, and still will take you months or years to pull off. The latter is the more likely to happen and it's why all those "highly abstract, functionality layers, config driven, business logic emerging) projects start perfect and flexible and end up as a "what the fuck is even this".
After a system is implemented, that emergent business logic becomes the language everyone will speak in. Having your organization speaking 2 or 3 completely irreconcilable languages is very painful and unless you have multiple folks up, down and sideways in the organization that can fluently translate between the 2, you'll be in a world of pain and wish you had some closer representation of your domain
I think what that quote is against is the common middle ground where states are expected to be 'impossible' and thus not handled and cause bugs when they are found to be not actually.
Either deciding that they are possible or must be impossible is usually better and which one to go with depends on the specifics
My general take is that you want relatively few control loops, in positions of high leverage.
That's also why it's possible to support all of these attributes. If you make say transparent data synchronization a core value prop of the platform, then all future development supports that first, and you evolve your feature set based on what's possible with that constraint. That feature set might not be exactly what your users want, but it's what you support. Your product appeals to the customers for whom that is their #1 purchase decision.