> If you’re building an abstraction-heavy API, be prepared to think hard before adding new features. If you’re building an abstraction-light API, commit to it and resist the temptation to add abstractions when it comes along.
You could always do both.
Provide a low-level abstraction-light API that allows fine control but requires deep expertise, and write a higher-level abstraction-rich API on top of it that maps to fewer simple operations for the most common use cases - which some of your clients might be implementing their own half-baked versions of anyway.
If you maintain a clean separation between the two, having both in place might mean there is less pressure to add abstractions to the low-level API, or to add warts and special-cases to the high-level API. If a client wants one of those things, it already exists - in the other API.
Bonus points for providing materials to help your clients learn how to move from one to the other. You can attract clients who do not yet have deep knowledge of payment network internals, but are looking to improve in that direction.
This. There should be a low level API to be able to do rarer more complicated cases, and a higher level simple API for common cases built on the lower-level API.
Just today I was working with the Web File System API, and e.g. just writing a string to a file requires seven function calls, most async. And this doesn't even handle errors. And has to be done in a worker, setting up of which is similar faff in itself. Similar horrors can be seen in e.g. IndexedDB, WebRTC and even plain old vanilla DOM manipulation. And things like Vulkan and DirectX and ffmpeg are even way worse.
The complexity can be largely justified to be able to handle all sorts of exotic cases, but vast majority of cases aren't these exotic ones.
API design should start first by sketching out how using the API looks for common cases, and those should be as simple as possible. E.g. the fetch API does this quite well. XMLHttpRequest definitely did not.
Edit: I've thought many a time that there should be some unified "porcelain" API for all the Web APIs. It would wrap all the (awesome) features in one coherent "standard library" wrapper, supporting at least the most common use cases. Modern browsers are very powerful and capable but a lot of this power and capability is largely unknown or underused because each API tends to have quite an idiosyncratic design and they are needlessly hard to learn and/or use.
Something like what jQuery did for DOM (but with less magic and no extra bells and whistles). E.g. node.js has somewhat coherent, but a bit outdated APIs (e.g. Promise support is spotty and inconsistent). Something like how Python strives for "pythonic" APIs (with varying success).
Tbf Vulkan is not intended for an endprogrammer. It is a deliberately low level standardization to allow directly control GPU hardware. The high-level approach (OpenGL) failed. The endprogrammer is supposed to use a third party middleware, not Vulkan itself.
I'm particularly fond of this pattern when you can implement the high level API that you want outside the library, which ensures that your low level API is sufficiently flexible and means you're dogfooding your own stuff as a user. It's far too easy to get used to the internal side of a tool you're building, and forget how people actually use it.
It is also important to guarantee that the two API designs are coherent and interoperable, and this kind of strict layering is the best strategy to avoid mistakes.
Also, to some extent, Emacs. There are thousands of functions (actually, a bit less than 10k in stock Emacs without packages, and over 46k in my Emacs) performing various low-level tasks, and much fewer commands (~3k in stock Emacs, almost 12k in my config), i.e., interactive functions, often higher-level, designed for the user.
It doesn't double the security surface area if the abstracted API goes through the low-level API. The outer one is just chrome, and so the risks of screwing something up there is far lower.
Unless you're using a trash language where even simple wrappers could buffer underrun or something.
Unlikely to double. The low level API exposes all capabilities, the high level API exposes a subset of those capabilities under a smaller surface. The high level API will not be as large as the low level.
Agreed. We have a rich API to build complex pipelines, but a lot of times, users do simple mappings. So there is an API for that common use case, which has a much simpler syntax, but internally uses the rich API
Problems can sneak in when you use the low-level API to do something to an object that can't be cleanly represented in the higher-level API. You need some kind of escape hatch, like a list of links to or ids of low-level details or a blob (Map<String,arbitrary-JSON> of miscellaneous data that can hold the low-level additions.
Hopefully the top-level important concepts like "amount_due" will still reflect the correct numbers!
Those problems usually present themselves by people overthinking the high level api and trying to be smart.
As an example, you can use chattr to make a file in Linux immutable. ls still shows that you have permission to write to the file, even though it will fail.
When people try to overthink the api and have it determine if you really can write to a file, people will try using the high level api first (chmod) and it won’t work because it has nothing to do with permissions.
Another example of this is AWS CDK. There are a few “levels” of constructs - high level ones that are simpler and apply “presets” that are enough for 80% of users, but the core low level ones still exist if you have a nonstandard use case.
I’m genuinely curious as to what an abstraction-rich api would look like and why it would be useful.
I’ve mainly worked in enterprise organisations or in startups transitioning into enterprise which is sort of where my expertise lies. I’ve never seen an API that wasn’t similar to the examples in this case.
I mean… I have… but they wouldn’t be labelled as high-abstraction api’s. If they needed a label it would be terrible APIs. Like sending table headers, column types in another full length array with a line for each column, and then the actual data/content in a third array. Even sending the html style that was used in what ever frontend, so that some data is represented as “some data” and other is represented as [“some data”, [“text-aligned”, “center”…],… . Yes, that is an actual example. Anyway I’ve never seen a high abstraction api and I feel like I’m missing out.
Compare GTK/Qt to raw xlib/wayland/win32/cocoa primitives - including the way that those toolkits abstract away the differences.
Or (as others have pointed out) the various `git` "porcelain" commands (checkout/add/commit/branch) compared to the primitive operations for dealing with various object types. Even just `git pull` as a combination of `git fetch` and `git merge`.
Or how a filesystem API of open/read/write/close/unlink is a very simple abstraction over block allocation and (in the old days) moving a disk head and waiting for the platter to spin under it in order to access the right sector. Not to mention the "directory" and "subdirectory" abstraction, instead of just having one giant table of inode numbers.
Edit: Or compare HTML with abstractions like "headings" and "paragraphs" to a raw typesetting language like troff or TeX.
I like the part that explains why Increase choose a different approach. Contexts matter a lot when you design something fundamental, but people usually don't appreciate this enough.
No Abstractions here really means "just use terms from the underlying system", which is a good naming principle in general.
Problems inevitably arise over time when there's multiple underlying systems and they have different names for the same thing, or, arguably worse, use both use a name but for different things. In this example, what if the underlying payment providers have different models? Also, what if the Federal Reserve, deprecates Input Message Accountability Data and switches to a new thing?
Maybe things are a lot simpler in the payment industry than they are in transportation or networking protocol. If I built a packet-switching product based on X.25 and later wanted to also support tcp/ip, what's the right abstraction?
For deprecations we're lucky in that the underlying systems don't change very much (the Input Message Accountability Data isn't going anywhere). But we'll run into collisions when we, for example, start issuing cards on Mastercard as well as Visa.
We have experimented with a couple of, um, abstractions, and may do so there. One rule we've stuck to, and likely would as well, is to keep the "substrate objects" un-abstracted but to introduce higher-level compositions for convenience. For example, there is no such thing as a "Card Payment" (https://increase.com/documentation/api#card-payments) - it's just a way to cluster related card authorization and settlement messages. But it's extremely useful for users and nontrivial to do the reconciliation, so we tried it. But we think it's essential that the underlying network messages (the "substrate objects") are also accessible in the API, along with all the underlying fields etc.
Unfortunately 100% of the public APIs I have worked on are in payments. I wish I had another lens!
How does that work if, for example, the example given of "Visa and Mastercard have subtly different reason codes for why a chargeback can be initiated, but Stripe combines those codes into a single enum so that their users don’t need to consider the two networks separately.". Unfortunately, the article doesn't explain how Increase handles that overlap. Presumably, as the article states, their customers are the sort that do care about Visa reason codes vs Mastercard reason codes, so what's the design of a "no abstraction" API in that case?
> No Abstractions here really means "just use terms from the underlying system"
Which sounds a bit like Domain Driven Design, although the "underlying system" in this case may be a bit too implementation-centered to be considered a real business domain.
To expand on that a bit: In DDD you generally defer to the names and conceptual-models the business-domain has already created. Trying to introduce your own "improved" [0] model or terms creates friction/miscommunications, adds opportunities for integration bugs, and ignores decades or even centuries of battle-tested specialized knowledge.
> the "underlying system" in this case may be a bit too implementation-centered to be considered a real business domain.
I tend to agree with this. The domain concepts would be things like charge-backs and the reasons for them. The details of the codes and categories are implementation-specific. Unless, as Increase seems to be implying, their domain is the payment networks and fintech and their customers care about them the same way a kernel programmer would care about the details of memory allocators or schedulers, while most application programmers just want them to exist and work in a consistent way.
If you love Stripe (and as a designer and tech entrepreneur I do – Stripe's simplicity and front-end skill is incredible) you might look at them and copy their ability to simplify and deliver polished experiences.
But the real mastery of Stripe is that they know their customers — and the simplicity they crave.
By this article is sounds like Increase does as well and has forged a similar laser-focus on what their customers need to build terrific design guidelines for making products. Inspiring to see.
Yeah I do think you can see in Stripes API places where there are differing tensions between “let’s make this potentially universal” and “let’s accept that this stuff is going to probably only apply for one payment method in one market”.
Personally I appreciate when the latter happens, but there’s an aesthetic decision there
This is similar to Domain-Driven Domain's "Ubiquitous Language" design pattern, making your implementation use the same real-world terminology used domain experts.
I was introduced to this concept a good while before DDD came along, when someone opined that if the nouns and verbs in your code don't match the problem domain that's an impedance mismatch and it's going to get you into trouble some day.
It really reads like a shame response to me. People are so pathologically allergic to saying "I was wrong" or "we were wrong" that they end up pushing their metaphors around like a kid trying to rearrange their vegetables on their plate to make it look like they ate some of them.
It's also smacks of the "No defects are obvious" comment in Hoare's Turing Award speech.
This is a great example of the concept “ubiquitous language” from Domain Driven Design.
Use language that your domain experts understand. If your users know about NACHA files, using other terms would mean they need to keep a mapping in their head.
On the other hand, in Stripe’s case, their users are not domain experts and so it is valuable to craft an abstraction that is understandable yet hides unnecessary detail. If you have to teach your users a language, make it as simple as possible.
Or to put it another way, they are domain experts in the kinds of transactions they want to perform, not how transactions are implemented in the financial system.
Readit News
You could always do both.
Provide a low-level abstraction-light API that allows fine control but requires deep expertise, and write a higher-level abstraction-rich API on top of it that maps to fewer simple operations for the most common use cases - which some of your clients might be implementing their own half-baked versions of anyway.
If you maintain a clean separation between the two, having both in place might mean there is less pressure to add abstractions to the low-level API, or to add warts and special-cases to the high-level API. If a client wants one of those things, it already exists - in the other API.
Bonus points for providing materials to help your clients learn how to move from one to the other. You can attract clients who do not yet have deep knowledge of payment network internals, but are looking to improve in that direction.
Just today I was working with the Web File System API, and e.g. just writing a string to a file requires seven function calls, most async. And this doesn't even handle errors. And has to be done in a worker, setting up of which is similar faff in itself. Similar horrors can be seen in e.g. IndexedDB, WebRTC and even plain old vanilla DOM manipulation. And things like Vulkan and DirectX and ffmpeg are even way worse.
The complexity can be largely justified to be able to handle all sorts of exotic cases, but vast majority of cases aren't these exotic ones.
API design should start first by sketching out how using the API looks for common cases, and those should be as simple as possible. E.g. the fetch API does this quite well. XMLHttpRequest definitely did not.
https://developer.mozilla.org/en-US/docs/Web/API/FileSystemS...
Edit: I've thought many a time that there should be some unified "porcelain" API for all the Web APIs. It would wrap all the (awesome) features in one coherent "standard library" wrapper, supporting at least the most common use cases. Modern browsers are very powerful and capable but a lot of this power and capability is largely unknown or underused because each API tends to have quite an idiosyncratic design and they are needlessly hard to learn and/or use.
Something like what jQuery did for DOM (but with less magic and no extra bells and whistles). E.g. node.js has somewhat coherent, but a bit outdated APIs (e.g. Promise support is spotty and inconsistent). Something like how Python strives for "pythonic" APIs (with varying success).
”The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.”
If only the “A” in API stood for “abstraction”. For many APIs, it probably stands for “accreted”. :)
There are high-level "porcelain" commands like branch and checkout.
And then there are low-level "plumbing" commands like commit-tree and update-ref.
[1] https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Po...
Also, to some extent, Emacs. There are thousands of functions (actually, a bit less than 10k in stock Emacs without packages, and over 46k in my Emacs) performing various low-level tasks, and much fewer commands (~3k in stock Emacs, almost 12k in my config), i.e., interactive functions, often higher-level, designed for the user.
Unless you're using a trash language where even simple wrappers could buffer underrun or something.
Hopefully the top-level important concepts like "amount_due" will still reflect the correct numbers!
As an example, you can use chattr to make a file in Linux immutable. ls still shows that you have permission to write to the file, even though it will fail.
When people try to overthink the api and have it determine if you really can write to a file, people will try using the high level api first (chmod) and it won’t work because it has nothing to do with permissions.
KISS is really needed for high level APIs.
One level of API for implementation model.
And second level for mental model.
I’ve mainly worked in enterprise organisations or in startups transitioning into enterprise which is sort of where my expertise lies. I’ve never seen an API that wasn’t similar to the examples in this case.
I mean… I have… but they wouldn’t be labelled as high-abstraction api’s. If they needed a label it would be terrible APIs. Like sending table headers, column types in another full length array with a line for each column, and then the actual data/content in a third array. Even sending the html style that was used in what ever frontend, so that some data is represented as “some data” and other is represented as [“some data”, [“text-aligned”, “center”…],… . Yes, that is an actual example. Anyway I’ve never seen a high abstraction api and I feel like I’m missing out.
Or (as others have pointed out) the various `git` "porcelain" commands (checkout/add/commit/branch) compared to the primitive operations for dealing with various object types. Even just `git pull` as a combination of `git fetch` and `git merge`.
Or how a filesystem API of open/read/write/close/unlink is a very simple abstraction over block allocation and (in the old days) moving a disk head and waiting for the platter to spin under it in order to access the right sector. Not to mention the "directory" and "subdirectory" abstraction, instead of just having one giant table of inode numbers.
Edit: Or compare HTML with abstractions like "headings" and "paragraphs" to a raw typesetting language like troff or TeX.
Deleted Comment
Problems inevitably arise over time when there's multiple underlying systems and they have different names for the same thing, or, arguably worse, use both use a name but for different things. In this example, what if the underlying payment providers have different models? Also, what if the Federal Reserve, deprecates Input Message Accountability Data and switches to a new thing?
Maybe things are a lot simpler in the payment industry than they are in transportation or networking protocol. If I built a packet-switching product based on X.25 and later wanted to also support tcp/ip, what's the right abstraction?
For deprecations we're lucky in that the underlying systems don't change very much (the Input Message Accountability Data isn't going anywhere). But we'll run into collisions when we, for example, start issuing cards on Mastercard as well as Visa.
We have experimented with a couple of, um, abstractions, and may do so there. One rule we've stuck to, and likely would as well, is to keep the "substrate objects" un-abstracted but to introduce higher-level compositions for convenience. For example, there is no such thing as a "Card Payment" (https://increase.com/documentation/api#card-payments) - it's just a way to cluster related card authorization and settlement messages. But it's extremely useful for users and nontrivial to do the reconciliation, so we tried it. But we think it's essential that the underlying network messages (the "substrate objects") are also accessible in the API, along with all the underlying fields etc.
Unfortunately 100% of the public APIs I have worked on are in payments. I wish I had another lens!
The article clearly says it also means "no unifying similar objects", which enables the naming decision.
Which sounds a bit like Domain Driven Design, although the "underlying system" in this case may be a bit too implementation-centered to be considered a real business domain.
To expand on that a bit: In DDD you generally defer to the names and conceptual-models the business-domain has already created. Trying to introduce your own "improved" [0] model or terms creates friction/miscommunications, adds opportunities for integration bugs, and ignores decades or even centuries of battle-tested specialized knowledge.
[0] https://xkcd.com/793/
I tend to agree with this. The domain concepts would be things like charge-backs and the reasons for them. The details of the codes and categories are implementation-specific. Unless, as Increase seems to be implying, their domain is the payment networks and fintech and their customers care about them the same way a kernel programmer would care about the details of memory allocators or schedulers, while most application programmers just want them to exist and work in a consistent way.
If you love Stripe (and as a designer and tech entrepreneur I do – Stripe's simplicity and front-end skill is incredible) you might look at them and copy their ability to simplify and deliver polished experiences.
But the real mastery of Stripe is that they know their customers — and the simplicity they crave.
By this article is sounds like Increase does as well and has forged a similar laser-focus on what their customers need to build terrific design guidelines for making products. Inspiring to see.
Personally I appreciate when the latter happens, but there’s an aesthetic decision there
https://thedomaindrivendesign.io/developing-the-ubiquitous-l...
It really reads like a shame response to me. People are so pathologically allergic to saying "I was wrong" or "we were wrong" that they end up pushing their metaphors around like a kid trying to rearrange their vegetables on their plate to make it look like they ate some of them.
It's also smacks of the "No defects are obvious" comment in Hoare's Turing Award speech.
Use language that your domain experts understand. If your users know about NACHA files, using other terms would mean they need to keep a mapping in their head.
On the other hand, in Stripe’s case, their users are not domain experts and so it is valuable to craft an abstraction that is understandable yet hides unnecessary detail. If you have to teach your users a language, make it as simple as possible.