Readit News logoReadit News
gls2ro · 3 years ago
I think the article is missing some points with regards to the REST.

If the API that the author is building is a REST API then the response for a non-existing resource is 404.

In case of REST the main idea is that if you try to GET a resource by ID then you assume that the resource exists. Thus if it does not exists => 404.

It does not matter too much which part of the URL is the one that is causing the URL to be wrong.

Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources.

In the first case, asking for `/api/v11/employees/1` is not a search or find to see if there is a V11 and if inside there is employee 1. Building an URL is an intentional choise that includes assumptions, like there is an 'api', that has a 'v11' and inside there are employees and one should have the ID 1.

The same goes for later case with employee ID 100. If you ask for an employee with an ID that means you know (or assume that you know) that employee exists. Thus if it does not it should be very level clear => 404.

In both cases responding with 200 means something like "I kind understand that you want an URL that is similar with some that I have so it is almost okish".

But in REST this is not the case. It is like you are serving some static folders and you want to get the file 100 under /api/v1/employee and that file does not exists.

Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example.

Of course this is IMHO and I am only addressing REST API. If the API is not supposed to be REST then do whatever you want but make sure you document it well and be consistent.

unregistereddev · 3 years ago
I agree - though I also agree with the author that this is an area where REST specifications are a little clunky.

Yes, `api/v1/employees/100` should return a 404, because that path represents the location of a specific entity and that entity does not exist.

Just as the author thinks it's clunky to return HTTP error codes representing application errors, I think it's clunky to apply application logic to HTTP method semantics. GET, POST, and DELETE were designed as instructions telling a web server how to handle static-ish content, and it shows in their design. Why would a GET have a body? It's a request for the server to return a specific file. This leads to breaking REST standards - for example, search endpoints that logically are GET calls (they return entities), but are implemented as POST methods because the search criteria is complex enough that you wanted a request body. Or bulk delete calls that are similarly implemented as POST methods.

REST works best in a "rules are meant to be broken" manner in my opinion. It's not a bad system (which is why it is so common), but mixing application logic with HTTP transport logic does lead to oddities.

dvlsg · 3 years ago
Some applications opted for using GET with request bodies as well, for better or worse. Elasticsearch comes to mind.

There have been some attempts to extend the list of http verbs to include something that fits those use cases more naturally - SEARCH and QUERY. QUERY got more traction, if I remember correctly, but I haven't heard much about it in a while.

https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Ver...

treis · 3 years ago
>Yes, `api/v1/employees/100` should return a 404, because that path represents the location of a specific entity and that entity does not exist.

A GET request to that path should return the current state of the resource at the path. As you note, the current state of that entity is that it does not exist.

Let's look at the spec:

>200 OK - The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

> GET an entity corresponding to the requested resource is sent in the response;

Did the request succeed?

Yes, we found the current state of the entity so we return 200

What do we return?

"An entity corresponding to the requested resource". In this case however we want to represent an entity that doesn't exist.

rglullis · 3 years ago
If you have POST for search and bulk delete, you are "doing REST wrong" and calling RPC by another name.
bachmitre · 3 years ago
this
that_james · 3 years ago
I very intentionally avoided the REST vs HTTP RPC debate.

This is _specifically_ about HTTP APIs. REST is not a synonym for HTTP but there are much better resources out there that rant on about the important of hypertext and URL support etc.

This is mostly about the perspective of consumers.

>It does not matter too much which part of the URL is the one that is causing the URL to be wrong.

Your consumer disagrees, they'd like to know if their URL was fat fingered or if a record was missing. My argument is 404 is inappropriate because the web service exists, but the record doesn't.

> Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources.

I can't say this is wrong, but it doesn't feel right. `/api/v11` straight up resolves to nowhere. Maybe this an instance where Gone is better than Not Found?

> Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example.

100% agreed. It's just a thought I had while debating with my team.

I posted this here for this exact kind of feedback :D good points raised

gls2ro · 3 years ago
(I think debating is good and healthy, so let me do a short rebuttal about the difference between `/api/v11/employees/1` and `/api/v1/employees/100` with an example specifically because you say that you are talking about _HTTP_ APIs.

So I will focus on HTTP then.

Say you install an nginx/apache and then have a static structure where you have the profiles of employees saved as PDFs directly on the disk.

Then you do `GET /public/v1/employees/1.pdf` and it works returns 200 with the content. Then you do `GET /public/v11/employees/1.pdf` in this case all servers will return 404. The same goes for `GET /public/v1/employees/100.pdf` will still be 404. What if someone asks for `GET /public/v1/employeea/1.pdf` the server will again respond with 404.

Then I go and I implement an webapp to replace that. I plan to keep the URLs the same but now there is an app that will return the .PDF as a datastream or file.

For me, I don't see any reason why to change the behaviour of the URLs just because I replaced a static app with a dynamic web app. Any HTTP server will respond in the same way thus the current one should respond the same.

Responding like this has a compatibility (let's say) reason behind that is not a personal nor related specifically to my project.

btilly · 3 years ago
If the people who designed the web didn't want information about the application code to show up as a status code, we wouldn't have status 500.

Originally, anything with a path was meant to simulate a directory tree of static files. We build it dynamically because that's easier to maintain. But making it look and act the same by returning 404s is historically correct.

Of course things evolve and move on. You're free to do as you wish. But to me you're making a bizarrely arbitrary distinction about what part of the application is allowed to return a 404 (routing code in the framework) and what aren't (your own code). Or did you not realize that a framework like Django isn't actually part of the webserver?

inanutshellus · 3 years ago
This is a bit weird to me.

Your article is entitled "I've been abusing HTTP status codes" ... but... you're not "abusing" them, you're "not using" them for your APIs. (Or, said another way, you're leaving them to their normal usage for HTTP servers.)

Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST even if you avoided mentioning it.

...

Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".

lamontcg · 3 years ago
> Your consumer disagrees, they'd like to know if their URL was fat fingered or if a record was missing.

Why?

How often does this really come up?

Who is typing in URLs like this manually?

If you're typo'ing it in code, are you not doing any kind of validation/testing against any kind of spec that can catch this?

Why is it up to the actual webservice returning a 404 to catch these kind of typo errors?

And I'm not saying I disagree with the argument -- I fully get the argument that was made, but practically the fact that you're caring about it suggests you're missing other components in your stack. You're producing a URL request which is outside of the spec of legal URLs for the webservice. You can validate that before you ever make a real web request against a real server.

jbverschoor · 3 years ago
What does this api do? Get? Update? Remove?

You can’t ignore the http verbs, so your article doesn’t make sens. You also shouldn’t ignore status codes.

You’ll also might get in trouble with caching.

You can easily use status codes, and provide all the detail + status field in the response. It makes consuming a lot easier

dragonwriter · 3 years ago
> Maybe this an instance where Gone is better than Not Found?

Gone means a resource identified by a URI existed, it no longer exists, and that resource (not the URI, necessarily) will never be available again.

smashed · 3 years ago
I think the author is aware and I noticed the term REST does not appear anywhere in the article.

The main idea of the article is exactly that he does not agree that http error codes be used for application level errors (as REST principles recommend).

I think at this point that ship has sailed. Most HTTP based API will use http error codes in various ways. I would be surprised to not receive a 404 when requesting a resource ID that does not exist for example when consuming a new API.

TimTheTinker · 3 years ago
> I think at this point that ship has sailed.

If you were talking about TCP/IP, then perhaps - because much of the internet's infrastructure is hard-coded (even burned into silicon) to use current standards. But application semantics aren't frozen in stone by any means.

Servers and clients built atop HTTP are still being written. Why not adopt better patterns that apply properly separate semantics for each layer?

I liked the REST ideas when they came out, particularly because they provided a much better and simpler alternative to SOAP. But I think improving patterns where we can is always a good idea.

yencabulator · 3 years ago
> In case of REST the main idea is that if you try to GET a resource by ID then you assume that the resource exists. Thus if it does not exists => 404.

This is actually more than a little scary.

If you're configuring the API client, and typo the base URL, your next automation run could decide that no resources exist anymore, and delete all kinds of things from your database.

I would really recommend designing APIs that communicate more than just the 404, and writing clients that check that extra part, to differentiate between "something is wrong" and "you requested item of type X from api Y, and that does not exist". If the client got a response that doesn't explicitly state X and Y, it should assume misconfiguration and give a more general error.

The response body could be used for this distinction, if you want to stick with 404.

treis · 3 years ago
>If the API that the author is building is a REST API then the response for a non-existing resource is 404.

The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying.

>Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid.

But then we've lost standardization which is the whole point of the error codes to begin with.

Feeble · 3 years ago
> But then we've lost standardization which is the whole point of the error codes to begin with.

Returning 200 and then using response body to denote missing resource is no different. So you have to choose, either 404 can be invalid path and missing employee; or 200 can be valid data or missing employee. Personally I would prefer the former as 200/OK then indicates success.

dec0dedab0de · 3 years ago
>If the API that the author is building is a REST API then the response for a non-existing resource is 404.

The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying.

As others have mentioned, you can return anything you want in the body of a 404 to clarify. However the other issues you all mentioned should really be in the 500s with errors.

quesera · 3 years ago
> this error then overlaps with server path routing issues, DNS problems, and general network issues.

No webserver will ever return a 404 for any of those cases.

0xCMP · 3 years ago
Exactly the point I bring up to not do this. However, a viable (but often difficult or unsupported) way to solve this would be changing the HTTP Message so it's clearer the intent of the error. Most frameworks like Django hardcode the response messages.
jehlakj · 3 years ago
Maybe I'm not understanding this, but wouldn't network issues take priority naturally? For example, if there's a problem with a database connection, you'd return 500. How would you know if the resource even exists?
imetatroll · 3 years ago
Historically 404 is returned for static websites. Ergo it is the standard.
guhcampos · 3 years ago
I'd say this makes the author's point entirely valid, it only shifts the argument in a broader direction: it's not the application abusing HTTP status codes: it's the REST standard itself.

Why, in all fairness, we always knew. It's an overloading of an existing technology, a bit hacky as it is, and stuff like GRPC are the answer to this, but still the simplicity of HTTP and REST are the demise of standardization.

dragonwriter · 3 years ago
> it's the REST standard itself.

REST is an architectural style, not a standard.

> It's an overloading of an existing technology, a bit hacky as it is

REST is not tied to a particular technology or protocol, and is not “overloading” whatever protocol(s) it is used with; the architectural style specifies using the subset of the available features of the underlying protocols that corresponds to REST semantics consistent with the specifications of those protocols.

vasilakisfil · 3 years ago
> Returning a 2xx code immediately tells the client that the HTTP response contains a payload that they can parse to determine the outcome of the business/domain request.

There is nothing in the protocol that mandates only 2xx status codes are parsable. Instead, the Content-Type pinpoints to the client what kind of content the body has, regardless the status code. And the content-type will most probably be what the client asked (if not, then the server announces that the response content is not subject to negotiation, something that you see rarely nowadays..)

In general I think this post reflects one of the mentalities that appeared around 10 years ago regarding HTTP APIs (another was the extreme hypermedia-driven APIs). But I really think nowadays we can do much better, we can deliver a practical API that works 99% of the times.

By the way in the latest HTTP RFC (RFC9110) status code 422 is finally officially defined (previously was part of webdav extention), which means that the war 422 vs 400 for validation and other semantic errors is over and we have a clear winner. But 200 vs 404 for something that was not found? Well that was has ended long ago I think..

conductr · 3 years ago
I think 404 being a common and useful server error is the issue. Had they/REST aligned on 204 No Content (for things like the employees/100 example) or something 2xx and less common I think it wouldn’t be much of an issue at all. I still think it’s actually not much of an issue. Of all the quirks out there, this creates little pain
dragonwriter · 3 years ago
> Had they/REST aligned on 204 No Content (for things like the employees/100 example)

Who is “they/REST”, and where have “they/REST” aligned on anything. All REST says is “use the underlying protocol without messing with its semantics, but only the parts that correspond to REST semantics”.

If the resource exists and is accurately represented by no data, then 204 fits for a gET, but that’s a different case than where there is no matching resource to the URL. 204 is mostly for modifying actions (e.g., PUT, DELETE) that don’t need to return more than the fact of success.

firebird84 · 3 years ago
I've never seen a response indicating the content is not subject to negotiation. I generally only see that the response is not acceptable (i.e. the client has indicated they can't accept it) and the server skips processing the request.
oliwarner · 3 years ago
> There is nothing in the protocol that mandates only 2xx status codes are parsable

I think a kinder reading of this point is "a 2xx response means you can parse the thing you were expecting to parse"

zenexer · 3 years ago
That should be indicated by the Content-Type header, not the status code. If you get a 2XX response but the Content-Type isn't what you expect, you probably shouldn't try to parse it. I've seen misbehaving APIs return 2XX when they really should've returned 503.

Often this it to get around the inability for some cloud-based load balancers to accept 5XX status codes as healthy responses. Take AWS ELB/ALB, for example. There are conditions under which a target instance knows the underlying issue isn't related to its own health, such as during planned maintenance, upgrades, or database failures. In these situations, it would be desirable to return 503 and configure ELB/ALB to accept that as a healthy response. Since AWS won't let you do that, some applications will just return an empty or bogus 200 response during upgrades or maintenance.

dragonwriter · 3 years ago
> I think a kinder reading of this point is “a 2xx response means you can parse the thing you were expecting to parse”

It doesn’t though.

Even with an Accept header on the request, it is not impermissible for a server to return a 2xx result with a different format indicated by the Content-Type header if it cannot, for whatever reason, return a result in a requested format (it may also, consistent with the HTTP spec, return a 406 Not Acceptable or, if it wants to be cagey about a resource representation existing if there is none matching the Accept header, 404 instead.)

If you want to know whether and how you can safely parse the response, the HTTP compliant way is to read the Content-Type header. Otherwise, you are relying on assumptions (which may be valid for out-of-band, non-HTTP reasons) about behavior that are outside of the spec.

agloeregrets · 3 years ago
This is a whole lot of nice sounding theory but ultimately in practice this is just a downright mess to handle in a real application calling the API. If you are using the Angular httpClient for example, a 404 immediately throws an observable error when your app really should be telling the user that there are no results for that query. This crosswires a potential server-level error (broken routing etc) with a request level error in error handling and would make it way harder to determine the cause of the error and lead a dev to just write `status.code ==="404" ? 'User does not exist': ....`

Did I mention that httpClient, by default, doesn't let you get 404 error bodies?

But ultimately, it is all just ideas on 'how neat it would it be to use codes!' when in practice it is so so much better to just drop it and use the codes for more literal values. Imagining a users/{X} as a 404 for invalid 'X' is fun..but like..the server actually defines it as something like /users/:userId and it isn't actually an invalid route and will not be caught by the default 404 handling. It's a conceptual dance.

SahAssar · 3 years ago
> Did I mention that httpClient, by default, doesn't let you get 404 error bodies?

Why would a http client not let you get http response bodies for statuses that usually send bodies? I could understand it for a 201, and definitely for a 204, but for a 404 it just seems like bad design of the client.

dragonwriter · 3 years ago
What I am hearing is “Angular httpClient is badly defined”, which is the kind of risk you run into a lot with big monolithic highly-opinionated frameworks.
nrdvana · 3 years ago
jQuery gives you the body. ExtJS gives you the body. Webix gives you the body. Maybe this is an Angular problem?
dariusj18 · 3 years ago
Don't get me started on HTTP 300 support
zinekeller · 3 years ago
> By the way in the latest HTTP RFC (RFC9110) status code 422 is finally officially defined (previously was part of webdav extention), which means that the war 422 vs 400 for validation and other semantic errors is over and we have a clear winner.

Unfortunately, the way 422 is written implies that the sent body/content has error(s) and not the header. It's close, but I still feel that for GET requests it's wrong.

bbatha · 3 years ago
The standard says nothing about bodies:

> The 422 (Unprocessable Entity) status code means the server understands the content type of the request entity (hence a 415(Unsupported Media Type) status code is inappropriate), and the syntax of the request entity is correct (thus a 400 (Bad Request) status code is inappropriate) but was unable to process the contained instructions. For example, this error condition may occur if an XML request body contains well-formed (i.e., syntactically correct), but semantically erroneous, XML instructions.

https://datatracker.ietf.org/doc/html/rfc4918#section-11.2

Additionally, bodies are allowed on GET requests by the standard though are not commonly used because of bad middleboxes. However, many GET requests include query params or other parts of the url to be parsed, and its completely reasonable interpretation of the standard to return 422 if those are not well-formed according to application rules.

shadowgovt · 3 years ago
There's nothing in the protocol, but in general, you should not assume non-2xx status codes have parseable payloads (more specifically, you should not assume the format of a non-200 status code).

Reason being that any intermediary step in the chain of resolving your request can return the response. If your load balancer has fallen over? You'll get a 500 with a plaintext body. Don't parse that as JSON.

(Technically, any intermediary step might also return you a 2xx with a non-parseable body, but that scenario is far, far more rare... Mostly see it in completely misconfigured servers where instead of getting the service you're trying to talk to you get the general "Set up your site" landing page).

zeveb · 3 years ago
> you should not assume the format of a non-200 status code

You should never assume format based on status code at all! You should detect it based on the Content-Type header.

> You'll get a 500 with a plaintext body. Don't parse that as JSON.

Any intermediary which returns plain text with an application/json Content-Type is badly, badly broken.

dragonwriter · 3 years ago
> There’s nothing in the protocol, but in general, you should not assume non-2xx status codes have parseable payloads (more specifically, you should not assume the format of a non-200 status code).

There’s absolutely something in the protocol, and you should absolutely use the Content-Type header to determine whether there is a body and whether it is in a parseable format irrespective of the status code, except for the minority of status codes defined as not having a body (e.g., 204 & 205.)

Spivak · 3 years ago
> There is nothing in the protocol that mandates only 2xx status codes are parsable.

I think my overly defensive view of this for real-life code is that error states are inherently situations where the normal code contracts break down and that I must make fewer assumptions about the response, for example that they are well-formed or even match the requested content-type.

The number of times that I've encountered a JSON API that can suddenly return HTML during error states or outages is too damn high. So unless you give me a 2xx I'm not going to assume I got something intelligible.

eadmund · 3 years ago
> I must make fewer assumptions about the response, for example that they are well-formed or even match the requested content-type.

I think that you should always assume that an HTTP response is a well-formed HTTP response (otherwise you can't even trust that the 404 itself is correct); and you should never assume that the received MIME type is the same as the ones you indicated you accepted; you should always check the Content-type header.

wizofaus · 3 years ago
I'd say most web APIs I've used or developed recently return JSON even with 4xx or 5xx error codes. What can be annoying is knowing what JSON schema to parse with depending on the status code, as not even the Content-Type header will tell you that. APIs (esp. those behind load balancers) that sometimes return HTML and sometimes JSON are far too common though - the problem there is that the JSON responses are appropriate for programmatic consumption (terse/semantically well defined) where HTML ones typically aren't. But even if the Content-Type header is abused (application/octet-stream anyone?) it's not hard to write code that copes with either. One API I used recently returned JSON in some cases and CSV format in others, with no distinction between status code OR content type!
dlisboa · 3 years ago
I don't see how using 404 is abusing it. If the client requested `/api/v1/employees/<non-existing-employee>` this means that the entire resource (identified by the full URL, including authority) is non-existing (i.e. Not Found). Both the technical and business requirement are the same here: what the client is trying to access cannot be found. There's no difference between that and `/api/v999/employees/1`, as that resource also does not exist, even if `/api/v1/employees/1` does.

You should detail the error further in the 404 response, and you can say "this entire path prefix could not be found" or just "the path prefix is fine, but this specific employee could not be found". The spec does not prevent you from informing that only the representation of that specific resource is not found. [1]

Using 200 for that is a bit of a cop out, you lose all of the semantics of the response and it's all pushed to out-of-band definitions (like your API's documentation). It could just as easily be argued in the article's example that every single 4xx error is just a 200 with a '{"result": false, ...}' body.

---

[1] - https://datatracker.ietf.org/doc/html/rfc7231#section-6.5.4

oceanplexian · 3 years ago
I tend to agree with this perspective. It's not HTTP's responsibility to distinguish between infinite permutations of non-existing resources, only that the resource does or doesn't exist.

Returning a 200 is counter-productive because now any sort of logging, metric system, or middleware is going to be ineffective unless you write some custom code to parse out the body and decode whatever custom error resource the developer decides to invent. And in that case why are you even using HTTP as a transport protocol in the first place if you're going to violate 30+ years of precedent.

brodo · 3 years ago
Exactly. And the same goes for all other status codes.
cpfohl · 3 years ago
Boy. This is the best explanation of this belief I've come across yet. It's coherent and reasonable, even if I disagree.

My highest scoring StackOverflow answer [1] (and my most controversial) is on exactly this topic and _no one_ in any competing answer have given _any_ sensible explanation of their reasoning. I've genuinely never understood their thought process, despite trying really, *really* hard. This explains it a bit, even though I totally disagree.

There are other tools for helping differentiate different varieties of 404 or 401. There are status codes, there's _nothing_ preventing you from returning a payload with explanation or overriding the Reason Phrase. If your HTTP client is too fragile to handle a very common usage pattern for HTTP status codes then pick a different one.

---

[1] - https://stackoverflow.com/questions/11746894/what-is-the-pro...

that_james · 3 years ago
I appreciate the kind words, thank you.

I've said it in other responses, but maybe I should have been specific that I was referring to HTTP RPC (explicitly not citing REST for a reason).

Perhaps a combination of both is best, but in my experience a nice clean break between these 2 layers at the protocol level helps clients simplify their handling of responses.

2xx: Go ahead and parse (another commenter pointed out Content-Type. That one always gets me)

4xx/5xx: Something bad happened and the request wasn't processed by the server, either because it blew up (5xx) or because you did something _very_ wrong (4xx)

The objective is clarity to the caller.

It's a little obtuse but I think it works well with that objective in mind.

nrdvana · 3 years ago
If it's not REST then shouldn't the url be /api/v1/employees?id=1000 instead of encoding the employee id in the URL? In this case the path is the endpoint, so a employee id is a parameter and not a controller action.
okamiueru · 3 years ago
I really enjoyed that SO thread, and your answer. In my opinion your take is correct, to the extent that it is fascinating that it is a controversial topic. The majority of people, (and by people, likely limited to software developers), seem to not fully understand HTTP status codes. This is of course fine, the fascinating part is the extent they insist they do. Whenever the majority, in any technically field, opine something different, I assume they are right, and I've missed something. But having designed APIs for years, I can safely say that it doesn't apply here.
okamiueru · 3 years ago
Also, the OP article does explain well what could cause this misunderstanding. HTTP resource nouns shouldn't ever be "guessed". The resource either exist or it doesn't. And if it exists, it either has a data, or it doesn't.

404: it doesn't exist 204: it exists and has no data to parse.

matsemann · 3 years ago
Wow, 452 upvotes and 89 downvotes on your answer. Very controversial, indeed!

While I agree with you, I sometimes misuse 200 OK even for actual errors. Most times because whatever internal library I've used to do the requests have made it hard to parse the error data if it happens (just throws a general exception instead when it encounters a 4XX or 5XX). Or sometimes because the request failing is somewhat expected, but the client library then insists on doing lots of error handling when it fails (logging, alarms, circuit breakers and whatnot), polluting everything.

XCSme · 3 years ago
TIL: You can view upvote/downvote count by clicking on the score. I only needed 12 years on SO to find this out...
dariusj18 · 3 years ago
Looks like HN people are changing that as we speak
stackbutterflow · 3 years ago
Reading this discussion on hn and stackoverflow, I feel sometimes software development is more philosophy than engineering.
NateEag · 3 years ago
Yes, software dev absolutely is closer to philosophy than engineering.

There is such a thing as software engineering, but that isn't what most of us do (nor is it what we should be doing, IMO).

NASA needs software engineers. Low-level critical systems need them, so the G-MAFIA has a few.

For most business purposes, though, the iterative exploratory "let's get a UI out there and see how it works" is exactly what we need.

It's a useful, cost-effective strategy when lives aren't on the line, and it isn't engineering.

(My current job title is "Software Alchemist")

dncornholio · 3 years ago
Only if you let it. Developers tend to overthin things. Be careful on where you spend you energy on.

Because honestly the answer is, as long as you are consistent, it really doesn't matter if you use status codes or not.

dgb23 · 3 years ago
I very much disagree with this. This is both semantically wrong and unpractical at the same time.

You can and probably should provide a payload for further processing in some cases, but that's an orthogonal concern.

You are also writing a HTTP server, which provides resources, expects certain formats, provides certain formats, tells intermediate layers and clients about what to cache for how long, says whether a new resource has been created or _will be_ created. You might be redirecting or you might be telling a client that they need to retry a request because there has been a conflict. Your response might be processed by different layers, some of which only care about the status codes, while others will inspect the payload to make further decisions. The list goes on.

Status codes are there so you can do all of these things and more in a standardized fashion. Even just from a human consumption perspective it is useful to just look at the code and immediately have an intuition of what happened.

Also many of the status codes make no sense at all if an application server doesn't provide them. A reverse proxy doesn't know whether there has been a transactional issue on the application server, so it cannot say 409 without that information. A caching layer doesn't know about the domain model so it cannot say whether a specific resource can be cached forever.

that_james · 3 years ago
I think the issue is here is that I should have explicitly stated this was about HTTP RPC (I am steering clear of REST for a reason).

In the context of RPC, is it still impractical? It feels clearer to me, but I might just be continuing my own confusion now

dragonwriter · 3 years ago
> In the context of RPC, is it still impractical?

If you are defining a bespoke RPC protocol that incidentally uses some parts of HTTP without much concern about the spec but only concern with what existing infrastructure will do with requests, you have…lots of freedom to design things however you want.

OTOH, a lot of work has gone into handling almost every conceivable aspect of information interchange, within a REST style (because REST was both derived from the architecture of the Web and used in updating HTTP for HTTP/1.1) in the HTTP standards (there are some gaps still, like that addressed by the draft QUERY [0] method), why reinvent the wheel, unless you are specifically dealing with use cases squarely within the gaps?

[0] https://www.ietf.org/archive/id/draft-ietf-httpbis-safe-meth...

dgb23 · 3 years ago
My comment makes sense if you lean on REST/HTTP and want to leverage the uniform communication and layered architecture. The more bespoke you get, the less you want to care about what I said above. I interpreted your post as general if that makes sense.

In any case, consistency is key. Personally I think it is easier to lean on standards in order to be consistent. Which is why I made some examples of status codes that can only be formed by an application server. I think "how do I avoid surprises, workarounds and inconsistencies" is a good question to make decisions around what we're discussing here.

efficax · 3 years ago
This is a terrible idea for observability. It would be difficult or impossible with most observabilty platforms to make it parse the http response to determine error responses, but when my systems are sending out a ton of 422s or 400s that's a useful signal to me that something is going on. I'd highly recommend using status codes to indicate application level behavior.
foreigner · 3 years ago
I think I agree with the article, because I just got bitten by exactly this. My API was returning 404 as a fairly common response when a user makes a typo, but Twilio's observability code was treating that as a serious error it needed to alert me about.
eadmund · 3 years ago
> My API was returning 404 as a fairly common response when a user makes a typo, but Twilio's observability code was treating that as a serious error it needed to alert me about.

Then Twilio's obesrvability code is broken. Requesting a non-existent resource is not a serious error if resources are requested by hand-constructed URLs.

dementiapatien · 3 years ago
Does that mean Twilio sends an alert every time some random webscraper tries to GET some favicon or /admin path that doesn't exist on the server? Doesn't that happen hundreds of times each day?
fishtoaster · 3 years ago
This is probably the biggest reason I like the author's approach: a lot of tools have assumptions about what, eg, a 404 means that might not match what it means as an application error. For example, my API was also returning quite commonly as my frontend checked the existence of various records. As a result, my chrome console was flooded with red error alerts about failing requests (404s), even though each request had "succeeded" just fine.

In another case, I had a site that used http basic auth. An xhr api request returned an expected 403, which resulted in the browser suddenly concluding the basic auth (which was unrelated to the api call) was invalid and the user needed to be reprompted for credentials again.

Both of these could be argued as browser problems, but that's the point: browsers (and observability tools and many other things) think they know what a given http status means. Using http statuses for app-level errors often breaks that.

intelijstupide · 3 years ago
> It would be difficult or impossible with most observabilty platforms to make it parse the http response

Yea, because they've all been built on top of the same faulty assumption. This isn't a counter argument to the OP in any way and I'm kind of surprised you're not the only one bringing this up as some kind of gotcha.

Obviously platforms would need to be handle and treat http and application layers differently. This is as much a given as the fact you'd have to update all your client code to wrap and unwrap the application layer on top of the http layer.

The observability of your application would actually increase as you'd be able to differentiate between an increase in application failure modes from an increase in http failure modes. That's actually really valuable.

that_james · 3 years ago
> when my systems are sending out a ton of 422s or 400s that's a useful signal to me that something is going on

Why not ping the monitoring when you encounter the business error?

From that perspective, a stream of HTTP errors could indicate a more specific kind of infrastructure failure vs just clients sending garbage.

Happy to admit I'm wrong though, it's why I posted it here, see what others think

agloeregrets · 3 years ago
Yikes, no. Other way around in fact!

Observable errors mean that there is an actual error and triggers error handling to the user. By saying, that for example, a search for a user in a list returns no results is a 4XX you are implying that the path is an error when the response is in fact, valid and useful. More than that, you now have cross-wired a REAL 4XX error (for example the API URL was changed and no result will work to it) with this fun pretend 4XX concept that ends up confusing your error handling ('Endpoint cannot be reached with this URL.') with ('Your result has no entries.'). Finally, conceptually speaking, it's not a 404. The URL you requested was handled by the endpoint you intended to reach. That is the exact opposite of a 404 for example.

To me, conceptually it's just not an error in most cases and even when it is, it's likely an error on the other side.

grose · 3 years ago
Imagine that we didn't have a fancy dynamic system and we were still using files for web pages like the good old days. In that scenario, what would happen if someone accessed the non-existent `/api/v1/employees/100` path? It would return 404 Not Found. From that perspective, I think it's hard to justify that the original intentions of HTTP would want you to return a 200.
jmbwell · 3 years ago
Indeed. The application contemplated by HTTP is hypertext. If we are being sticklers for the RFCs, modern applications implementing non-hypertext business logic via HTTP are abusing more than just response status codes.

But if we relax the “hypertext” presumption a bit and consider HTTP a protocol for the transfer of resources, surely an API can be implemented such that using response codes is sensible and useful. REST and SOAP for CRUD operations can still work just fine, and response codes can still make sense in the context of the API itself, even if the content is JSON describing business logic state.

It’s a messy world, so we have to be liberal in what we accept, but using status codes is not inherently abuse. Any ambiguity might just be a symptom of leaky abstractions in the API.

afiori · 3 years ago
In this context you would also have to include discoverability of URLs.

In this philosophy you would not suppose how employees are named and would ask for a list of valid employees URLs to a URL previously returned by `/api/v1/`.

In this way you only ever call URLs that the api provided you with, so that you are not expected to find out by trial and error whether `/api/v1/employees/100` exists or not; the api itself has already told you this and a 404 is the response to trying to build your own URLs by hand.

treis · 3 years ago
>non-existent `/api/v1/employees/100` path

But the path does exist. The employee record represented by that path doesn't exist.

eadmund · 3 years ago
No, the resource /api/v1/employees/100 does not exist.
quesera · 3 years ago
And 404 is the appropriate response code for a resource that was not found at the path requested.
kweingar · 3 years ago
I guess in a pedantic sense, all possible paths “exist.”
Spivak · 3 years ago
It's the difference between "There is nothing handling this route at all" and "there is something handling this route but the object isn't found." To me the former is an HTTP 404 and the latter is an application "Not Found."

But I also very much don't like playing the "if you expose your app over HTTP you should assimilate HTTP semantics and do a fuzzy lossy map of your application to HTTP verbs and HTTP status codes." I leave the HTTP semantics to the front end web server and live on top.

zeveb · 3 years ago
> It's the difference between "There is nothing handling this route at all" and "there is something handling this route but the object isn't found."

'Nothing handling this route' has no meaning, because routes have no meaning in a hypertext application. Clients should not generally be constructing URLs by hand; they should be receiving them from the servers they communicate with.

In the example in the article, an application dealing with employees should not be constructing URLs by appending employee IDs to strings; rather, every reference to an employee in the application should be to a URL rather than an ID. So when it requests a list of employees, it receives the equivalent of {/api/v1/employees/1, /api/v1/employees/2 … /api/v1/employees/N} rather than {1, 2 … N}.

> I also very much don't like playing the "if you expose your app over HTTP you should assimilate HTTP semantics and do a fuzzy lossy map of your application to HTTP verbs and HTTP status codes."

If you are building a hypertext application, then you should build a hypertext application. It's completely possible. Off the top of my head, protocols such as ACME (used by Let's Encrypt) are good examples to follow.

yread · 3 years ago
Isn't that enough that there are originalists for constitution, do we now have to have "what did the founding fathers^H^H^H^H^H creators of HTTP mean"-people? That was 30 years ago. Web has evolved since then. What was a good solution then may not be one now. How people work with HTTP now and then is completely different.
rglullis · 3 years ago
I'll take "People who really need to understand the principle of of Chesterton's fence" for $200, Alex...
dragonwriter · 3 years ago
The most recent specification of HTTP semantics is about a month old.
thr0wawayf00 · 3 years ago
I've definitely seen RFCs that specify HTTP status codes should be used to represent the actual application layer. The SCIM protocol, for example, specifies to return a 404 if a user cannot be found, not 200. That's definitely in the business domain, not the network domain.

One other small gripe I have with this approach is that it reduces the visibility of a failing request in the application layer, which is useful for debugging. It's not uncommon for one of my apps to issue many requests to the app server and if everything returns a 200, I have to look through the various payloads to find the request that actually failed.

I guess my question is: is it really abusing HTTP statuses if it works well overall? I rarely find myself wondering whether a request failed in the network layer or the application layer, I could probably count on one hand the number of times this scenario bitten me in my career. I don't really see the point in throwing out something that works over semantics in an RFC.

kmeisthax · 3 years ago
In the original context of HTTP being a way to distribute interlinked documents, a web application backend sending a 404 for an invalid database identifier is equally as valid as sending a 404 for an unrouteable path. The HTTP spec does not support a distinction between the two cases.

If you absolutely needed separate status codes for the two cases, you probably could use 410 Gone or 501 Not Implemented for unrouteable paths and 404 Not Found for bad IDs.