Readit News logoReadit News
vesinisa · 5 years ago
Klarna is no stranger to criminally lax attitude towards data privacy and security. In Finland, they implemented a checkout flow based only on your SSN (personal ID number). By simply entering someone else's SSN (which is not hard to guess/pry) you can reveal anyone's official home address.

Further, they enable a "pay later by invoice" checkout flow, again by just knowing someone's SSN. Scammers use this to order items from web stores to automated pick-up lockers with someone's else's SSN for payment info. The victim usually only becomes aware about this activity when they start getting debt collection notices for unpaid invoices from multiple stores for thousands and thousands of euros. The debt collection process in Finland is famously unfair and harsh towards the supposed "debtor" (here: victim of fraud).

Unless the "debtor" (victim) actively opposes each and every individual collection, the cases will eventually end up in court with summary judgement. This will ruin the victim's credit rating, which has devastating results for just about all aspects of life. People are known to have collapsed under the burden of all this and ended up taking their own life.

Klarna's response to all this is that they want convenient checkout experience and some fraud is unavoidable. Although there are excellent technical means available to strongly identify users in Finland, they add a minor layer of inconvenience compared to just typing in your SSN. This is OK for Klarna since they give exactly zero fucks about security as long as they can make a little buck from it.

avereveard · 5 years ago
Good to hear, I failed an interview with them at the sixth step or so because I make a point not to remember easily googleable trivia, bet their engineers are traversing rb trees on paper off memory like crazy right now trying to find a solution
2rsf · 5 years ago
In Sweden you can ask them to require Mobilt BankID confirmation to every buy, their competitors (like qliro) don't have that yet so Klarna are only half bastards. But they did get a lot of criticism from the Swedish government about the same things you have presented.
simon1573 · 5 years ago
Qliro has that too, which I know since somebody bought shoes with my SSN. I don't know if it's a general feature or if you have to contact them, but the functionality is implemented at least.
tapland · 5 years ago
Yeah. Klarna has BankID but it doesn't make them half bastards as they log in to users banks as the user, where they can see all account balances and purchases.

That makes them double bastard.

vesinisa · 5 years ago
Wait, so you have to explicitly ask Klarna to not sell stuff under your payment info to random internet strangers? That's not too good either - I think ordering without BankID should be opt-in rather than opt-out.
sneak · 5 years ago
The actual victim of the fraud is the creditor who was defrauded by the criminal; they are just leveraging the unfair legal system to push the liability onto someone who was not party to the fraud in any way (the person whose name was used in the fraud by the criminal against Klarna).

This is the lie of "identity theft". It's not identity theft, it's money/goods fraud, from a bank that didn't do proper authentication.

tapland · 5 years ago
And guess who's their own bank?

Klarna \o/

sly010 · 5 years ago
I am not sure this makes sense. Shouldn't Klarna provide proof of the transaction to the court? Won't the court look at it and throw it out as baseless? If Klarna were actually on the hook for their own money, it wold only have to happen a few times before they realize it's not worth it. edit: definitely not a finnish lawyer
vesinisa · 5 years ago
Problem is the invoice itself is real. You have to contest it actively to the debt collector and give at least some evidence as to why the debt is invalid. If you do not actively contest the collection, it will soon end up in court. This is a very routine case for a district judge where they will give a default judgement in favor of the plaintiff.

The problem is that the law in Finland is written so that even if the collection is baseless the supposed debtor needs to actively manage it or end up in legal jeopardy. Which is rather unfair if you are a victim of identity theft.

AdamJacobMuller · 5 years ago
> If Klarna were actually on the hook for their own money

Aren't they?

> it wold only have to happen a few times before they realize it's not worth it

That's an assumption. Someone did some A/B testing and said this payment flow results in X% increased sales, resulting in Y% more revenue for Klarna, it results in Z% more fraud. I don't even object to this per-se.

If Y is greater than Z, they will do it.

If they are especially awful they will consider that of the Z% of fraud it will result in A% still being paid for and B% being recovered in debt collection. That is awful.

dpatterson2008 · 5 years ago
This reminds me of this BBC article I came across: https://www.bbc.co.uk/news/business-55829879

> To use Klarna's pay later service, which defers payments for up to 30 days, shoppers only have to provide a name, email, date of birth, mobile number and billing address.

It’s mind blowing that’s all the information you need to process a payment via Klarna.

sneak · 5 years ago
The benefits of reduced friction in high-trust societies are incredible sometimes. The failure modes are predictable.
pylon · 5 years ago
This is one of the reasons I wish governments in the world implement proper digital authentication instead of relying on static identifiers like name, address, or SSN.
rjzzleep · 5 years ago
The Baltic states have had proper digital authentication for years. Priv/pub key pair on the Xth iteration digital identity card that is checked against your passport physically. The problem isn't that governments don't have proper digital authentication. It's that most countries want to reinvent it every time. The German version is a clusterfuck that they then had to force into existence by mandating it by law and yet normal citizen services can't be done with it.
vesinisa · 5 years ago
Finland offers a state of the art digital authentication system. It's just that Klarna doesn't want to use it because it adds an auhtentication step to their checkout process. It's just easier for them to take the random internet user's word for who they are (!!).

I am not sure how this is even legal under the PSD2 in EU. It might not be. But Klarna does not seem to care, and I really hope someone will take them to court over this.

grishka · 5 years ago
I'm somewhat happy that my country is so much behind on all this digital stuff. You usually have to physically present your ID to do something serious, or at least provide a picture of it. We do also have an official "government services" website, and it implements a proper oauth flow that many other government sites use and uses SSN + password for login.
tedd4u · 5 years ago
At a large site I used to work for circa 2011, before everyone had gone fully HTTPS, we received similar panicked reports from users: "I'm logged in as someone else!" Turns out an ISP in the Philippines decided to just ignore `cache-control` and `vary` headers and forcibly started caching logged-in responses along with auth cookies. Bad times. Made it clear to me why the whole web would have to go HTTPS.
NullPrefix · 5 years ago
Yeah but what about the saved traffic? Think of the poor routers that have to do all this transferring job.
Scoundreller · 5 years ago
Reminds me of a primitive web filter at work that blocked me from something. So I look at the URL, add an “s” to http and voila. I think they MITM everything now.
generic_dev_47 · 5 years ago
I worked in a project over 10 years ago where something very similar happened!

We had built and authentication service that, among other things, was used by a SyncML service that was used back in the day of feature phones to syncs contacts etc. You can imagine that getting someone else's contacts on your phone isn't exactly ideal. This was how we came to know about the problem, from customers getting other customers data!

The error was caused by a CDN switch. Our instructions to the the CDN team responsible for the switch was "Make sure the CDN honors our cache headers, if our HTTP responses say something can be cached do so, if they say that the response should not be cached then don't". We were in at least three meetings where we repeated this mantra.

I believe that the CDN team thought that they had setup the CDN correctly but they had missed an edge case. The CDN was in fact setup to cache even uncacheable responses, and served those, _only_ when it could not reach our servers.

So if there was a traffic spike and the CDN determined that our authentication servers were unreachable it would fall back to serving data that should never have been cached in the first place! Happily returning tokens to random users that had authenticated just before the traffic spike...

matsemann · 5 years ago
Something similar happened a few years ago in Norway, when the yearly tax returns were released. Everyone of course logs in at the same time. It goes down, and the cache serves someone else's data instead.
Silasdev · 5 years ago
Happened for the danish tax authority about 10 years ago as well. Although I think the issue for them was that the unique login token was based on a timestamp that several users happened to share during very busy peaks.
AtNightWeCode · 5 years ago
I would expect this to happen if an option in the line of "serve stale content if target server is unreachable" is enabled.
generic_dev_47 · 5 years ago
Yes, you are right!
corroclaro · 5 years ago
Ouch.
mrweasel · 5 years ago
Klarna is a weird company. Last I interacted with them it was clear that they are completely designed to operate within Sweden, but have no idea of how to deal with the outside world. Maybe that have changed.

I talked to Klarna maybe 10 years ago. One of the things I wanted to know was how they dealt with abuse in Sweden, given you just need the social security number of a person and then you can do purchase as that person, and Swedish SSNs are not secret.

The friendly Klarna rep. had no idea what I meant, as you could only get stuff delivered to the address associated with the SSN. Based on how that would be abused in Denmark we suggested ordering a box of random sex toys to any random person in Sweden. The only answer I got was "Why would anyone do that?"

It took less than six month for Klarna to start asking us to block addresses, because they had no way to prevent abuse.

2rsf · 5 years ago
> "Why would anyone do that?"

That's such a typical Swedish answer... but they do allow (but not as default!) to block orders and request digital confirmation

neuronic · 5 years ago
Of course, Sweden's largest export are lessons about morality.

Interesting for a country that slowly eradicates their indigenous people btw.

https://en.wikipedia.org/wiki/S%C3%A1mi_people#Discriminatio...

https://en.wikipedia.org/wiki/Swedification#Swedification_of...

rightbyte · 5 years ago
Isn't this how post order used to work? You just send a pre-printed form to the company and fill in the address and name? However, with computers automated scams are instant and could have a greater scale. I.e. instead of having some random person have a delivery pizza, you could order 1000 pizzas in 1000 towns.
randomswede · 5 years ago
I suspect that depends on where in the world.

In Sweden, the classic "order by post" required paying at the post office as you picked your parcel up, with only the pick-up slip (with the total to pay) being delivered to the address.

I have seen a few Swedish companies who didn't use the Swedish postal order system, instead opting to send a package with a giro slip and an "pay within X days" inside.

My understanding is that the post office took a small cut of the postal order payment, as a fee for guaranteeing payment. And the companies instead sending a giro slip had enough compliance with paying that it netted more money.

zaphirplane · 5 years ago
Sometime people pretend that it doesn’t make sense. Had a real-estate say offers have to be a signed contract I say it needs a expiry period so people don’t come back in a month and demand the contract is followed. He kept doubling down on “I don’t understand the problem”
AtNightWeCode · 5 years ago
I believe the user id actually was the email address when they started out.
cerved · 5 years ago
typiskt danskt att skicka massa knullsaker
mrweasel · 5 years ago
Naive, dejlige svenskere.
mavster · 5 years ago
I'm just guessing, but...

"developer gets a great idea - let's push an update to the API as a GET request so we can cache this on the CDN... forgetting that the JWT token is potentially returned in the call. Now, whoever makes the call first gets their JWT token stored for everyone else to load instead when the API call is made."

Ta-da, Klarna.

akamia · 5 years ago
I worked with a team that owned a service that resizes images. An engineer was assigned a task to add support for auto rotating images. His solution involved saving the image to a file and then using a library to handle the rotation. He used a hardcoded value for the file name. In a local environment where requests are sparse this looked fine to him and other engineers on the team missed it in code reviews. It wasn't until it went out to prod that he realized the error in this. Users started seeing other users' images because the file's content was constantly being overwritten.

When you test features like this or caching a response with a JWT it can be very easy to default to the happy path or ignore the impact of a large volume of concurrent users.

auggierose · 5 years ago
"An engineer was assigned"

Nope. That definitely wasn't an engineer.

dminor · 5 years ago
Years ago I added varnish in front of a website to cache image requests, not realizing that if the response included 'set-cookie' that was also cached.

We immediately started getting reports of random products appearing in our customers' shopping carts, as people's sessions got merged with random strangers.

Puts · 5 years ago
Just feel the urge to point out that Varnish by default do specifically not cache requests with a set-cookie header. :)
miohtama · 5 years ago
I expect something exactly like this happened. I had a similar bug long time ago. Apache was somehow incorrectly caching the request and the session cookie in the request ended up in a cache. But it happened only about 1/10,000th of the time so it was impossible to figure out the root cause.

However, one common source for this kind of bugs is to ”cache any URL ending .pdf as a static file” and then you are in fact serving logged in PDFs like customer invoices that come with the session cookie.

I think CloudFlare used to come with a default rule to treat .pdf as a static content. The responses were cached when you hit their ”cache the good stuff” checkbox.

AtNightWeCode · 5 years ago
I doubt that Klarna, a bank, have OSI layer 7 proxies in the cloud, with TLS termination in their CDN solution, on AWS. I would assume this traffic is outside of that. But then again, I know they wasted 25M+ Euros on a garbage NodeJS platform. They also created an own cloud once. Yes, it is in the trash bin.
mekkkkkk · 5 years ago
I'd actually bet against you on that one. They are still stuck with one foot in the startup mindset.
darthrupert · 5 years ago
Surprisingly many IT companies tried to create their own clouds, or at least their own kubernetes.
jordanbeiber · 5 years ago
They didn’t “create” their own cloud - they wanted to host their own hardware using an api layer to provision resources. That stuff was not built in-house.

Manhandled in-house though...

piva00 · 5 years ago
What makes you doubt that?
irjustin · 5 years ago
I can 100% see this being the cause if this comes out as the root.

But... API's really shouldn't be cached? At least not at the CDN level. The risk of serving up stale dashboard data alone makes users go ????... and we definitely don't want - not even mentioning the problem here, that's crazy.

beejiu · 5 years ago
100% agree with this. A database is, in some form, a cache of its own. If you have to add additional cache on top, it's an additional source of complexity and risk. If you are building a financial platform, you should DESIGN around this.
chrisrogers · 5 years ago
Depends on the scope of the API of course, but it's a good rule of thumb for any API with private auth
toredash · 5 years ago
Of course you can cache it, but your assuming it should never. Nothing wrong with caching API calls on the CDN forever as long as your purge the cache once you need it. Event based purging.
elamje · 5 years ago
This reminds me -

A couple of years back, I was making https://lifeboxhq.com which involved users uploading quite a bit of content. I was happily testing security with some url resource enumeration and for some reason, I could non-deterministically access user uploads via url, even on accounts I didn't own. I spent several days looking at my Flask code, javascript, etc. to debug....

I knew it wasn't my code, but I was getting more and more frustrated, then I remembered I set up Cloudflare....

Remember to exclude certain routes from Cloudflare if you want to avoid arbitrary user content from being cached without authentication.

growt · 5 years ago
I introduced a similar bug into one of my products in the past (Be honest, who hasn't?). But I'm surprised here because Klarna is a quite mature product and something like this shouldn't really happen at that stage.
yawaramin · 5 years ago
Oh, it can definitely happen even in mature products. One I worked on had pretty much the same issue as Klarna (people seeing others' info) when someone updated a web client library we were using to a new version that subtly changed how it handled concurrency.
zitterbewegung · 5 years ago
I remember something similar when there was a load balancing issue with some website where it would randomly assigning a user with someone else's account.

Deleted Comment

iratewizard · 5 years ago
To get around this, one could include the request IP address in the JWT and required a refresh token to be sent when the user's IP switches.
secureleaf · 5 years ago
This is not a safe method for protecting against this type of cache vulnerability. IP addresses are regularly shared by multiple users, especially when behind NAT (even mobile ISPs are doing carrier grade NAT these days).
remram · 5 years ago
In this context, this would just prevent everybody from logging in. The JWT would correctly get rejected but people would still be getting the wrong token from the CDN over and over.
shaan7 · 5 years ago
Ha, one time I was debugging an issue that only happened to a particular user. Lazy as I was, I hardcoded his auth token in the code "just to test". Having found the bug quickly, I was excited and did not realize I checked-in the auth token too. Bypassed reviews, pushed to prod and then reports started coming in "Hey, users are saying they are all logged in to this random guy's account".

Lessons learned the hard way ;)

lupire · 5 years ago
Did you compensate the victim of your personal and corporate negligence?

Deleted Comment

passerby1 · 5 years ago
Just out of curiosity. Is it a bad question for some reason or why is it downvoted?
dkersten · 5 years ago
Here's their official statement:

https://www.klarna.com/uk/blog/written-statement-on-app-bug/

Although I dunno about "According to GDPR standards, only non-sensitive data was exposed." since in the twitter thread someone said:

This is definitely not a test environment. I was called by someone who was logged in to my account and saw all my personal data including bank details, Klarna card etc.

And while I'm told the bank details are obfuscated (I don't use Klarna, I dunno), I would consider the phone number to be a clear breach of my privacy under GDPR.

Although, the twitter account that said that has 0 followers, so maybe its not true. I dunno. I know someone who works for Klarna and he told me: "Full investigation will take time. There's a LOT of engineers working on this. Only confirmation I have currently is that the firstname was visible."

Going by the screenshots, first name and account balance. Doesn't seem that bad from a GDPR point of view. Still bad, of course, but not suuuper sensitive.

EDIT: Nevermind: https://twitter.com/esraefe/status/1397843949985931265

skeeter2020 · 5 years ago
And this is both maddening AND make the problem worse (from the CEO):

"We are truly sorry for any inconvenience..."

dkersten · 5 years ago
Oof, yes, its not about inconvenience...
henvic · 5 years ago
As a software engineer, I hate when I add a check for something "that will never happen" but that if happens is awful, and people complain.

A classic example: you need to get a user from a session, check against a database, and continue if they're signed in.

Then I add a simple if databaseUser.Username != form.Username and people will say "if that happens we've something worse wrong". Geez, something might be wrong and such double checking might provide to be useful.

On a smaller scale, bits flip due to cosmic rays and so on. Of course, there must be a limit where we stop, but people are used to actively avoid doing such "silly assertions" even for important steps.

¯\_(ツ)_/¯

jojohohanon · 5 years ago
A lifetime ago I was writing code for airline data processing. The specs are very clear about what the valid representation of every field was (less so about what they meant, but...).

So we generated our parser to fail if field ORG/1457 (made up) was not numeric max 8 digits. Or missing where mandatory.

Even if we never touched the data in that field.

Turns out that no-one else used the spec that way. No two were the same, so we had to basically implement two layers of parsing. One to put the data in a common parse tree, and the other to per-sending-mainframe interpret the data as how the sender had implemented.

We assumed that the mainframe would never send illformed data, and indeed that-could-never-happen. But they differed in what they thought was well formed.

rad_gruchalski · 5 years ago
Klarna uses erlang and erlang enforces exhaustive conditionals and pattern matching so the error / unsupported case is always handled.
rightbyte · 5 years ago
I like defensive programming. Even though I think the state is unreachable, it feels nice to add a panic assert just in case.
YeBanKo · 5 years ago
If it is due to cache, then extra check like you described probably would not help.
anticristi · 5 years ago
Most people I met who do double checks would simply return "not loggen in" and issue a WARN deep within the other 200 WARNs-per-second. That is IMHO a very bad usage of double checks. It gives a false sense of security and masks the deeper problem until it's too late.

However, if you make the assertion fail loud, then it provides an additional security layer and should be used as often as makes sense.

jacquesm · 5 years ago
This is very good practice as far as I'm concerned. Functions should treat their arguments as potentially hostile input.
cerved · 5 years ago
maybe if it helps to fail fast and only public functions
geofft · 5 years ago
I think there's merit in objecting to "that will never happen" checks in some cases (though, to be clear, I'm not saying the people objecting to your code are thinking about the same thing I am).

Specifically, if you have data that is loaded from some other source, your extra safety check might be checking data that's loaded from the same source, in a way where if something did go wrong, it went wrong in both places you're checking.

In this case, it seems pretty unlikely that Klarna's bug was that they ran "SELECT * FROM users WHERE Username = 'joeuser'" and they got back a row where Username != 'joeuser'. I don't think there's a recorded case of that ever happening with databases.

However, it seems much more likely that Klarna's bug was in HTTP caching or something, that results were returned for the wrong user. Then there's no opportunity to see databaseUser.Username != form.Username: that check would have indicated that things are correct, but the username being passed into this code was wrong in the first place. That sort of problem definitely happens in the wild - see the "Kenneth" story elsewhere in these comments, or off the top of my head https://blog.zulip.com/2021/03/20/zulip-cloud-security-incid... from two months ago.

And if it is, somehow, a database bug, why do you trust the database at that point? What if the database returns part of one row and part of another? What if it returns the username you sent in because of some optimization to avoid copying data, but thanks to a bug (or a cosmic ray) it reads in the rest of the data from an unrelated row? In the unlikely but not totally impossible case that you need to protect yourself against this, validating the username isn't enough; you'd better sign the entire database row and validate the signature before trying to use any of the data that's been returned. (And come up with some reason why you trust your own app code more than the database.)

The problem with such "silly assertions" is that they make you feel like you've added test coverage, when the thing you're testing is something like a database that is extensively tested by its vendor and by everyone else using the database, and there are other seams in your code which are much more likely to break. Meanwhile, they make the code longer and harder to read, which prevents readers of the code from easily identifying what those seams are.

(And by slowing down the API endpoint that talks to the database, it motivates other developers to try to put some caching in front of that endpoint, which may actually cause this sort of problem!)

henvic · 5 years ago
> I don't think there's a recorded case of that ever happening with databases. > and there are other seams in your code which are much more likely to break.

One such thing is the abuse of layers and layers of abstractions. For example, many people (unfortunately, in my view) love to use ORMs and query builds, and things like these are much more easier to happen when things are too generic.

And signing the entire database row and validating it, and so on, might be unjustified for most people, especially if you already count with correction from a TLS layer, and you can just have the trade-off of adding a simple conditional to check if the data you receive is sane.

This is not something essential for everything, but that is nice to have, especially the further you're out of control.

For example, if you retrieve data from an external API you should not trust it blindly, but rely on your internal references (security concerns aside, I'm talking about other kind of erratic behavior or bad data).

mekkkkkk · 5 years ago
I agree, and I've also been called out for doing "stupid" defensive assertions. It's almost certainly not a code-level issue this time though. This whole thing reeks of infrastructure/caching issues.
bagacrap · 5 years ago
it's fine to make the check but I hope you don't sweep it under the rug with an early out without at least logging the occurrence
henvic · 5 years ago
uh? Why would you make the check, find a critical internal inconsistency, and skip logging it? :)