Just checked our Stripe dashboard and it looks like this has quietly been doing good work for us for many months now blocking suspicious charges. It took me a few clicks to find https://dashboard.stripe.com/search/rules?rule_token=block_i... and after going through a few of them, the per-charge risk factor descriptions are really helpful too. The high-risk reasons are messages like: "This card has been used from an unusually large number of IP addresses across the Stripe network over the last 24 hours." and "This email has been linked to an unusually large number of cards across the Stripe network over the last hour."
Thanks to Stripe for making it not-a-black-box! I hope others who build machine learning systems also find a way to make its decisions understandable by humans (when possible).
One thing to note is that the risk factor surfaced in the dashboard is the primary risk factor, but the final risk level is a combination of many different factors--it's not accurate to say that the charge was blocked only because the card country did not match the IP country. That said, if you are certain that there are areas where there are high numbers of false positives, Radar does provide tools for making corrections (by writing rules to allow or send charges to review, for example). I'm happy to talk more about the specifics in your case if you send me a note (and definitely let me know if there's a way we can make this clearer in docs/product) -- I'm tara@stripe.com.
Usually fraud systems are black boxes to prevent abuse, not because they can't be human readable.
I agree however that to my eye it does usually seem excessively black-box, as it's not like most fraudsters are idiots, they already know what tools are arrayed against them.
Prime example is MaxMind's minFraud. How long do you think it took for someone to pay $500 and test their card details + billing information + shipping information + sock5/rdp before submitting payment on a MaxMind "protected" webstore?
MaxMind isn't a black box either. You can pay 0.03 USD to get the full break down of scores on an inquiry. You can register and get an ID number to token your fraudulent VM/RDP with the card before submitting a real payment. That partially helps defeat their device tracking [1].
If a system requires that it be a black box to prevent abuse, you should use another system. A black box with flaws that allow abuse is difficult to fix. With an open, auditable system, methods of abuse are more easily exposed and fixed.
I like the rotating 3D model in the landing page very much. Are they using some sort of pre-baked library which lets you create such an visualization with 30 lines of Javascript, or is it 100% custom? Maybe someone can point me to a good resource for such elegant WebGL renderings.
Glad you like it! We used Three.js (https://threejs.org) to handle rendering the icosahedron itself. Three.js actually includes an icosahedron as one of its built-in primitives, however we also wanted to add some subtle details to the model such as rounded edges. So, we created a rounded version in Cinema 4D, and then rendered both that model as well as an invisible copy of the object using the Three.js primitive. The primitive gives us easy access to things like the vertex coordinates, that are then used to position the labels, which are plain DIVs and not rendered with WebGL.
Great work, result is awesome. Love how it pops in and fits with the page background nicely. I find your web pages and documentation to be pretty much the nicest on the internet.
They're using Three.js, but it looks like the spinning icosahedron is custom-made with about 300 lines of code. That includes calculating the vertices of the icosahedron, loading the model from a 3D model file, setting up the surface material characteristics, setting up the lights, animation, moving HTML nodes with the labels along with the vertices, and mouse dragging.
Stripe's product pages and FE technical excellence are second to none. Maybe Apple, but they tend to over do it sometimes whereas Stripe does a great job of making the perfect amount of 'subtle flashy'.
I think the "golden age" of online fraud is coming to an end quickly. I've posted quite heavily on Stripe and fraud threads on HN previously if you want to read my comment history.
This is a big step for Stripe. I've often asked why they didn't have an integration with MaxMind or SiftScience already set up. They've been building their own behind-the-scenes the entire time! This feature is fantastic if you are a merchant and want to avoid fraud.
To me, the more interesting side of online credit card fraud is the merchant/payment processor side. Stripe has a cult-like following in the fraud world because it's known as the the easiest target. They make it so easy to sign up and process transactions compared to other services like Authorize.net/BrainTree/etc. They've shed this label recently, in part because the biggest forum thread discussing it was closed. The other reason was because it became so much more difficult. With this release, I think it's simply because they could identify accounts with high numbers of suspected fraudulent transactions. All the fraudsters were used to just signing up, running charges on their webstore with sock5, and waiting 2 days for bank transfers. Now Stripe can identify those transactions well in advance and assign each account a risk score. Previously, Stripe had to identify the account risk by sales volume, chargebacks, bank account provider, sign up IP, and every one's favourite privacy invader IESnare.
Fraudster's have one last shining hope against Stripe. Passing their card data to Stripe via API, instead of Stripe.JS/Checkout. Radar only works with Stripe.JS/Checkout. Setting up your own web server to pass card information prevents them from ever seeing any IP address except the web server. All you have to do to get them to be okay with this is to turn over a PCI self-compliance form. Rumour on the internet has it that there's a pre-built web application specifically for charging Stripe accounts via API.
I'm still looking for a job in fraud prevention friends at Stripe :D
Using Spreedy to act as a 'white' proxy to their API would make it an even harder job for Stripe to detect the fraudsters. What would be your solution to the problem?
It seems Spreedy is the solution I'm talking about. I can't say with absolute certainty. Their website says it uses api.stripe.com, which is great. Basically, any code that passes directly to Stripe's API instead of to Stripe.JS/Checkout.
If you strip away the "buyer's" IP then a lot of their ability to detect fraudulent transactions goes away. They still have account based limits and other methods, but my personal opinion is the anecdotal increase in difficulty of creating fraudulent Stripe accounts is due to Radar based detection.
AlphaBay. It had a 200+ page thread on Stripe and methods for creating Stripe accounts to process fraudulent payments. I feel confident saying this time 1 year ago they were easily pushing 500k-1m USD in transactions a month. There were several users who posted screen shots of single accounts reaching 50k-180k.
Threads deleted, but only to stop newbies and methods from leaking out to public readers like myself.
It seems one of Stripe's biggest risks is the impending PSD2/XS2A changes within the EU/UK. This means banks/merchants/retailers will ditch traditional card networks (and their fees) to instruct P2P payments directly. This probably opens up a host of very effective anti-fraud measures too (eg. 2FA with mobile devices).
I wonder how Stripe will react to this major change in the market?
This is why Stripe is my favorite startup out of the so-called unicorns. They are really good at finding ways to make more money, while at the same time improving customer experience.
Agreed, their design gets high marks on executing current trends well, having a distinct brand, and being functional to interact with. And that's on top of serious talent and execution on the tech and business angles as well.
I can't wait to see how they expand as a company going forward and would absolutely love to work with them if I wasn't preoccupied with more personal pursuits.
I have some holiday reading to plan and would love a book recommendation from their UX/UI people. The state-of-the-art in HCI has moved on from what I learned as a novice developer in the '90s. (I already have Tufte's VDQI of course)
(I work at Stripe) Stripe's already actioning charges based on the feedback from the machine learning models. We hope that they'll take care of most fraud for you.
If you do want to write custom rules on top of what the models are doing, we've actually built in a testing interface to the rule creation process. When you test a rule, we'll actually simulate what the rule would have done had it been active for the past 6 months. Using that information, you'd be able to tell the # of legitimate, fraudulent, or already blocked payments that would have matched the rule & make a decision on what's best for your business.
That said, we're looking to make our opinion on a given rule more clear (rules are still in beta) and would love more feedback on how we can make this better. Feel free to drop me a line (tara@stripe.com) if you have feedback!
> we've actually built in a testing interface to the rule creation process. When you test a rule, we'll actually simulate what the rule would have done had it been active for the past 6 months.
> On its own, a bimodal distribution does not tell you that a model is good. (A vacuous model that randomly assigns probabilities of just 0.0 and 1.0 would also have a bimodal score distribution.) However, in the presence of evidence that transactions with a low score are not fraudulent and transactions with a high score are fraudulent, an increasingly bimodal distribution is a sign of improved efficacy for a model.
To do this more precisely, a scoring rule (https://wiki.lesswrong.com/wiki/Scoring_rule) gives a system credit for both (1) making accurate predictions and (2) being confident at the right times.
Readit News
Thanks to Stripe for making it not-a-black-box! I hope others who build machine learning systems also find a way to make its decisions understandable by humans (when possible).
This has become clear now....but Stripe has thus been blocking legitimate payments from our users with no insight for us
80% of our clients are travelling at the time of purchase... Now we understand why we were getting so many BLOCKED transactions!!
Although this is a welcome addition, a heads up would have been nice...
I agree however that to my eye it does usually seem excessively black-box, as it's not like most fraudsters are idiots, they already know what tools are arrayed against them.
MaxMind isn't a black box either. You can pay 0.03 USD to get the full break down of scores on an inquiry. You can register and get an ID number to token your fraudulent VM/RDP with the card before submitting a real payment. That partially helps defeat their device tracking [1].
https://www.maxmind.com/en/minfraud-device-tracking
This is a big step for Stripe. I've often asked why they didn't have an integration with MaxMind or SiftScience already set up. They've been building their own behind-the-scenes the entire time! This feature is fantastic if you are a merchant and want to avoid fraud.
To me, the more interesting side of online credit card fraud is the merchant/payment processor side. Stripe has a cult-like following in the fraud world because it's known as the the easiest target. They make it so easy to sign up and process transactions compared to other services like Authorize.net/BrainTree/etc. They've shed this label recently, in part because the biggest forum thread discussing it was closed. The other reason was because it became so much more difficult. With this release, I think it's simply because they could identify accounts with high numbers of suspected fraudulent transactions. All the fraudsters were used to just signing up, running charges on their webstore with sock5, and waiting 2 days for bank transfers. Now Stripe can identify those transactions well in advance and assign each account a risk score. Previously, Stripe had to identify the account risk by sales volume, chargebacks, bank account provider, sign up IP, and every one's favourite privacy invader IESnare.
Fraudster's have one last shining hope against Stripe. Passing their card data to Stripe via API, instead of Stripe.JS/Checkout. Radar only works with Stripe.JS/Checkout. Setting up your own web server to pass card information prevents them from ever seeing any IP address except the web server. All you have to do to get them to be okay with this is to turn over a PCI self-compliance form. Rumour on the internet has it that there's a pre-built web application specifically for charging Stripe accounts via API.
I'm still looking for a job in fraud prevention friends at Stripe :D
If you strip away the "buyer's" IP then a lot of their ability to detect fraudulent transactions goes away. They still have account based limits and other methods, but my personal opinion is the anecdotal increase in difficulty of creating fraudulent Stripe accounts is due to Radar based detection.
Edit: I lied my email is rwmurray @ [vt.edu].
what was that forum?
Threads deleted, but only to stop newbies and methods from leaking out to public readers like myself.
It seems one of Stripe's biggest risks is the impending PSD2/XS2A changes within the EU/UK. This means banks/merchants/retailers will ditch traditional card networks (and their fees) to instruct P2P payments directly. This probably opens up a host of very effective anti-fraud measures too (eg. 2FA with mobile devices).
I wonder how Stripe will react to this major change in the market?
For example: https://developer.americanexpress.com/products/accept-amex
Probably not, as they are quite US-focused and 3D-Secure is still in closed beta. Probably better margins in the US.
I can't wait to see how they expand as a company going forward and would absolutely love to work with them if I wasn't preoccupied with more personal pursuits.
How would I ever know if the rule I've built is too constraining, or too loose in accepting payments?
Payment is not exactly an area of my business that I want to do a lot of trial and error..
If you do want to write custom rules on top of what the models are doing, we've actually built in a testing interface to the rule creation process. When you test a rule, we'll actually simulate what the rule would have done had it been active for the past 6 months. Using that information, you'd be able to tell the # of legitimate, fraudulent, or already blocked payments that would have matched the rule & make a decision on what's best for your business.
That said, we're looking to make our opinion on a given rule more clear (rules are still in beta) and would love more feedback on how we can make this better. Feel free to drop me a line (tara@stripe.com) if you have feedback!
That is slick.
To do this more precisely, a scoring rule (https://wiki.lesswrong.com/wiki/Scoring_rule) gives a system credit for both (1) making accurate predictions and (2) being confident at the right times.