One thing I love about the HFT world is the race to be fast. You cant get better by scaling out, you have to make your trading server as fast as possible. This leads to some whacky but clever hacks. I've always felt like I was working on a F1 race team when working in HFT.
I never came across isolating CPUs, TCP offload, FPGAs, layer 1 switching until I joined a HFT shop. Its like a whole different way of looking at computing system architecture.
I'm always impressed how direct the relationship between profit and mathematics/physics/electronics is in HFT. It feels like a marker of civilizational advancement – it's not enough abstraction to just have fungible means of payment and slices of enterprises, not enough to have instruments derived from these, not even enough to have the ability to trade from afar using electricity. At this point there's value even in making the trades as close to instant as possible. Is it _good_? That's what I don't know – in some way it seems more divorced from reality than the engineering of F1, in another it seems more grounded
Optimization for latency is a completely different game than optimization for throughput. The flip side has similar but opposite hacks and you can get a lot of hints on what to do by looking at that polar opposite as things to avoid.
Necessity is an amazing thing: at a company I worked for, the dev team (which I was a part of) "couldn't make things cheaper" than $14k/m for a small app.
We spun them into their own org, and now its "their money" they are spending. Their bill is $2k/m! Amazing!
It depends on the market. Most trading venues don't really require a latency much below 100us. The only ones that do are the ones that are fully deterministic. If fastest guy wins, then all you have to do is throw a good tech team at the problem and then money comes in.
Targeting a non-deterministic venue comes with its own set of challenges which are not purely technical. You don't know how long it will take for your order to reach the matching engine nor the likelihood of things changing before it gets there. Simulation is also more difficult. So you have to rely on alpha instead, which is a different game.
HFT provides liquidity and depth to the market tho. It does serve a purpose. How much? Not sure. Does it need to be super duper fast? Not sure. But that is natural in industries like this, because if you don't do it then someone else will.
In theory, HFT firms boost growth for the entire economy by reducing the liquidity premium portion of the cost of capital. But in practice it's hard to determine whether that social benefit actually outweighs the value they extract. This would be an interesting area for further economics research.
For retail investors like me, HFT firms allow me to get better trade execution.
It's not unproductive at all - you just don't like what it's being used for, and you should explicitly say that rather than proclaiming an industry to be unproductive.
It's kind of funny because, on the one hand, yes, you need to be fast and have clever hacks.
On the other hand, your edge can be explained in very simple terms: in the article, their edge comes from the fact that they had a closer colo than other market participants. ITG made money by front running customers[0], cheaper financing means you can arb an ETF more aggressively than the other guy etc.
Like, there are straightforward mechanical reasons why you're making money. No amount of clever hacks can help you if you're at a structural speed, informational or cost disadvantage.
I've found that most of the HFT strategies are pretty simple. Its exercising a proven mathematical stat/arb condition in the market. The tricky part is finding it first and getting to execute on it first.
However, there are plenty of firms that operate in the low frequency space. Long/short shops are typical for this type of trading. They don't care about order speed. They spend all their time trying to find inefficiencies or hidden correlations between financial instruments and profit from them.
Low frequency tends to be highly research compute focused, while HFT is highly trade compute focused.
It's like wearing glasses and seeing things As They Should Be whilst everybody else is telling you to just accept and deal with their blurry abstractions.
This is the best post I've found that actually describes what it's like to run a HFT firm / desk. Probably best article ever written about HFT.
For me HFT is the most interesting job one can have and it requires you to be good at a variety of things to make money: low level tech, quant modeling, risk management, business sense so you can cut good deals, etc.
Unfortunate the industry is unfairly maligned by people who really don't know what they're talking about (like Michael Lewis in Flash Boys). It's cheaper to trade now (especially for retail) than it ever has been in history and the positive effect this has had on the capital markets and the economy in general cannot be understated.
Had a skim. The "An asset manager’s conclusion" section is very interesting. If you are the HFT, you may not be selling to idiots. Something I always wondered about Market Making HFT, what happens when the market moves suddenly because of fresh info, all those orders you have to fulfill that are smart.
As an MM, you manage your spread, the distance away from the midprice (say) that you're quoting. The tighter your spread, the more trades you pick up, the more profit you make against noise trading. The looser, the fewer you pick up and, importantly, the more time you get to react to seeing informed/toxic flow.
So that's a huge part of the game, try to detect toxicity early and widen your spread as much as needed to stay out of it. In practice it means that whenever toxic flow hits a book the book's spread (the distance between the bid and ask) will widen as well as the new price is sought.
Pay for Order Flow is a big tool here because you pretty much know that all trades done on free brokers are non-informed. If you subtract that out you increase the signal-to-noise ratio on predicting toxic flow.
Finally, lets say you are hit and take on a bunch of toxic inventory. Now you need a place to dump it to. Your options depend heavily on your situation, but you might either hope you have complementary non-MM strategies which can absorb this inventory or you might hope that you can quickly shovel it off to another venue (another exchange, a dark pool) where you won't get hurt so badly.
It’s called adverse selection and informed order flow, and it’s one of the major challenges of market making. I believe they use trade secrets to try to prevent it as much as possible but they also instantly hedge if it does happen.
This is why HFTs pay so much for order flow from retail firms. Large orders on the books at public exchanges are subject to informed traders, while retail orders from Robinhood are random (or worse).
I got sucked into a finance rabbit hole and I have been reading about some "tricks" in market making and I don't see why these are problematic? Even if it is a problem, how would you even prove it? Is it wrong to place an ask order at much lower price, then backing off if it doesn't work out? Should I just put my limit order and let it stay there?
It’s all about intent. If you place a bonafide order with the intent of it getting, but at a later time (1ms is fine) decide that is no longer a good price, you are cancel it.
However, if you place an order that you do not intend to get filled to make the market think there is demand when there actually isn’t, that is illegal.
So, how do they prove this? Well, proving intent is hard, unless you’re dumb enough to send emails saying, for example, you want a smart order type that won’t get filled.
No, placing orders and then canceling them is very common and makes up the majority of orders. It is not spoofing. Spoofing is about the intent to mislead the market.
Gosh, seeing someone actually posting in the public about their experience using Orc gives me very strange feeling.
Some context here. Software design-wise, Orc was stuck in 00s. Not scalable, hard to support, hard to add new features, etc. And the corporate structure did not allow for innovations.
Somewhere in the end of 00s group of engineers left Orc (the company) to rebuild the trading platform from the ground up, and formed a new company, called Tbricks. Tbricks was such a success that Orc (again, the company) spend a lot of cash to acquire it, put key people from Tbricks to the leading positions and sunset legacy (classic, as we called them) Orc products.
After the merger, the company was renamed to Itiviti. I came there ~a year after the merger and worked for five years as an SWE in the Tbricks team. The goal of the merger was to migrate all Orc customers to Tbricks in a year or two. Five years later I left the company, and there still were a few of major customers not migrated yet. Orc was supported by one or two persons full-time, and a few of greybeards who moved on to the new projects, but still had the knowledge. And I think it is still bringing few millions USD a year in license costs.
While I'm generally pro-HFT and market making, I'm seeing some comments that are equating market making and high frequency trading. They're not the same thing. As an HFT firm, you can have two kinds of strategies: Liquidity providing and liquidity taking. This article calls these two strategies market making and "being an aggressor" in the "Strategy Zero" and "Survival!" sections. The HFT describes their aggressor strategy as follows:
"Mr. F., one of the original hires, coded what we called Strategy Zero. It was simple: have no position, wait for an option edge of a reasonable size, at least a tick. Then, hit it. Wait. Dump it out and take a tick profit."
In this case, the strategy involves 1) seeing that you can buy a security at $10, 2) knowing that the security is actually worth $10.01 and will move to this price in a few seconds, 3) lifting someone's $10 offer, 4) waiting the few seconds for the market to move, and 5) hitting someone's $10.01 bid for a $.01/share profit.
This liquidity taking strategy is not similar to a liquidity providing strategy. Liquidity providing is leaving orders on the order book for anyone to take and gaining money from one person who buys your shares for $10 and from another who sells shares to you at $10.01. This involves more risk to your business.
I think that if you want to debate the merits of whether working at an HFT firm is an actually useful service, you have to look at how often the HFT firm provides liquidity versus takes liquidity. Providing liquidity more obviously tightens spreads and improves prices, as you can't be a successful liquidity provider unless you provide the best prices first. Liquidity taking, however, involves lifting orders from exchanges that you know are incorrectly priced, which seems to me like a more obvious "bad" (or at least less good than liquidity providing).
> Liquidity providing is leaving orders on the order book for anyone to take and gaining money from one person who buys your shares for $10 and from another who sells shares to you at $10.01.
I think you have that backwards. If you always sell at $10 and buy at $10.01 you lose on every trade.
For real, especially considering since I contributed to the kickstarter or whatever. I’ve dm’d him once or twice over the years on Twitter asking about it and no response.
Readit News
I never came across isolating CPUs, TCP offload, FPGAs, layer 1 switching until I joined a HFT shop. Its like a whole different way of looking at computing system architecture.
We spun them into their own org, and now its "their money" they are spending. Their bill is $2k/m! Amazing!
Making things fast though is a whole other world.
Targeting a non-deterministic venue comes with its own set of challenges which are not purely technical. You don't know how long it will take for your order to reach the matching engine nor the likelihood of things changing before it gets there. Simulation is also more difficult. So you have to rely on alpha instead, which is a different game.
For retail investors like me, HFT firms allow me to get better trade execution.
Avoid the tendency to exaggerate your feelings.
On the other hand, your edge can be explained in very simple terms: in the article, their edge comes from the fact that they had a closer colo than other market participants. ITG made money by front running customers[0], cheaper financing means you can arb an ETF more aggressively than the other guy etc.
Like, there are straightforward mechanical reasons why you're making money. No amount of clever hacks can help you if you're at a structural speed, informational or cost disadvantage.
[0]: https://www.reuters.com/article/us-securities-regulation-itg...
However, there are plenty of firms that operate in the low frequency space. Long/short shops are typical for this type of trading. They don't care about order speed. They spend all their time trying to find inefficiencies or hidden correlations between financial instruments and profit from them.
Low frequency tends to be highly research compute focused, while HFT is highly trade compute focused.
For me HFT is the most interesting job one can have and it requires you to be good at a variety of things to make money: low level tech, quant modeling, risk management, business sense so you can cut good deals, etc.
Unfortunate the industry is unfairly maligned by people who really don't know what they're talking about (like Michael Lewis in Flash Boys). It's cheaper to trade now (especially for retail) than it ever has been in history and the positive effect this has had on the capital markets and the economy in general cannot be understated.
So that's a huge part of the game, try to detect toxicity early and widen your spread as much as needed to stay out of it. In practice it means that whenever toxic flow hits a book the book's spread (the distance between the bid and ask) will widen as well as the new price is sought.
Pay for Order Flow is a big tool here because you pretty much know that all trades done on free brokers are non-informed. If you subtract that out you increase the signal-to-noise ratio on predicting toxic flow.
Finally, lets say you are hit and take on a bunch of toxic inventory. Now you need a place to dump it to. Your options depend heavily on your situation, but you might either hope you have complementary non-MM strategies which can absorb this inventory or you might hope that you can quickly shovel it off to another venue (another exchange, a dark pool) where you won't get hurt so badly.
And if you can't do any of that, you take a loss.
1. https://en.wikipedia.org/wiki/Spoofing_(financhttps://en.wik...
2. https://en.wikipedia.org/wiki/Layering_(finance)
3. https://en.wikipedia.org/wiki/2010_flash_crash
However, if you place an order that you do not intend to get filled to make the market think there is demand when there actually isn’t, that is illegal.
So, how do they prove this? Well, proving intent is hard, unless you’re dumb enough to send emails saying, for example, you want a smart order type that won’t get filled.
Some context here. Software design-wise, Orc was stuck in 00s. Not scalable, hard to support, hard to add new features, etc. And the corporate structure did not allow for innovations.
Somewhere in the end of 00s group of engineers left Orc (the company) to rebuild the trading platform from the ground up, and formed a new company, called Tbricks. Tbricks was such a success that Orc (again, the company) spend a lot of cash to acquire it, put key people from Tbricks to the leading positions and sunset legacy (classic, as we called them) Orc products.
After the merger, the company was renamed to Itiviti. I came there ~a year after the merger and worked for five years as an SWE in the Tbricks team. The goal of the merger was to migrate all Orc customers to Tbricks in a year or two. Five years later I left the company, and there still were a few of major customers not migrated yet. Orc was supported by one or two persons full-time, and a few of greybeards who moved on to the new projects, but still had the knowledge. And I think it is still bringing few millions USD a year in license costs.
"Mr. F., one of the original hires, coded what we called Strategy Zero. It was simple: have no position, wait for an option edge of a reasonable size, at least a tick. Then, hit it. Wait. Dump it out and take a tick profit."
In this case, the strategy involves 1) seeing that you can buy a security at $10, 2) knowing that the security is actually worth $10.01 and will move to this price in a few seconds, 3) lifting someone's $10 offer, 4) waiting the few seconds for the market to move, and 5) hitting someone's $10.01 bid for a $.01/share profit.
This liquidity taking strategy is not similar to a liquidity providing strategy. Liquidity providing is leaving orders on the order book for anyone to take and gaining money from one person who buys your shares for $10 and from another who sells shares to you at $10.01. This involves more risk to your business.
I think that if you want to debate the merits of whether working at an HFT firm is an actually useful service, you have to look at how often the HFT firm provides liquidity versus takes liquidity. Providing liquidity more obviously tightens spreads and improves prices, as you can't be a successful liquidity provider unless you provide the best prices first. Liquidity taking, however, involves lifting orders from exchanges that you know are incorrectly priced, which seems to me like a more obvious "bad" (or at least less good than liquidity providing).
I think you have that backwards. If you always sell at $10 and buy at $10.01 you lose on every trade.
I'd pay $100 for the book