I am an aspiring technologist who is interested in understanding how the tech and techniques in High frequency trading systems. Can you recommend me some reading resources?
I was in this area from 2003 to 2013. Can't think of any reading materials, but it all boils down to ultra low latency (think < 100 microseconds within the local network) and high volumes (think > 30,000 transactions per second). The state of the art may have moved since then, but:
1) Network layer: raw TCP sockets with binary packed messages (sometimes structs directly cast to and from raw buffers, keeping network/host byte order in mind); separate message data dictionary to avoid metadata overhead inside messages.
2) Processing: optimised C++ code; pointers; multi-threaded processes that utilise all available cores; minimal locking and contention between threads (queues with spin locks and bulk transfers); threads directly bound to CPU cores (prevent context switching overhead); prevent heap locks by allocating large object pools at startup;
3) Disk I/O: minimise disk I/O in logic-heavy, low latency threads; application level in-memory disk write queues; prefer append mode writes; application level in-memory caches (basically hash maps of reference data loaded at startup)
4) None of the "trendy" programming practices: no OOP design patterns; no dependency injection; functional programming, but without fluent APIs;
5) No popular frameworks: internally developed solutions optimised for our specific use cases, instead of off the shelf libraries or frameworks (e.g. the messaging middleware was in-house)
There is more, but this is what I could think of, off the top of my head.
re (2) for execution code, its actually the opposite. Typically the execution hotpath is done on a single core and some places go as far as disabling hyperthreading to eke out as much single core performance as possible.
Having fast C++ code is always good even if its not directly responsible for execution, but yes typically order entry and trade parsing at really serious HFT firms is done with an FPGA/ASIC. Sometimes a trade packet wont even hit the software and instead be processed entirely on the FPGA.
Some neat tricks people have done is to actually process packet bytes in order as they come in on the FPGA, and if it has some actionable header they just send an action before the entire trade packet is received. With that you can go down into the few ns for your w2w latency.
When I left, they were experimenting with an FPGA implementation of the TCP stack (!). Obviously a very challenging undertaking, not sure how far they got.
Interestingly, other than direct cabled network connections and trying to get ground stations physically closer to the satellites, which is obviously impossible, this is almost exactly what we did for first-level ground processing of geointelligence data for the NRO. Major difference would be data from the sensor platform doesn't even bother with TCP. It's just pure packed structs, but also encrypted with decryption handled by dedicated hardware devices. No need for PKI since it isn't public. The collection metadata from the tasking station still uses TCP.
You can troll through the job postings of most of these companies and learn about their org structures and tech stacks. When I looked almost nothing is public other than older documentaries. Nanex had some neat wiring diagrams and a few videos clips if you google them. Don't get too excited you're not going to learn about some super secret strategy or anything.
This book was good https://www.amazon.ca/Advances-Financial-Machine-Learning-Ma... in that is sort of explains at a high level how you'd build a company to look at something like this (think it was one of the last chapters). You'll need extremely deep/skilled hardware/networking groups, data acquisitions (think packet capture to data) folks, people to clean and maintenance massive the data repos, internal tool teams, quantitative analysts, traders, management/admin, etc. At this level just getting the fiber to each exchange likely costs many many millions. When you go and look through those job postings you can sort of slot these into your mental model of this flow and start to see how it might be put together.
That book is good but if you are new to the topic it will be almost unreadable. At least it was for me. I had to read it 3-4 times over the years and still find new things as my knowledge grows.
I assume from the comment they mean from one exchange location to another, but even then it’s not true. Those routes are _overbuilt_ precisely because of HFT so getting on dark fiber for that is cheaper than other backhaul fiber.
I’ve been out of HFT for a long time but when I was in it you could get onto the state of the art microwave networks between Chicago and the east coast for ~25k a month, which isn’t cheap but also not the biggest expense an HFT firm would have.
Worse fiber is a terrible choice over what’s typically used in low latency trading (which is different than but related than high frequency) - typically microwave networks are used because you get good line of sight advantages and latencies are better. HRT, stryker, and others are heavily dependent on their microwave advantage.
I think some US firms may have dug tunnels beneath the Hudson river to be able to locate in Jersey City rather than Manhattan. That probably cost millions.
That’s a hard question to answer. The question is far too broad.
HFT is a combination of lots of things: low-level computer architecture knowledge, hardware engineering, computer networking, numerical computing, machine learning, super-computing, market simulation, market microstructure knowledge, risk management, financing, psychology/game-theory, etc etc.
Usually no one person is in expert in all of those areas. Usually, a firm is successful because they put together a world class team with all of those skillsets AND they’re willing to invest heavily in the right things.
And even then, it’s so competitive that you can still fail. It’s quite challenging to get everything right. There are far more ways to fail than to succeed.
I’d suggest you pick some sub-area you’re interested and focus heavily on that.
I don't know very much about HFT, and these books are old, so take this with a cow lick of salt, but I've seen these mentioned on a blog I found here on HN (by 'yummyfajitas), and I found them illuminating:
Algorithmic Trading & DMA by Barry Johnson
Trading and Exchanges: Market Microstructure for Practitioners by Larry Harris
You might also be interested in 'yummyfajitas blog:
If people have up to date replacements for these books, I'd be super interested.
If you keep asking around, people are gunnuh recommend Reminiscences of a Stock Operator, and it's an enjoyable book but keep in mind that it's basically the fictionalized biography of a problem gambler who happened to operate in the stock market, he traded recklessly and died destitute (by suicide - when you reread it, you start to notice how he never mentions enjoying anything, or talks about the people in his life in more than passing). Don't take it too seriously.
These are still fairly canonical, and what I got recommended when I started recently. Trading at the Speed of Light by MacKenzie is very good too- a lighter read, but more recent.
I went down this road a few years ago. Reading up on algorithmic trading is a great start. HFT often employs normal algorithmic trading techniques optimized for low latency. There are some algorithms that only work when you have a temporal edge. That is the software side. The hardware side is really interesting as well. If a connection to an exchange can be reduced by a few milliseconds it gives a competitive advantage. There is a lot of money spent to be the fastest because for some types of algorithmic trading being fastest almost means free money. Because of how lucrative a minor edge can be you are not likely to find many details on the bleeding edge of HFT.
Beware, the more you read about the practices of HFT firms the more some of it looks like fraud on a grand scale. For example, front running orders is a common practice that goes unpunished for HFT firms but will land you or I in jail.
I got into algorithmic trading and HFT when I was dabbling in bot trading cryptocurrencies. The wild west of crypto bot trading is pretty much gone now but man it was fun while it lasted.
Do you work at/have worked at a trading firm? Frontrunning is indeed illegal and several firms get fined for it.
I was at a top trading shop and we and other places take compliance pretty seriously nowadays. Otherwise (if you aren't a prop firm) you'll lose clients, get fined, lose reputation, etc.
Also fyi being the very fastest has diminishing returns. You are at the whim of the exchange's matching engine processing order, which may be nondeterministic.
There are places nowadays that derive their edge from fast hw/sw, but increasingly, the most successful places are ones with the smarter strategies, higher volume, and more products traded.
I found Headland's blog post about quant trading useful [0] and as well as their other blog posts [1].
Jane Street's podcast and blog are insightful too [2]. I read somewhere that Dr. Simons (one of the founds of RenTech) has this book in his office [3].
Readit News
1) Network layer: raw TCP sockets with binary packed messages (sometimes structs directly cast to and from raw buffers, keeping network/host byte order in mind); separate message data dictionary to avoid metadata overhead inside messages.
2) Processing: optimised C++ code; pointers; multi-threaded processes that utilise all available cores; minimal locking and contention between threads (queues with spin locks and bulk transfers); threads directly bound to CPU cores (prevent context switching overhead); prevent heap locks by allocating large object pools at startup;
3) Disk I/O: minimise disk I/O in logic-heavy, low latency threads; application level in-memory disk write queues; prefer append mode writes; application level in-memory caches (basically hash maps of reference data loaded at startup)
4) None of the "trendy" programming practices: no OOP design patterns; no dependency injection; functional programming, but without fluent APIs;
5) No popular frameworks: internally developed solutions optimised for our specific use cases, instead of off the shelf libraries or frameworks (e.g. the messaging middleware was in-house)
There is more, but this is what I could think of, off the top of my head.
userspace networking with one thread busy polling, all power state management + ht disabled with receive packet steering
Some neat tricks people have done is to actually process packet bytes in order as they come in on the FPGA, and if it has some actionable header they just send an action before the entire trade packet is received. With that you can go down into the few ns for your w2w latency.
This book was good https://www.amazon.ca/Advances-Financial-Machine-Learning-Ma... in that is sort of explains at a high level how you'd build a company to look at something like this (think it was one of the last chapters). You'll need extremely deep/skilled hardware/networking groups, data acquisitions (think packet capture to data) folks, people to clean and maintenance massive the data repos, internal tool teams, quantitative analysts, traders, management/admin, etc. At this level just getting the fiber to each exchange likely costs many many millions. When you go and look through those job postings you can sort of slot these into your mental model of this flow and start to see how it might be put together.
That book is good but if you are new to the topic it will be almost unreadable. At least it was for me. I had to read it 3-4 times over the years and still find new things as my knowledge grows.
I don't know if rentec is a HFT firm or what strategies they use but this was a really good interview with one of the founders https://www.youtube.com/watch?v=QNznD9hMEh0.
I've never worked in US financial markets, but that's def not true in many countries.
< 400 USD / month can set you a server + connection within a B3 datacenter (Brazil's largest and only exchange).
I’ve been out of HFT for a long time but when I was in it you could get onto the state of the art microwave networks between Chicago and the east coast for ~25k a month, which isn’t cheap but also not the biggest expense an HFT firm would have.
Deleted Comment
HFT is a combination of lots of things: low-level computer architecture knowledge, hardware engineering, computer networking, numerical computing, machine learning, super-computing, market simulation, market microstructure knowledge, risk management, financing, psychology/game-theory, etc etc.
Usually no one person is in expert in all of those areas. Usually, a firm is successful because they put together a world class team with all of those skillsets AND they’re willing to invest heavily in the right things.
And even then, it’s so competitive that you can still fail. It’s quite challenging to get everything right. There are far more ways to fail than to succeed.
I’d suggest you pick some sub-area you’re interested and focus heavily on that.
Algorithmic Trading & DMA by Barry Johnson
Trading and Exchanges: Market Microstructure for Practitioners by Larry Harris
You might also be interested in 'yummyfajitas blog:
https://www.chrisstucchio.com/blog/2012/hft_apology.html
If people have up to date replacements for these books, I'd be super interested.
If you keep asking around, people are gunnuh recommend Reminiscences of a Stock Operator, and it's an enjoyable book but keep in mind that it's basically the fictionalized biography of a problem gambler who happened to operate in the stock market, he traded recklessly and died destitute (by suicide - when you reread it, you start to notice how he never mentions enjoying anything, or talks about the people in his life in more than passing). Don't take it too seriously.
Also read up on branchless programming, lock-free data structures and avoiding heap-allocations.
Beware, the more you read about the practices of HFT firms the more some of it looks like fraud on a grand scale. For example, front running orders is a common practice that goes unpunished for HFT firms but will land you or I in jail.
I got into algorithmic trading and HFT when I was dabbling in bot trading cryptocurrencies. The wild west of crypto bot trading is pretty much gone now but man it was fun while it lasted.
In addition to the algorithmic trading books suggested I found these links helpful: https://www.quantstart.com/articles/Beginners-Guide-to-Quant...
https://www.youtube.com/watch?v=rB5jJuMP84E - just neat
https://www.youtube.com/watch?v=oUVchh37AO8
https://www.youtube.com/watch?v=bo-F61ZuBDg - 2010 flash crash
I was at a top trading shop and we and other places take compliance pretty seriously nowadays. Otherwise (if you aren't a prop firm) you'll lose clients, get fined, lose reputation, etc.
Also fyi being the very fastest has diminishing returns. You are at the whim of the exchange's matching engine processing order, which may be nondeterministic.
There are places nowadays that derive their edge from fast hw/sw, but increasingly, the most successful places are ones with the smarter strategies, higher volume, and more products traded.
Jane Street's podcast and blog are insightful too [2]. I read somewhere that Dr. Simons (one of the founds of RenTech) has this book in his office [3].
0. https://blog.headlandstech.com/2017/08/03/quantitative-tradi...
1. https://blog.headlandstech.com/
2. https://blog.janestreet.com/ and https://signalsandthreads.com/
3. The Econometrics of Financial Markets, https://www.amazon.com/gp/product/0691043019/