Been following signals and threads for a while, and it’s amazing. Have to be in a learning mood, but it gets pretty technical. Podcasts are a tough medium to do that in but if you really zone in, signals and threads is fantastic
> In Young Cho thought she was going to be a doctor but fell into a trading internship at Jane Street
Genuinely perplexing how they always try to show each multi-million earning engineer as some normal person and not someone that went to Exeter and Harvard
Graduating from a SUNY school, Jane Street gave me an interview and a fair shake. I didn't get the sense of elitism there. There does happen to be a lot of really smart engineers, mathematicians and scientists going to Harvard, MIT, etc.
There are places that without that Ivy League or Target School degree you don't hear back, I don't think Jane Street is one of them.
I don't really see a lot of elitism in these circles - there's a chunk of such engineers that did not in fact go to Exeter and Harvard - but what they have in common is they are all to a fault very bright technical people that can produce complicated things quite quickly, and can communicate effectively with people around them - to make sure that what they produce is in fact useful :)
I genuinely don't think people that went to Exeter and Harvard (or MIT or Stanford or Penn or other JS feeder schools) see the rest of us as human unless there's another common factor like being employed by Jane Street and having those quirky hobbies the rest of them do - and then they rationalize it away by saying that the exceptions didn't apply to top schools, or were admitted but didn't attend for "financial reasons".
I went to an "elite" school, but I worked at one HFT where the local team of 5 people had only me with a background from a similar school, and another HFT where I was one of three on my immediate team of ~12 engineers. At one point, one of my colleagues at the second firm asked me why I was there as opposed to the alternatives of (1) a company with more of a focus on the elite and cultivating power or (2) being an artist supported by my well-off parents.
Very interesting podcast, I find that the guest was very candid so it was great to hear what it is really like working at Jane Street.
The reproducible-Python notebook problem/notebook for researchers mentioned in the podcast inspired me to create a new project, branch-pad https://github.com/alexyorke/branch-pad which is an interactive Python notebook environment that allows you to create and explore multiple branches of code execution.
Ten years ago, all the $1mn+ earning engineers were in trading. Now they are in LLM/AI. Glad to see this happen, Jane Street has increased their advertising here because of this.
I read this and think: "I would love to spend a day with Jane Street and teach them how to use notebooks." So much effort is wasted because of knowledge gaps or systems that don't encourage best practices.
I just taught a course to a client this week helping them with this (and other best practices for Python).
I've been back and forth whether notebooks are really "best practice".
Currently leaning towards no, as they are hard to test, compose, and version control (it is possible, but you need bespoke notebook-specific tools, and most just don't).
Why are you trying to test and compose notebooks? Notebooks don't really fit into that lifecycle of work, they're implicitly script-like code.
Any time a notebook needed to be tested, refactoring it into a module and testing that was the better choice.
Notebooks in my opinion are by far the best tool for interactive and exploratory data work. I've been using Jupyter/IPython for 10 years and it takes very little discipline to keep them clean and clear. I've never bothered trying to deploy them in any meaningful way.
The quirks with version control are annoying, that's one reason I've switchex to marimo in the last few months.
I’m intrigued as well.. My experience is notebooks struggle as a format for production code. We encourage people who work heavily in notebooks to use them for exploratory work, but choose other tools when it comes time to ship.
When you are exploring something, experimenting, showing.. it’s great; train-of-thought structure, APIs like Pandas optimised for writing and terseness etc.
But when you have a piece of code that will lose a million dollars a minute if someone ships a bug, and which will be maintained by many engineers over many years, then you really want a format that’s optimised for long-term maintenance, incremental change, testability, and APIs optimised for readers.
My book, Effective Pandas 2, has many of them. There's a few conference talks of mine floating around on YouTube that also mention some.
Writing clean data code is one aspect. Filling in knowledge gaps is another. Covertly teaching software engineering best practices to folks who "aren't programmers" yet sit down and write code in Jupyter all day is another.
I should probably write a blog post or record a short video. (I just taught a week long course on this for a client last week.)
Readit News
Genuinely perplexing how they always try to show each multi-million earning engineer as some normal person and not someone that went to Exeter and Harvard
There are places that without that Ivy League or Target School degree you don't hear back, I don't think Jane Street is one of them.
I don't think they'd ever call someone like me back, not that I'd ever be able to pass a coding round with them.
Dead Comment
People like me are "NPCs" in their parlance.
Deleted Comment
The reproducible-Python notebook problem/notebook for researchers mentioned in the podcast inspired me to create a new project, branch-pad https://github.com/alexyorke/branch-pad which is an interactive Python notebook environment that allows you to create and explore multiple branches of code execution.
I just taught a course to a client this week helping them with this (and other best practices for Python).
Currently leaning towards no, as they are hard to test, compose, and version control (it is possible, but you need bespoke notebook-specific tools, and most just don't).
Any time a notebook needed to be tested, refactoring it into a module and testing that was the better choice.
Notebooks in my opinion are by far the best tool for interactive and exploratory data work. I've been using Jupyter/IPython for 10 years and it takes very little discipline to keep them clean and clear. I've never bothered trying to deploy them in any meaningful way.
The quirks with version control are annoying, that's one reason I've switchex to marimo in the last few months.
Do you have this written down anywhere or on video anywhere? I'd love to learn more about what you mean.
When you are exploring something, experimenting, showing.. it’s great; train-of-thought structure, APIs like Pandas optimised for writing and terseness etc.
But when you have a piece of code that will lose a million dollars a minute if someone ships a bug, and which will be maintained by many engineers over many years, then you really want a format that’s optimised for long-term maintenance, incremental change, testability, and APIs optimised for readers.
Writing clean data code is one aspect. Filling in knowledge gaps is another. Covertly teaching software engineering best practices to folks who "aren't programmers" yet sit down and write code in Jupyter all day is another.
I should probably write a blog post or record a short video. (I just taught a week long course on this for a client last week.)