These pieces of advice are useful. However, they don't touch the bottleneck: mental health. And no, it is not "like any other demanding job". A PhD hits on two fronts - one is "all or nothing". If you spend years and still haven't submitted your dissertation, it is a career-ending failure. The other is its tie to one's identity. You put sweat, blood, and tears into your research, only to be rejected at a journal or conference because the result is "technically correct but not significant enough". Sure, there are similar parts in other careers - from talking with people, it works a bit similarly in medicine (when it comes to "all or nothing") and art (when it comes to this identity).
If people fail, it is mostly because they burn out. If they succeed, it is not unlikely that they will need to heal their burnout wounds anyway.
I am sure Karpathy's experience is different. But most people starting their PhDs are not Karpathy.
The peer review paper requirement puts you in a situation where if your topic of research happen to not be interesting for the reviewers (that you have no control over), you can be a talented student that worked very hard and still fail due to being out of time after multiple successive rejections.
Your supervisor may not understand this until it’s too late, and you may not have the ability to judge your adviser's ability to do so until you are committed.
The main problem is that you were raised in a school system where if you show up, study and do your assignments you are pretty much guaranteed to succeed sooner or later. A PhD is not like that.
Too many people stay in academia out of inertia and being comfortable with the "school" mode of existence and are afraid of the broad wide world and the decisions involved. They finish their masters and liked the classes and the thesis topic and so they stay.
But as you said, a PhD is quite different than all schooling before that. And that's good. A PhD is supposed to signify that you contributed new scientific value as judged by the expert international community, not just your teacher. Of course there are many wrinkles on this story like sloppy knee-jerk reviews etc.
But anything in life where you "just show up" and fulfill some explicit assignments tends not to be very valuable. If just showing up and doing what someone else decided for you is enough for a thing, that thing will lose value very soon. Similarly if you make sure almost everyone can do it, it won't have value, but will become a participation trophy.
But nothing in real life work like that. School is fake. You don't get a job just by showing up or having a diploma. Nobody will fall in love and start a relationship/family with you for showing up and fulfilling some list of criteria. Nobody will fund your startup or strike a business deal with your company because you showed up and did some assigned tasks.
In almost all aspects of life being proactive and exercising agency will get you much further than the teacher's pet mindset that school instills. And unfortunately rather than selecting for it, the PhD selects against such agency again because it's the safe option and people who are ready for an adventure usually dislike the academic environment. Not all of couse, I obviously don't mean every single person fits this. But in my experience this explains part of the mismatch in expectations and reality for the "I was a good student so a PhD felt natural" people. Not those come into the PhD with a well thought out plan, and knowing exactly why they want to pursue it, the upsides and downsides etc.
> The main problem is that you were raised in a school system where if you show up, study and do your assignments you are pretty much guaranteed to succeed sooner or later. A PhD is not like that.
It was not my case and bold of you to assume so. I had peer-reviewed publications before I even applied for PhD.
While I do know some people who expected PhD to be "more classes with more difficult assignments", the mast majority of PhDs I know had nothing to do with mentality you described.
Your point is valid in many ways. The picture can be a little brighter. The PhD path does not have to be an all or nothing.
1. You can hedge your bets by submitting your work to various conferences of various qualities (without going 3rd-tier, you can bet across 1st-tier and 2nd-tier)
2. You can spend time choosing the professor and the topic before going all in
3. You can seek advice and social interactions within your research group, departement and school
None of this is a silver bullet, but it compounds in the right direction.
PhD programs are very different. The environment Karpathy describes is fairly similar to what I saw as a math PhD in a good school, but not an ivy. My theoretical physics PhD friends had the same setup as I had, but experimentals lived in a different world, long hours in the lab every day, including weekends.
My advisor was well established, tenured prof with a number of students. I had to teach, but the effort was light. We taught large, basic courses that are boring for tenured profs. We usually requested the same 1-2 classes to teach and after the first round had all the materials (homework, quizzes, etc.) and could teach on autopilot. University gave us undergrad graders to grade assignments but I never used them since I wanted to see what my students wrote. Which is a testament that the load was light; if I was drowning I would use all free help I could.
But there was a cult of academia. "Get an academic job or you are a loser" mentality was prevalent. My advisor was disappointed, but OK when I decided to go into industry after PhD, but a friend's (Physics PhD from Harvard, CEO of a profitable startup now) advisor does not talk to him anymore because he did not stay in academia.
And I only realized long after finishing my PhD how incredibly lonely PhD path is. You live in your bubble for years, with minimal interactions outside a few other folks at the same grad school. Stipend was enough for basic living, but not much else. No good vacations, ski trips with friends, etc. And a few somewhat creepy characters that grow in this lifestyle. This is all surmountable, but the mental toughness required is certainly something to keep in mind. I did not have that mental toughness, but was an introvert, which helped a lot. But looking back I see that I also could have gone off the rails. My 2c.
It depends highly on the field. In history, sure. The point of getting a history PhD is to become a history professor, and you can't do that if you don't get the PhD, and meanwhile history PhDs don't meaningfully open up any other job prospects, so attempting and failing to get a PhD provides negative value.
In CS and many engineering disciplines, there is a long history of people dropping out of PhDs and landing in industry. The industry is therefore much more accustomed to, and therefore accommodating to, people taking this path. Whether it's a maximally efficient use of time is another question, but it's certainly not wasted effort.
But I do agree that it's stressful nonetheless because it still feels like a failure even if it is not actually in reality. I wrote about this when I put down my own PhD journey here [1]. In particular after the control replication (2017) paper, I very nearly quit out of academia entirely despite it being my biggest contribution to the field by far.
Eh. It's different, but framing it as uniquely challenging seems silly. There are very few other jobs where you don't need to deliver any specific, measurable results for months or years. And your "career-ending" outcome is that you go and get a cozy industry job in the same field because you already have a degree. Now, you might have a difficulty adjusting to that because they will want you to get stuff done.
The one piece of advice I give new PhD students is to maintain a list of your references for a bibliography ahead of time. For every paper you read, copy the citation in BibTeX format and write a couple of sentences to remind yourself what the paper was about. Do this for every source, even if it doesn't seem important at the time.
Use zotero and betterbibtex. By all means type a comment so you know which ideas came from where but I'm a big advocate of taking notes by hand when you really want to understand something, as opposed to reminding your future self about something you already understand.
Not within a PhD, but as a side project I work on a research project on wikiversity about grammatical gender in French. It does reference a bunch of books and academic works, like probably a hundred I guess. The most tedious work though is to check which nouns are used only in a single gender of do have some epicenic or specific inflection used in the wild and giving a reference that attest that when it's not already so consensual that most general public dictionary would already document the fact. For that the research refers to thousand of webpages. I'm glad that most of the time I just need to drop the DOI, ISBN, or page URL and MediaWiki will handle the filing of the most relevant fields. That's not perfect, it generates the output with many different models currently (some don't have an excerpt field), and some required fields might be left blank, url to pdf won't work, and so on. But all in all it make the process of taking note of the reference quick and not going too much in my way. Creating a structured database out of it can certainly be done later.
Zotero and AI have this covered now. If there's one thing AI is good at it's summarising crappy formatted papers. Never understood the 2 and 3 column thing. Horrendous way to format something.
2-column format has narrower columns, which means that your eyes move more vertically than horizontally while reading it. That is considered conducive to “skimming” long texts if you’re a “speed reader”.
Do you mean that you’re using AI as a search engine for your local bibliography? I haven’t seen any AI plugins for Zotero.
The comments you write in to Zotero are not what paper is about - abstract covers this well enough - it’s about what you found interesting or useful about the paper.
I have had some fun exhuming my old LaTeX skills and assembling a BibTeX bibliography from which I automatically extract the right entries presented in whichever style is needed for a given paper and for my own (HTML) site. I even publish the collection in Zenodo in case useful to others. I use the 'annote' field for the reminder you suggest.
Ha! You just made me remember how much I used JabRef (open source bibtex reference app) back in 2004 when I did my PhD.
It was the best/worst 4 years of my life. I studied overseas (uk), met my future wife and got a PhD that really wasn't useful for much to me. Fortunately it was under a scholarship.
The lack of good tools to have good research notes with good search is kind of mind-boggling. I have reverted to having a website for myself, a private one that I run on my machine, using mkdocs which comes close to what I would want.
Presumably the idea is that you put the relevant parts of the list in your thesis. You need to convince your examiner that you understand the background to the original research you did, and a solid reference list (with supporting text in the introductory/background section of your thesis) is part of doing that.
Personally I did the references at the end and didn't feel like I suffered from that decision, but the key references in my particular area were a relatively small and well-known set.
I discounted the remainder of the piece after reading this:
Personal freedom. As a PhD student you’re your own boss. Want to sleep in today? Sure. Want to skip a day and go on a vacation? Sure. All that matters is your final output and no one will force you to clock in from 9am to 5pm. Of course, some advisers might be more or less flexible about it. . .
For some programs, this is untrue. Your advisor, your experiments, or your conference deadlines govern your schedule.
I'd go further and say it's dangerously untrue. What I advise people is that your results are constantly decaying. Only a rate of progress that exceeds the rate of decay will get you out the door. Decay happens for a number of reasons:
* Your records are never good enough to completely replace your memory of what you did. The longer it takes, the more studies, readings, etc., that you will have to repeat.
* In physical and biological sciences, equipment breaks down, gets taken away, facilities get moved, etc. This stuff happens at a constant rate, and is a pure time cost.
* Technological progress gradually raises the bar for the minimum quality of some results, e.g., in computation. Even "theory" is highly computational these days.
There are also risks of career-ending accidents that can be treated as a constant risk per unit time:
* Your advisor dies, retires, gets promoted to administration, loses funding, changes jobs, gets embroiled in ethical / legal issues, etc.
* Some unexpected new result from another team or industry erases the relevance or novelty of your work.
* You get sick, have family crisis, etc.
* Burnout
Results are the wrong unit of measure. A better KPI is results per unit time. The people who look like they fucked around for 4 years then submitted a brilliant thesis were either working hard all along, or were just brilliant, which I certainly wasn't.
My then-fiancee and I were both grad students. We made a pact to meet at 7:00 every morning in the cafe across from the research building for coffee, to force both of us to stay on a work schedule.
This is the kind of advice I give incoming graduate students. The sooner you start to treat grad school like a full-time job, the better. I was in a similar boat: my wife and I were both in grad school at the same time. We worked 9-5 every day, even if we weren't going in to the office. We both finished on time, and generally didn't have a difficult time with our degrees.
He is obviously talking about computer science. Yes, I know in biology or medicine you can often only access the experimental devices during set hours and the lab may not be accessible 24/7 etc. But in computer science the schedule is mostly free, except for meetings and teaching duties but those are specific time slots not a regular clock-in clock-out job like a cashier or bus driver.
In my part of the world (central Europe), the vast majority of PhD-students is actually employed by the university they aim to obtain the PhD from. So in addition to working on your thesis you most likely have to support other research projects as well as do a lot of teaching. The model of a free PhD student certainly exists, but it is rare.
It's pretty common in ivy-leave US universities (which is what the article is biased towards). There, you only have to TA a bit and you certainly don't work for the uni or the dept.
The flip side of the coin is that if you don't have a determined rest time, you are always working. During my PhD, I couldn't feel the difference between weekdays and weekends; thus, I felt guilty on weekends when I was not making progress.
Karpathy is an interesting case of PhD gone industry and he mentions this topic in the article. In my field of computational social science it is sadly very taboo to happily leave the academy. Yet, they don’t do much to make it more appealing. My biggest win was to find a group of people outside of my research group that I liked collaborating with. Research is more fun as a team sport.
What was the motivation? Honestly, I was too lazy to get a job and staying in academia for another 3+ years seemed amazing (probably not recommended, but it worked out OK for me).
What helped get me through it:
1) Doing something I genuinely enjoyed - I approached the Computer Vision professor who gave me some ideas. I super enjoy writing code, and the idea of processing gigabytes of video to produce answers seemed cool. I treated it as a super difficult programming project.
2) Breaking my leg - Just before starting, I broke my leg badly. And that meant working from home with a weekly visit from the professor with a stack of reading papers. That time spent understanding state of the art was super useful.
3) Funding - At some point, DARPA gave enough money for me not to worry about funding, so I never had to work a job or get distracted.
4) Marriage - The final straight of writing a thesis was tough and I was super lucky to have a supportive wife who pushed me to get-shit-done.
As if "A" or "C" defined a person capacity. I know some straight A's that went directly for a repetitive and boring but well paid and stable job. Other stayed in academia and turned top scientists.
Academia is a very particular dynamic very difficult to find elsewhere, and some people dig it. You can watch some people finding the same dynamic at Google for example, where they are allowed and encouraged to fiddle around and keep publishing (e.g. the Attention paper, why would Google allow such publication?). Such dynamics are explored in Terence Kealy book "The economic laws of scientific research".
This varies widely between fields and institutions. Getting a PhD position nowadays in ML or computer vision is much harder. You need to already have publications when you apply and need to have experience specifically in the subfield, give a good talk, an interview, a good motivation letter / research statement, recommendation letters from good internships and multiple PIs you worked with, good grades, etc.
It can be different in other fields an in lower tier colleges.
I am a bit surprised that this article talks so much about actual PhD stuff than high level guidance. Maybe it has to do with the author's personal background/experience or field.
Something I didn't see in the article:
Depending on your field, it can be extremely hard to get tenure. Unless you are a genius and are seeing signs you are well on your way to getting professorship and tenure (publishing good and important papers, really good at communication, checking all the other boxes), you'd better consider an off-ramp as early as possible.
In some fields, 100 people compete for 1 open positions, and that's rough. Having been involved in such a hiring process, I know it's extremely challenging.
I was smart enough to realize I'd never be a good researcher relatively early during my PhD and started preparing for job interviews. Sadly, I see too many people still having unrealistic dreams about being a professor late in their PhD. They even do postdoc and do that for many years until they finally discover they are not hireable. Good news is that they often find an industry job quickly after switching.
I could see that clearly -- their research, originality, communication and understanding of the field is just not there, and one doesn't become a professor without being completely in control of the direction of their academic research.
But it's hard to tell people "you are just not good enough for this". In most situations, you'd be considered unsupportive. However, in this case, it's the best thing you can say to a struggling PhD/postdoc.
(Many professors are completely incapable of advising on students' career. They often find it surprising that a student wants to go into industry. They hold completely incorrect assumptions like a PhD student just goes into a management role as soon as they graduate. Rarely a professor tells you that you should not pursue tenure.)
* Much of that was direct feedback from my advisor -- that I did't understand my research subject thoroughly, I could not explain or describe things very clearly, and even someone outside the field could ask a question which I have trouble responding.
* I had difficulty coming up with my own ideas. A good PhD student should develop a good sense of direction halfway through their research PhD, and by the end they should be able to conduct their own research independently. I was nothing like that.
* I attended talks from fellow students that were invited to the department and went to conferences. I know what good researchers are like -- they do good work, publish papers relatively early in their PhD, know everything about the field and what are they doing, have confidence, communicate with others well and have great delivery in presentation.
-- * This might be the one that is most compelling. You could tell who is going to be a professor and who is not. (As I am typing this, I look up the guy I have in mind. He just got assistant professorship last year. I am not all surprised.)
Granted, I am far from a model PhD student. My research is mediocre. But I have seen people who deliver even worse research and have even less chance of ever getting tenure spending more time in academia. So I hope my words can be helpful.
All that said, I would say you don't need to self doubt. If you are smart and constantly think about your career, and if you go to conferences, you'll know whether this is for you and what to do.
“How to get into a top PhD program: get ~3 famous professors to write letters saying you’re one of the five best students they’ve ever worked with.”
I feel like this particular advice applies to a very small subset of people. If I’d had professors telling me that I certainly would have considered doing a PhD!
That's what used to be thought about any school at all, then about high school diplomas then about a university diploma. Each time it was decided that by expanding the number of people they would get uplifted to a better standard of life, a higher class etc. But social status is relative and mostly zero-sum, so the value of a diploma simply goes down when everyone has one. Chasing credentials without actual value contributions cannot cash out in anything real.
"How to be a top YouTuber" by MrBeast. "How to be a top athlete" by C. Ronaldo. "How to become a popstar" by Ed Sheeran. You know that such advice will have limited usefulness to most of the aspiring people.
Karpathy is an exceptional person, maybe not as much as Ronaldo in football but taking advice from him similarly won't be guaranteed to work. You can't have guarantees in such things.
In truth, the more literal but not fully literal thing that happens regarding surviving a PhD is that you try to publish a paper in a top venue but after several rejections you publish in a lower tier one, then you do two more followup in similarly second tier but not terrible venues and you get a "magna cum laude" or perhaps a "cum laude" once you reach 5 years and the prof wants to avoid the embarrassment of not having graduated you.
Of course many people don't come into the PhD with such plans, they expect a summa cum laude and papers in top venues and talk invitations and so on, since they've always been a top student so far.
Interesting. A couple of questions:
- How young are the kids?
- How do they behave, especially with essentials like eating and sleeping habits?
- Could you carve out a morning and/or evening routine for yourself?
- How much outside help could you rely on (grandparents nearby, lovely neighbours...)?
My kids now are 7,5,2.
They were obviously younger when I was doing it.
2 years was during COVID so I never really had to travel for classes which really saved time.
I did most of the coding and ideation on either paternity leave or holidays. Evenings were often running smaller experiments or eventually endless iteration of a paper.
My wife was great and helped me focus usually 8-11pm most nights and some weekends 1-4pm.
We don't live nearby any family and friends were not much help.
Readit News
If people fail, it is mostly because they burn out. If they succeed, it is not unlikely that they will need to heal their burnout wounds anyway.
I am sure Karpathy's experience is different. But most people starting their PhDs are not Karpathy.
See also "The Lord of the Rings: an allegory of the PhD?" http://danny.oz.au/danny/humour/phd_lotr.html
Sure you may survive. But even if all goes well, you succeed, there will be a void in you after the quest.
Your supervisor may not understand this until it’s too late, and you may not have the ability to judge your adviser's ability to do so until you are committed.
The main problem is that you were raised in a school system where if you show up, study and do your assignments you are pretty much guaranteed to succeed sooner or later. A PhD is not like that.
But as you said, a PhD is quite different than all schooling before that. And that's good. A PhD is supposed to signify that you contributed new scientific value as judged by the expert international community, not just your teacher. Of course there are many wrinkles on this story like sloppy knee-jerk reviews etc.
But anything in life where you "just show up" and fulfill some explicit assignments tends not to be very valuable. If just showing up and doing what someone else decided for you is enough for a thing, that thing will lose value very soon. Similarly if you make sure almost everyone can do it, it won't have value, but will become a participation trophy.
But nothing in real life work like that. School is fake. You don't get a job just by showing up or having a diploma. Nobody will fall in love and start a relationship/family with you for showing up and fulfilling some list of criteria. Nobody will fund your startup or strike a business deal with your company because you showed up and did some assigned tasks.
In almost all aspects of life being proactive and exercising agency will get you much further than the teacher's pet mindset that school instills. And unfortunately rather than selecting for it, the PhD selects against such agency again because it's the safe option and people who are ready for an adventure usually dislike the academic environment. Not all of couse, I obviously don't mean every single person fits this. But in my experience this explains part of the mismatch in expectations and reality for the "I was a good student so a PhD felt natural" people. Not those come into the PhD with a well thought out plan, and knowing exactly why they want to pursue it, the upsides and downsides etc.
It was not my case and bold of you to assume so. I had peer-reviewed publications before I even applied for PhD.
While I do know some people who expected PhD to be "more classes with more difficult assignments", the mast majority of PhDs I know had nothing to do with mentality you described.
1. You can hedge your bets by submitting your work to various conferences of various qualities (without going 3rd-tier, you can bet across 1st-tier and 2nd-tier)
2. You can spend time choosing the professor and the topic before going all in
3. You can seek advice and social interactions within your research group, departement and school
None of this is a silver bullet, but it compounds in the right direction.
To a point. If you stray too far from your personal interest, you increase the risk of burnout.
My advisor was well established, tenured prof with a number of students. I had to teach, but the effort was light. We taught large, basic courses that are boring for tenured profs. We usually requested the same 1-2 classes to teach and after the first round had all the materials (homework, quizzes, etc.) and could teach on autopilot. University gave us undergrad graders to grade assignments but I never used them since I wanted to see what my students wrote. Which is a testament that the load was light; if I was drowning I would use all free help I could.
But there was a cult of academia. "Get an academic job or you are a loser" mentality was prevalent. My advisor was disappointed, but OK when I decided to go into industry after PhD, but a friend's (Physics PhD from Harvard, CEO of a profitable startup now) advisor does not talk to him anymore because he did not stay in academia.
And I only realized long after finishing my PhD how incredibly lonely PhD path is. You live in your bubble for years, with minimal interactions outside a few other folks at the same grad school. Stipend was enough for basic living, but not much else. No good vacations, ski trips with friends, etc. And a few somewhat creepy characters that grow in this lifestyle. This is all surmountable, but the mental toughness required is certainly something to keep in mind. I did not have that mental toughness, but was an introvert, which helped a lot. But looking back I see that I also could have gone off the rails. My 2c.
It depends highly on the field. In history, sure. The point of getting a history PhD is to become a history professor, and you can't do that if you don't get the PhD, and meanwhile history PhDs don't meaningfully open up any other job prospects, so attempting and failing to get a PhD provides negative value.
In CS and many engineering disciplines, there is a long history of people dropping out of PhDs and landing in industry. The industry is therefore much more accustomed to, and therefore accommodating to, people taking this path. Whether it's a maximally efficient use of time is another question, but it's certainly not wasted effort.
But I do agree that it's stressful nonetheless because it still feels like a failure even if it is not actually in reality. I wrote about this when I put down my own PhD journey here [1]. In particular after the control replication (2017) paper, I very nearly quit out of academia entirely despite it being my biggest contribution to the field by far.
[1]: https://elliottslaughter.com/2024/02/legion-paper-history (written without any use of LLMs, for anyone who is wondering)
Do you mean that you’re using AI as a search engine for your local bibliography? I haven’t seen any AI plugins for Zotero.
Add any paper you pick up to your tracking system before you read it. Make that part of your reading ritual.
Save the PDF right away, too. You may later lose access to the journal. Or, CiteULike (where I^Hyou uploaded all those articles) may go away.
[1] https://en.wikipedia.org/wiki/Citeulike
It was the best/worst 4 years of my life. I studied overseas (uk), met my future wife and got a PhD that really wasn't useful for much to me. Fortunately it was under a scholarship.
1. It reduces the odds of missing a key reference in your papers and accelerates the write-up of the (often mandatory) Related Work section
2. It helps you maintain a mental map of the field as your research progresses
Personally I did the references at the end and didn't feel like I suffered from that decision, but the key references in my particular area were a relatively small and well-known set.
* Your records are never good enough to completely replace your memory of what you did. The longer it takes, the more studies, readings, etc., that you will have to repeat.
* In physical and biological sciences, equipment breaks down, gets taken away, facilities get moved, etc. This stuff happens at a constant rate, and is a pure time cost.
* Technological progress gradually raises the bar for the minimum quality of some results, e.g., in computation. Even "theory" is highly computational these days.
There are also risks of career-ending accidents that can be treated as a constant risk per unit time:
* Your advisor dies, retires, gets promoted to administration, loses funding, changes jobs, gets embroiled in ethical / legal issues, etc.
* Some unexpected new result from another team or industry erases the relevance or novelty of your work.
* You get sick, have family crisis, etc.
* Burnout
Results are the wrong unit of measure. A better KPI is results per unit time. The people who look like they fucked around for 4 years then submitted a brilliant thesis were either working hard all along, or were just brilliant, which I certainly wasn't.
My then-fiancee and I were both grad students. We made a pact to meet at 7:00 every morning in the cafe across from the research building for coffee, to force both of us to stay on a work schedule.
What was the motivation? Honestly, I was too lazy to get a job and staying in academia for another 3+ years seemed amazing (probably not recommended, but it worked out OK for me).
What helped get me through it:
1) Doing something I genuinely enjoyed - I approached the Computer Vision professor who gave me some ideas. I super enjoy writing code, and the idea of processing gigabytes of video to produce answers seemed cool. I treated it as a super difficult programming project.
2) Breaking my leg - Just before starting, I broke my leg badly. And that meant working from home with a weekly visit from the professor with a stack of reading papers. That time spent understanding state of the art was super useful.
3) Funding - At some point, DARPA gave enough money for me not to worry about funding, so I never had to work a job or get distracted.
4) Marriage - The final straight of writing a thesis was tough and I was super lucky to have a supportive wife who pushed me to get-shit-done.
This is actually how I view academia. "Couldn't get a job"
It really lowered the prestige of a PhD for me. Heck, if I think through my PhD friends... none of them were A students. They were all C-tier.
Academia is a very particular dynamic very difficult to find elsewhere, and some people dig it. You can watch some people finding the same dynamic at Google for example, where they are allowed and encouraged to fiddle around and keep publishing (e.g. the Attention paper, why would Google allow such publication?). Such dynamics are explored in Terence Kealy book "The economic laws of scientific research".
It can be different in other fields an in lower tier colleges.
Something I didn't see in the article:
Depending on your field, it can be extremely hard to get tenure. Unless you are a genius and are seeing signs you are well on your way to getting professorship and tenure (publishing good and important papers, really good at communication, checking all the other boxes), you'd better consider an off-ramp as early as possible.
In some fields, 100 people compete for 1 open positions, and that's rough. Having been involved in such a hiring process, I know it's extremely challenging.
I was smart enough to realize I'd never be a good researcher relatively early during my PhD and started preparing for job interviews. Sadly, I see too many people still having unrealistic dreams about being a professor late in their PhD. They even do postdoc and do that for many years until they finally discover they are not hireable. Good news is that they often find an industry job quickly after switching.
I could see that clearly -- their research, originality, communication and understanding of the field is just not there, and one doesn't become a professor without being completely in control of the direction of their academic research.
But it's hard to tell people "you are just not good enough for this". In most situations, you'd be considered unsupportive. However, in this case, it's the best thing you can say to a struggling PhD/postdoc.
(Many professors are completely incapable of advising on students' career. They often find it surprising that a student wants to go into industry. They hold completely incorrect assumptions like a PhD student just goes into a management role as soon as they graduate. Rarely a professor tells you that you should not pursue tenure.)
What are some things that made you realize that? Im considering going into research and certainly have some doubts
* I had difficulty coming up with my own ideas. A good PhD student should develop a good sense of direction halfway through their research PhD, and by the end they should be able to conduct their own research independently. I was nothing like that.
* I attended talks from fellow students that were invited to the department and went to conferences. I know what good researchers are like -- they do good work, publish papers relatively early in their PhD, know everything about the field and what are they doing, have confidence, communicate with others well and have great delivery in presentation.
-- * This might be the one that is most compelling. You could tell who is going to be a professor and who is not. (As I am typing this, I look up the guy I have in mind. He just got assistant professorship last year. I am not all surprised.)
Granted, I am far from a model PhD student. My research is mediocre. But I have seen people who deliver even worse research and have even less chance of ever getting tenure spending more time in academia. So I hope my words can be helpful.
All that said, I would say you don't need to self doubt. If you are smart and constantly think about your career, and if you go to conferences, you'll know whether this is for you and what to do.
I feel like this particular advice applies to a very small subset of people. If I’d had professors telling me that I certainly would have considered doing a PhD!
How To Get Rich?
Step 1: be rich
Step 2: you're now rich
Karpathy is an exceptional person, maybe not as much as Ronaldo in football but taking advice from him similarly won't be guaranteed to work. You can't have guarantees in such things.
In truth, the more literal but not fully literal thing that happens regarding surviving a PhD is that you try to publish a paper in a top venue but after several rejections you publish in a lower tier one, then you do two more followup in similarly second tier but not terrible venues and you get a "magna cum laude" or perhaps a "cum laude" once you reach 5 years and the prof wants to avoid the embarrassment of not having graduated you.
Of course many people don't come into the PhD with such plans, they expect a summa cum laude and papers in top venues and talk invitations and so on, since they've always been a top student so far.
(I tried doing a PhD while working full time, and quit the idea after 3 years.)
When things went smooth (accepted paper) I tried to roll that motivation forward as much as possible.
2 years was during COVID so I never really had to travel for classes which really saved time.
I did most of the coding and ideation on either paternity leave or holidays. Evenings were often running smaller experiments or eventually endless iteration of a paper.
My wife was great and helped me focus usually 8-11pm most nights and some weekends 1-4pm.
We don't live nearby any family and friends were not much help.