I understand the frustration, but at the end of the day I think they do serve a purpose. They're not great at that purpose, but they're good enough and they generally produce false negatives (smart people fail) rather than false positives (you hire an idiot). And this makes sense, because a false negative is low cost (maybe you spend 50% longer interviewing candidates) but a false positive is high cost (you hire an idiot and have to spend months establishing that, firing them and starting the hiring process again).
The side effect of this though, is that it's an extremely painful process for the applicant. Even if you're a great software engineer, you're going to freeze up, miss something, have a bad day, and fail a few interviews. Now for most software engineers failure is an unusual and harrowing experience, so they react really badly to it. Especially ina
scenario where it's difficult to blame anyone but yourself. But just don't! It's fine! just move on, there's plenty of jobs and the interview process is a blunt tool, not a final evaluation of your worth in life.
At all the companies I’ve run we’ve used a simple whiteboard example for this reason. Don’t worry about typos; how does the person think.
“Trick” questions are dumb, but you want to get an idea of if the candidate knows their stuff (and let the candidate know what we’re like — interviewing is a sales process in both directions). If someone asks a question like “is it ok to modify the argument” (or says “I’ll assume I can’t modify the argument”) that’s great (or says “I can’t remember if strlen includes the 0 byte so I’ll add 1 — normally I’d look it up”) - again, no trick questions.
When I say “simple” it’s something like atoi or “shuffle a deck of cards”: basic, but not 100% trivial like fizzbuzz.
On of our best hires was a guy who made a fundamental mistake — when we asked him if it world work as intended he said “I’m a fucking idiot” and fixed it. You’re not supposed to talk like that in an interview I supposed but we read it as a strong positive signal.
We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time.
> We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time.
Trying to debate the interview format or reject interview questions is one of my hard-stop rejection triggers in interviews now.
The couple times I’ve been part of hiring pipelines where candidates argued about the interview itself and then got hired anyway, they became extremely difficult employees within the company. Arguing about interview questions turned into arguing about every other ticket, code reviews, and every architecture choice.
I’ve had the same experience where we take an actual problem we solved in our codebase and turned it into a small interview problem. Same thing: Some candidates will try to debate the relevance or try to wriggle out of it.
I have come to like this approach reasonably well. My current company uses a similarly simple problem to evaluate candidates. For me, it was a breeze, and I thought we’d be hiring slouches left and right. Once I started administering the interview, I realized that the majority of candidates absolutely bomb this simple question for one reason or another.
It seems that “able to code a simple problem well” is a far less ridiculous proxy for good software engineer than “able to code a hard problem at all”. Much to my surprise.
Fwiw this is surprisingly subtle and non-trivial to get right. A naive "random swap" approach does not produce uniform permutations. Sorting based on a random comparison function can likewise produce biased results depending on the implementation of the sort. Tagging each element with a random number and then sorting based on the tag works (or so I think?) but you need to deal with the case where tags collide. Fisher-yates shuffle is bulletproof but I would not trust myself to write it from memory lest you end up writing Sattolo's algorithm instead.
That I wouldnt mind. I didnt do a full on CS degree, that never stopped me from being a good contributor to various teams over 8 years. I have learned the best candidates are the ones who like you say are open about how they messed up, and can get along with your team. It takes one toxic dev to ruin productivity for an entire team.
I agree with all of this. Leetcode questions are fine as long as they aren't actually difficult maths puzzles or brain teasers. We also used atoi/itoa loads and it was pretty much the perfect difficulty.
Though I still do start with a short fizzbuzz-level question because you would be surprised how many people fail that and it makes it way less awkward if you start with atoi and they can't write a loop.
If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice.
Offtopic, but there's an interesting parallel between making a mistake like that and AI hallucination. In a human, that behavior is a positive signal, but for many people, that same behavior is proof of how LLMs are just a toy that will never rival human intelligence.
Okay, do do quizzes, but let people google goddamn stuff because that's just how software engineers operate. Don't make it an exam. Don't mimic the worst thing about the education systems. Both in the school and in the university, these goddamn exams were the worst because they tested memory first and everyone else second, and I'm such a kind of person that I could never remember things on command. It was a real struggle. I can't be the only one.
I've never been through the "regular" IT hiring process myself, but I've interviewed several candidates for an Android developer position. I liked asking questions and looking at how the person thinks. I didn't expect exact correct answers for every question, I just wanted to see whether they have a fundamental understanding of Android. After that, they had to build a small app at home (not my initiative) to show their skills.
What are you going to google though? For most of these LC questions you only need to know how to use arrays, maps and loops. If I'm interviewing you and you tell me you need to google how to append something to an array or check if the element is in a map then I'll think you just haven't written enough code yet.
There are better questions where I do think google should be allowed but they aren't LC-style. Some examples are building APIs or operating on files
I like this idea as the more advanced version of whiteboard programming from the year 2000. Even if you still do whiteboard programming, give them a PC / tablet, and allow them to do some Google searches. Then, you could even make the problem a bit harder to solve. Again: You are looking at the candidate's problem solving process holistically. I also very much agree with your second paragraph. Some of the most pleasant surprises I had during interviews were candidates who had very limited domain knowledge, but tried as hard to possible to make educated guesses, or ask intelligent questions. It shows a lot about a candidate.
> They're not great at that purpose, but they're good enough and they generally produce false negatives (smart people fail) rather than false positives (you hire an idiot).
I've seen this reasoning, but I'm wondering if this is actually true, especially if you also believe the other myth in hiring: most people applying are not qualified for the job. Since most hiring actually requires you to fill the position in the end (in other words, you can't leave it open indefinitely if no one qualifies as eventually the pressure to hire mount up), every false negatives does increase the odd of you hiring a false positives as well. Just take the example at the extreme, if there is only ever one person on Earth qualified for your job and you got a false negatives on him, there is no other way but getting an idiot in the job.
In my years of hiring experience, it doesn't hold up. I've hired folks who could leet code all day long, but couldn't develop a feature from scratch, or figure things out without enormous hand holding. I want to hire problem solvers who also know how to code. I stopped doing leet-code interviews about 7 years ago. By all means have people code in interviews, but don’t do leet code.
> if you also believe the other myth in hiring: most people applying are not qualified for the job.
Depends where you’re looking in the hiring pipeline. If you’re looking at raw applications submitted to a publicly listed job posting, I can say without reservation that the majority of candidates are not qualified. The pre-screening candidate pool is extremely bad on average, especially for remote job listings. Some people spend all day clicking the apply button on every job listing they see.
Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true.
They do produce plenty of false positives - just not "you hire an idiot" kind. You will hire a person completely unsuitable for the actual job, but is good at leet. Issue is not the lack of intelligence. Issue is not being software engineer despite being good at algorithms. Or not being good at whatever your position requires.
I do work in a team where majority was hired by puzzles of sorts. All of them are smart. They are not software engineers and it shows massively.
>They are not software engineers and it shows massively.
Coming from Real Engineering and making 2x more programming, I lol at people who say there is any Engineering in software.
We are programmers dealing with layers upon layers of abstraction. Knowing to optimize for time by using vectors is more of an art, than a science.
I did do safety critical C code and Assembly which are probably my hole in the whole 'its not engineering'. But javascript/python/backend work: programming.
Hard leetcode questions hire grinders, not thinkers. Many of those questions were thesis/publication-worthy decades ago when first solved; it's unlikely for even a very intelligent person to produce a similar result in just tens of minutes. So if someone solves it, it's much more likely to be because they saw it or a similar question while grinding practice questions than that they genuinely derived the optimal solution from scratch.
> Many of those questions were thesis/publication-worthy decades ago when first solved
The hare and tortoise (cycle detection via Floyd's) is a legendary impossible interview question. It wasn't solved until the 1960s with a PhD thesis. It is a hard problem to face in an interview. I had seen it twice in my career. I failed the first time. The second time I had the answer memorised, then put on my very best Morgan Freeman stage acting to pretend I was discovering the answer during the interview. What a joke. I was laughing to myself as I was "stumbling" towards the answer while the interviewer was providing minor hints.
I disagree with this. I can solve the majority of leetcode hard problems in under an hour despite not having seen them before and I'm far from the smartest person I've met.
>because a false negative is low cost (maybe you spend 50% longer interviewing candidates)
this would make sense if they filled these positions in 2,3 weeks tops. But that's the insane part; I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks. At that point I feel you are no longer avoiding false positives, but simply hiring the most desperate programmers willing to jump the hoops. The best candidates will get an offer mid interview at that rate.
>But just don't! It's fine! just move on,
I've been "just moving on" for some 9 months now. There's only so much morale in a tank, no matter how BS you know the whole process is. I just wanna move on in life, not be bogged down on how many permutations you can color an N x M grid with.
First, I am sympathetic to your experience. It must be difficult.
> I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks.
From this story, it is clear that your market is clearly in favour of the employer. One year ago, it was a different story: half the time. Another thing that will hurt: If you are amazing, that 6-10 weeks will suddenly become 1-2 weeks. In my experience, nothing hurries the interview process more than telling an employer about late round interviews (or offers) at another competitor.
> a bad hire can be an existential threat to the team or the entire company.
Totally, but leetcode wont filter them out.
Incompetent is fairly easy to navigate around, pathological bastardness is impossible to work past. I know that some companies use psychometric tests, but they are not based in anything remotely scientific, but are also super easy to game.
That false negative can be pretty damn expensive if the good engineer you didn't hire could have saved your ass from the situation the less-good engineer you did hire couldn't handle.
FWIW, I have come across examples in the wild of engineers who were extraordinarily polished and facile at everything leetcode, and poor at computer science. Not many, but some. This is a somewhat recent phenomenon.
Best tests are simple and common computer science problems in algorithms and data structures with no tractable solutions. There is no facile answer, you truly have to reason from first principles for any particular scenario because there is no alternative.
The problem with Leetcode-style test nowadays is that there's too much cramming. Otherwise, leetcode-style interview has certain correlation with one's geekiness and raw talent. Just this week my team solved a long-tail performance issue by first building a simulation of a queuing system, and also built a customized Coffman–Graham algorithm to resolve the order of execution of a complex task graph. I would certainly appreciate that my team members just get it when someone mentions basic CS concepts like topological order in a graph or queuing theory.
If, after all of the normal interviews, you randomly selected candidates who passed and do not make them offers . . . you generally produce false negatives (assuming the interviews mostly worked).
How do you know they produce false negatives over false positives? How do you know if your top candidates simply didn't cheat over the honest candidates that produced worse but honest code?
There's an entire industry of bootcamps built on the premise of teaching you how to pass this exact test in like 6 weeks.
And because they ask pretty irrelevant questions, the 6 weeker will do better then the recent Stanford PhD who has been studying a specific problem for 5 years or the multimillionaire who spent the past 10 years building and selling successful software companies.
Those people will fail and your 6 weeker who's never heard of cmake, diff or gdb will pass.
When I see there's leetcode as the hiring process I presume the place is full of bozos because the process actually strongly preferences them.
Well said, like IQ tests, if you train for it you're better at it, but it the meaning of the results is dubious.
Recently I failed the first round of a fintech analyst position because I didn't complete the 9 undergrad (or high-school, in certain talented schools) math problems (I have a math PhD) in the 1 hour time frame. In retrospect, I should've cheated using computer/online assistance, and my true failure was my unwillingness to cheat. Not because fintech analysts are cheaters, but because the testing process is nonsense, so recognizing that, one should cheat it (indeed, an analyst mindset!) I'd rather be jobless than to adopt that mentality though. (and in fact I am, the only job I was able to find was a minimum wage service-type job, it can't even pay for daycare, I'd rather raise my child on my own while wife works thank God.)
We have high-interest rates and global crises resulting in companies not hiring while simultaneously governments give orders to news media to tout the strong economy. Ineffective hiring practices will always exist, but people don't complain when jobs are available; you only see this type of discussion online when things have gone sideways.
I believe this is a common myth. I've spend around seven years teaching programming and people who can go from zero to decent problem solving in just six weeks are virtually non-existent.
Either it wasn't a true "zero" but rather they were really good e.g. at advanced math or physics and managed to see some parallels, or it has to be a genius. I'd be very interested in hiring such a person.
Not my experience at all. Ask a bootcamper a slight variation on a problem they're clearly doing from memory and they fold. Ask a solid CS person, and they usually get intrigued, and dive right in.
I once had an interview that I really liked. It was for a bank.
They give the candidate some messy, done-in-a-hurry code (but not purposefully obfuscated) and ask them to refactor it to the best of their ability. The interviewer sits next to the candidate and talks to them throughout the whole process. It's pretty much pair programming, but the candidate has the initiative.
I found it to be a breath of fresh air after all the leetcode interviews. Of course, they have the luxury of doing this, because they know what exact technologies they are using and they don't need to measure candidate's ability to learn new tech, etc. In their situation they are more interested in topics like whether the candidate can produce maintainable code, name things properly or communicate with co-workers.
To be fair I think it is a proof that the person can learn all of the leetcode stuff and therefore they might be clever/disciplined enough to learn new tech as well.
Although I don't find it very relevant. When I was fresh out of university, I was good with algorithms / data structures (neither of which I use much in actual work), therefore able to pass leetcode interviews just fine. Yet it was at the time challenging for me to learn new frameworks and I found all of the tooling I never heard of rather confusing.
This is an interview style I do often.
The first bit is just super basic shit (since I mostly do database stuff) and then a code review section.
What's super interesting is I have actually had people pretty much fail the basic stuff (which was probably nerves) and then go on and kill the code review section in ways that shows their serious expertise.
IMO interviews are pretty bad at demonstrating people's ability to work, but if you can zoom in and out on a problem and do good enough on the coding stuff its usually a win.
Leetcode style interviews probably serve two functions:
1) A way to suppress wages and job mobility for SWE. Who wants to switch jobs when it means studying for a month or two? Also, if you get unlucky and some try hard drops an atomic LC hard bomb on you now you have an entire company you can no longer apply to for a year.
2) A way to mask bias in the process while claiming that it’s a fair process because everyone has a clear/similar objective.
Meet someone who went to your Alma mater? Same gender? Same race? Give them the same question as everyone else, but hint them through it, ignore some syntax errors, and give them a strong hire for “communication” when they didn’t even implement the optimal approach…
Or is it someone you don’t like for X reason? Drop a leetcode hard on them and send them packing and just remain silent the entire interview.
To the company this is acceptable noise, but to the individual, this is costing us 100s of thousands of dollars, because there’s only a handful of companies that pay well and they all have the same interview process. Failing 3 interviews probably means you’re now out $200-300k of additional compensation from the top paying companies.
I’ve interviewed for and at FAANGs. I can’t believe the low bar of people that we’ve hired, while simultaneously seeing insane ridiculous quad tree/number theory type questions that have caused other great engineers to miss out on good opportunities.
Someone will reply to me “if you know how to problem solve you will always pass.” Ok, come interview with me and I will ask you verbatim one of those quad tree/number theory/inclusion exclusion principle questions and I’d love to see you squirm, meanwhile another candidate is asked a basic hash map question.
I'm sure anyone determined to do so can act unfairly regardless of what process is in place, but the fact that there is a standardized test in my mind does the opposite and makes the process much fairer. Assuming a fair-minded interviewer, the process gives a chance to a candidate whose resume may have less vaunted names on it to demonstrate their skill. I'm quite sure that I'd never have had some of the opportunities I have, not having a CS degree, if it weren't for whiteboarding interviews. I can't imagine any possible process that would thwart interviewers intentionally subverting it to hire their friends.
> the fact that there is a standardized test in my mind does the opposite and makes the process much fairer
The problem isn't standardized tests, it's that leetcode questions are about having the time to have learnt the answer beforehand, rather than raw ability for problem solving.
A leet-code test would be much more standardized if candidates could solve it at home. Just send me a link to the quiz and let me solve it within a specified time frame.
I've done tests like this for some companies. It felt a lot fairer and more closely resembling the actual work environment than live leet-code interviews, with biased interviewer(s) and a stress factor that's not a part of the actual job.
> I'm sure anyone determined to do so can act unfairly regardless of what process is in place, but the fact that there is a standardized test in my mind does the opposite and makes the process much fairer.
This is what privilege looks like. The inability to see barriers that affect others in a worse position.
Standards do not imply fairness only consistency.
You got a test that filters out people who lack the time to train for these tests. Basically, devs with a life (see family)
So are we just going with a base assumption that interviewers can NEVER be trusted with anti-bias training and learning how evaluate people fairly? The examples mentioned in this comment section are all blatantly intentional biases that people are choosing to use. The amazing part is that all the “standard test eliminates bias” people seem to the most ignorant to where bias helps THEM. Forcing people to study for two months is blatantly discriminatory to age and family status, at a systemic level. While “this white straight guy might explicitly choose to give the other white straight guy an easy question,” is very subjective and intentional on the individual level. Like, employees can always choose to do bad things, in any situation. That’s why we have at-will employment…
Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that?
> the fact that there is a standardized test in my mind does the opposite and makes the process much fairer.
It can make the process fairer but it's not a given. You can do the classic "no dogs, no blacks no Irish," and that was a standard across pubs in England. It certainly wasn't fair, unless you were racist.
If you're committed to fairness, then a standard will help. It gives you a clear point to fix and improve things and something you can use to measure if you're achieving your stated goals. That's definitely something you can't do if you're just making things up as you go along.
And yes, I made a deliberately provocative statement. I'm obviously not saying whiteboard tests are the equivalent of segregation.
The problem you are describing is interview variance and hiring bias, not leetcode. This happens irrespective of interview style.
Many companies have question banks that are specially designed to be fair/have some contextual relevance (ideally) to some "realistic" problem. Or at least, many of the companies I've interviewed at follow this model. I consider these coding questions to be "leetcode" style because at the end of the day they are an isolated coding problem, even though they may not be a problem from leetcode verbatim.
Companies that execute on that style of interview well are generally fairly pleasant interviews, at least in my experience. Good companies/interviewers will gauge more than just the final code to determine a hire or no hire. And a large portion of companies also have hiring ratings on a scale to make it less binary.
I take issue with it because Leetcode gives tech people a fake feel good, “yeah but at least it’s fair” illusion, when really it’s probably just as biased as any other hiring method.
This is ironic considering these companies have forced mandatory DEI seminars (which I have no problem with btw), inclusive language, #EveryoneCanCode, and so on.
But despite all this, you end up with teams and organizations that are 99% of the same X somehow. Replace X with race, school, even state from home country sometimes.
You know there are websites where people share all the interview questions to these hard interviews / referrals exclusively in their language and behind a pay or rep wall? And there are Telegram groups where international students leaks the questions or do interviews in place for one another.
It’s inevitable these types of issues arise when there’s so much at stake, ex: in just 5 years, a $200k TC advantage at a top company, becomes $1m or more with appreciation.
I just really dislike the veneer of “fairness” when there are so many problems with the process, even beyond the questions that have nothing to do with the job.
Question banks that are too big: huge variance, and OP's point stands.
Question banks that are too small: leaked on eastern forums immediately, candidates show up reading answers out to you (some of the guides include guidance on when to pretend to think, I am not kidding).
The idealized version of "question banks" might work. The real one does not; you'd require employees constantly scouring forums in every language known to mankind, immediately removing anything that gets leaked. On top of that you'd probably require a competent committee overseeing all questions in the bank constantly and ensuring the lack of variance in difficulty.
I don't think the problem is the format, i.e. a 30-50 minute interview on simple coding with DS&A problems, but the escalation.
The reality is, fizz buzz got us 75% of the way there. It turns out when pressed, a lot of people can't write code. Yes, there's false positives, but there's also people brute focing their way through via copy & paste.
This doesn't manifest as a person who can't do any task, just as a person that's slow, delivers weird abstractions, and would take a lot of your time to get anything useful from.
But those people are also making those arguments, because, as you said, there's hundreds of thousands of dollars in it.
A lot of people couldn’t even read this thread if someone was watching closely. They’d recognize words and idioms, but couldn’t make sense of these or think. It’s called anxiety and rarely has anything to do with actual performance. Anxiety is a frequent guest in a smart guy’s home.
As an early 2000s integrator/analytics I learned to write code when someone watches out of interest or straight time pressure (still taxing sometimes). But most developers that I know intellectually curl up in such situation, regardless of their skill or performance levels. We worked with one very math-y low-level-skilled guy whom our clients described as “literally freezes for an hour without moving, should we pay for it?” when he did field work. He was a very strong developer otherwise.
Not that a company must hire or want these people, but the idea of writing code under uncanny pressure all day makes as much sense as that swordfish scene.
It's also a great way to filter out anyone who has any kind of commitment like kids, aging parents to take care of, or have health issues themselves that makes it hard to cram leet code non stop on top of a busy career.
For companies this is very rational, you want non-distracted employees who can work over time.
It's really not that rational unless you want someone only familiar with being the top code monkey.
In all my years of sw dev the number of times that would have helped vs being able to communicate and manage expectations across a swath of people is like 1:1000.
> To the company this is acceptable noise, but to the individual, this is costing us 100s of thousands of dollars, because there’s only a handful of companies that pay well and they all have the same interview process.
If you want to earn top money, you gotta play the game. This is true in every profession, you think lawyers that want to work at top law firms have it any easier?
There's a lot of valid criticism with respect to today's tech hiring practices (especially because it doesn't only affect top paying companies), but I don't feel like "I can't get ridiculously wealthy without selling my soul" is a particularly compelling one.
Although I find it disappointing, this is the truth. It is a function of supply and demand. There is always a gating function when you have an abundance of supply; sometimes it is leetcode, sometimes it is school, take home work, resume formatting, a mixture, all of the above, etc.
Everything is a proxy because, in most scenarios, there is no way to understand effectiveness of an employee until they have worked for you for a while.
Of course there are other ways, but they all carry their own risks.
That’s why referrals, imperfect as they are, carry so much sway. In theory they reduce uncertainty and require some expenditure of political capital.
I'm not sure there's so much agency in any of this. Most of it is probably cargo-culting. Every software shop wants to be Google, so they do what Google does (or what they think Google is doing). It doesn't matter if they do it well, or understand (or agree with) the ultimate purpose of the process.
If Google engineers all wore pink robes and top hats while interviewing then we'd see that everywhere.
I think you’re overthinking it. Hanlon’s razor applies - the industry as a whole is incompetent at interviewing. And to be clear I’m not being smug here - every time I’ve ran recruitment rounds the interview process was a mess. It’s an extremely difficult equation to balance and as an industry we have never made the commitment to figure out a golden standard for how to do it.
>because there’s only a handful of companies that pay well
If this was just FAANG I wouldn't even mind it. Bigger compensation means bigger hoops to jump through, artificial or otherwise. At least those bigger companies come with interview guides.
But I've been dropped Leetcode mediums on some Unity game dev jobs barely paying 40/hr with minimal benefits. Game interviews already have an absurd number of topics to try and guess you'll be quizzed on. (last interview wanted low level C memory/bit manipulation question... the one before that did Unreal Engine trivia... then the one before that decided to give me vector math and ray intersection questions), adding in random string manipulation questions when you will barely use more than concatenation on the job doesn't help with studying.
If you are applying for game dev jobs, C memory management, the gnarly parts of Unreal engine, vector math, and ray intersection are all things you could be doing on the job.
Game dev is lower paying, less structured, and more involved than typical web dev jobs.
I’d really only want to get into game dev early in my career or as an independent creator. It’s a touch exploitative.
This is a good assessment. 1 and 2 are why the system won't change, but I don't think they were intentionally designed with that in mind. I think it's a hang over from academia, bearing in mind how many of the top engineers at FAANG are PhDs.
Well, that and how almost nobody who successfully finds employment after "grinding" leetcodes wants to remove the barriers for entry.
I think Leetcoders can't envisage a better way to assess someone than by subjecting them to the same kind of hoop-jumping you get made to do in university. They're not interviewing you for a job as there's no module on interviewing candidates on the CS curriculum, and don't have much professional experience outside of academics or software engineering. They're simulating a dissertation defence, because that's how they were assessed for their competence.
That's my charitable interpretation. If I'm being cynical, it's elitism - a way of making sure you're "one of us" (read: obnoxiously academic, Type-A personality, "logic over feelings").
> how many of the top engineers at FAANG are PhDs.
Not that many, but its interesting how many are from "elite" universities. I'm in a research org for a FAANG and we often get to see all the handwringing about how we can't recruit more of people type x.
Well, if you only hire from MIT, Stanford, Oxford etc, then they are all going to look the same.
For those outside the research org, its a bit better, but its still the most uneven place I've worked.
Engineers typically optimize for their own learning, and the false negative rate can reflect more about how aggressively the interviewing panel wants specific peers versus how much they actually want the job done well.
Suppose there was a SWE union and the union has to create their own reference compensation bands. Would the union again lean towards leetcode or some other standardized test? Or just use years of experience and maybe past projects?
When weighing the “effectiveness” of leetcode interviews, it’s important to weigh that thus far SWEs have failed to effectively unionize despite e.g. the past class-action no-poaches clearly showing C-suites should be paying engineers a larger share.
That depends very much on your world view. If you get the job, it would imply you've just cost a lot of other people $100ks. That simply can't be true, because there's a small number of such jobs.
The only way this can possibly hold is if you hold that only the "best" candidate should get the job, and by "best" you mean: the one that gets most of the leetcode questions right. But then there's no us.
Edit: I'm not in favor of leetcode interviews, and I do understand that there's a bias in interviews (which won't go away when you drop the leetcode).
> Meet someone who went to your Alma mater? Same gender? Same race? Give them the same question as everyone else, but hint them through it, ignore some syntax errors, and give them a strong hire for “communication” when they didn’t even implement the optimal approach…
I do not know how hiring is done at megacorps; but on the few occasions I participated in tech interviews, we were looking for candidates to join the team that I was on. So, what I was interested about the candidates was how well they were prepared for the kind of work our team does; and how much I would like to work with them. I am sure this second question introduces all sorts of subjective biases; but then, hey, aren't you going to spend countless future hours with that person? At least the first concern incentivised us to make sure candidates were sufficiently competent. BTW, we didn't ask leetcode-type questions.
> but to the individual, this is costing us 100s of thousands of dollars, because there’s only a handful of companies that pay well and they all have the same interview process
I am not sure I am following the argument. If there is only a handful of companies that pay well, and if you only consider companies out of that list for employment, isn't it reasonable to predict that there will be lots of other candidates who want the same, far more than the number of available positions at those companies. How, then, should those companies select from such a pool of candidates?
There is no grand conspiracy. People try to develop what they feel is a fair interview with good signals. It simply turns out that many people are not good at that and have biases that they may not even be aware of. Tackle on some cargo culting and "best practices" and you have your typical broken process.
For those wondering what the alternative is to leetcode: a work sample test. Take a real problem your team solved recently, distill it down to remove internal-specific knowledge, and give candidates 3x the time to solve it compared to what it takes an internal dev to solve. Bonus points for having a rubric guide the scoring.
Casual gamedev here.
Ive written like 100 quadtrees from scratch so its a goto algorithm Ive got down by memory. I used to think this datastructure was very basic and common because young gamedevs often share it online with eachother. Highschoolers etc.
One time in a technical interview I realized a quadtree would solve the problem they were asking. I also happened to be reading SICP and practical common lisp on the plane on the way over to the interview. (specifically the sql sdl macro chapter) Unfortunately this gave me an idea. I narrated, sketched out and pseudocoded a plist based quadtree on the whiteboard and felt pretty good about myself.
I turned around to see my interviewers were not impressed. Actually they thought I was an idiot. They didnt know what a quadtree was, and they were quite upset I used "tuples".
I assured them its very efficient, and that the compiler would take care of "the tuples".
They failed me. Moral of the story don't use common lisp in a javascript interview.
What about a type of blind interview where neither of the two know the problem or solution in advance. After, their solution is blindly reviewed/graded by a third party.
- Input from the interviewer reflects communication ability, cultural fit, and so on of the candidate.
- Input from the blind grader reflects ability of them as a team.
- Low grades count against the interviewer as well.
This is very much like the "real world" situation. OTOH, I think this leaves the interviewee even more vulnerable to the unfortunate situation where they correctly solve the problem but the interviewer is convinced that a different, wrong solution is correct. Live long enough and you'll have that happen to you.
OTOH, seeing how the interviewer reacts to that situation will tell you a lot about whether you really want to work there.
Naive questions ahead, if hires weren’t made scarce by some absurd filter, why would they pay $200-300k extra? Feels like the whole idea of stellar salaries must be based on something stellar. Afair, before Google(?) made it normal, developers were sort of dirt cheap. Weren’t developers in abundance at all times?
The reason they pay $200-300k extra is to attract the best they can. Say you got the same salary working in an ethical company than in a FAANG, would you go for the FAANG?
The absurd filter is just some kind of lottery. They could have a different one: at the end of the day, it's only when the person starts working that you can actually see what they are worth.
The thing with a filter like this one is that it filters out a whole category of people who may actually be good. And it reinforces itself: you hire people who are good at leetcode, who will themselves possibly be good at hiring people who are good at leetcode. Does a company of leetcoders perform better than a company made of a diversity of good engineers? Not clear.
We’re hiring for an entry level position where I work. So entry level, that their first month is going to be dedicated to simply learning the framework we use.
However, we do require a base level of competency. We give out a 40-ish minute assessment. Two multiple choice and three coding problems.
And they’re easy. One of them is essentially “John has a 5 gallon bucket and 3 gallon bucket, how many buckets does he have?”. All you have to do is rewrite the clearly labeled circuit diagram as a Boolean expression.
Not a single person this round has passed. Several have failed the simple ones.
So while grilling people on the minutiae of a language or asking them to solve the traveling salesman is not beneficial, neither is nothing.
I interviewed at a company known for consistently asking one of the same four questions in a specific interview round. These questions were widely shared on forums like Blind, Leetcode, and Glassdoor. The recruiters also provided strong guidance on the type of problems to expect.
I prepared thoroughly for all four main questions and any other plausible ones I could think of. I practiced writing solutions to ensure I was fast enough for the interview. Additionally, I pre-prepared ideal answers for each question in case I got stuck.
When the interview came, I got a total curveball: a question that was significantly harder than the usual ones. It didn't fit the round's theme (it was a DSA question, but I'd already aced the DSA round), was obscure enough not to be on LeetCode, and required writing a solver for a hard variant of a known algorithm. I panicked, copied the prompt into ChatGPT (despite being instructed not to use it), transcribed the result, and pretended I had recently studied the relevant algorithm.
I passed the round, nailed the other interviews, got the offer, and accepted. Later, I found out that interviewers are instructed to pick one of four specific questions for that round, and the one I got wasn't in the list.
I'm left wondering if the interviewer was trying to sink me or was just bored with the usual questions. The whole experience raised several questions for me:
Is it cheating if I already had pre-prepared answers for the questions they were supposed to ask? What's the difference between using pre-prepared answers and using Google or ChatGPT during the interview?
If the interview had gone according to plan, what was I actually demonstrating? My ability to use Google?
When the interviewer asked an impossibly difficult question, I would have failed if I answered it legit, even though I'm a good engineer. Failing such an unfair interview round doesn't serve the company's interests.
What is this interview process meant to demonstrate? My true value as an engineer lies in my ability to communicate clearly, think outside the box, identify and address technical tradeoffs, mentor juniors, and propose technical solutions that meet requirements while minimizing risks. Yet, I'm expected to solve a hard variant of the Traveling Salesman Problem in 45 minutes or I don't get the job? Why?
The whole process seems broken, but I'm not sure how to fix it.
> A way to suppress wages and job mobility for SWE. Who wants to switch jobs when it means studying for a month or two?
If what Google search tells me is true the average SWE tenure is only a couple of years or so.
I might need a month or two of study (or more!) to be able to handle the harder Leetcode problems, but if I was expecting to have to do this every couple of years or so I'd consider taking some steps to maintain that knowledge between jobs.
That would probably be considerably less overall effort than forgetting it and having to cram for months every couple of years.
The proof of some of this is that these companies could come up with a standardized test maybe with, say, smaller-scale refreshers every few years (this is normal in some professional certs) and save a ton of money if it were really about ensuring a base level of competence using these sorts of questions as the yardstick.
But that would dramatically increase worker mobility, for one thing. So they don’t do that.
(“No it’s because they don’t want you to be able to test prep” that makes no sense because 99% of the way to prep for these is… exactly like test prep)
Leetcode feels meritocratic. I suspect many people (engineers perhaps more so) don't trust their ability to "size someone up" from 45 minutes of casual conversation alone. Or worse, they worry that they'll have to defend their hire/donthire based on what amounts to a casual conversation whereby they were trying to judge character, affability, confidence, drive, motivation....
This reads very similarly to consulting case interview questions. 1) It requires a lot of free time beforehand to study the frameworks so it filters intent but also removes those who need to work and 2) it seems objective on the surface but you can essentially pick winners by choosing difficulty and nudging along.
I don't buy the analysis on 1). Leetcode interviews restrict the pool of SWEs. You have a lot of SWEs who fail to qualify any more, but the few that do are more coveted and get to command higher wages
I totally agree with A and B and have further suspected they use leetcode as a means to support a H1B candidates employment prospects (for those who do pass)
> A way to suppress wages
> A way to mask bias
> Same gender? Same race?
> someone you don’t like
Ok fair points. it really could be a thing when you've already decided and are just playing around the process. But ya'know even when there isn't an inherent bias from race, gender, alma maters and overall like ability, the whole tech interview process still sucks.
The shit soup presented at a coding interview, leetcode or take home style ones are a result of wholesale cargo cult stupidity than malice.
There's no incentive at any level in BigCo. Margins are too big for top level to care. its hard to measure, middle management layer can easily fudge metrics while driving the org effectively into the ground. Lower layer that made it through the filter, sick of this grind and are busy filing tasks sipping free coffee.
The elephant in the room is that you cannot make hiring a process driven activity.
This is org level dysfunction. There's more such exhibits - DEI, promo criteria, ERGs, re-org politics, vendor selection to name a few.
> there’s only a handful of companies that pay well
with all due respect I believe this is about as far from the truth as it gets. I believe the issue is that people think they have to work at FAANGs in order to be paid REALLY well which is just nuts.
I know many people making (very) high 6 figures working at places most have never heard of. instead of looking for FAANGs people should be looking at companies that have been in business for 20+ years (if publicly-traded company the easiest "measure" would be whether you would invest your own dough into that company) and then make your self indispensable there (this is much easier than it sounds) to a point where you are more valuable to the company they company is to you. This is un-achievable at FAANG but this should be everyone's career goal. once you are worth more to a company then company is to you, compensation-wise sky is the limit :)
> I know many people making (very) high 6 figures working at places most have never heard of
You know people making ~800-900k at places most people have never heard of? That is what "high 6 figures" means. I have to assume you mean more like ~$180-190k?
yeah i agree that "if you know how to problem solve you will pass" statement is a joke. you absolutely need to memorize most of these problems as you'll never encounter them out in the real world. i think we need to get better at the behavioral side of interviewing - this should be the juice of getting at whether or not an engineer is good. and if they're really good at lying... CEO material? lol.
And you think without leetcode questions the despicable biased interviewer from your example would not be able to hire someone from their Alma mater over someone else?
The important thing to screen a software engineer for is not knowledge but the strength of their problem solving and ability to build mental models of problems.
Whiteboard interviews are a good way to evaluate these skills - assuming the candidate hasn't seen this particular problem before and the interviewer understands that this is what they're supposed to look for.
Goodhart's law applies - the measure becomes a target and it ceases to be a good measure.
People who have weak problem solving skills or poor abilities to form mental models still want high paying software jobs so they "grind leetcode" until they can pass these interviews.
Then over time these grinders get high enough up they start becoming interviewers themselves and they don't understand that they're even supposed to screen for talent, they conduct the interview like a memory test since to them that's what it was.
In spite of this, there's no better interview type I've seen proposed given the constraints most companies are under (don't waste candidate's time, have a consistent rubric that HR likes, etc.) so for now we're going to keep using it.
There's a bunch of better ones that are readily accessible.
Ask the candidate to build a known quantity like say, a calculator, calendar, whatever or give them a bunch of code and have them modify it in some way or discover a bug.
Something closer to building software. Leet code is too far away from it and the performance on it too poorly correlated with actual suitability.
> Then over time these grinders get high enough up they start becoming interviewers themselves and they don't understand that they're even supposed to screen for talent, they conduct the interview like a memory test since to them that's what it was.
It follows that better leetcode scoring is a good indicator of who can be replaced with LLM.
> Then over time these grinders get high enough up they start becoming interviewers themselves and they don't understand that they're even supposed to screen for talent, they conduct the interview like a memory test since to them that's what it was.
They say As hire Bs and Bs hire Cs . . . and to combat that we've added more leetcode to the interview process.
> In spite of this, there's no better interview type I've seen proposed given the constraints most companies are under
Yeah, right. Why would companies truly care about this issue and put any effort into making their hiring process feel less hostile when they still have loads of candidates who are more than willing to go through whatever tests they get asked during interviews? It's a leaky bucket and they're happy with the status quo while developers keep grinding away... I doubt it will change anytime soon, but I'm glad I can afford to put my foot down and refuse to do such interviews when I need to find new jobs.
A friend once in the middle of a technical interview stopped speaking, stood up and went out. They ran after him, "what's wrong, you're doing fine!" and he told them it depresses him too much and he's not interested anymore. Sad, given that this is a fun profession for most of us (I assume)
I've done the same thing, at a games company well-known for its perks and benefits, but also its crazy production schedules .. I thought I wanted to work there, and up until about half-way through the interview, I did, but then out came the whiteboard and in came the peers to see me implement basic things in pen, and actually it felt like a ritualized mobbing, because as I progressed through their challenges, things got more and more antagonistic and downright rude, and I realized they were testing me to see if I could deal with vitriol and hostility while producing technically minor results .. I simply stopped the interview, stood up, and said "I've made my decision, I don't really want to work with you guys now that I've seen how things are done, thanks for the opportunity and sorry to waste our time" .. and walked out.
The CEO chased me out into the parking lot and begged me to tell him what I saw and why the interview went awry. He was legitimately shocked that an interviewee could make such a decision, mid-interview .. All I could tell him was that it wasn't going to be a great fit, the culture didn't seem aligned, and the people who were brought in to the interview seemed desperate and hostile to each other - and that simply wasn't the way I had hoped that the company operated.
People forget that job interviews are a two-way street.
I realized that day that I have every right to inspect all aspects of a company that I might work for, and that I don't really have to make compromises if I see something negative during the process. It was the right decision because the very next day, at another competing firm, I met folks who have continued to be good, close friends and colleagues, whom I actually really enjoy being around.
This profession is fun. But when the fun gets farmed and exploited, its enslavement like any other.
Fortunately I never had a leetcode style interview. But I once interviewed at a place a friend worked at and they needed someone to bring their development up to speed (re-architecture, establishing processes, technical lead) and he recommended me for that. The interview partner was their new CTO (joined the company 2 months prior). From the start of the talk he was acting almighty and acted like I should be thankful for the opportunity I have been given. He was asking misleading technical questions. I was super confused first because I assumed malice first ("He is trying to trick me!") but as the interview continued I figured out that he really had no clue what he was taking about. Later during the interview I should describe my experience and I told him about the retail app suite I was the lead dev of for about 2y (POS software + back office + SAP interfaces). He proceeded to tell me that sounds quite nice, but the previous company he worked at did one million transactions per month - hah, eat that stupid peon! I was confused yet again, because that is really tiny? I told him that the project I led did that in a day. He accused me of lying. At that point I politely pointed out that we are not a good fit and wished him the best. 30 mins later my friend called me to ask what happened. The CTO apparently told everyone how poorly I behaved and how measly my skills are. When I told him about what really happened he was quite shocked. He talked to the CEO who did call me next day. I told him a sugarcoated version of what happened. A week later the new CTO was gone. According to my friend he also behaved to other employees in a similar way. The whole story is bizarre.
I had an interview for a company in the south of England a long time ago. The first half went okay, then in the second half the whiteboard came out, and I was asked to "implement C++ object oriented method dispatch in C" with a chunky marker pen. I mean, it's not exactly a rocket science question, but it's not easy either (on a whiteboard, with a chunky pen), and in the end it just felt like it was some weird hazing rather than any test of my actual software development or programming ability.
I stayed in the interview, went home, never heard from them again. I ended up getting a job at a different studio that had a slightly saner technical interview, and after I started there, I heard the first place had gone bankrupt! Sometimes you just dodge bullets...
> I realized that day that I have every right to inspect all aspects of a company that I might work for, and that I don't really have to make compromises if I see something negative during the process.
I think this is really contingent on your negotiating position.
I've been unemployed for over a year. The lower my savings get, the less picky I can be about potential employers.
Done a similar thing in a Zoom interview. I had a horrible hard Leetcode question, was completely stuck and was getting nowhere (neither did the interviewers seemed keen on helping me out). I told them hey thanks this isn't my day I want to quit the interview - at this point they really tried hard to make me stay for the rest of the time. I think if a candidate wants out - you should let him.
5 years later, I still refuse to do them and I don't want to put myself through that process ever again. I got bullied, harassed, laughed at and taunted by people who use this as a one-hour game of kicking people around and I'm not interested in entertaining them any more. I have the following disclaimer both on my LinkedIn [1] and GitHub [2] profiles because I had plenty of surprise-leetcode interviews, so I want to make it clear up front what's acceptable and what isn't.
> I have a personal policy against any type of live coding or online coding tests during interviews and I don't enjoy or engage in any form of competitive programming. Otherwise, I am happy to work offline on coding assignments with reasonable goals and deadlines and to have in-depth technical discussions about software architecture and design as well as relevant technologies.
I like the idea of your approach. Can you share anything about how well it's worked out for you?
My main concern is that with the allegedly rampant cheating on take-home assignments, the number of employers willing to use them may drop precipitously.
I can't say this will work for everyone given the scarcity of jobs. I can afford to miss out on certain opportunities and I'm perfectly happy to invest longer stretches of time in networking and doing whatever homeworks if the assignment makes sense and it's something I am interested in. Let me put it this way: I'd very much rather change professions than go through another Leetcode-style interview again. So far it hasn't come to that, thankfully. There are enough opportunities out there once you spend some time building a portfolio and people in the open source space get to know you.
PS: What this approach gives me is piece of mind since I know I won't get a surprise-Leetcode if I go through an interview. It has happened to me in the past, where the recruiter wasn't really aware what the process is and I haven't told them up front what I'm not OK with. Once I'm there in front of people, it's very awkward and frustrating for me to have to get up and walk out and I'd very much rather avoid that situation. Also, it's a waste of my time.
I see these posts a lot, and I sort of disagree with where they are coming from so I'll play devils advocate. Leetcode is used to select for one of (or both of these) 2 things for the average IC hire:
1. Do you work hard (i.e a person who studies many problems, memorizes solutions, and regurgitates then)
2. Are you smart (i.e a person who doesn't do any leetcode but knows the fundamentals and can sythesize past knowledge well enough to answer a new question)
Now in a vaccuum, my takeaway from someone who doesn't pass a leetcode question is that they are more likely to not be either of those (given no other information about them). In my opinion, solving at least one leetcode easy - medium question (maybe with some direction from the interviewer) should be the standard for any role where you need to code.
And realistically, companies index less on leetcode skill as you become more senior or apply to more niche roles. And plenty of companies out there have bug squash rounds, system design, behaviorals, take homes, trivia, background specific questions, etc. Leetcode is usually not the be all end all for hires above entry level. Leetcode is not meant to magically select for the best engineers. Its simply another signal for the hiring committee to use.
It's skills that a 4-year college student learns at approximately 19 and then promptly does not use for their next 50 years of actually building things.
It leaves out looking at the engineering skills it takes to put something past the finish line on budget.
It's at best, an adjacent sideshow and at worse a proxy for ageism to preference people closer to 19 and less capable at getting quality software out the door.
And maybe that's why the industry has such a damn hard time getting quality software out the door.
Maybe.
Here's a far better idea. Give them a broken build or crashed code or some other failure and ask them to fix it. If they can navigate around a system, diagnose a defect, find the cause, offer a patch, and match the coding/commenting style etc, that's an actual player you need, not someone who can still solve an algo question from their CS101B midterm exam at 37.
This version can go deep as well. You can have separate branches and an issue tracker, maybe with the conversation of the bug already going. Maybe the comments have a patchfile that needs to be manually applied. Maybe the patch has a new bug in it ... Perhaps it's a red herring and addressing a similar but unrelated issue. Maybe it's a closed bug and a regression and nobody wrote a unit test for it. Maybe the unit test is there and it's up to the candidate to discover it or the unit test is out of date, buggy and useless.
This is the real stuff.
Get them to communicate after. Saying something like "the unit tests were broken" versus "I was unable to understand how to get them to work right so I moved on" ... That's a huge difference in attitude - only one of them is a team player.
also if you're going to "timebox" be very very generous. If you think it takes 30 minutes, give them 5 hours and say you're being generous.
I've known people who program things like jet engines and stuff that goes into space. They know their stuff, are super methodical and take their damn time to do it well.
Quality programming is a rarely practiced time consuming art and you'd be lucky if you have 10% of your workforce doing it. Heck, you'd be lucky if you have 1%. Those people save the ship.
I'm not one of them. When you meet them, they're like a different kind of human.
If you timebox them to 30 minutes, the people who write mission-critical quality code will fail to get it in on time and be filtered out.
These are the people who write those 10,000 word articles that appear on HN with carefully constructed schematics, equations, and code that you struggle to understand and bookmark to study later.
On the other hand, if you give people 5 hours, I assure you, the morons will still obviously be morons and you give the "True" Engineers who take their craft dead seriously a chance to shine.
> 1. Do you work hard (i.e a person who studies many problems, memorizes solutions, and regurgitates then)
Most people applying already have a job. And if they work hard its actually quite uncommon for them to have ample energy to do leetcode questions on the side. So you might get the opposite - you'll get candidates who aren't great performers in their current roles but used their extra energy to grind leetcode.
2. Are you smart (i.e a person who doesn't do any leetcode but knows the fundamentals and can sythesize past knowledge well enough to answer a new question)
Leetcode questions can by definition never come close to real life problems - they are simply out of context. Working on a project even a few weeks will grant a smart person any insights he might actually need. The kind of out of the blue puzzle problem a leetcode presents is more akin to crosswords in old leisure magazines.
> 2. Are you smart (i.e a person who doesn't do any leetcode but knows the fundamentals and can sythesize past knowledge well enough to answer a new question)
I'm sorry but it's incredible hubris to think you'll come up with say Dijkstra's algorithm during the interview from the first principles. The only reason you can do it is you studied it before.
So someone coming up with the right answer and claiming they've seen it is useless.
Right answer and claiming they hadn't seen it is basically a guaranteed liar.
You can use it to filter out the liars I guess. But you'll also filter out your Claude Shannons as well.
Either way, it's a dumb test unless you're genuinely getting a steady stream of Princeton and Cambridge magna cum laudes waltzing through the door and can reliable bump the last two priors by a few decimal points.
>And realistically, companies index less on leetcode skill as you become more senior or apply to more niche roles.
when does this start? I'm over 8 years in and I'm tired of being told to make a dang NxM Bingo solver in 30 minutes.
>Leetcode is not meant to magically select for the best engineers.
I'd agree with you in 2022. in 2024, I'm being way more filtered out for questions that I get right but didn't solve fast enough, or with zero hints. I was never a leetcode savant (too many other things to study for, and LC is maybe 20% of my interviews) but I don't feel that much worse than 2 years ago. Companies just have sky high expectations and are on a bargain bin for 15+ year roles paying 5 year salaries.
I know people in the four quadrants of {no-hire, hire} x {would not pass LC, would pass LC}. In my world, conditioning on [would pass LC] does not improve the hire odds.
Even worse: it misses out on the hire people with no formal training too busy to waste time on skills with little use outside of the interview (in the real world, they'd use a library).
Spend enough time around hiring teams or recruiters and you'll see that 99% of them are simply very bad at hiring. They put almost no effort into a search. They can definitely find someone. But finding someone that is hired for their actual ability to solve the problems that the shop or team is working on is rare. Leetcode is just a way of culling the herd. The idea that it somehow reflects the requirements of the job are a useful fiction to obscure how little effort most teams and shops actually put into hiring.
Yeah, no. Show me your GitHub commit history, and point out where you used some leetcode magic. Should be easy, right? Like every other week you'd be using it? Or... is 95% of code somewhat boring, and ideally maintainable?
> 1. Do you work hard (i.e a person who studies many problems, memorizes solutions, and regurgitates then)
This isn't working hard. And you are competing with the people that are working smart. Such as people having a separate computer/screen to either look up the answer to a problem or ask an LLM to regurgitate some code you can transcribe. It selects for people that are dishonest.
> 2. Are you smart (i.e a person who doesn't do any leetcode but knows the fundamentals and can sythesize past knowledge well enough to answer a new question)
See above. A lot of the problems have nothing to do with real world coding scenarios. Do I need to pull out memoization and know how to work with grammar rules off the top of my head? Or the functionality of quad trees and how it applies to partitioning? No; I've literally never used them in my career and likely never will. What I do need as a software engineer is understanding the high level concepts and when to apply them and refresh myself on the use case when and if appropriate.
Readit News
The side effect of this though, is that it's an extremely painful process for the applicant. Even if you're a great software engineer, you're going to freeze up, miss something, have a bad day, and fail a few interviews. Now for most software engineers failure is an unusual and harrowing experience, so they react really badly to it. Especially ina scenario where it's difficult to blame anyone but yourself. But just don't! It's fine! just move on, there's plenty of jobs and the interview process is a blunt tool, not a final evaluation of your worth in life.
“Trick” questions are dumb, but you want to get an idea of if the candidate knows their stuff (and let the candidate know what we’re like — interviewing is a sales process in both directions). If someone asks a question like “is it ok to modify the argument” (or says “I’ll assume I can’t modify the argument”) that’s great (or says “I can’t remember if strlen includes the 0 byte so I’ll add 1 — normally I’d look it up”) - again, no trick questions.
When I say “simple” it’s something like atoi or “shuffle a deck of cards”: basic, but not 100% trivial like fizzbuzz.
On of our best hires was a guy who made a fundamental mistake — when we asked him if it world work as intended he said “I’m a fucking idiot” and fixed it. You’re not supposed to talk like that in an interview I supposed but we read it as a strong positive signal.
We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time.
Trying to debate the interview format or reject interview questions is one of my hard-stop rejection triggers in interviews now.
The couple times I’ve been part of hiring pipelines where candidates argued about the interview itself and then got hired anyway, they became extremely difficult employees within the company. Arguing about interview questions turned into arguing about every other ticket, code reviews, and every architecture choice.
I’ve had the same experience where we take an actual problem we solved in our codebase and turned it into a small interview problem. Same thing: Some candidates will try to debate the relevance or try to wriggle out of it.
It seems that “able to code a simple problem well” is a far less ridiculous proxy for good software engineer than “able to code a hard problem at all”. Much to my surprise.
Fwiw this is surprisingly subtle and non-trivial to get right. A naive "random swap" approach does not produce uniform permutations. Sorting based on a random comparison function can likewise produce biased results depending on the implementation of the sort. Tagging each element with a random number and then sorting based on the tag works (or so I think?) but you need to deal with the case where tags collide. Fisher-yates shuffle is bulletproof but I would not trust myself to write it from memory lest you end up writing Sattolo's algorithm instead.
Though I still do start with a short fizzbuzz-level question because you would be surprised how many people fail that and it makes it way less awkward if you start with atoi and they can't write a loop.
If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice.
I've never been through the "regular" IT hiring process myself, but I've interviewed several candidates for an Android developer position. I liked asking questions and looking at how the person thinks. I didn't expect exact correct answers for every question, I just wanted to see whether they have a fundamental understanding of Android. After that, they had to build a small app at home (not my initiative) to show their skills.
There are better questions where I do think google should be allowed but they aren't LC-style. Some examples are building APIs or operating on files
I've seen this reasoning, but I'm wondering if this is actually true, especially if you also believe the other myth in hiring: most people applying are not qualified for the job. Since most hiring actually requires you to fill the position in the end (in other words, you can't leave it open indefinitely if no one qualifies as eventually the pressure to hire mount up), every false negatives does increase the odd of you hiring a false positives as well. Just take the example at the extreme, if there is only ever one person on Earth qualified for your job and you got a false negatives on him, there is no other way but getting an idiot in the job.
Depends where you’re looking in the hiring pipeline. If you’re looking at raw applications submitted to a publicly listed job posting, I can say without reservation that the majority of candidates are not qualified. The pre-screening candidate pool is extremely bad on average, especially for remote job listings. Some people spend all day clicking the apply button on every job listing they see.
Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true.
As someone who’s done a lot of interviews, this is very true for SWE roles.
I do work in a team where majority was hired by puzzles of sorts. All of them are smart. They are not software engineers and it shows massively.
Most programming work needs common sense a lot more than it needs theoretical algorithmic knowledge.
And almost nobody values good documentation, especially as a competetive advantage.
Coming from Real Engineering and making 2x more programming, I lol at people who say there is any Engineering in software.
We are programmers dealing with layers upon layers of abstraction. Knowing to optimize for time by using vectors is more of an art, than a science.
I did do safety critical C code and Assembly which are probably my hole in the whole 'its not engineering'. But javascript/python/backend work: programming.
this would make sense if they filled these positions in 2,3 weeks tops. But that's the insane part; I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks. At that point I feel you are no longer avoiding false positives, but simply hiring the most desperate programmers willing to jump the hoops. The best candidates will get an offer mid interview at that rate.
>But just don't! It's fine! just move on,
I've been "just moving on" for some 9 months now. There's only so much morale in a tank, no matter how BS you know the whole process is. I just wanna move on in life, not be bogged down on how many permutations you can color an N x M grid with.
Totally, but leetcode wont filter them out.
Incompetent is fairly easy to navigate around, pathological bastardness is impossible to work past. I know that some companies use psychometric tests, but they are not based in anything remotely scientific, but are also super easy to game.
Best tests are simple and common computer science problems in algorithms and data structures with no tractable solutions. There is no facile answer, you truly have to reason from first principles for any particular scenario because there is no alternative.
But it's entirely pointless.
Deleted Comment
And because they ask pretty irrelevant questions, the 6 weeker will do better then the recent Stanford PhD who has been studying a specific problem for 5 years or the multimillionaire who spent the past 10 years building and selling successful software companies.
Those people will fail and your 6 weeker who's never heard of cmake, diff or gdb will pass.
When I see there's leetcode as the hiring process I presume the place is full of bozos because the process actually strongly preferences them.
Recently I failed the first round of a fintech analyst position because I didn't complete the 9 undergrad (or high-school, in certain talented schools) math problems (I have a math PhD) in the 1 hour time frame. In retrospect, I should've cheated using computer/online assistance, and my true failure was my unwillingness to cheat. Not because fintech analysts are cheaters, but because the testing process is nonsense, so recognizing that, one should cheat it (indeed, an analyst mindset!) I'd rather be jobless than to adopt that mentality though. (and in fact I am, the only job I was able to find was a minimum wage service-type job, it can't even pay for daycare, I'd rather raise my child on my own while wife works thank God.)
We have high-interest rates and global crises resulting in companies not hiring while simultaneously governments give orders to news media to tout the strong economy. Ineffective hiring practices will always exist, but people don't complain when jobs are available; you only see this type of discussion online when things have gone sideways.
Either it wasn't a true "zero" but rather they were really good e.g. at advanced math or physics and managed to see some parallels, or it has to be a genius. I'd be very interested in hiring such a person.
They give the candidate some messy, done-in-a-hurry code (but not purposefully obfuscated) and ask them to refactor it to the best of their ability. The interviewer sits next to the candidate and talks to them throughout the whole process. It's pretty much pair programming, but the candidate has the initiative.
I found it to be a breath of fresh air after all the leetcode interviews. Of course, they have the luxury of doing this, because they know what exact technologies they are using and they don't need to measure candidate's ability to learn new tech, etc. In their situation they are more interested in topics like whether the candidate can produce maintainable code, name things properly or communicate with co-workers.
Well leetcode interviews do not measure the candidate's ability to learn new tech either, do they?
Although I don't find it very relevant. When I was fresh out of university, I was good with algorithms / data structures (neither of which I use much in actual work), therefore able to pass leetcode interviews just fine. Yet it was at the time challenging for me to learn new frameworks and I found all of the tooling I never heard of rather confusing.
If you are looking to hire people to write < 50 line snippets, it works fine. Pretty much useless for everything else.
What's super interesting is I have actually had people pretty much fail the basic stuff (which was probably nerves) and then go on and kill the code review section in ways that shows their serious expertise.
IMO interviews are pretty bad at demonstrating people's ability to work, but if you can zoom in and out on a problem and do good enough on the coding stuff its usually a win.
1) A way to suppress wages and job mobility for SWE. Who wants to switch jobs when it means studying for a month or two? Also, if you get unlucky and some try hard drops an atomic LC hard bomb on you now you have an entire company you can no longer apply to for a year.
2) A way to mask bias in the process while claiming that it’s a fair process because everyone has a clear/similar objective.
Meet someone who went to your Alma mater? Same gender? Same race? Give them the same question as everyone else, but hint them through it, ignore some syntax errors, and give them a strong hire for “communication” when they didn’t even implement the optimal approach…
Or is it someone you don’t like for X reason? Drop a leetcode hard on them and send them packing and just remain silent the entire interview.
To the company this is acceptable noise, but to the individual, this is costing us 100s of thousands of dollars, because there’s only a handful of companies that pay well and they all have the same interview process. Failing 3 interviews probably means you’re now out $200-300k of additional compensation from the top paying companies.
I’ve interviewed for and at FAANGs. I can’t believe the low bar of people that we’ve hired, while simultaneously seeing insane ridiculous quad tree/number theory type questions that have caused other great engineers to miss out on good opportunities.
Someone will reply to me “if you know how to problem solve you will always pass.” Ok, come interview with me and I will ask you verbatim one of those quad tree/number theory/inclusion exclusion principle questions and I’d love to see you squirm, meanwhile another candidate is asked a basic hash map question.
The problem isn't standardized tests, it's that leetcode questions are about having the time to have learnt the answer beforehand, rather than raw ability for problem solving.
I've done tests like this for some companies. It felt a lot fairer and more closely resembling the actual work environment than live leet-code interviews, with biased interviewer(s) and a stress factor that's not a part of the actual job.
This is what privilege looks like. The inability to see barriers that affect others in a worse position.
Standards do not imply fairness only consistency.
You got a test that filters out people who lack the time to train for these tests. Basically, devs with a life (see family)
Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that?
It can make the process fairer but it's not a given. You can do the classic "no dogs, no blacks no Irish," and that was a standard across pubs in England. It certainly wasn't fair, unless you were racist.
If you're committed to fairness, then a standard will help. It gives you a clear point to fix and improve things and something you can use to measure if you're achieving your stated goals. That's definitely something you can't do if you're just making things up as you go along.
And yes, I made a deliberately provocative statement. I'm obviously not saying whiteboard tests are the equivalent of segregation.
...a standardized test? No. There are tests. They sure as heck aren't standardized.
Maybe they should be, since everyone seems to be doing the same thing.
Many companies have question banks that are specially designed to be fair/have some contextual relevance (ideally) to some "realistic" problem. Or at least, many of the companies I've interviewed at follow this model. I consider these coding questions to be "leetcode" style because at the end of the day they are an isolated coding problem, even though they may not be a problem from leetcode verbatim.
Companies that execute on that style of interview well are generally fairly pleasant interviews, at least in my experience. Good companies/interviewers will gauge more than just the final code to determine a hire or no hire. And a large portion of companies also have hiring ratings on a scale to make it less binary.
This is ironic considering these companies have forced mandatory DEI seminars (which I have no problem with btw), inclusive language, #EveryoneCanCode, and so on.
But despite all this, you end up with teams and organizations that are 99% of the same X somehow. Replace X with race, school, even state from home country sometimes.
You know there are websites where people share all the interview questions to these hard interviews / referrals exclusively in their language and behind a pay or rep wall? And there are Telegram groups where international students leaks the questions or do interviews in place for one another.
It’s inevitable these types of issues arise when there’s so much at stake, ex: in just 5 years, a $200k TC advantage at a top company, becomes $1m or more with appreciation.
I just really dislike the veneer of “fairness” when there are so many problems with the process, even beyond the questions that have nothing to do with the job.
Question banks that are too small: leaked on eastern forums immediately, candidates show up reading answers out to you (some of the guides include guidance on when to pretend to think, I am not kidding).
The idealized version of "question banks" might work. The real one does not; you'd require employees constantly scouring forums in every language known to mankind, immediately removing anything that gets leaked. On top of that you'd probably require a competent committee overseeing all questions in the bank constantly and ensuring the lack of variance in difficulty.
Source: I interviewed at and for Goog and Pltr.
The reality is, fizz buzz got us 75% of the way there. It turns out when pressed, a lot of people can't write code. Yes, there's false positives, but there's also people brute focing their way through via copy & paste.
This doesn't manifest as a person who can't do any task, just as a person that's slow, delivers weird abstractions, and would take a lot of your time to get anything useful from.
But those people are also making those arguments, because, as you said, there's hundreds of thousands of dollars in it.
As an early 2000s integrator/analytics I learned to write code when someone watches out of interest or straight time pressure (still taxing sometimes). But most developers that I know intellectually curl up in such situation, regardless of their skill or performance levels. We worked with one very math-y low-level-skilled guy whom our clients described as “literally freezes for an hour without moving, should we pay for it?” when he did field work. He was a very strong developer otherwise.
Not that a company must hire or want these people, but the idea of writing code under uncanny pressure all day makes as much sense as that swordfish scene.
For companies this is very rational, you want non-distracted employees who can work over time.
In all my years of sw dev the number of times that would have helped vs being able to communicate and manage expectations across a swath of people is like 1:1000.
If you want to earn top money, you gotta play the game. This is true in every profession, you think lawyers that want to work at top law firms have it any easier?
There's a lot of valid criticism with respect to today's tech hiring practices (especially because it doesn't only affect top paying companies), but I don't feel like "I can't get ridiculously wealthy without selling my soul" is a particularly compelling one.
Everything is a proxy because, in most scenarios, there is no way to understand effectiveness of an employee until they have worked for you for a while.
Of course there are other ways, but they all carry their own risks.
That’s why referrals, imperfect as they are, carry so much sway. In theory they reduce uncertainty and require some expenditure of political capital.
If Google engineers all wore pink robes and top hats while interviewing then we'd see that everywhere.
If this was just FAANG I wouldn't even mind it. Bigger compensation means bigger hoops to jump through, artificial or otherwise. At least those bigger companies come with interview guides.
But I've been dropped Leetcode mediums on some Unity game dev jobs barely paying 40/hr with minimal benefits. Game interviews already have an absurd number of topics to try and guess you'll be quizzed on. (last interview wanted low level C memory/bit manipulation question... the one before that did Unreal Engine trivia... then the one before that decided to give me vector math and ray intersection questions), adding in random string manipulation questions when you will barely use more than concatenation on the job doesn't help with studying.
Game dev is lower paying, less structured, and more involved than typical web dev jobs.
I’d really only want to get into game dev early in my career or as an independent creator. It’s a touch exploitative.
Well, that and how almost nobody who successfully finds employment after "grinding" leetcodes wants to remove the barriers for entry.
I think Leetcoders can't envisage a better way to assess someone than by subjecting them to the same kind of hoop-jumping you get made to do in university. They're not interviewing you for a job as there's no module on interviewing candidates on the CS curriculum, and don't have much professional experience outside of academics or software engineering. They're simulating a dissertation defence, because that's how they were assessed for their competence.
That's my charitable interpretation. If I'm being cynical, it's elitism - a way of making sure you're "one of us" (read: obnoxiously academic, Type-A personality, "logic over feelings").
Not that many, but its interesting how many are from "elite" universities. I'm in a research org for a FAANG and we often get to see all the handwringing about how we can't recruit more of people type x.
Well, if you only hire from MIT, Stanford, Oxford etc, then they are all going to look the same.
For those outside the research org, its a bit better, but its still the most uneven place I've worked.
Suppose there was a SWE union and the union has to create their own reference compensation bands. Would the union again lean towards leetcode or some other standardized test? Or just use years of experience and maybe past projects?
When weighing the “effectiveness” of leetcode interviews, it’s important to weigh that thus far SWEs have failed to effectively unionize despite e.g. the past class-action no-poaches clearly showing C-suites should be paying engineers a larger share.
That depends very much on your world view. If you get the job, it would imply you've just cost a lot of other people $100ks. That simply can't be true, because there's a small number of such jobs.
The only way this can possibly hold is if you hold that only the "best" candidate should get the job, and by "best" you mean: the one that gets most of the leetcode questions right. But then there's no us.
Edit: I'm not in favor of leetcode interviews, and I do understand that there's a bias in interviews (which won't go away when you drop the leetcode).
I do not know how hiring is done at megacorps; but on the few occasions I participated in tech interviews, we were looking for candidates to join the team that I was on. So, what I was interested about the candidates was how well they were prepared for the kind of work our team does; and how much I would like to work with them. I am sure this second question introduces all sorts of subjective biases; but then, hey, aren't you going to spend countless future hours with that person? At least the first concern incentivised us to make sure candidates were sufficiently competent. BTW, we didn't ask leetcode-type questions.
> but to the individual, this is costing us 100s of thousands of dollars, because there’s only a handful of companies that pay well and they all have the same interview process
I am not sure I am following the argument. If there is only a handful of companies that pay well, and if you only consider companies out of that list for employment, isn't it reasonable to predict that there will be lots of other candidates who want the same, far more than the number of available positions at those companies. How, then, should those companies select from such a pool of candidates?
For those wondering what the alternative is to leetcode: a work sample test. Take a real problem your team solved recently, distill it down to remove internal-specific knowledge, and give candidates 3x the time to solve it compared to what it takes an internal dev to solve. Bonus points for having a rubric guide the scoring.
One time in a technical interview I realized a quadtree would solve the problem they were asking. I also happened to be reading SICP and practical common lisp on the plane on the way over to the interview. (specifically the sql sdl macro chapter) Unfortunately this gave me an idea. I narrated, sketched out and pseudocoded a plist based quadtree on the whiteboard and felt pretty good about myself.
I turned around to see my interviewers were not impressed. Actually they thought I was an idiot. They didnt know what a quadtree was, and they were quite upset I used "tuples".
I assured them its very efficient, and that the compiler would take care of "the tuples".
They failed me. Moral of the story don't use common lisp in a javascript interview.
- Input from the interviewer reflects communication ability, cultural fit, and so on of the candidate.
- Input from the blind grader reflects ability of them as a team.
- Low grades count against the interviewer as well.
Bingo; now both are stakeholders in success.
OTOH, seeing how the interviewer reacts to that situation will tell you a lot about whether you really want to work there.
The absurd filter is just some kind of lottery. They could have a different one: at the end of the day, it's only when the person starts working that you can actually see what they are worth.
The thing with a filter like this one is that it filters out a whole category of people who may actually be good. And it reinforces itself: you hire people who are good at leetcode, who will themselves possibly be good at hiring people who are good at leetcode. Does a company of leetcoders perform better than a company made of a diversity of good engineers? Not clear.
However, we do require a base level of competency. We give out a 40-ish minute assessment. Two multiple choice and three coding problems.
And they’re easy. One of them is essentially “John has a 5 gallon bucket and 3 gallon bucket, how many buckets does he have?”. All you have to do is rewrite the clearly labeled circuit diagram as a Boolean expression.
Not a single person this round has passed. Several have failed the simple ones.
So while grilling people on the minutiae of a language or asking them to solve the traveling salesman is not beneficial, neither is nothing.
We should be testing floors, not ceilings
higher education, even job experience is no longer considered a sufficient barrier to entry.
reminds me when the guy who wrote homebrew failed the apple interview. if i recall.
ironic because they use brew so much internally.
I prepared thoroughly for all four main questions and any other plausible ones I could think of. I practiced writing solutions to ensure I was fast enough for the interview. Additionally, I pre-prepared ideal answers for each question in case I got stuck.
When the interview came, I got a total curveball: a question that was significantly harder than the usual ones. It didn't fit the round's theme (it was a DSA question, but I'd already aced the DSA round), was obscure enough not to be on LeetCode, and required writing a solver for a hard variant of a known algorithm. I panicked, copied the prompt into ChatGPT (despite being instructed not to use it), transcribed the result, and pretended I had recently studied the relevant algorithm.
I passed the round, nailed the other interviews, got the offer, and accepted. Later, I found out that interviewers are instructed to pick one of four specific questions for that round, and the one I got wasn't in the list.
I'm left wondering if the interviewer was trying to sink me or was just bored with the usual questions. The whole experience raised several questions for me:
Is it cheating if I already had pre-prepared answers for the questions they were supposed to ask? What's the difference between using pre-prepared answers and using Google or ChatGPT during the interview?
If the interview had gone according to plan, what was I actually demonstrating? My ability to use Google?
When the interviewer asked an impossibly difficult question, I would have failed if I answered it legit, even though I'm a good engineer. Failing such an unfair interview round doesn't serve the company's interests.
What is this interview process meant to demonstrate? My true value as an engineer lies in my ability to communicate clearly, think outside the box, identify and address technical tradeoffs, mentor juniors, and propose technical solutions that meet requirements while minimizing risks. Yet, I'm expected to solve a hard variant of the Traveling Salesman Problem in 45 minutes or I don't get the job? Why?
The whole process seems broken, but I'm not sure how to fix it.
IIUC, the interview deviated from the company's interview policies.
Could the answer simply be that the company has no intentions regarding the aberrant interview process that you experienced?
If what Google search tells me is true the average SWE tenure is only a couple of years or so.
I might need a month or two of study (or more!) to be able to handle the harder Leetcode problems, but if I was expecting to have to do this every couple of years or so I'd consider taking some steps to maintain that knowledge between jobs.
That would probably be considerably less overall effort than forgetting it and having to cram for months every couple of years.
But that would dramatically increase worker mobility, for one thing. So they don’t do that.
(“No it’s because they don’t want you to be able to test prep” that makes no sense because 99% of the way to prep for these is… exactly like test prep)
Leetcode is lazy, easy.
I don't think this is an issue of bad luck, its quite probably to get difficult interviews especially if you're applying to the popular names.
The shit soup presented at a coding interview, leetcode or take home style ones are a result of wholesale cargo cult stupidity than malice.
There's no incentive at any level in BigCo. Margins are too big for top level to care. its hard to measure, middle management layer can easily fudge metrics while driving the org effectively into the ground. Lower layer that made it through the filter, sick of this grind and are busy filing tasks sipping free coffee.
The elephant in the room is that you cannot make hiring a process driven activity.
This is org level dysfunction. There's more such exhibits - DEI, promo criteria, ERGs, re-org politics, vendor selection to name a few.
with all due respect I believe this is about as far from the truth as it gets. I believe the issue is that people think they have to work at FAANGs in order to be paid REALLY well which is just nuts.
I know many people making (very) high 6 figures working at places most have never heard of. instead of looking for FAANGs people should be looking at companies that have been in business for 20+ years (if publicly-traded company the easiest "measure" would be whether you would invest your own dough into that company) and then make your self indispensable there (this is much easier than it sounds) to a point where you are more valuable to the company they company is to you. This is un-achievable at FAANG but this should be everyone's career goal. once you are worth more to a company then company is to you, compensation-wise sky is the limit :)
You know people making ~800-900k at places most people have never heard of? That is what "high 6 figures" means. I have to assume you mean more like ~$180-190k?
See, this is true eventually. But it's _definitely_ not true in 45 minutes.
Why don't you list them here to illustrate your point?
Whiteboard interviews are a good way to evaluate these skills - assuming the candidate hasn't seen this particular problem before and the interviewer understands that this is what they're supposed to look for.
Goodhart's law applies - the measure becomes a target and it ceases to be a good measure.
People who have weak problem solving skills or poor abilities to form mental models still want high paying software jobs so they "grind leetcode" until they can pass these interviews.
Then over time these grinders get high enough up they start becoming interviewers themselves and they don't understand that they're even supposed to screen for talent, they conduct the interview like a memory test since to them that's what it was.
In spite of this, there's no better interview type I've seen proposed given the constraints most companies are under (don't waste candidate's time, have a consistent rubric that HR likes, etc.) so for now we're going to keep using it.
Ask the candidate to build a known quantity like say, a calculator, calendar, whatever or give them a bunch of code and have them modify it in some way or discover a bug.
Something closer to building software. Leet code is too far away from it and the performance on it too poorly correlated with actual suitability.
It follows that better leetcode scoring is a good indicator of who can be replaced with LLM.
They say As hire Bs and Bs hire Cs . . . and to combat that we've added more leetcode to the interview process.
Yeah, right. Why would companies truly care about this issue and put any effort into making their hiring process feel less hostile when they still have loads of candidates who are more than willing to go through whatever tests they get asked during interviews? It's a leaky bucket and they're happy with the status quo while developers keep grinding away... I doubt it will change anytime soon, but I'm glad I can afford to put my foot down and refuse to do such interviews when I need to find new jobs.
The CEO chased me out into the parking lot and begged me to tell him what I saw and why the interview went awry. He was legitimately shocked that an interviewee could make such a decision, mid-interview .. All I could tell him was that it wasn't going to be a great fit, the culture didn't seem aligned, and the people who were brought in to the interview seemed desperate and hostile to each other - and that simply wasn't the way I had hoped that the company operated.
People forget that job interviews are a two-way street.
I realized that day that I have every right to inspect all aspects of a company that I might work for, and that I don't really have to make compromises if I see something negative during the process. It was the right decision because the very next day, at another competing firm, I met folks who have continued to be good, close friends and colleagues, whom I actually really enjoy being around.
This profession is fun. But when the fun gets farmed and exploited, its enslavement like any other.
I had an interview for a company in the south of England a long time ago. The first half went okay, then in the second half the whiteboard came out, and I was asked to "implement C++ object oriented method dispatch in C" with a chunky marker pen. I mean, it's not exactly a rocket science question, but it's not easy either (on a whiteboard, with a chunky pen), and in the end it just felt like it was some weird hazing rather than any test of my actual software development or programming ability.
I stayed in the interview, went home, never heard from them again. I ended up getting a job at a different studio that had a slightly saner technical interview, and after I started there, I heard the first place had gone bankrupt! Sometimes you just dodge bullets...
I laughed when I read that - turning around the same BS they spew for why they claim they didn't hire someone.
I think this is really contingent on your negotiating position.
I've been unemployed for over a year. The lower my savings get, the less picky I can be about potential employers.
> I have a personal policy against any type of live coding or online coding tests during interviews and I don't enjoy or engage in any form of competitive programming. Otherwise, I am happy to work offline on coding assignments with reasonable goals and deadlines and to have in-depth technical discussions about software architecture and design as well as relevant technologies.
[1] https://www.linkedin.com/in/mtodor
[2] https://github.com/mihaitodor
My main concern is that with the allegedly rampant cheating on take-home assignments, the number of employers willing to use them may drop precipitously.
PS: What this approach gives me is piece of mind since I know I won't get a surprise-Leetcode if I go through an interview. It has happened to me in the past, where the recruiter wasn't really aware what the process is and I haven't told them up front what I'm not OK with. Once I'm there in front of people, it's very awkward and frustrating for me to have to get up and walk out and I'd very much rather avoid that situation. Also, it's a waste of my time.
1. Do you work hard (i.e a person who studies many problems, memorizes solutions, and regurgitates then)
2. Are you smart (i.e a person who doesn't do any leetcode but knows the fundamentals and can sythesize past knowledge well enough to answer a new question)
Now in a vaccuum, my takeaway from someone who doesn't pass a leetcode question is that they are more likely to not be either of those (given no other information about them). In my opinion, solving at least one leetcode easy - medium question (maybe with some direction from the interviewer) should be the standard for any role where you need to code.
And realistically, companies index less on leetcode skill as you become more senior or apply to more niche roles. And plenty of companies out there have bug squash rounds, system design, behaviorals, take homes, trivia, background specific questions, etc. Leetcode is usually not the be all end all for hires above entry level. Leetcode is not meant to magically select for the best engineers. Its simply another signal for the hiring committee to use.
It leaves out looking at the engineering skills it takes to put something past the finish line on budget.
It's at best, an adjacent sideshow and at worse a proxy for ageism to preference people closer to 19 and less capable at getting quality software out the door.
And maybe that's why the industry has such a damn hard time getting quality software out the door.
Maybe.
Here's a far better idea. Give them a broken build or crashed code or some other failure and ask them to fix it. If they can navigate around a system, diagnose a defect, find the cause, offer a patch, and match the coding/commenting style etc, that's an actual player you need, not someone who can still solve an algo question from their CS101B midterm exam at 37.
This version can go deep as well. You can have separate branches and an issue tracker, maybe with the conversation of the bug already going. Maybe the comments have a patchfile that needs to be manually applied. Maybe the patch has a new bug in it ... Perhaps it's a red herring and addressing a similar but unrelated issue. Maybe it's a closed bug and a regression and nobody wrote a unit test for it. Maybe the unit test is there and it's up to the candidate to discover it or the unit test is out of date, buggy and useless.
This is the real stuff.
Get them to communicate after. Saying something like "the unit tests were broken" versus "I was unable to understand how to get them to work right so I moved on" ... That's a huge difference in attitude - only one of them is a team player.
I've known people who program things like jet engines and stuff that goes into space. They know their stuff, are super methodical and take their damn time to do it well.
Quality programming is a rarely practiced time consuming art and you'd be lucky if you have 10% of your workforce doing it. Heck, you'd be lucky if you have 1%. Those people save the ship.
I'm not one of them. When you meet them, they're like a different kind of human.
If you timebox them to 30 minutes, the people who write mission-critical quality code will fail to get it in on time and be filtered out.
These are the people who write those 10,000 word articles that appear on HN with carefully constructed schematics, equations, and code that you struggle to understand and bookmark to study later.
On the other hand, if you give people 5 hours, I assure you, the morons will still obviously be morons and you give the "True" Engineers who take their craft dead seriously a chance to shine.
Most people applying already have a job. And if they work hard its actually quite uncommon for them to have ample energy to do leetcode questions on the side. So you might get the opposite - you'll get candidates who aren't great performers in their current roles but used their extra energy to grind leetcode.
Leetcode questions can by definition never come close to real life problems - they are simply out of context. Working on a project even a few weeks will grant a smart person any insights he might actually need. The kind of out of the blue puzzle problem a leetcode presents is more akin to crosswords in old leisure magazines.
I'm sorry but it's incredible hubris to think you'll come up with say Dijkstra's algorithm during the interview from the first principles. The only reason you can do it is you studied it before.
Chance of someone
- seeing it before: 10%
- being a liar: 5%
- being the next Claude Shannon: 0.000001%
So someone coming up with the right answer and claiming they've seen it is useless.
Right answer and claiming they hadn't seen it is basically a guaranteed liar.
You can use it to filter out the liars I guess. But you'll also filter out your Claude Shannons as well.
Either way, it's a dumb test unless you're genuinely getting a steady stream of Princeton and Cambridge magna cum laudes waltzing through the door and can reliable bump the last two priors by a few decimal points.
when does this start? I'm over 8 years in and I'm tired of being told to make a dang NxM Bingo solver in 30 minutes.
>Leetcode is not meant to magically select for the best engineers.
I'd agree with you in 2022. in 2024, I'm being way more filtered out for questions that I get right but didn't solve fast enough, or with zero hints. I was never a leetcode savant (too many other things to study for, and LC is maybe 20% of my interviews) but I don't feel that much worse than 2 years ago. Companies just have sky high expectations and are on a bargain bin for 15+ year roles paying 5 year salaries.
Even worse: it misses out on the hire people with no formal training too busy to waste time on skills with little use outside of the interview (in the real world, they'd use a library).
This isn't working hard. And you are competing with the people that are working smart. Such as people having a separate computer/screen to either look up the answer to a problem or ask an LLM to regurgitate some code you can transcribe. It selects for people that are dishonest.
> 2. Are you smart (i.e a person who doesn't do any leetcode but knows the fundamentals and can sythesize past knowledge well enough to answer a new question)
See above. A lot of the problems have nothing to do with real world coding scenarios. Do I need to pull out memoization and know how to work with grammar rules off the top of my head? Or the functionality of quad trees and how it applies to partitioning? No; I've literally never used them in my career and likely never will. What I do need as a software engineer is understanding the high level concepts and when to apply them and refresh myself on the use case when and if appropriate.
It's been proven many times that the severity of an initiation ceremony significantly boosts the commitment of those admitted to the group.
What is a better capitalistic company hiring programmers:
>Lots of smart similar people who believe in correct answers
>Lots of random skilled people
If you said the latter, you are idealistic and your opinion genuinely doesnt matter. Nature will replace you with someone smarter and realistic.
Citation needed