Hello fellow HN users.
Is it my idea or are interviews more harder nowadays? If you had to take a technical interview for your current position would you be able to pass it and get the job?
If you were the one doing the interview, would you be strict and request more things so as to filter out candidates? It’s understandable because somehow you need to find the best ones for a given position, but there are some soft skills that C/Java/Python interview problems can’t identify.
I’m asking because I’ve seen many interesting positions in which I could imagine myself into (having ~20 years experience in telecoms/soft engineering), but the entry requirements with regards to programming languages are set too high.
What are your experiences?
Consequently, I have a lot of successful open source work, but most companies still don’t care and fail me for lack of fake puzzle quiz skills.
It’s utterly baffling to me that companies don’t want creative employees that build successful products out in front of everyone. Where you have a clear view of all their interactions and code.
I’ve tried to change this thinking at my current company and the consensus was it would be biased. Biased?! Code quizzes are highly biased! I’m producing useful functionality to the world!
I believe this and transparency are the new remote work. Some companies will get it early and for others it will take a major upheaval to get them to a place of reasonableness.
I already have to be so focused with what little time I have in my life, and sacrifice a lot, to just make the things I want to make, I just can't imagine finding all the time to do this meta work/training.
Every day I grow more comfortable with the idea that my love of software dev and oss will forever be avocational. Perhaps though, financial instability aside, I'll be happier this way anyway
Example:
Breadth first search
Recursion
Binary Tree
Etc.
Then it’s more a classification problem.
Take-homes get a lot of criticism on HN, but in my experience they're superior for all the reasons you mentioned.
I don't actually see pushback against take-home problems in the real world, mostly just online. When given the choice, most people would prefer a practical take-home over a whiteboard style interview.
For example, it's very tempting for a company to send out tests before doing much resume screening. Why should I spend 30 seconds looking at a resume when I could have the candidate spend an hour on a test, and save myself 30 seconds?
Or what if I set a take-home test which is supposed to take 1 hour, but also supposed to really to stretch candidates' ability and give the best candidates a chance to shine? Probably a lot of average candidates will give themselves a bit of extra time - and before you know it, I'm sending a "1 hour" test that takes an average of 4 hours.
Take-home tests are a lot less objectionable if you can properly guard against these problems.
Many interview at a much less aggressive pace than that and I would prefer a take home when I am out of leetcode practice.
as an applicant, ive refused takehomes that were auto-sent on receiving the application, but if its after a phone call with the hiring manager or engineer its not so bad. but then, i did end up having to defend why i didnt do a bunch of parts ancillary to the assignment - 'why didnt you build a config loading system?' 'uh cause i timeboxed it and that gives you fairly little signal compared to the business logic'
When I went through the job search, I did close out a few applications because I found out I wasn't too interested in their description of the role (after a conversation with more details than that was in the job posting), or simply if their screening process wasn't worth my time. The one I lamented about pass through those interest tests as the job genuinely fascinated me.
A take home that is 1-2 hours (what might be the length of an in person) is fine.
A take home that is 10 hours of work on the other hand
1. They do not scale horizontally. You do a takehome and get rejected, then your effort has become worthless when applying to other companies. Versus grinding leetcode - it scales across a huge breadth of many different companies.
2. Often times a takehome is not. instead of an leetcode interview. It's in addition to the leetcode interview. So you end up needing to practice leetcode anyways. Now you just have more work to do - work that doesn't horizontally scale.
Coincidentally, I've started working on a product to help this[2]. The general idea is that tasks like pull-requests, bug fixes and realistic take home projects are a better way to test candidates. Everything is done using Github, the tasks are all realistic and candidates can use their own laptop and development environment. It's not fully ready yet, but if anybody wants to chat about it just use the info@ address for the site.
[1] https://www.sciencedaily.com/releases/2020/07/200714101228.h... [2] https://devscreen.io/
No offense to anyone, but it strikes me a little bit as armchair expert hot takes saying "why not just X". Personally I think that rather than complaining about leetcode-style loops, it'd be more productive to think in terms of what the problem domain is: namely, many of these companies are typically large and want some level of standardization in the hiring process: you have to contend with things like some ethnicities being culturally prone to "hiring down" to inflate subordinate count, talk-the-talk-but-dont-walk-the-walk types, thinly disguised sexism/ageism/other biases that are excused under the "cultural fit" umbrella, mismatch in terms of desire to provide technical interviewing skills vs volume of candidates, etc. It's easy to attack any one methodology simply because if such a thing as a silver bullet existed, we'd all already be using it. All interview methodologies are flawed, and for better or worse, leetcode-style happens to predominate interview loops because it has a semblance of standardization (though ironically, IMHO a good interviewer ought to be looking between the lines more than seeking specific answers, and a lot of leetcode-style interviewers don't get that nuance)
If anything, my two cents is that leetcode ain't going anywhere, so the next best thing we should be focusing on is making it more widespread that memorization culture isn't and shouldn't be the evaluation criteria (this goes both for the interviewer and interviewee sides)
I've been doing this long enough to know that the things that matter in every job at every company are almost never technical ability. It's skill in managing deadlines, collecting requirements and dealing with changes, divvying up work with teammates, things like that. Things that are difficult to test for, even on a whiteboard.
Asking questions about experiences and posing hypotheticals to see what gets candidates interested, or what stories they have to tell, these have been a better predictor of eventual success in my experience than any sort of technical test.
We had a known problem set that all candidates had to work and this consistency kept HR happy (although it was an uphill battle to convince HR that it wasn't a "test"). The problem was sized so it would be just barely possible to complete the entire thing in 45 minutes. In fact, I think in all the years we did it, only two people every completed the exercise, and one of those we suspect knew about it ahead of time.
However, the problem wasn't a blind "do this on a whiteboard" exercise. We didn't really care whether or not they finished: rather we wanted to see the process they took, the clarifying questions they asked, where they got stuck, how they asked for help, etc.
The candidate could choose to work in complete silence, or work with the interviewer as a team exercise, or a mix of both. The point of the exercise was to see if they could (a) program their way out of a wet paper bag and (b) do so in a way that didn't completely alienate the people they would have to work with. Leetcode? Obscure algorithms? We couldn't care less.
It was surprisingly effective.
Sorry, what? Which ethnicities are these?
This is the redacted exchange:
Hi [PeopleOps], I started the assessment and was very disappointed to the point where I didn't complete it. I've been an Engineering Manager for about three years and still have a lot to learn, but I've never encountered anything that looks like a set of brain teaser puzzles & psychometric analysis similar to this test. I have to interpret this a huge mismatch between how [Company Name] defines the best skills and aptitudes for the role and my own, so I'm going to withdraw from the process. I can't imagine a lot of candidates are prepared to jump through these sort of hoops but that's just my single perspective.
MONTHS LATER
Thanks for the time earlier this week.
I wanted to give you a heads up as to where we are at in our Sr. Engineering Manager hiring. We had an Engineering manager join us earlier this week and as of right now, we're ok on the hiring front.
I'd love it if you could provide us some feedback on the process as we're always looking to improve. Here's a link to a very brief survey that would really help us out. Thanks!
The issues that candidate employers are concerned about are (should be) more along higher level constructs such as ensuring quality or how to decide on a new technology or platform. What's a really interesting bug you found -- where really interesting probably means it was a system level issue, not just some bad code.
The interview becomes a detailed discussion between peers who are each trying to understand if they want to work with the other.
If someone started out with leetcode crap I'd probably react exactly as you did.
I'm a script hack and sysadmin/SRE, not a software puzzle guru. I look up the syntax I need for the language I'm writing today. I may end up programming a different language next week, month or year. It's not worth the trouble memorizing the triviata of every language I ever used.
There are only so many ways to cast variables and write loops. Everything else is nomenclature and punctuation.
Ask me to address a system problem, don't give me a BS puzzle.
In general, I dislike the interview process too, but there just isn't a better way to do these, as is clear from this large thread. If you look at it from the company's perspective, you have to conduct interviews at scale, and it can't hand off the same system problem to everyone.
You're being tested on your problem solving ability. It doesn't correlate directly to your job, but the general assumption is, if you can solve puzzles after studying for them, the chance that you can solve any random problem thrown at you is high.
How do you know this? The signal of them dumping you with some leetcode should be an indication that it wasn't meant to be?
The job description was top notch. It was a clear description of what they needed this engineering role to do. Inspire of it being a bit of a niche, I had almost perfect set of experiences from my prior jobs that made it very interesting.
Fun because it would have given me an opportunity to _improve_ upon my already bootstrapped skills. Productive because, I already have a head start than another engineer getting into the groove of things. This is HN, I can be tech specific :-) I have worked a lot on performance from HPC perspective. The job was at a High Frequency Trading platform that needed a sys.engineer who would work on extracting performance with things like pinning a CPU core to a process and several optimisations at the OS and kernel that would shave off a few milliseconds in an already fast application.
I recently did a round of hiring for my team, and we had several excellent candidates, but could only hire one, so we made a choice.
It’s always sad (for me) because I tend to see the potential in almost everybody, and wish we could hire all the people.
But the enemy has the high ground at the moment.
An implication of this is that the optimial interview process -- from the employer perspective -- will have a lot more false negatives (good candidates rejected) than false positives (bad candidates accepted).
So it could correctly be optimal for the interview process to be so onerous that most job-holders would fail their own interview!
A process where you optimize for avoiding false positives makes sense.
Most companies are not Google though, and if they are <50 employee - it's absolutely the opposite. 1. It's fast to fire people 2. Missing out on a great programmer for an average one makes a huge difference!
So, it follows that they should optimize for avoiding false negatives (rejecting a great candidate), without worrying too much if they got a bad apple snuck in.
This is true also for places like Germany (and Europe in general afaik) where the cost of firing someone past their probation (6 months) is very very costly, especially for small companies.
IMO, at-will employment should be something the employer and the employee should negotiate and not imposed by the state or the employer. Unfortunately, it's usually the employer who has the upper hand.
Sure, but the really bad hires usually aren't bad because they lack technical skills. They're bad for reasons that end up in HR. You can always reassign someone with people skills but short on algorithms.
Firing an asshole who is incompetent is one of the easiest things to do and it can be done well within the probationary period, heck even an asshole who is fairly competent isn't a big deal and can be a complete morale booster and likely will not have resulted in much lost productivity.
I'd say most of the people I fire are not so called "HR liabilities", and in general most people are good people. But most software developers are not competent and in some cases that realization can take a year or longer in domains that require a lot of ramp up time, training, and investment. It's often a lot better to avoid hiring people who have even a 20% chance of not working out.
My advice for these sorts of tests is to make it as short as possible, and use the test to give you something to talk about in the follow up in person interview. Replace "Invert a binary tree/solve this riddle" with "Why did you do this in the take home? What would you do differently if $FOO was now a requirement".
I find it effective.
To answer the OG question, I'm sure I would do well. Depends on the other candidates on whether I would get the job though.
I've gotten data science ones where you could spend anywhere from an hour to an entire career working on that one particular problem. If it's meant to be a simple pass-fail exercise, say so up front and reserve the interview for discussing how the solution could be improved.
I really liked this since it gives plenty of room for discussion, leaves you a lot of freedom to chose your tech stack and make a case for it.
The thing that I find the most useful are the responses I get when I point something out and ask "why did you take this approach" or "what would you do to go faster" or if there's a bug, "I don't think this is doing what you want it to".
I absolutely grade this on a curve -- Entry level and data science backgrounds almost always go to pandas right off, front end people will be focused more on the UI side of it. But if you're senior and come at me with an O(N) or O(N^2) solution then I'm going to really want to know what you have to say about the performance question.
Deleted Comment
... and?
I wasn't suggesting that we had existing devs do the take home, realize it took 30 hours, and then thought that was a reasonable request :D
Something relevant to the work I do all the time? yes, I'd probably pass.
Something about some data structure or algorithm I haven't needed to use in four years that i now have 30 minutes to implement in code? I'd likely fail.
When something comes up in my actual job where i need to solve a problem i'm not familiar with, i first do some research on the problem and learn/remember what i need to know about it and related algorithms/data structures before doing any actual coding.
i certainly do not "immediately write code as fast as possible" when presented w an unfamiliar problem.
if you must ask candidates to work on coding problems, i believe that you should give them 3 or more problems and let them pick which one to work on for the actual "coding test" part of it.
Then maybe have a conversation about the other problems to get a sense of how they would think about/approach them w/o actually making them write code.
At this point though I just stop the interview if they’re asking Big-O or data structure questions. If the company can’t come up with meaningful real world examples that actually explores my experience, I don’t want to work there.
I do the following in JS:
- General graph algorithms (shortest path, backtracking based)
- Traversals for graphs / trees (depth first, breadth first, iteratively, recursively)
- Heaps (min, max)
- Tries
- Matrix manipulation / traversal. This is a very error prone for a lot of folks even though it looks simple when they need to write it by hand, and I always need to putz around with 5-10 problems.
I have enough exposure to common interview questions that the above is generally enough.
If you want to talk privately you can reach me via twitter DM at https://twitter.com/@jitl or jake@makenotion.com
Thus, if you interview those promoted employees, they are not going to match external candidates unless they have been in the role for a while and are now excelling.
Additionally, you evaluate external candidates much more heavily on raw skills b/c they have no internal knowledge/experience, but you include that internal knowledge/experience when evaluating employees.
Finally, interviewing skills != working skills. I've been writing software for 20yrs, and interviewing SDEs for almost as long, and I'd still do interview practice to prepare. In a normal job I can go search the details of what a red-black tree is, and find a library to use it (or spend a few days implementing it). In an interview I need to already know it and be able to implement it myself in less than an hour. (yes, that sucks, but it's the reality of interviewing today)
Other close people I've asked before:
A, at Google, got in 10+ years ago. Told me she would not pass the current interviews.
B, at Google, an aquihire, 5+ years ago. Told me he would never have gotten in the front door (leetcode) way.
C, at Netflix, got in 5+ years ago. Told me odds are he would not pass if he were interviewing today. He says he is in a very interview un-ready state, and none of his realworld experience working at Netflix would help him pass his team's interviews.
The technical interview is broken and has been for a long time. I'm sad that as a society we accept leetcode or unrelated system design questions as a means to measure proficiency.
The standard continues to rise in certain companies and lowers in others.
I'm sure if I was in "interview mode", I could pass, but my position is not about knowing how to invert binary trees.