We need to admit that the interview process isn't about determining the relative qualifications of the applications in the pool, because we all know that 3 hours and whiteboard can't tell anything you need to know about whether or not that person will perform over the years. We need to acknowledge that we perform this pantomime in order to satisfy layers of management that are unable to accept that you can't hire for these positions as you would a traditional sales position.
Further, I think it needs to be said that at smaller companies (read: startups) whats really happening is that applicants are being evaluated for whether or not they're cool enough to join the club. We call it "culture fit" but we're really just trying to vet their personalities.
EDIT: To be clear I do think people's personalities need to be evaluated, especially with smaller teams. I just think we should drop the pretense that "implementing" a depth first tree search on a whiteboard tells us anything other than that they could summon that algorithm at that moment in time with that amount of coffee and whether or not they had a pleasant morning. First of all, I want engineers that are good at working with others, and second of all it takes months to truly evaluate an engineers ability to do their job. We gain nothing by pretending this isn't the case.
The vast majority of applicants cannot code at all. And I mean that literally: they're at a loss at how to write a function that adds two numbers or counts the number of elements in a list.
Worse is that these guys can be employed as developers (even 'senior' ones!) for years and years in 'serious' enterprises.
How, you ask? By using copy-paste and cleverly navigating their enterprise processes and dodging responsibility.
Maybe this is what you mean by 'being good at working with others', but it's definitely not what I want in a software developer.
Source: I've interviewed a great deal of people for lots of positions over the years.
The problem is there are a lot of people who are still good coders who suck at white-boarding for one reason or another. I became one of them due a combination of age, rustiness and an escalation of nervousness after failing a couple whiteboards out of the gate.
Of course once I did land a job it took about a week to shake off the rustiness, and the company that hired me is thrilled.
The point is that companies like Google and Facebook can afford to miss out on those devs. But smaller companies should be looking for diamonds in the rough, not trying to mimic the FAANGs and getting their leftovers.
> And I mean that literally: they're at a loss at how to write a function that adds two numbers or counts the number of elements in a list.
Seriously, Where are you finding these candidates? seriously.
I've worked at a number of mid-sized companies, and interviewed dozens of candidates, and I have never, ever, ever come across a candidate that couldn't write code on this level: "write a function that adds two numbers or counts the number of elements in a list".
+1. I’ve interviewed senior guys, with medium to high salaries, who couldn’t do fizzbuzz. What’s worse, a lot of them were fully confident in awful solutions, and didn’t even want to test them.
Talented people frustrated at the process just don’t get how bad bad coders are. I would never have believed it myself until I experienced it.
"The vast majority of applicants cannot code at all. And I mean that literally: they're at a loss at how to write a function that adds two numbers or counts the number of elements in a list."
I'm genuinely curious how you manage to find all these folks. I've been on the interview team at my company for a several years now(mostly in house, some first pass phone screens) and I've never encountered a single person who was literally unable to code a trivial problem. The last time I met a "programmer" who couldn't code was first semester university, and I thought most of them quickly flunked out/changed majors. I wonder if there is something about your company/recruiting process that is particularly attractive to them, or if our prescreen(which I'm admittedly no expert on) is just particularly good at filtering them out, or if there's some other explanation.
The vast majority of applicants cannot code at all. And I mean that literally.
No, you mean that hyperbolically.
Not only does it simply not happen that "the vast majority of applications cannot code at all" -- this literally has never happened at all, in my experience.
What does happen is that you get a range of people on a spectrum. And yeah, a fair number of them can't code very well. They're slow, they don't see smart solutions, whatever - or are just plain sloppy. But that's quite different from "not being able to code at all."
As to those people who (supposedly) can't "write a function that adds two numbers or counts the number of elements in a list" -- most likely they're simply freezing up from the anxiety of being whiteboarded by a perfect stranger for the first time in a great while - or perhaps ever. (In fact that's exactly what happened to me, on my very first on-site interview after college).
Or that is to say: they haven't internalized -- and produced defenses for -- the (intentionally) awkward and humiliating ritual of the modern tech interview process.
And again, you should only be actually seeing these people once in a blue moon. Unless the people running your incoming "pipeline" are utterly incompetent, and are constantly feeding you a stream of unqualified candidates. In which case your companies much bigger problem a lack of engineers who are able to "ace" HackerRank problems in 59 minutes or less.[1]
[1] Which, lest be honest now -- basically can only happen after extensive time spent on practicing these problems in advance. Or that is, by blatantly gaming your hiring "filter".
And one more thing:
How, you ask? By ... dodging responsibility.
No - their jobs just have different metrics for "responsibility" than yours. That's just the way many businesses are run, whether you like it or not.
I've screened people like you describe, but the only time I've interviewed them face-to-face was when they didn't have a technical phone screen for whatever reason.
FWIW, one of the ways I screen companies when I'm looking is whiteboard problems. I refuse them and move on. In my experience, only HugeCos and places with problems use them. I'm sure that's not true, but I have a necessarily small sample-size, and skipping over firms that do it has worked well for me so far, and there are plenty of fish in the sea. (I do in fact suck at writing on whiteboards, I just don't consider it a skill worth developing to pursue jobs I probably don't want.)
You're probably using a different definition of "whiteboard problem" than is common for what is used a places like Google/Facebook/etc.
I agree with you 100% if "whiteboard problem" means, sit with them while they type up a function in an IDE that does something common (e.g. validates a string, implements some error handling, do a failure backoff, etc).
I disagree if it means, ask them to implement an algorithm on a whiteboard to steer a robot through a maze in a time with optimal algorithmic complexity. This is completely useless and the people that can do this have little overlap with people that can implement easy to read/debug code worthy of production and maintenance.
> The vast majority of applicants cannot code at all.
I've been interviewing devs for years, and this is not my experience at all. The vast majority of applicants that I've interviewed can code, although they tend to be minimally competent at it.
>The vast majority of applicants cannot code at all.
You kinda set up a strawman here. If the purpose of the whiteboard problem was just to establish some very low baseline of coding ability then I doubt many people would argue about their effectiveness. But companies don't use whiteboard problems for that purpose. In my experience (on both sides of the table) they are given with increasing levels of difficulty to see how far the candidate can go. They do not simply ask a few basic questions like "how would you write a function that returned the sum of two numbers" or "count the number of elements in a given array."
I'm not saying there is a really good answer to this. The best I've seen is that some people just seem to be good at hiring and others are not. I am one who is not. I am also a terrible interviewee. The whole process whigs me out.
The number of people I've had fail simple questions (not binary node problems, I mean problems where the optimal solution is basically a for loop with an if statement) is absolutely insane. I would say at least 50% of applicants fail to solve the coding question, and this is interviewing for a 100k+ US job in medium sized city (so low cost of living, and good salary for the area).
We have 'hard' questions in our pool we can ask (where optimization actually comes into play) but I've found that the easy questions weed out so many candidates it's not worth it. There's no room for debate if someone tries to write 15+ if statements rather than creating a loop and one if statement.
May be the company you hire for has a mediocre reputation. So that people with average programming skills don't mind applying. People are generally good at self sorting. At some level they must be thinking they can get this job and end up being surprised that all the rounds are not behavioral rounds.
My frustration is that this is the only industry I know of that makes you perform a pre-hiring test. Can you imagine if a construction worker had 1 hour to assemble a dog house as proof that he can build?
Absolutely untrue in my experience, I can't speak for other people. To imply that this is absolutely untrue in the global space would require that I have interviewed everyone.
Whiteboard problems absolutely do work in my interviews. Again, use of the word absolute indicates that I've never interviewed without a whiteboard. Given the high number of candidates I've interviewed, this might indicate a flaw in the interview process.
The vast majority of applicants I select for interviews cannot code at all. And I mean that literally: they're at a loss at how to write a function that adds two numbers or counts the number of elements in a list. I should consider the possibility that I'm selecting the wrong people for interviews.
Worse is that these guys can be employed as developers (even 'senior' ones!) for years and years in 'serious' enterprises. Clearly other companies are making the same mistakes I am making in their candidate selection process.
How, you ask? By using copy-paste and cleverly navigating their enterprise processes and dodging responsibility.
Maybe this is what you mean by 'being good at working with others', but it's definitely not what I want in a software developer.
Source: I've interviewed a great deal of poorly selected people for lots of positions over the years.
‘Culture fit’ (without clearly specifying what that means) is such an enormous backdoor for all manner of bias and personal prejudice I’m surprised the concept is even legal.
As usual, the root problem is that workplaces are made of humans. Yes, "culture fit" can absolutely be used to excuse nastiness. It is also a real thing that seriously matters, and ignoring interpersonal dynamics is a very quick way to kill a startup.
I'd just note that you can smuggle nasty behavior into any formal mechanism. Witness the legal system (which you'd engage here) - whining about bad-faith arguments in court is basically a national sport in the US, until the whiner is the plaintiff.
I've seen "culture fit" used in an attempt to reject female candidates because the interviewer didn't want to stop telling dick jokes around the office.
Fortunately, the management chain caught this and ... corrected the issue.
So if I make a startup, I should be forced to hire some brilliant programmer whose personality I clash with over over a slightly less brilliant programmer who I can tell will become a fast friend just because the former has demonstrated more technological prowess?
tbh I don't see a huge issue with a small team of friends wanting to stay a small team of friends. imo, it starts to be problematic when a large (for some definition of "large") employer hires on the basis of culture fit.
Completely agree. I've been lucky to have been surrounded by pretty reasonable people in my 8-10 years in the startup world on the east coast but "culture fit" was one of the rare situations that I got in a really impassioned arguments with my superiors. It's code that allows you to avoid hiring a guy who wears Fubu or has a Kentucky drawl.
What’s wrong with evaluating whether you’d get along with someone as a coworker? In my experience, being friendly and getting along has aided cooperation far more than hiring the most qualified candidate at the detriment of group dynamic. Yes, there are potential biases, but at a smaller company I’d say that’s a huge factor.
Culture fit does make a lot of sense though. I interviewed at a company where everyone was a gamer. Would I have fit in? Probably not. Another company everyone looked about 25 or younger.
I don't blame them for not wanting to hire someone pushing 50. They just don't know if they're going to get the cool 50-year-old or the grumpy old troll who won't listen to anyone.
Of course they can never come out and say that for obvious reasons.
>First of all, I want engineers that are good at working with others
Yes !
The worst engineers I have had to work with were sometimes pretty skilled technically, but their ego or shitty personality was preventing them from being somebody the team could benefit from.
I would have thought that it is why we have cultural fit interviews though.
Personally I would sooner drop whiteboards than cultural fit interviews, but the later probably need way more training for the interviewer than what I got.
THIS - my experience with interviewing with startups has been very poor. Generally startups are staffed by younger people.
Sometimes those people conducting the interviews are on their first or second job, and generally those younger coders have a tendency to emphasize things like obscure syntax for whatever programming language and the latest programming paradigms. It is overly language centric rather than dealing with how to solve problems. That is a terrible metric for the issue at hand "will this person be an effective at their job".
Yeah that's fair. Im just frustrated with how low-bandwidth those things are. I definitely get a bit of a sense of how a person thinks by watching them do those CS 201-style problems, which isn't nothing.
I find them useful if you use a real world problem from your product, and the whiteboard is just a tool to help you collaborate as they brainstorm and discuss possible solutions with you. That reduces the pressure and you also get a read on their collaboration skills.
The ideal would be pair programming for an afternoon.
> We call it "culture fit" but we're really just trying to vet their personalities
I've seen this end up in teams that lack diversity several times. My last company was big on interviewing for fit. Then a year goes by and you realize most of the people you've hired are a clone of everyone else. I had a co-worker that once argued that you can't judge people based on personality in an interview and I tend to agree with that now
Eh I see where you're coming from, and most interviews are this way, but I've definitely seen and organized interviews that eschew leetcode problems for very simplified every day problems, and at least imo they can tell you a decent amount. Have 3 or 4 of those in varying skills and you can get a pretty decent idea.
coding portions of an interview should probably be considered as like a bloom filter. it doesn't tell you for sure whom to hire, but it can certainly expose whom not to hire.
It's not easy to see in the moment, and sometimes an individual's options are limited (though this seems rare for senior devs), but why would anyone want to continue to create value for someone who holds biases against them that are hidden as not being a 'cultural fit'?
All interviewing techniques are broadly disliked. A quick summary of complaints found on HN every time we discuss this (about once a week):
- Whiteboarding: algorithmic knowledge not relevant to daily tasks, we have google, obtuse coding environment
- Take home exams: companies have less incentive to respect time of candidate, favors candidates with lots of time to spend interviewing
- Small consulting gigs: not practical for programmers with existing jobs, draws out job search
- Informal conversations, reading resumes: not stringent, susceptible to talented bullshitters
To be 100% clear, none of the above is my opinion, I am simply restating what is regularly posted at this online watering whole.
A discussion of the failings of white-boarding without the context of alternatives is meaningless because interviewing techniques are search functions that all have precision/recall tradeoffs. That there are negatives to white-boarding is a given.
For example, consider that white boarding interviews are short (3 hours). This naturally limits the company's ability to evaluate candidates (less precision), BUT it saves time for the candidate and company (more recall).
So what happens here at HN every week is you get five people all bad mouthing a different interviewing technique, but we never get any closer to a consensus on a technique that would please even a simple majority of programmers (let alone everyone).
TLDR; you don't like white boarding, so what about making a compelling case for something else?
I explicitly tell candidates that they should use pseudocode. Nitpicking stupid syntax that the compiler can catch in a whiteboard interview is pretty pointless.
I presume you haven't submitted candidates to a fizzbuzz test. Try it. The test is well known and out there in the wild. It's almost part of programmer culture. Yet it is absolutely shocking how many candidates that apply to jobs that involve programming will fail that simple test or some minor variation of it. Almost as shockingly, they will also fail to spot their bug after producing a solution that doesn't work as expected.
From what I gather, what is most shocking of all is that candidate still fail this test, despite it being so common, despite there being an several github repos devoted to it, as well as other pages and articles about it (somewhere out there was a page or a repo or something I encountered that had it written in almost every programming language out there, include whitespace and brainf*ck).
Even when given as a "take home" challenge, they can't even copy-pasta an example off the internet...
I interviewed ~500 people in a fairly large (~2000 devs) company.
Interviews used to take about 6-8 hours of my week, and I was usually completely destroyed by the context switch. I remember many days when a deadline was very close, and I absolutely did not want to do the interview, but had to.
In general some days I was in good mood, some in bad mood and even though I tried to make the interview as objective as possible, I am absolutely sure I failed. Keep in mind, one interview has 5 interviewers and guess what they had bad days too.
From those 500 interviews, I was certain for about ~5 cases, for the rest I could not say if it was my fault, or the candidate's fault that the interview went south, in that case I pretty much voted as the majority was voting.
I still feel very bad about people's livelihood being decided in such shitty process, there is absolutely no way that in 1 hour I can find out if someone is good. I knew that, my managers knew that, their managers knew that, and yet we kept going..
I feel that adding more time/people to the process only increases the variance, adding more structure to the process only increases the bias (like we get people who can pass interviews, but not do their job).
I noticed that my job enjoyment took a hit during a period of heavy interviewing for new devs for all the reasons you mention.
A one hour interview required a minimum of 30 minutes prep to read the resume, etc and generate questions. Then at least 30 minutes to write up my findings in a way that would allow me to participate in the roundup, which could be up to a month away if I was the phone screen. The roundup itself accounted for at least another 30 minutes. So you were 2 hours and 30 minutes into it assuming everything goes perfectly. It's a huge time suck.
But beyond the time that I lost, it's a very difficult and crushing experience to have candidate fail and have to be the one that says "no" (and that is happening way way more than yes, especially if you are interviewing early in the pipeline). But it's equally disheartening to see a spark of something in an interview, tease it out, imagine that the person can be raised up to become a good developer only to have the candidate get shot down by all your colleagues because they didn't have a similar experience.
I disliked losing time during the day, but I really disliked the emotional burden of interviewing and saying "no." I empathize with the person on the other side as I've been there.
I really hated the cases were when "no" can not be explained.
In some cases when there is "no" it comes with clear instructions like: hey you need to know v8 more or understand hotspot better and try again in 6 months, but then there were those cases where the committee was incapable of giving feedback.
I am pretty sure I'm a terrible interviewer and interviewee. I've had generally bad experiences all the way around. Interviewing (both sides of the table) has been a big time-suck, yielded mostly unsatisfactory results, and has been an emotionally draining process.
I take a completely different approach to hiring, and I've gotten fantastic results.
Instead of throwing tricky algorithm questions at a candidates, I scour their detailed employment records for the most relevant experience for the first project. In other words, I'm looking for relevant experience rather than top-of-the-head algorithmic brilliance. In the interview, I pose our project problem, and the candidate who gives me the most impressive proposal to get that done gets a chance to solve it.
I hire from an international pool, often on Upwork, so I can start developers on a project basis.
If the developer does a great job, we hire him/her for another project, and so on. At some point, this becomes a full-time relationship, with stock options and other perks.
Using this approach, we value experience over "raw intelligence," per se, and we end up with a team of self-directed developers who are fabulous at delivering great finished products.
It's amazing how well this has worked out for us. I think there's an arbitrage opportunity to avoid coding tests and hire on this basis.
Yeah this somehow still works for pretty much every other profession, and worked fine for programming until a few years ago when everyone decided timed programming challenges were the way to go.
I wish all jobs could just hire devs for short contracts then convert the keepers to full time. I'd be more than happy in that scenario because I know they'll want to convert me and now it's up to me.
The big problem with this is that the short contract does not guarantee continuity of health insurance coverage, and this is a dealbreaker for a lot of folks. The pool of applicants who would accept these offers excludes the top-performing folks who have much better alternatives than contract-to-hire.
> I hire from an international pool, often on Upwork, so I can start developers on a project basis.
I wish I could always hire developers to start on a project basis, but that's just not possible for many (most?) of the best local candidates. Someone who is great who has a full time job at company A is very, very rarely interested in leaving for company B on a project basis.
For our last hire, I was the only one on the hiring committee to ask about past projects. The others defaulted to "fundamentals" questions.
I think the reason people do this is a mix of 1) not knowing what traits to look for 2) not comfortable with unstructured conversation 3) frankly that it takes work to evaluate each resume and research projects enough to have a useful conversation.
For me though, how they achieve success at past work is the best indicator of future success.
Here's the problem with doing this in the US: if you're out of a job, you're out of health insurance. If your whole family is on your health insurance plan, you'll never accept a project-based gig that might not become full time. So with this approach, you're filtering out a new and different subset of the applicant pool before they even interview.
Yeah this is the M.I.T. and Harvard "process". Hire success, instead of fostering success. Works well for small companies. If you need to hire 10.000 to 30.000 people a year, this approach is absolutely useless.
Also just because someone can't come up with the coolest solution for your project, doesn't mean they aren't good developers. This is even more biased than algorithmic interviews, because algorithmic interviews have structure. I actually CAN solve a puzzle in 20 mins. You definitely can not solve a project in 20 minutes or even an hour. It is impossible. And the best candidates will be those who sit back, analyze the problem for days or even weeks and come up with a good solution for your project. This approach is so infeasible for general interviewing that I don't even know where to begin. You are basically filtering out EVERYONE and take the one person who knew enough upfront to accidentially solve your problem best...
He said "impressive proposal", not "coolest solution".
To me, "impressive proposal" means "articulation of a workable plan", the components of which are relatively small, actionable, and take into accounts requirements, and that demonstrates a good understanding of risk. In fact I'd argue that the ability to do this kind of top-level break-down well is one of the best indicators of seniority. The downside is that to do it well requires knowledge of a very specific process, architecture, and technology stack. That is, if you're good at breaking down problems use stateful Erlang and OpenBSD running on bare-metal with web clients, you might have an issue breaking down using stateless C# on Azure with Android native clients. Some combinations are more compatible than others, of course, but any senior dev in one stack is going to have to recapitulate the learning curve of another stack before they can regain this superpower! (We may like to think that only architecture matters, but it is relatively rare that an architecture rendered in one stack is actually isomorphic to the same architecture rendered in another!)
This is seriously f'd up. When I was hired by IBM years ago, they also had a test called the IPAT (Initial Programming Aptitude Test) but, guess what?, there was no programming in it! Instead, the test was designed to see if one had the basic reasoning and logic skills which would be necessary to solve a wide variety of programming-related problems -- not one stupid, specific problem in a particular language. That's just asinine.
Making sure that developers have the "needle in a haystack" solution that you're looking for almost guarantees that you'll weed out the most talented folks -- the ones who are programming generalists that can reason about and solve any kind of programming problem.
I believe these run into legal grey areas because they have to be obviously relevant to the job in question, which likely leads to the tests getting progressively more domain-specific as time goes on.
Programming does seem to be a field in which general intelligence is a big predictor of success. Whereas a field like management probably personality and general intelligence are more equally weighted.
Well yes. And that is what the current crop of "programming puzzle" interviews essentially are; the next iteration of attempting to measure people on a cardinal scale instead of an ordinal scale.
The key to remember is that evaluating on a cardinal scale is much more efficient than evaluating on an ordinal scale, so companies will do everything they can to transform the "secretary problem" into one where they measure absolute metrics instead of relative metrics.
Now, whether those metrics align with business value is a separate issue. All evidence suggests that they are largely independent of business value.
HR told me I couldn't ask such questions without proving they were useful first.
Basically a scientific experiment, have someone else ask and grade the questions, but those results were not considered for hire/nohire. Then 6 months after a bunch of people were hired we pull the results of the test and compare to their performance reviews. If the questions were a good predictor or future success we could use them on future interviews.
The overhead of this was more than we were willing to pay so we went back to HR's allowed questions.
My previous company issued the Wondelic Test (https://en.wikipedia.org/wiki/Wonderlic_test) for everyone. I don't know the details but there was a minimum for different positions. There was also a personality portion and they were looking mostly for "A" type personalities, and a small subset of others. I don't have the "A" type personality, but I'd like to think the IQ portion made up for it LOL.
1) there are not even enough generalists, good generalists are expensive, and bad generalists... you don’t want them.
2) as a generalist, I am quite OK with hiring specialists, if they are really good at what they do. I am a “full-stack”, which means that while I can do both frontend and backend, I can’t shoot React and CSS like a machine gun, nor can I squeeze the last bits of performance from JVM. I hire specialists to do that, after I got the ball rolling.
Am generalist, I hire generalists. My favourite feature is that they/we don't know what we're not supposed to do. So we just go, damn the torpedoes. Smart generalists GSD
Sometimes you really do need particular skills and experience that take decades to learn. Otherwise you don't even know what you don't know and you don't know what mistakes are usually made in a given area.
Most opportunities look for some high degree of specificity. For contract work or a pre-defined task, that makes sense but for companies that need a rich talent pool, I don't know why there appears to be little opportunity for generalists.
That seems much worse than programming tests IMO. If I have a bachelor's degree, and you're asking me to retake the SAT, you're just insulting my qualifications. At least programming tests have some legitimate rationale. Let's not pretend IBM has any fucking idea what they are doing.
It wasn't an IQ test, and IBM definitely knew what it was doing in ways today's "move fast and break things" culture cannot fathom. Their software was and still is used in many mission-critical applications because the company was rigorously focused on quality. When was the last time you heard of a bug in mainframe software affecting anyone?
This was also back when companies were actually willing to train people. Using a test like that, they could try to hire people who had "potential" and then give them a career where they could grow, improve, and bring value to IBM
That doesn't mesh so well with the modern era of employers hating spending a dime on training, even if it would save them a dollar, and employees wanting to jump ship every other year for that sweet sweet 10% compensation bump
I think my team has gotten pretty good at this. We address the "flood of applicants" by tossing out any resumes that don't have some sort of CS or programming on them (about 25%), and favoring, in order, people who have held a programming job, people who have done programming internships, and people who have taken CS classes. A decent GitHub profile will bump you up in the two later categories.
Next is a five minute phone screen. We're a Java shop, so I ask them something dumb, like "what's the difference between public, private, and protected?" Something any Java dev would know; I'm just trying to find out if they have ever actually used the language.[1]
People that pass the phone screen get a Skype interview, where they write code in an IDE. The first half hour or so is chat and trivial problems like "sort this array" or "return true of this String starts with a letter between A-Z, inclusive." They're allowed to Google and use the standard libraries.
Finally, we have a "close to real world" problem for them to work on. It's a standalone, mostly-toy CRUD application, and we'll ask them to add a feature that represents the kind of work they'd be doing. Again, they have their IDE of choice, Stack Overflow, etc.
I don't think we've had a bad hire since we started using this process. Have we turned aside some all-stars that interview poorly? Maybe, but the team we've built is really good at what they do, so I'm pretty happy with the results.
[1] We have hired Senior devs that don't know Java. One of our team leads was a C# guy for a decade or so, but he was smart and available, so we scooped him up. Java is easy to learn. Programming well isn't.
About your last comment - c# was explicitly made similar to java (at least at first, it diverged in time but the basics are close to being a superset of java).
My first job using c# was in 2010, and I have been programming in java since 2002 at that point. I was productive pretty much since the first day & not because I'm a genius - c# is that similar to java...
I suspect that if you'd have hired a php or a python expert they would have taken more time to get used to java (not that that would have meant you shouldn't hire them!)
Your last point is interesting and hits close to home for me, from the other end. My current gig is python, but when I got hired I new exactly zero python. But I had been writing software a long time and in the same area (networking & automation) so they scooped me up. Let's just say it's worked out very well, for everyone involved.
filtering for cs is short sighted imo. You're going to see less and less folks holding a cs degree as more folks wisen up to the trap of student debt in the us.
Instead put folks on trials and keep the good performers.
It does scare me when experienced developers can't write FizzBuzz though, in any language of their choosing, in a reasonable time frame. The thing I tell myself is that there are so many "gotcha" style programming interview questions that people might just be nervous that there's some trap waiting for them in the question.
A few years ago I instituted an "interviewing code of conduct" for my teams with a few tenants that we've refined over the years, but the first one was "We will treat every candidate with respect and empathy in our interview process". The team has adopted some attitudes and techniques to do so while also not compromising on the talent or skills we are looking for. We've gotten very positive feedback on our interviews, so we think it's working.
I've been on both sides of this. As an applicant, sometimes it's a crapshoot in terms of getting lucky that they asked me the right question. On the other hand, as a hiring manager or interviewer, I see lots and lots of candidates who can't write out a fizzbuzz solution, and it wastes a lot of my time (I'm a sucker and will take a lot of time guiding interviewees to the right solution, even on a simple problem when I know the interview is over, but I don't want to bruise their ego too much).
I don't have a silver bullet solution to this, but I think a few things go in the right direction:
1) Candidates should have to write code in interviews. But they shouldn't have to solve puzzle problems with "gotcha" solutions. If there's a specific trick that requires an "aha" moment, you are really testing how well a candidate solves puzzles under pressure, not how they code.
2) Test candidates on what they are best at. If someone has been working in C# for the last 5 years, don't ask them to whiteboard in Python, which they used in college. Picking up new languages/frameworks is quick for someone who knows what they are doing [0].
3) Offer candidates a choice between an in person interview or a take home coding test, the latter of which would take more time. Some candidates don't want to deal with doing a 6 hour take-home coding problem [1]. Other candidates suck at whiteboarding under pressure. So more options seems better.
[0] There are exceptions to this. You might have a unique problem and the budget/resources to hire a rockstar for a specific role. Desirable companies willing to dole out big salaries do this all the time. But much more often, I see companies offering average salaries for very very specific roles. One company near me told me that I was one of the best candidates they've seen, but they are looking specifically for someone with 1+ years of Java experience. I could have picked up the basics of Java in a month, and been fairly proficient in 2-4 months. Meanwhile, they are still looking to fill that role and it's been over 2 years.
[1] I've had a few companies that insist on this, but I haven't had a period of unemployment where I have the time for this. Good developers tend to be/stay employed, so if you are looking to hire senior devs, you probably need to consider their schedules. Unless I'm desperate to leave a job, I can only make so much time for interviews.
> they shouldn't have to solve puzzle problems with "gotcha" solutions.
I’ve never come across this myself, but I always figured that sort of interviewing would correct itself over time - if you ask questions that nobody is going to know the answer to, eventually when three years have gone by and you still haven’t hired anybody, you’re going to have to adjust your tactics.
I used to just look at someone's resume and then start asking them, in a collegial engineer fashion, about technical questions that come to mind about things they've worked on, and go from there, perhaps in speculative directions. But that's not good, now that everyone is concerned about real or perceived IP taint.
There's now a better way, now that a lot of people have open source contributions. Look at their open source contributions, especially if they're involved in public discussions as well as in code. Then you can followup and ask them about those.
Open source behavior is not necessarily quite the same as workplace behavior, especially if their participation was unpaid, and they had limited time, and the team dynamics were different, but there can be a lot of overlap.
(Simple example, using someone famous: if you did not know anything about Linus Torvalds, and didn't trust his resume and references, you could learn from looking at his open source participation that he knows how to code, has managed ambitious projects with cat-herding, is knowledgable and conscientious, historically has a very frank manner that some might find discouraging, and has recently reflected on manner and is modifying it. If that isn't enough, start discussing a technical topic with him that doesn't seem to involve proprietary IP.)
One engineer taking a quick glance at open source participation, and then asking questions about that, is arguably more useful than the engineer spending the same amount of time asking some contrived question and sitting in a stuffy room while the candidate does a theatre performance under conditions that aren't representative of real work.
Also, before considering dumping many hours of take-home makework programming on someone, it's respectful to first take a look at their open source. (Especially with a person who does open source on the side. There's an extra frustration with takehome, which is that they probably have backlogs of unpaid open source things they'd like to spend time on, and the takehome is hours of similar work in free time, but gets thrown away.)
It's a sad fact that interviewing sucks for both sides. Nobody trains us to conduct interviews, so we all just wing it.
I hate wasting time on candidates that can't answer rudimentary programming questions and I hate being in interviews where I'm asked questions which I feel are silly or irrelevant. I can't even trust my results sometimes because there's always that feeling that perhaps a candidate missed a question because I was a poor communicator.
Listening in on interviews with my team mates has really opened my eyes to the random nature of hiring. I'm certain I would not have passed the bar if interviewed by Team Member B instead of Team Member A.
> I can't even trust my results sometimes because there's always that feeling that perhaps a candidate missed a question because I was a poor communicator.
This sentence shows a level of awareness sadly uncommon, that I don't believe I've encountered in... thousands of interviews.
Not only that, but they sometimes even yield False-Positives, like in my experience:
I have been in charge of smallish engineering teams for around 5 years (as head of engineering for different startups). In the past, I used to actually do the 3-HackerRank-timed question as a 1st automated filter.
The problem is that, by testing for "fundamentals" (algorithms and data structure really) I skewed my hiring pool to the ones that were best at those.
Who is the people that are best at solving those kind of puzzle problems? (like the ones in HR, Codility, CodeFight, Codewars, etc), those most likely are Jr developers that are in Uni or recently graduated and spent their Uni free time in coding competitions.
The problem with that is that, these are a very particular type of programmers: They are super-effective in writing tiny one-of code to solve a specific "closed" problem. They usually don't care about testing, code reading quality, interactions, maintainability, etc. Given that they optimize for time and "pass the test cases".
Because of that, suddenly I had like 10 devs that were very good at algorithms but very Jr with regards to Software Engineering, architecture, maintainability, business understanding, etc.
Nowadays I have developed an automated challenge that 1. Requires coding, 2. Requires HTTP requests interaction, 3. Requires thinking and allows me to filter out people that really don't know what they are doing ( https://paystand.ml/challenge/ ).
WRT the 1 hour interview, I have always used a modified version of ( https://sijinjoseph.com/programmer-competency-matrix/ ) to be as objective as possible, and to be able to score Developers in a wide range of skills, and not only "they don't know how to solve the problem of returning a correct sub-tree from a BST within certain ranges". Sure, Algorithms and Data Structures are part of the requirements, but even knowing little of that should not disqualify you.
I straight up refuse coding challenges. I code in the open on GitHub quite often and have built plenty of very high quality (and complex) web applications which I can go into intricate detail about. If your company can’t figure out I’m a good hire without having me jump through your hoops then your process is trash and I have no interest in working for you.
In fact, I’ve been thinking how I would approach interviewing candidates. It seems to me a much better way to interview would be to have candidates do a bit of a show and tell with a project they found challenging. Come in, bring your computer, show me what you’ve built. I’ll do my best to understand it, then I’ll ask questions about it: which aspect was the most technically challenging and why, what was your favourite part to work on, least favourite and why, what would be your scaling strategy and have them go into detail about that, etc. I’d much rather hear how a person answers questions about a project they’re very familiar with than have them do a bunch of arbitrary code problems. What’s wrong with this approach?
There are _tons_ of very good engineers who have spent the majority of their careers in extremely restrictive IP environments and cannot talk about the projects they are working/worked on recently without violating contracts.
Further, there are _tons_ of very good engineers that cannot do 'a bit of show and tell' easily. They basically shut down. Doing the process you outline will optimize for talkers, and the industry is full of people that can talk but can't do.
> There are _tons_ of very good engineers who have spent the majority of their careers in extremely restrictive IP environments
Don't take those jobs if you want to be hireable I guess? You'd have to be joking to think a HackerRank quiz is going to help you glean any information about their expertise. Such expertise, by the way, is something I'd like to know about. Figure out a way to talk about it. Tell me the situation (you can't disclose real details) and then tell me about a hypothetical project with similar parameters in a way that doesn't break your contract. Or make up an imaginary project -- if you're actually a good programmer you can come up with something that sheds light on your technical ability and understanding.
> Further, there are _tons_ of very good engineers that cannot do 'a bit of show and tell' easily.
I'm sorry, soft skills are essential at my imaginary company. You need to be able to explain your thinking at an abstract level. I don't care if you can't do whiteboard problems, I get that - I can't either - but you do have to be able to walk through problems with other people. Especially problems that you know and understand well. If it's the interview environment that's the problem, well I'd like to informalize that process as well too so people don't feel so nervous about the whole thing. But if you can't explain a problem to somebody 1 on 1 then you'll probably be a very annoying person to work with.
> Doing the process you outline will optimize for talkers
It would optimize for "explainers", not "talkers". Talkers are usually people who can't explain something, and they're fairly easy to weed out from the people who actually know their stuff.
That’s great. But for some reason every github profile I get to review is just an empty shell or a bunch of one commit Mooc templates. I’ve almost stopped looking at them.
The best interview I had, we worked through a real world project - a like button widget to allow 3rd party sites to embed. We talked about the DB choice, the front end structure, etc. It was great.
Unfortunately that was the company that was all 25-year-olds and had a terrible commute, and when they heard my previous salary all interest dried up. But all interviews should be like that imo.
What ever happened to using a portfolio and references? If a candidate is 10+ years in the business and they don't have a long list of products built and people who can verify the quality of those products, then what information is a fizz-buzz going to give you?
Most egregious example: I interviewed at a household name company, whose entire web stack was built upon a technology I created, and they still gave me a whiteboard test.
I reject senior software engineers all the time. I give interviews on whatever language you're most comfortable in, I'm not interested in time or space complexity, I just care about how well you can use your tools and debug.
I'm continually astounded at the number of people who can't sort a list (with their language's standard library), clone an array, or do basic string manipulation. I've had a staff engineer at Google ask what the sleep command is in JavaScript. A senior engineer from Twitter didn't know how to use arrays ("We don't really use arrays in Scala").
I don't disagree that many companies do interviews wrong. But there's also a huge number of "senior" engineers who simply shouldn't be senior in the first place. Perhaps it's a result of companies handing out promotions to keep folks around rather than to recognize talent.
Lol. At my job, I code in several different languages. Sometimes, a new project requires me to to learn to write in a new language. It's not a big deal... Programming is pretty general, so new syntax doesn't slow me down too much.
But I can't tell you what the sleep command is any of these languages. I look it up every single time :P
I’m starting to experience that too, where as the breadth of languages learned widens I forget the language specifics and really keep the generic terminology keen. Searching questions like how to sleep in LanguageX, how to iterate, create a thread, whatever need be.
I don't think there's a sleep command in Javascript. It is against the "entire browser runs in a single thread" model, and it would make no sense up to very recently before promises were created (what it would do? start a call-back?).
If the question is from somebody with extensive experience in JS, that's a clear red flag (I don't have any extensive experience in JS). If it's from somebody with passing experience, it's not that bad. That said, I have no idea how to call sleep in any of the ~4 languages I'm currently using either.
I'm in exactly the same boat. I've used 4 different stacks in the last year. It's too much. I can easily get in and debug and write code in basically any environment, but I don't have time to learn all the string methods of any particular language. Google all the way, and official docs (if they're any good). I rewrote an entire huge module in C#/.NET, have zero experience with that environment in the past. I would absolutely flunk a C# interview but I'm pretty confident in my ability to get work done.
This. A million times over. I lead the engineering and run the hiring and I too just turn down senior engineers left and right at times.
We run about as practical a process as I can imagine. A short take home exercise where you use your own environment and build a very small project and are free to even have a starter ready to go ahead of time to focus on the question asked.
In house we have a project you work on on our code base. Very small. Maybe ends up requiring less than 20-30 lines of code for the day. Most people who come in are always familiar with our stack.
The amount of people that just fall on their face is astonishing. Many people will be fine with Greenfield development but then have zero skills being able to deal with an existing code base with places stubbed out with //todos and the ability to pair with people on the team.
I'm convinced people are getting turned down not because of the interview process, but because they just aren't that good at the end of the day.
Yes. I do not need a bunch of code wizards, we aren't solving if p=np daily. But that doesn't mean I should accept people who can't code I'm hopes of them growing into fine developers. I can do that worth one person, but not when bringing on 8.
Memorizing the method for sleep in javascript is the least of a staff engineer's concerns at Google. Besides, with something as easy to google and aided by the IDE, do you REALLY care? What matters is that the guy is able to design and structure a solution, google whatever he needs and build it. At his level, he'd guide a team working on such projects and be concerned with high level design, working with management and managing large $$$ projects.
Do your engineers work out of a cave with no internet and code on cave walls?
There is no sleep command in JavaScript. He fundamentally didn't understand that JavaScript used callbacks and wanted to poll for an XHR's completion.
The job was to write JavaScript, not to do any architecting. There was nothing unclear about the job role. And they had claimed to have built [a very important dashboard that I won't mention to keep them anonymous] mostly themselves, which was quickly disproved.
I've written in so many languages that I wouldn't be able to answer how to sleep in language X off the top of my head. Is it time.Sleep()? delay()? wait_for()? nsleep()? timer().delayUntil()? something else? No clue, but I know how to find out.
You're not hiring me for my encyclopedic knowledge; you're hiring me because I get the job done quickly and effectively, using a proper design and clean, tested code.
Not to mention, there is no sleep command (I assume he meant setTimeout, which has very different functionality than a typical language's sleep command) in javascript, which kind of makes me question the credibility of someone insisting on gatekeeping the senior developer title so hard.
You're missing the forest for the trees here. In these larger companies, senior engineers tend not to work directly with code that much anymore, they're usually working on higher level problems like system design or management. These require different skill sets such as leadership and social skills that take years to develop; the ability to google quickly for a function name isn't really valued as much.
If you're working in a small company or a startup, then you don't need BigCo senior engineers, you can get away with people who haven't developed these skill sets. However if you have intentions of growing, either in your system size or in your headcount, you shouldn't discount these engineers over minor details like this. Ask them higher-level design questions if you actually want to test their abilities, otherwise you're looking at the wrong metrics.
> you shouldn't discount these engineers over minor details like this.
Then they should apply for a job where their skills are relevant. If you're applying for a job where you're writing code (like all of the ones I described), you'd better be able to code.
Edit: and we let you Google whatever you need in the interview. If you can't figure out how to clone an array (an implementation detail of the interview question) on your own, I don't know what role you think you're applying for.
I've met a ton of people like this. They were all people who used to be engineers, but moved into more management centric roles. Their title was still "Senior Engineer" or something similar, but their skill set was almost entirely cat wrangling.
I had one guy get very angry with me when I asked him a programming question; he insisted that that just wasn't what a senior dev did.
I call shenanigans on this comment. Weeding out folks for not knowing how to sleep in a particular language or array syntax in another is not warranted. I think the response that Scala doesn't have arrays is totally legitimate. You should have asked them what was the project/problem they were most proud of and suss out the details of that.
Moreover, keep in mind that for some orgs a Senior dev is actually someone that does more systems design, i.e. an architect. Make sure you ask them those sorts of questions before discounting their abilities. You don't want to be having an architect refactoring your frontend, that's a silly way to spend your money.
In all of the above cases, the candidate had no fundamental knowledge of their chosen language. All of the jobs I was hiring for were for positions that involved writing code. None of these were roles that involved more thought leadership than coding.
I should have been more clear about the JavaScript issue: there is no sleep command. The candidate didn't know how callbacks worked. In JavaScript. Callbacks in JavaScript are how _everything_ works.
I can write fizzbuzz and while I don’t code in JavaScript I could take a guess at sleep from ruby, python and java. I’ve interviewed lately so I’m trained up and could print all paths in a binary tree or all permutations of a set, or detect cycles in a linked list. It’s not all rote, I can adapt and apply these techniques. I have done so at work.
But I couldn’t find all matching sub trees within a binary tree at the whiteboard in an hour (45 min really). I haven’t done a formal survey, but my extensive experience and discussions with a lot of devs tells me that this really isn’t about fizzbuzz or other simple things.
It’s time to stop pretending this is about weeding out people who can’t code. Companies by their own admission set a very high false negative rate.
Then they claim there is a shortage and successfully lobby the government to create a shadow immigration system putting tech companies in charge of who gets to work in the US and the conditions under which they are allowed to remain.
When the next recession hits, this will be even more of a problem.
I think Software Development has a problem in that there is no space for a middle-ground career path. I typically compare myself to a studio musician. I'm never going to be a Rock Star, but you can plug me into any band and we'll make a good recording. That's why I like contract work... nobody cares about my age or history, I can just hit my marks and do what I love. I'm not Senior, but I don't know if I really ever want to be either.
Someone who's been writing code professionally at a software company (especially google or twitter), definitely has at least a moderate if not high level of proficiency in software engineering.
if these highly competent people can't answer your questions, you really need to take a second look at your Questions.
None of the examples were trivia. These were examples of what the candidates got stuck on in a larger interview question, with the ability to use Google to look things up.
I don't care what kind of abstractions you have at your current company. We don't (/didn't) have those abstractions. If you can't show me that you can do CS101 basics (appending to an array?) in the language that you claim to write day to day, how can I trust you to be able to write the code that we describe in the job description?
None of the examples were trivia. These were examples of what the candidates got stuck on in a larger interview question, with the ability to use Google to look things up.
Readit News
Further, I think it needs to be said that at smaller companies (read: startups) whats really happening is that applicants are being evaluated for whether or not they're cool enough to join the club. We call it "culture fit" but we're really just trying to vet their personalities.
EDIT: To be clear I do think people's personalities need to be evaluated, especially with smaller teams. I just think we should drop the pretense that "implementing" a depth first tree search on a whiteboard tells us anything other than that they could summon that algorithm at that moment in time with that amount of coffee and whether or not they had a pleasant morning. First of all, I want engineers that are good at working with others, and second of all it takes months to truly evaluate an engineers ability to do their job. We gain nothing by pretending this isn't the case.
Whiteboard problems absolutely do work.
The vast majority of applicants cannot code at all. And I mean that literally: they're at a loss at how to write a function that adds two numbers or counts the number of elements in a list.
Worse is that these guys can be employed as developers (even 'senior' ones!) for years and years in 'serious' enterprises.
How, you ask? By using copy-paste and cleverly navigating their enterprise processes and dodging responsibility.
Maybe this is what you mean by 'being good at working with others', but it's definitely not what I want in a software developer.
Source: I've interviewed a great deal of people for lots of positions over the years.
Of course once I did land a job it took about a week to shake off the rustiness, and the company that hired me is thrilled.
The point is that companies like Google and Facebook can afford to miss out on those devs. But smaller companies should be looking for diamonds in the rough, not trying to mimic the FAANGs and getting their leftovers.
Seriously, Where are you finding these candidates? seriously.
I've worked at a number of mid-sized companies, and interviewed dozens of candidates, and I have never, ever, ever come across a candidate that couldn't write code on this level: "write a function that adds two numbers or counts the number of elements in a list".
Talented people frustrated at the process just don’t get how bad bad coders are. I would never have believed it myself until I experienced it.
I'm genuinely curious how you manage to find all these folks. I've been on the interview team at my company for a several years now(mostly in house, some first pass phone screens) and I've never encountered a single person who was literally unable to code a trivial problem. The last time I met a "programmer" who couldn't code was first semester university, and I thought most of them quickly flunked out/changed majors. I wonder if there is something about your company/recruiting process that is particularly attractive to them, or if our prescreen(which I'm admittedly no expert on) is just particularly good at filtering them out, or if there's some other explanation.
No, you mean that hyperbolically.
Not only does it simply not happen that "the vast majority of applications cannot code at all" -- this literally has never happened at all, in my experience.
What does happen is that you get a range of people on a spectrum. And yeah, a fair number of them can't code very well. They're slow, they don't see smart solutions, whatever - or are just plain sloppy. But that's quite different from "not being able to code at all."
As to those people who (supposedly) can't "write a function that adds two numbers or counts the number of elements in a list" -- most likely they're simply freezing up from the anxiety of being whiteboarded by a perfect stranger for the first time in a great while - or perhaps ever. (In fact that's exactly what happened to me, on my very first on-site interview after college).
Or that is to say: they haven't internalized -- and produced defenses for -- the (intentionally) awkward and humiliating ritual of the modern tech interview process.
And again, you should only be actually seeing these people once in a blue moon. Unless the people running your incoming "pipeline" are utterly incompetent, and are constantly feeding you a stream of unqualified candidates. In which case your companies much bigger problem a lack of engineers who are able to "ace" HackerRank problems in 59 minutes or less.[1]
[1] Which, lest be honest now -- basically can only happen after extensive time spent on practicing these problems in advance. Or that is, by blatantly gaming your hiring "filter".
And one more thing:
How, you ask? By ... dodging responsibility.
No - their jobs just have different metrics for "responsibility" than yours. That's just the way many businesses are run, whether you like it or not.
I've screened people like you describe, but the only time I've interviewed them face-to-face was when they didn't have a technical phone screen for whatever reason.
FWIW, one of the ways I screen companies when I'm looking is whiteboard problems. I refuse them and move on. In my experience, only HugeCos and places with problems use them. I'm sure that's not true, but I have a necessarily small sample-size, and skipping over firms that do it has worked well for me so far, and there are plenty of fish in the sea. (I do in fact suck at writing on whiteboards, I just don't consider it a skill worth developing to pursue jobs I probably don't want.)
I agree with you 100% if "whiteboard problem" means, sit with them while they type up a function in an IDE that does something common (e.g. validates a string, implements some error handling, do a failure backoff, etc).
I disagree if it means, ask them to implement an algorithm on a whiteboard to steer a robot through a maze in a time with optimal algorithmic complexity. This is completely useless and the people that can do this have little overlap with people that can implement easy to read/debug code worthy of production and maintenance.
I've been interviewing devs for years, and this is not my experience at all. The vast majority of applicants that I've interviewed can code, although they tend to be minimally competent at it.
>The vast majority of applicants cannot code at all.
You kinda set up a strawman here. If the purpose of the whiteboard problem was just to establish some very low baseline of coding ability then I doubt many people would argue about their effectiveness. But companies don't use whiteboard problems for that purpose. In my experience (on both sides of the table) they are given with increasing levels of difficulty to see how far the candidate can go. They do not simply ask a few basic questions like "how would you write a function that returned the sum of two numbers" or "count the number of elements in a given array."
I'm not saying there is a really good answer to this. The best I've seen is that some people just seem to be good at hiring and others are not. I am one who is not. I am also a terrible interviewee. The whole process whigs me out.
Deleted Comment
We have 'hard' questions in our pool we can ask (where optimization actually comes into play) but I've found that the easy questions weed out so many candidates it's not worth it. There's no room for debate if someone tries to write 15+ if statements rather than creating a loop and one if statement.
[0] https://www.codility.com/
Absolutely untrue in my experience, I can't speak for other people. To imply that this is absolutely untrue in the global space would require that I have interviewed everyone.
Whiteboard problems absolutely do work in my interviews. Again, use of the word absolute indicates that I've never interviewed without a whiteboard. Given the high number of candidates I've interviewed, this might indicate a flaw in the interview process.
The vast majority of applicants I select for interviews cannot code at all. And I mean that literally: they're at a loss at how to write a function that adds two numbers or counts the number of elements in a list. I should consider the possibility that I'm selecting the wrong people for interviews.
Worse is that these guys can be employed as developers (even 'senior' ones!) for years and years in 'serious' enterprises. Clearly other companies are making the same mistakes I am making in their candidate selection process.
How, you ask? By using copy-paste and cleverly navigating their enterprise processes and dodging responsibility.
Maybe this is what you mean by 'being good at working with others', but it's definitely not what I want in a software developer.
Source: I've interviewed a great deal of poorly selected people for lots of positions over the years.
I'd just note that you can smuggle nasty behavior into any formal mechanism. Witness the legal system (which you'd engage here) - whining about bad-faith arguments in court is basically a national sport in the US, until the whiner is the plaintiff.
Fortunately, the management chain caught this and ... corrected the issue.
Culture is preference between two otherwise value-neutral positions.
For example: encouraging collaboration vs. encouraging independent work.
Or supporting self-organizing teams vs. all work having a WBS/charge code.
Culture is not choosing between being respectful and not; or choosing between being openminded and not; or choosing between being honest or not.
I don't blame them for not wanting to hire someone pushing 50. They just don't know if they're going to get the cool 50-year-old or the grumpy old troll who won't listen to anyone.
Of course they can never come out and say that for obvious reasons.
Yes ! The worst engineers I have had to work with were sometimes pretty skilled technically, but their ego or shitty personality was preventing them from being somebody the team could benefit from.
I would have thought that it is why we have cultural fit interviews though.
Personally I would sooner drop whiteboards than cultural fit interviews, but the later probably need way more training for the interviewer than what I got.
Sometimes those people conducting the interviews are on their first or second job, and generally those younger coders have a tendency to emphasize things like obscure syntax for whatever programming language and the latest programming paradigms. It is overly language centric rather than dealing with how to solve problems. That is a terrible metric for the issue at hand "will this person be an effective at their job".
The ideal would be pair programming for an afternoon.
I've seen this end up in teams that lack diversity several times. My last company was big on interviewing for fit. Then a year goes by and you realize most of the people you've hired are a clone of everyone else. I had a co-worker that once argued that you can't judge people based on personality in an interview and I tend to agree with that now
- Whiteboarding: algorithmic knowledge not relevant to daily tasks, we have google, obtuse coding environment
- Take home exams: companies have less incentive to respect time of candidate, favors candidates with lots of time to spend interviewing
- Small consulting gigs: not practical for programmers with existing jobs, draws out job search
- Informal conversations, reading resumes: not stringent, susceptible to talented bullshitters
To be 100% clear, none of the above is my opinion, I am simply restating what is regularly posted at this online watering whole.
A discussion of the failings of white-boarding without the context of alternatives is meaningless because interviewing techniques are search functions that all have precision/recall tradeoffs. That there are negatives to white-boarding is a given.
For example, consider that white boarding interviews are short (3 hours). This naturally limits the company's ability to evaluate candidates (less precision), BUT it saves time for the candidate and company (more recall).
So what happens here at HN every week is you get five people all bad mouthing a different interviewing technique, but we never get any closer to a consensus on a technique that would please even a simple majority of programmers (let alone everyone).
TLDR; you don't like white boarding, so what about making a compelling case for something else?
Whiteboards are for writing down algorithms and quick charts and other design-phase stuff, not writing compilable code.
Deleted Comment
Even when given as a "take home" challenge, they can't even copy-pasta an example off the internet...
We need to admit that interviewing is hard, and we aren't good at it. In fact, we don't need to admit it, we know it already.
Interviews used to take about 6-8 hours of my week, and I was usually completely destroyed by the context switch. I remember many days when a deadline was very close, and I absolutely did not want to do the interview, but had to.
In general some days I was in good mood, some in bad mood and even though I tried to make the interview as objective as possible, I am absolutely sure I failed. Keep in mind, one interview has 5 interviewers and guess what they had bad days too.
From those 500 interviews, I was certain for about ~5 cases, for the rest I could not say if it was my fault, or the candidate's fault that the interview went south, in that case I pretty much voted as the majority was voting.
I still feel very bad about people's livelihood being decided in such shitty process, there is absolutely no way that in 1 hour I can find out if someone is good. I knew that, my managers knew that, their managers knew that, and yet we kept going..
I feel that adding more time/people to the process only increases the variance, adding more structure to the process only increases the bias (like we get people who can pass interviews, but not do their job).
A one hour interview required a minimum of 30 minutes prep to read the resume, etc and generate questions. Then at least 30 minutes to write up my findings in a way that would allow me to participate in the roundup, which could be up to a month away if I was the phone screen. The roundup itself accounted for at least another 30 minutes. So you were 2 hours and 30 minutes into it assuming everything goes perfectly. It's a huge time suck.
But beyond the time that I lost, it's a very difficult and crushing experience to have candidate fail and have to be the one that says "no" (and that is happening way way more than yes, especially if you are interviewing early in the pipeline). But it's equally disheartening to see a spark of something in an interview, tease it out, imagine that the person can be raised up to become a good developer only to have the candidate get shot down by all your colleagues because they didn't have a similar experience.
I disliked losing time during the day, but I really disliked the emotional burden of interviewing and saying "no." I empathize with the person on the other side as I've been there.
In some cases when there is "no" it comes with clear instructions like: hey you need to know v8 more or understand hotspot better and try again in 6 months, but then there were those cases where the committee was incapable of giving feedback.
Instead of throwing tricky algorithm questions at a candidates, I scour their detailed employment records for the most relevant experience for the first project. In other words, I'm looking for relevant experience rather than top-of-the-head algorithmic brilliance. In the interview, I pose our project problem, and the candidate who gives me the most impressive proposal to get that done gets a chance to solve it.
I hire from an international pool, often on Upwork, so I can start developers on a project basis.
If the developer does a great job, we hire him/her for another project, and so on. At some point, this becomes a full-time relationship, with stock options and other perks.
Using this approach, we value experience over "raw intelligence," per se, and we end up with a team of self-directed developers who are fabulous at delivering great finished products.
It's amazing how well this has worked out for us. I think there's an arbitrage opportunity to avoid coding tests and hire on this basis.
I wish all jobs could just hire devs for short contracts then convert the keepers to full time. I'd be more than happy in that scenario because I know they'll want to convert me and now it's up to me.
I wish I could always hire developers to start on a project basis, but that's just not possible for many (most?) of the best local candidates. Someone who is great who has a full time job at company A is very, very rarely interested in leaving for company B on a project basis.
I think the reason people do this is a mix of 1) not knowing what traits to look for 2) not comfortable with unstructured conversation 3) frankly that it takes work to evaluate each resume and research projects enough to have a useful conversation.
For me though, how they achieve success at past work is the best indicator of future success.
You'll get the talented, flexible people that can get things done.
Also just because someone can't come up with the coolest solution for your project, doesn't mean they aren't good developers. This is even more biased than algorithmic interviews, because algorithmic interviews have structure. I actually CAN solve a puzzle in 20 mins. You definitely can not solve a project in 20 minutes or even an hour. It is impossible. And the best candidates will be those who sit back, analyze the problem for days or even weeks and come up with a good solution for your project. This approach is so infeasible for general interviewing that I don't even know where to begin. You are basically filtering out EVERYONE and take the one person who knew enough upfront to accidentially solve your problem best...
To me, "impressive proposal" means "articulation of a workable plan", the components of which are relatively small, actionable, and take into accounts requirements, and that demonstrates a good understanding of risk. In fact I'd argue that the ability to do this kind of top-level break-down well is one of the best indicators of seniority. The downside is that to do it well requires knowledge of a very specific process, architecture, and technology stack. That is, if you're good at breaking down problems use stateful Erlang and OpenBSD running on bare-metal with web clients, you might have an issue breaking down using stateless C# on Azure with Android native clients. Some combinations are more compatible than others, of course, but any senior dev in one stack is going to have to recapitulate the learning curve of another stack before they can regain this superpower! (We may like to think that only architecture matters, but it is relatively rare that an architecture rendered in one stack is actually isomorphic to the same architecture rendered in another!)
Making sure that developers have the "needle in a haystack" solution that you're looking for almost guarantees that you'll weed out the most talented folks -- the ones who are programming generalists that can reason about and solve any kind of programming problem.
I believe these run into legal grey areas because they have to be obviously relevant to the job in question, which likely leads to the tests getting progressively more domain-specific as time goes on.
Programming does seem to be a field in which general intelligence is a big predictor of success. Whereas a field like management probably personality and general intelligence are more equally weighted.
Well yes. And that is what the current crop of "programming puzzle" interviews essentially are; the next iteration of attempting to measure people on a cardinal scale instead of an ordinal scale.
The key to remember is that evaluating on a cardinal scale is much more efficient than evaluating on an ordinal scale, so companies will do everything they can to transform the "secretary problem" into one where they measure absolute metrics instead of relative metrics.
Now, whether those metrics align with business value is a separate issue. All evidence suggests that they are largely independent of business value.
Basically a scientific experiment, have someone else ask and grade the questions, but those results were not considered for hire/nohire. Then 6 months after a bunch of people were hired we pull the results of the test and compare to their performance reviews. If the questions were a good predictor or future success we could use them on future interviews.
The overhead of this was more than we were willing to pay so we went back to HR's allowed questions.
Legality...not sure.
Most opportunities look for some high degree of specificity. For contract work or a pre-defined task, that makes sense but for companies that need a rich talent pool, I don't know why there appears to be little opportunity for generalists.
That doesn't mesh so well with the modern era of employers hating spending a dime on training, even if it would save them a dollar, and employees wanting to jump ship every other year for that sweet sweet 10% compensation bump
Thought in the US you will run into problems due to Griggs v. Duke Power Co.
1. These "dev gate" programming challenges are filtering out senior devs, talented devs, creative devs etc. people who would be great at the role.
2. There are people applying for these roles who can't knock out a decent Fizzbuzz solution (in any amount of time).
3. For many roles, there's a flood of applicants
Any solutions to this need must address all three of these things simultaneously which seems astoundingly difficult.
Next is a five minute phone screen. We're a Java shop, so I ask them something dumb, like "what's the difference between public, private, and protected?" Something any Java dev would know; I'm just trying to find out if they have ever actually used the language.[1]
People that pass the phone screen get a Skype interview, where they write code in an IDE. The first half hour or so is chat and trivial problems like "sort this array" or "return true of this String starts with a letter between A-Z, inclusive." They're allowed to Google and use the standard libraries.
Finally, we have a "close to real world" problem for them to work on. It's a standalone, mostly-toy CRUD application, and we'll ask them to add a feature that represents the kind of work they'd be doing. Again, they have their IDE of choice, Stack Overflow, etc.
I don't think we've had a bad hire since we started using this process. Have we turned aside some all-stars that interview poorly? Maybe, but the team we've built is really good at what they do, so I'm pretty happy with the results.
[1] We have hired Senior devs that don't know Java. One of our team leads was a C# guy for a decade or so, but he was smart and available, so we scooped him up. Java is easy to learn. Programming well isn't.
My first job using c# was in 2010, and I have been programming in java since 2002 at that point. I was productive pretty much since the first day & not because I'm a genius - c# is that similar to java...
I suspect that if you'd have hired a php or a python expert they would have taken more time to get used to java (not that that would have meant you shouldn't hire them!)
I still don't know Java that well, though ;-)
Instead put folks on trials and keep the good performers.
A few years ago I instituted an "interviewing code of conduct" for my teams with a few tenants that we've refined over the years, but the first one was "We will treat every candidate with respect and empathy in our interview process". The team has adopted some attitudes and techniques to do so while also not compromising on the talent or skills we are looking for. We've gotten very positive feedback on our interviews, so we think it's working.
I don't have a silver bullet solution to this, but I think a few things go in the right direction:
1) Candidates should have to write code in interviews. But they shouldn't have to solve puzzle problems with "gotcha" solutions. If there's a specific trick that requires an "aha" moment, you are really testing how well a candidate solves puzzles under pressure, not how they code.
2) Test candidates on what they are best at. If someone has been working in C# for the last 5 years, don't ask them to whiteboard in Python, which they used in college. Picking up new languages/frameworks is quick for someone who knows what they are doing [0].
3) Offer candidates a choice between an in person interview or a take home coding test, the latter of which would take more time. Some candidates don't want to deal with doing a 6 hour take-home coding problem [1]. Other candidates suck at whiteboarding under pressure. So more options seems better.
[0] There are exceptions to this. You might have a unique problem and the budget/resources to hire a rockstar for a specific role. Desirable companies willing to dole out big salaries do this all the time. But much more often, I see companies offering average salaries for very very specific roles. One company near me told me that I was one of the best candidates they've seen, but they are looking specifically for someone with 1+ years of Java experience. I could have picked up the basics of Java in a month, and been fairly proficient in 2-4 months. Meanwhile, they are still looking to fill that role and it's been over 2 years.
[1] I've had a few companies that insist on this, but I haven't had a period of unemployment where I have the time for this. Good developers tend to be/stay employed, so if you are looking to hire senior devs, you probably need to consider their schedules. Unless I'm desperate to leave a job, I can only make so much time for interviews.
I’ve never come across this myself, but I always figured that sort of interviewing would correct itself over time - if you ask questions that nobody is going to know the answer to, eventually when three years have gone by and you still haven’t hired anybody, you’re going to have to adjust your tactics.
There's now a better way, now that a lot of people have open source contributions. Look at their open source contributions, especially if they're involved in public discussions as well as in code. Then you can followup and ask them about those.
Open source behavior is not necessarily quite the same as workplace behavior, especially if their participation was unpaid, and they had limited time, and the team dynamics were different, but there can be a lot of overlap.
(Simple example, using someone famous: if you did not know anything about Linus Torvalds, and didn't trust his resume and references, you could learn from looking at his open source participation that he knows how to code, has managed ambitious projects with cat-herding, is knowledgable and conscientious, historically has a very frank manner that some might find discouraging, and has recently reflected on manner and is modifying it. If that isn't enough, start discussing a technical topic with him that doesn't seem to involve proprietary IP.)
One engineer taking a quick glance at open source participation, and then asking questions about that, is arguably more useful than the engineer spending the same amount of time asking some contrived question and sitting in a stuffy room while the candidate does a theatre performance under conditions that aren't representative of real work.
Also, before considering dumping many hours of take-home makework programming on someone, it's respectful to first take a look at their open source. (Especially with a person who does open source on the side. There's an extra frustration with takehome, which is that they probably have backlogs of unpaid open source things they'd like to spend time on, and the takehome is hours of similar work in free time, but gets thrown away.)
I hate wasting time on candidates that can't answer rudimentary programming questions and I hate being in interviews where I'm asked questions which I feel are silly or irrelevant. I can't even trust my results sometimes because there's always that feeling that perhaps a candidate missed a question because I was a poor communicator.
Listening in on interviews with my team mates has really opened my eyes to the random nature of hiring. I'm certain I would not have passed the bar if interviewed by Team Member B instead of Team Member A.
This sentence shows a level of awareness sadly uncommon, that I don't believe I've encountered in... thousands of interviews.
I have been in charge of smallish engineering teams for around 5 years (as head of engineering for different startups). In the past, I used to actually do the 3-HackerRank-timed question as a 1st automated filter.
The problem is that, by testing for "fundamentals" (algorithms and data structure really) I skewed my hiring pool to the ones that were best at those.
Who is the people that are best at solving those kind of puzzle problems? (like the ones in HR, Codility, CodeFight, Codewars, etc), those most likely are Jr developers that are in Uni or recently graduated and spent their Uni free time in coding competitions.
The problem with that is that, these are a very particular type of programmers: They are super-effective in writing tiny one-of code to solve a specific "closed" problem. They usually don't care about testing, code reading quality, interactions, maintainability, etc. Given that they optimize for time and "pass the test cases".
Because of that, suddenly I had like 10 devs that were very good at algorithms but very Jr with regards to Software Engineering, architecture, maintainability, business understanding, etc.
Nowadays I have developed an automated challenge that 1. Requires coding, 2. Requires HTTP requests interaction, 3. Requires thinking and allows me to filter out people that really don't know what they are doing ( https://paystand.ml/challenge/ ).
WRT the 1 hour interview, I have always used a modified version of ( https://sijinjoseph.com/programmer-competency-matrix/ ) to be as objective as possible, and to be able to score Developers in a wide range of skills, and not only "they don't know how to solve the problem of returning a correct sub-tree from a BST within certain ranges". Sure, Algorithms and Data Structures are part of the requirements, but even knowing little of that should not disqualify you.
In fact, I’ve been thinking how I would approach interviewing candidates. It seems to me a much better way to interview would be to have candidates do a bit of a show and tell with a project they found challenging. Come in, bring your computer, show me what you’ve built. I’ll do my best to understand it, then I’ll ask questions about it: which aspect was the most technically challenging and why, what was your favourite part to work on, least favourite and why, what would be your scaling strategy and have them go into detail about that, etc. I’d much rather hear how a person answers questions about a project they’re very familiar with than have them do a bunch of arbitrary code problems. What’s wrong with this approach?
Further, there are _tons_ of very good engineers that cannot do 'a bit of show and tell' easily. They basically shut down. Doing the process you outline will optimize for talkers, and the industry is full of people that can talk but can't do.
Don't take those jobs if you want to be hireable I guess? You'd have to be joking to think a HackerRank quiz is going to help you glean any information about their expertise. Such expertise, by the way, is something I'd like to know about. Figure out a way to talk about it. Tell me the situation (you can't disclose real details) and then tell me about a hypothetical project with similar parameters in a way that doesn't break your contract. Or make up an imaginary project -- if you're actually a good programmer you can come up with something that sheds light on your technical ability and understanding.
> Further, there are _tons_ of very good engineers that cannot do 'a bit of show and tell' easily.
I'm sorry, soft skills are essential at my imaginary company. You need to be able to explain your thinking at an abstract level. I don't care if you can't do whiteboard problems, I get that - I can't either - but you do have to be able to walk through problems with other people. Especially problems that you know and understand well. If it's the interview environment that's the problem, well I'd like to informalize that process as well too so people don't feel so nervous about the whole thing. But if you can't explain a problem to somebody 1 on 1 then you'll probably be a very annoying person to work with.
> Doing the process you outline will optimize for talkers
It would optimize for "explainers", not "talkers". Talkers are usually people who can't explain something, and they're fairly easy to weed out from the people who actually know their stuff.
Dead Comment
Unfortunately that was the company that was all 25-year-olds and had a terrible commute, and when they heard my previous salary all interest dried up. But all interviews should be like that imo.
Most egregious example: I interviewed at a household name company, whose entire web stack was built upon a technology I created, and they still gave me a whiteboard test.
I'm continually astounded at the number of people who can't sort a list (with their language's standard library), clone an array, or do basic string manipulation. I've had a staff engineer at Google ask what the sleep command is in JavaScript. A senior engineer from Twitter didn't know how to use arrays ("We don't really use arrays in Scala").
I don't disagree that many companies do interviews wrong. But there's also a huge number of "senior" engineers who simply shouldn't be senior in the first place. Perhaps it's a result of companies handing out promotions to keep folks around rather than to recognize talent.
But I can't tell you what the sleep command is any of these languages. I look it up every single time :P
I'm good with the sleep command in my many different languages. It's dealing with dates in any language...
I suspect 10% of of StackOverFlow hits are developers remembering how to parse a date in some form or another.
https://en.wikipedia.org/wiki/Interference_theory#Retroactiv...
If the question is from somebody with extensive experience in JS, that's a clear red flag (I don't have any extensive experience in JS). If it's from somebody with passing experience, it's not that bad. That said, I have no idea how to call sleep in any of the ~4 languages I'm currently using either.
Deleted Comment
We run about as practical a process as I can imagine. A short take home exercise where you use your own environment and build a very small project and are free to even have a starter ready to go ahead of time to focus on the question asked.
In house we have a project you work on on our code base. Very small. Maybe ends up requiring less than 20-30 lines of code for the day. Most people who come in are always familiar with our stack.
The amount of people that just fall on their face is astonishing. Many people will be fine with Greenfield development but then have zero skills being able to deal with an existing code base with places stubbed out with //todos and the ability to pair with people on the team.
I'm convinced people are getting turned down not because of the interview process, but because they just aren't that good at the end of the day.
Yes. I do not need a bunch of code wizards, we aren't solving if p=np daily. But that doesn't mean I should accept people who can't code I'm hopes of them growing into fine developers. I can do that worth one person, but not when bringing on 8.
Do your engineers work out of a cave with no internet and code on cave walls?
The job was to write JavaScript, not to do any architecting. There was nothing unclear about the job role. And they had claimed to have built [a very important dashboard that I won't mention to keep them anonymous] mostly themselves, which was quickly disproved.
You're not hiring me for my encyclopedic knowledge; you're hiring me because I get the job done quickly and effectively, using a proper design and clean, tested code.
If you're working in a small company or a startup, then you don't need BigCo senior engineers, you can get away with people who haven't developed these skill sets. However if you have intentions of growing, either in your system size or in your headcount, you shouldn't discount these engineers over minor details like this. Ask them higher-level design questions if you actually want to test their abilities, otherwise you're looking at the wrong metrics.
Then they should apply for a job where their skills are relevant. If you're applying for a job where you're writing code (like all of the ones I described), you'd better be able to code.
Edit: and we let you Google whatever you need in the interview. If you can't figure out how to clone an array (an implementation detail of the interview question) on your own, I don't know what role you think you're applying for.
I had one guy get very angry with me when I asked him a programming question; he insisted that that just wasn't what a senior dev did.
Moreover, keep in mind that for some orgs a Senior dev is actually someone that does more systems design, i.e. an architect. Make sure you ask them those sorts of questions before discounting their abilities. You don't want to be having an architect refactoring your frontend, that's a silly way to spend your money.
I should have been more clear about the JavaScript issue: there is no sleep command. The candidate didn't know how callbacks worked. In JavaScript. Callbacks in JavaScript are how _everything_ works.
But I couldn’t find all matching sub trees within a binary tree at the whiteboard in an hour (45 min really). I haven’t done a formal survey, but my extensive experience and discussions with a lot of devs tells me that this really isn’t about fizzbuzz or other simple things.
It’s time to stop pretending this is about weeding out people who can’t code. Companies by their own admission set a very high false negative rate.
Then they claim there is a shortage and successfully lobby the government to create a shadow immigration system putting tech companies in charge of who gets to work in the US and the conditions under which they are allowed to remain.
I think Software Development has a problem in that there is no space for a middle-ground career path. I typically compare myself to a studio musician. I'm never going to be a Rock Star, but you can plug me into any band and we'll make a good recording. That's why I like contract work... nobody cares about my age or history, I can just hit my marks and do what I love. I'm not Senior, but I don't know if I really ever want to be either.
if these highly competent people can't answer your questions, you really need to take a second look at your Questions.
Have you worked in the industry for longer than 2 years?
> ask what the sleep command is in JavaScript.
> We don't really use arrays in Scala
These all seem like genuine things. In more complex companies, people may use more exotic data structures and/or have abstractions around data access.
Silly comment overall.
I don't care what kind of abstractions you have at your current company. We don't (/didn't) have those abstractions. If you can't show me that you can do CS101 basics (appending to an array?) in the language that you claim to write day to day, how can I trust you to be able to write the code that we describe in the job description?