Dogfooding their process is going to give them adverse results here: Microsoft is paying them to take all the time they want, but good candidates won't be so willing to spend so much time. Already, they're up to a full day of interviews, which is ridiculous. One of the worst parts of current FAANG interviewing is how long it takes to get to the day of interviews, needing to take off from work to attend the interview day, and then getting rejected nonetheless. Not only does this process ask for too much commitment on the part of candidates, it selects for candidates for whom such a commitment is less relatively costly - i.e. people who are currently unemployed, students, etc, and while there are many such people with great talent and potential, respectively, they're also less likely to be strong candidates than people who are currently employed.
One of the key issues in recruiting is how to make hiring decisions quickly. Good candidates don't have the patience for your internal politics to resolve over a month or two to figure out whether they're getting an offer or not; they're just going to interview with other places in the meantime who will beat you to the punch.
Take a resume off the pile. Email or text to schedule a call. During the call, talk about relevant experience / past projects, what each side is looking for, etc. Make a decision during the call whether you want to invite the person for a two-hour interview, and schedule it during the call, first offering an evening time slot for the interview. In the first hour, talk about culture, problem-solving and teamwork approaches, engineering attitudes. In the second hour, pull out an actual current problem and solve it collaboratively. If all goes well, send an offer, and offer to set up a dinner to talk about / negotiate the offer.
It should not take more than a few hours to make a decision on someone; it should not require a candidate to take paid time off. Your hiring pipeline's average lead time should be measured in days.
One company wanted approached to interview me for the position of a "scientist". Their initial step of the interview process was to spend 4-6h on some "challenge" they designed. I told them, I can't spend 4h of my time on the 1st step where I have no clue what my hiring manager even does! I made a counter to their proposal as all my research projects are publicly available. My counter was similar to your proposition. I asked them to pick any of my research projects and talk about it, I told them I just need a 5 minute heads up. Of course, they didn't take me up on it because they required "commitment from the candidate". I chuckled on the irony in that last phrase. Interview is a 2-way process and it seems a lot of people have bought into some of the kool aid.
I can share with you one hiring insight I have that goes against your sentiment. I've been hiring for hedge fund research roles for the last three years interviewing ~30 candidates over that period.
What I've learned is that it is a very weak signal to talk to candidate about "their research" unless it's immediately and directly applicable to what we were doing as a company, so that we can talk about it as equal experts, which never really happened in our case. Previous research done by candidate was, like research almost always is in a narrow and specialize niche. Every candidate I was talking to was very good with talking about their research and from my perspective at the time of the interview it was impossible to validate and evaluate their claims. That conversation could convey the character of the candidate but I couldn't evaluate their ability to do new research.
Instead, presenting candidates with the same problem, that in our case took around 3 hours to solve, in our case proved a good way to compare candidates a bit more objectively and test how they approach solving problems they haven't seen before which is more applicable to what they'll be doing on the job.
The downside of that process is that during three hours you can only test so much, but hiring is a really hard thing to do right and we didn't want to take more of candidates time as we did value it highly and found that to be the sweet spot for us.
>>I told them, I can't spend 4h of my time on the 1st step where I have no clue what my hiring manager even does!
You had a lucky escape. I had the same from a company. Which was arguably a 2 full nights worth work if done right.
I took the challenge. Did it, it passed their functional test suite which they provided. Unit test cases, git repo, documentation etc etc all in place.
They came back saying it was not enough. So it's not like if you do the take home project, they let you in.
Also its a tad little irritating to make people work for so long, make them do everything. And even then with perfectly working solution turn them down.
Once I wanted to interview for a company that wanted me to implement a FULL chat client, including the full range of auxiliary features, for step one...
>Interview is a 2-way process and it seems a lot of people have bought into some of the kool aid.
What too many people choose to forget or ignore is employment itself (at least in CA) is a 2-way process. The employer is not doing you a favor by employing you, despite how many people have drank the kool-aid to the contrary. You and the company have entered into a mutually beneficial agreement. Unfortunately, many people don't have the means or desire to walk away when an employer veers into "you should be thankful we let you work here" territory.
Yeah, if only it was all unicorns and rainbows. What you describe is how I imagine they interview for director / VP positions. With hundreds (thousands?) of applications per day for software engineering positions, all with perfect resumes, what you describe just doesn’t scale. Software engineers do 1-2 interviews per week each, it’s part of the job. I might be a minority, but I would refuse to spend two of my evenings per week interviewing people. I’d rather spend time with my family instead. I think asking candidates to get a day off for interviews, which happens like once per year or even less often for each candidate is more reasonable than asking your employees to work evenings each week.
> Software engineers do 1-2 interviews per week each, it’s part of the job. I might be a minority, but I would refuse to spend two of my evenings per week interviewing people.
Why are software engineers doing interviews at all? What makes software engineers qualified to interview people (especially when you consider the need to avoid hiring bias, problematic statements by interviewers that expose the company to legal suit, etc.)?
I'm not saying that companies should go to the other extreme and have all interviews conducted by HR - that presents its own set of problems in that HR doesn't know how to evaluate candidates for technical skill. But expecting hiring managers to open up a few evenings per week in the irregular and relatively uncommon circumstance (if it's common, then you have problems with churn on that team, and you should consider firing the manager) in which positions open up on their team to evaluate potential direct reports is not even remotely unreasonable.
if i am looking for a different job, i'll not just interview with one company, but several. even more so if i still have a job and i really need to weigh the options to decide if the new offer is really better.
there is only so many days i can take off. with two weeks holidays, i might spend all of my holiday budget for this year.
I completely agree with this. It's a process that's practical, fair and scalable.
I'm currently interviewing and so far I have had a 48 hour weekend project, a 3 hour code assessment, and I'm expected to endure 3 days of interviews. I'm fairly close to throwing in the towel and saying I can't be arsed with this.
I very much disagree with you: the more experienced the candidate is, the more the interview is about the company and the work environment, not just the candidate. I.e. it is a two way street, and the candidate has to decide in the end whether he/she wants to work for that company or not. There is simply no way a 1 or 2 hour interview can provide this information - this is why you need more time, and this is why I do think the process described in the post is a great way to interview.
This may not apply to all companies or situations, but for a Microsoft-sized organization this is the way it should be done.
If you're Microsoft-sized, then if anything, your interview process should be less strict, not more strict. At organizational scale, enforcement processes become more important than culture. That means building out custom standardized entrance exams, regular "induction days" with mandatory internal courses for new hires (wherein it is your job in the first few weeks or even months to just go to those courses during normal working hours), deployment pipelines which guide new junior hires by enforcing minimum test coverage, style conventions, passing static code analysis, etc.
Your hiring needs should reflect the size of your organization, and your hiring process should reflect your hiring needs. If you hire relatively infrequently then it makes sense to go over individual resumes and have infrequent evening interviews. If you need to hire hundreds of people this month just to negate churn then you have the resources to build out the kind of enforcement-process-oriented above since you surely need that kind of infrastructure anyway. Everything in between is just a matter of degree and where your company is on the process-vs-culture balance as a function of its size.
I sorta agree on the latency although there are a lot of unavoidable reasons that happens including but not limited to travel schedules of the decision makers and the need to interview multiple candidates.
However TBH an in-person day with 4 or so interviews (perhaps post a phone screen) is absolutely standard for all sorts of professional positions and has been for many decades. I can only think of one case where I just had an informal in-person lunch instead--and that was a case where I knew the company's owner very well.
I sometimes wonder if a shortcut would be to hire & Fire quickly but only for short-term contract work. After X time as a contractor devs can upgrade to a full position.
> But the aha moment for me was that not everyone does well in those fast-paced brainstorming sessions. A lot of people (including me) prefer to sit with a cup of coffee and some data and try to think things through.
This is me. Hard to get that across in an interview, but once people work with me, they're cool with me coming back an hour later in an email with some thoughts on the last meeting topic. They know and respect that that's the way I work[0].
[0] I'm not the only one on the team who does this, so that helps.
I'm like this and I know why it happens for me. I cannot think my thoughts while listening to others. Either I'm listening, or I'm thinking, I cannot do both at once. I can follow others and sometime I can make a contribution, but it would be a small contribution, a deal less than I could do given a time to think.
I tried to find some way around it, and the only way I see is to learn how to think loudly. The only way is to supress others from thinking by speaking aloud my thoughts as they come. Let others follow my thoughts, not the other way around. I figured it out while making some group work with ~20 yo students being a student myself. I'm older (I was 35-40 over that period) and therefore smarter and in the most cases I was able to do this group work myself a way better then they do.
I tried different approaches. I tried to take a leader role in a discussion and to talk everytime I feel like this. I tried to let someone else to become a leader and to follow her. I tried to just get out of the discussion, to sit quietly with my own thoughts and to come with my own solution to compare it with the group one. The conclusion is: I cannot think AND listen at the same time.
Given that many people report that their best thinking is done while walking or taking a jog, it might not be too wild to try out interviews which allow the candidate to take a walk before they give an answer.
Hijacking this thread to say that blog post isn't about Microsoft as a whole, but only about Microsoft's Developer Division (Visual Studio, etc.). The rest of Microsoft still uses the standard whiteboard/Leetcodey approach.
My goal when interviewing candidates is to ellicite this response at least once. I want someone who is honest that they don't know the answer and won't bullshit me about it. Also like to hear how the candidate would go about figuring it out, e.g. what resources they would use. I'm also fine with speculative answers as long as they're clearly stated as such, "I dont know, but I think it'd be something along these lines..."
That same nugget was in The Pragmatic Programmer which I recently have been reading. In regards to estimates or questions about things you don't know, if you can't give a good answer on the spot the second best answer is "Let me get back to you.". The bad answer or estimate will just cause pain for your team/org.
I’ve never done any leetCode style interviewing or studying, but from what I have heard, it’s all about “pattern matching” and knowing how to solve similar problems.
I have done plenty of architecting. A large percentage of architectural problems that you are asked to solve have already been solved. After enough years at enough different companies and reading enough architectural books, it’s just pattern matching and figuring out the correct tools.
This is actually something I've encountered outside of interviews in day-to-day work with teams.
When the topic is something where I know some underlying complexity exists, or the stakes are high, I often want to spend a bit of time thinking it through by myself because that's how I think best about those sorts of problems.
I've encountered friction at times with those that aren't familiar with and/or care about the underlying complexities and just want to talk high-level and make decisions and move on. Sometimes it is important to make quick decisions to avoid analysis paralysis, but often it is worth giving those who do best thinking through things on their own a chunk of time to go off and do what they do best. I wish more people took that into consideration when conducting brainstorms.
I'm not the best with Python, but if someone gave me that answer I'd feel like I avoided a land mine. Dicts, and analogous data types, are incredibly useful.
The most charitable explanation I can think of is that they have a very narrow idea of "practical application" where the performance of looking stuff up in hash maps all the time is not good enough. If that's the case, they should have been more specific, and asked for specific improvements. Otherwise, they're full of shit.
This is fine in most situations, but sometimes companies want people who actually have experienced in a given problem domain, and generally speaking, if you have to "sit with a cup of coffee and some data and try to think things through" that implies a lack of experience, which may not be desirable to the interviewer.
I'm one of a couple domain experts on my team. I am well respected as such. The reason I "sit ... coffee.. think things through" is because I am doing a mental impact analysis. How are the suggestions going to affect the system as a whole. One simply cannot know these things off the top of their head in a meeting for a system of any meaningful size.
It is precisely because of my experience that my team is OK with me taking an hour to ponder.
Edit: Let me clarify one thing. I'm not talking about situations where, say, the PO comes and asks, "Hey, JustSomeNobody, why does the system do this under these conditions?" I know the answer to this. If they want logs for that specific instance, I can get them. What I'm talking about is, when the PO calls a meeting and says, "I just got off the phone with <Entity>, they want these changes to the system. What will it take?" I don't answer these questions, definitively, without thinking about them. That does NOBODY any good. I'll sometimes give a few initial thoughts for the sake of the meeting, but that's it.
i’m like this but admittedly, in some cases, you’re interviewing for roles where it’s critical you identify that candidate can think on their feet (eg devs that might need to serve in 24/7 ops roles), where “let me get back to you” isn’t a feasible option
Usually those who optimize for thinking on their feet do so at great compromise in quality. Maybe you can afford that, but it's kind of "shooting from the hip" so to speak.
It's probably not that these candidates can't think on their feet, it's just that they generally aren't willing to compromise so greatly on accuracy and quality of the answer.
That's reactive thinking and it honestly sucks.
While you might encounter unexpected situations from time to time, a good ops person will be preventing problems a lot more than he firefights. If you only hire a good firefighter, he will do the only thing he knows ad infinitum.
I've worked with people that were reactive and would love to be "the fixer" when, in reality, 99% of the issues he fixed were caused by his ignorance/incompetence in the first place.
Specially for crucial 24/7 roles, you need people that think ahead and not reactive ones.
in a situation like that i'd like to see both happening. start the emergency procedure, and while it's running think through if you missed anything or if there is an alternate approach.
just had that happen. we had an overloaded server, and while we tried to figure out how to make the server faster, after some thinking we realized that the better long term solution was to change some of the architecture so that the server would get less load to begin with.
Great, now can Netflix, Google and Facebook do this, too? Not because I want to work in these places, but because they influence everyone else and as a senior engineer in the systems space I feel I shouldn't need to study days or weeks for fizzbuzz sorting algorithms questions that are designed to test comp sci recent grads. I have a proven career, and was never suddenly stumped in a project due to not being able to.. sort letters in some obscure order based on arbitrary rules. How about we look at that, and compare the candidate's past experience to the current needs and interview based on that -- You DID look at the CV before bringing the candidate in, didn't you?
I've interviewed and been interviewing for over a decade. I have had the displeasure of interviewing many people who have very impressive resumes yet when asked how they would judge their skillset in an area, and being told from them that they would say "expert", being shown exactly the opposite when it came time to answer some questions in that area.
One data point... Senior engineer that founded the local java user group and wrote a published book on the java language. Couldn't code a simple reverse string algorithm, nor explain whether, after with help devising an answer, the method was thread-safe.
Another data point... A self-described expert in SQL with it all over their resume. Couldn't write a simple join, inner outer or other.
Weed-out questions exist because our industry is flooded with people who don't know, nor care to know, their craft. And there are plenty of employers who continue to hire these people, and promote them. I interviewed numerous "architects" (with development experience all over their resume) who couldn't come up with a naive solution, let alone the optimal, to a very simple problem. Want to fix the interview process in our industry? Figure out a way to weed out the liars? I'm all for it. I would rather not have to do basic algorithm questions, but that's the current environment.
Totally reasonable response. However, it seems like we're on an endless loop with this discussion. People talk about how they need technical interviews because someone who looks good on paper can't code reverse string, or write a basic join. But here's the thing, absolutely none of my technical interviews have involved anything remotely this simple. They are "find all matching matching subtrees in a binary search tree", or "find all possible matrices with a negative determinant within an NxM matrix of arbitrary size", and it's supposed to be done in about 45 minutes, at the whiteboard.
Really, it's not fizz buzz or string reversal that causes people to re-study for their algorithms and data structures exam before an interview.
In many ways, I think this is similar to requiring that senior actuaries re-study integration by parts prior to every interview. They don't have to, because they have a proper exam that is widely accepted in their field. We don't. So instead, we are taken through full day whiteboard exams, under conditions of great secrecy, over and over, every time we interview.
I have very little experience with interviewing candidates, but how can you be certain these people were liars? Maybe they simply were having a bad day, or got too much in their head with stress during the interview. There are many factors at play when it comes to a bad performance and it's not always simply that the engineer doesn't know the answer.
As a hiring manager, I agree 100%. These people are everywhere. I've also given someone the benefit of the doubt -- he wrote a Java book! once we hire him, I'm sure it will work out -- only to find, nope, he can't code on the job either.
Other hand, I'd suggest instead of "lying" that people are just myopic about their skill sets. I'd been writing JavaScript code for years, for example, but on an interview learned about a whole world of "modern" JavaScript that I didn't know existed. Clearly they thought I was a fraud. But I'd just been living on the 3rd floor without ever visiting the basement where all the pipes and boilers were.
So, I agree you want to see evidence of what's on the resume. But why isn't an actual work project of some sort better than whiteboard coding exercises?
Why do you assume that the person is lying? Why do you start your relationship with your new co-worker on distrust?
There is a way to weed out the liars - you hire them and then when it does not work out you fire them. You would be astonished how little people lie on their CV.
They may exaggerate claims - yes, that is why you gave them a call in the first place. But then again you exaggerated your claims in the job posting of which skills are required for the job.
The one thing that I noticed is that only the US companies have this awkward parallel reality 'technical-interview' hiring process. In other countries like Germany or France the interview process is mostly focused on the person and the desire to work for the company and the willingness to learn the skills needed for the job. Just because you fail to answer immediately to a random question it does not mean you are unqualified for the job.
When I conduct an interview I try to only ask open questions like:
* What is your favorite IDE? - Whatever the person answers, tells a lot about how he works, what tools he uses, and if he is aware of the common tools
* What feature of the new Version of <Programming Language> do you like the most? - A) you may learn about some weird feature, b) you can discuss how that feature will improve code
* Do you prefer JavaScript or TypeScript (dynamic vs typed language)?
There is no right answer. It is all about how the candidate argues his answer. You learn more about the person and if you can deal with him on a day to day basis (and if he has the brains to think about things).
Fizzbuzz is a trivial test to detect people who are basically fraudulent in their claims of programming ability. You shouldn’t need to study for it. Sorting algorithms, sure.
I remember a colleague who was affronted at being asked to do Fizzbuzz. I told him it was just a simple screen for fraudsters and bullshit artists and not to take it personally, they didn't know him from Adam, after all.
Yup, my experience is that FizzBuzz level complexity is roughly what you want for interview problems. I find that and some questions about algorithms and data structures tell you all you need to know.
I interviewed at Facebook and the tech part was completely sane and appropriate, the technical task was a bit fizzbuzzy but not something you'd need to spend nights digging into obscure algorithms book, but rather simple task which required some thinking, but not too much of it. No experience with Google or Netflix.
As of looking at CV, I've seen people with great CVs that proved utterly useless and of course the opposite too. CV can be very misleading, both unintentionally and intentionally. You need to speak with the person.
The issue with that is that if my CV is incredibly specialized in one area that happens to match the job I’m going for (eg compilers), why test my knowledge in designing things like a distributed system and in how great I do dynamic programming (that is not useful in my area)??? It feels like there is no verification at all of what my CV claims, nor does it give me a chance to show off the skills I’ve been working on for them last 20 years, both of which are very relevant to the job. Instead, I’m tested on what I knew when I graduated with my BS, and that’s it; it’s like my PhD was a waste.
I had a junior/intermediate level engineer start asking me minutiae about HTTP and HTTPS...
After about 5 minutes of this I literally said:
"Dude, have you seem my resume? I literally built a petabyte scale full text search engine and wrote 1.5M lines of code to do so and a HTTP framework that parses fetches more than 2PB of HTML per month. If there's some edge case that I might miss I can Google it in 30 seconds".
I no longer keep small tactical issues in my head. It's pointless. Now I try to understand the system as a whole.
It could be _because_ they read your resume that they asked those questions, to see if they believe it. People heavily embellish their degree of involvement in projects all the time.
Or they could just be asking gotcha trivia. It’s hard to tell.
A lot of people will write whatever makes them sound good on their resumes, so we have to ask questions about minutiae to make sure candidates aren’t lying.
It sucks, but this is what happens when the job you’re interviewing for has no license or certification requirements (but pays six figures).
1.5M lines is the output of a team of hundreds of developers over years. I don't believe anyone would write a million lines during a job.
2PB a month, is 125B documents at 16kB each, or 50k documents per second. That's decent. That's where it starts to be interesting in elasticsearch or hadoop.
I guess people who allowed hacks like adding -1 fridge to get a free tv in earlier online stores could also claim "but they have build online stores that moved thousands of sales every minute"
> You DID look at the CV before bringing the candidate in, didn’t you?
I would like to speak to this after interviewing at a few places recently, including Google. I don’t think a single interviewer looked at my resume. I’ve developed a sizable open source project that is used by real people for over a decade. If the company really wanted to know if I could write code they could just go and look at it. Instead they ask puzzle questions that I have no interest in and end up failing. These companies expect the candidate to spend weeks preparing for their hoop jumping, but can’t be bothered to actually read the CV. Multiple people didn’t seem to realize where I lived even when my address was at the top of the paper.
Netflix already does this. Just interviewed with them for a full stack role. All interviews are dependent on the team, but generally feature less whiteboarding than FB/Google. I did a takehome technical exercise, in person interviews were tailored to my experience. They were mostly about system design, general concepts, and a heavy heavy focus on interpersonal skills. Their approach seems to be:
1. Make sure you actually know what’s on your resume
2. Make sure you have the strategic thinking and interpersonal skills to own the system you’ll be working on
3. Make sure you’re a good match for their fairly unique work culture
By "fizzbuzz" I wasn't referring to the "divide by 3 and 5" one that seems common, but something much worse. I once interviewed at a place where they presented a jumble of characters, and about 8 rules, and I was on the phone being watched (after an hour of successfully troubleshooting a broken AWS instance with a bunch of quirky Linux issues). The 8 or so rules involved something like, "You must print each line with one character longer than the last / c must never come right before vowel / x's must be on alternate rows / f's and z's ... " while parsing all of this stuff, I had 10 minutes total since we were almost out of time when we started the questioning. The interviewer said, "Don't worry, just do it in pseudocode."
That's bad interviewing.
I can do the problem easily without someone looking over my shoulder, and I need more than 10 minutes just to think about the 8 or whatever rules he gave me.
He failed me, and I ended up working there anyway a year later after the guy quit (true story).
This sounds like you're having the wrong conversations. With all your experience, why are you still going into companies through the front door? Where's your network?
Fundamentally: Why are you having the "Can you code your way out of a wet paper bag" conversation instead of the "How much value can you add to this company" conversation?
Perhaps my experience as a consultant/freelancer is breaking my perspective. It's been several years since I last had a coding interview as an applicant.
If you apply for a full-time software engineering position at a large company today, you will be run through the all-day series-of-hourlong-interviews, even if you are a referral. It is possible for the hiring manager to work closely with recruiter and recruiting coordinator to craft a more customized interview process, but that's rare (and, impossible at a place like Google, which decentralizes hiring). The value of network is the candidate wasn't subjected to the usual resume review weeding-out, but their info went straight to one or more hiring managers for consideration.
> Fundamentally: Why are you having the "Can you code your way out of a wet paper bag" conversation instead of the "How much value can you add to this company" conversation?
Because people lie. It's that simple. Referrals lie, CVs lie, friends lie to get friends in a position. Do understand that there's a fundamentally different set of problem a 40.000 FTE company has to deal with in comparison to a 30 person startup.
Also is doing a full day interview really such a horrible sacrifice for most paid jobs in the world? Professions like laywers and doctors need to go through significantly longer hazing rituals to get to FAANG income... is a month of study to be essentially among the most paid people in the world REALLY such a horrible thing?
You still have to go through a standard interview loop at Google/Facebook/Netflix/etc... even if you get in via a recommendation. The recommendation (and your cv) helps you in getting a serious phone screen and in getting a good choice of the team you want to join, but isn’t considered very much during the interview itself.
Well said. It occurs to me that my last job interview that involved convincing the other party that I was a good developer was back in 1999 when I had just a few years of experience.
Since then every gig has come via an introduction by a former boss or co-worker, or by introducing myself to somebody with hiring authority and using a pile of impressive artifacts to skip the "can this guy do the work" questions.
The first several years of your career should be getting yourself into a position where nobody would ever dare ask you to sort things on a whiteboard.
You shouldn't have to only be able to go through your network to get a job. We should judge based upon the same criteria every other damned job judges ...how well they did their last job.
The problem is that resumes can be deceiving. If you've worked long enough in the industry, you've met a few incompetent people with stellar resumes. It's why having google on your resume opens a lot of doors. Everyone respects google's vetting/interviewing system.
Also, fizzbuzz isn't anything you really need to study for (certainly not for days or weeks). It's something you should be able to work out during the interview if you truly have developing experience. But then again, I've known people with amazing resumes who couldn't even understand the fizzbuzz question itself.
The Googles of this world don't fizzbuzz you, they want you to implement (close to optimal solutions!) and use tries and heaps & co. during whiteboard interviews. Neither of which I've ever had to implement in 10+ years.
Did you, in your "proven career", ever work on the systems the size of Google Cloud and having same amount of users?
Because cheap stuff you can run away with on 1mil users suddenly falls apart when you need to serve more across continents and suddenly you kinda need to know what all those words in documentation actually mean.
Also, as for CV, everytime I've interviewed people I've gotten more than I few people that lied in their CVs. Persumed "senior developers" who couldn't do a fizzbuzz in their chosen language and tooling. CVs are utterly worthless from interviewers perspective - there will always be someone with exact same CV like you have but will be utterly incompetent.
The original article [1] describes new process in the context of PM interviews, which are different from software developer interview, no matter how technical PM is.
The changes that I particularly like are: improvement in coordination between interviewers themselves, and stop sharing feedback between themselves until the very end. The latter really affects how next interviewers view the interviewee.
After reading about wisdom of the crowds, we started doing this with estimates. We always break up the project into small tasks together then seperate and estimate them individually.
We've found we're a lot more accurate because we don't influence each other, as well as don't go into autopilot on an estimate assuming someone else has it under control, or someone just deferring to the more Sr. Dev when the more Sr. Dev made a mistake.
I could see how this would also work well with interviews.
Well, FB did that at least 6+ months ago when I got trained for interviewing.
As for the questions asked, there is an internal tool with various questions you are supposed to ask, with info about what signals you can get, how to extract more info and how to formulate your feedback.
I was always told that I need to help the person I'm interviewing solve the problems.
> Under the new process, Microsoft shares the interview questions in advance so that candidates can prepare. During the interview itself, a candidate might run through a real scenario or problem the team is trying to solve.
LOVE IT.
I think best when I have a chance to actually think, not when I’m participating in some odd job interview game show.
Here's what they are doing, from the linked blog post:
> Our dev teams had taken to working with candidates to solve a bug or feature as part of the interview process. It was a collaborative effort with the candidate and the team working together to solve a real problem.
Sounds like a great idea.
Except it's so variable. What if you get lucky and you get a easy bug, whereas someone else gets something much harder. Let's standardize that by giving everyone the same task to work on.
And what if we want to hire people who aren't fluent in your project's programming language? That would be a big handicap for them. Let's deal with that by having minimally sized "projects" in all the languages, and allowing the candidate to choose their preferred language.
And we don't want the interviews to be biased against people who don't already have domain knowledge in a specific field. So let's make the task generic enough to be approachable by any good programmer.
Congrats, if you did all of the above, you've reinvented leetcode style interviews.
There's some cool stuff that MS has put into practice. I like the focus on reading/understanding existing code, instead of just writing code. I also like the relaxed pacing and "open book" approach.
But for the most part, I don't think this is really as revolutionary as people think it is.
Fixing a simple bug in your own software is an order of magnitude faster than doing it in someone else's codebase. Make no mistake, it would be cheaper for them to fix it themselves than guide someone else through a paired-programming exercise. This isn't unpaid labor, it's more expensive labor.
A little bit of unpaid labor is honestly fine. Especially since I can’t imagine this being easy to exploit in a way that really impacts a company’s bottom line.
It’s like getting a free sample of food at the store, before committing to a full purchase
When I interviewed at Redmond 15 years ago, it was the recruiter who asked me a bunch of techie questions then decided to bring me out for a whole day of onsite interviews. During that I recall it was only a director of SW who asked me some mentally crushing questions about subtleties in how the Windows kernel fiddles with files. As I said, I was a UNIX developer who didn't know those bottlenecks in the Windows kernel. Still he persisted. He did take me for a spin in his awesome BMW sports car after though!
My thoughts on Redmond were that it rains all the time and the inside of the Microsoft buildings is pretty depressing.
I've had something similar happen to me once. I'm DevOps, very systems focused. I was leaving Cisco and got on an interview panel with 4 network engineers at some place.
I caveated I know networking better than most but I'm by no stretch a network engineer. They agreed and explicitly stated the position was on the systems side and it was fine.
The interview was only an hour and half. About 30 minutes into they ran out of questions and started to get bored. They tell they're all studying for their CCIE's and decided to start borrowing questions from the final exam.
They decided to play "lets stump the Cisco guy". And did they? YES! Did they stop? NO! This was 7+ years ago and I'll never forget how uncomfortable and bad that interview was. They managed to dredge up all these arcane, stupid questions, some of which bore no practical function.. and listened to me "I dont know" for a solid hour
I'm sure it left them with a bad impression of me, and it certainly left me not wanting to work there.
Good riddance to all those mostly useless brain-teaser questions.. They require countless hours of practice for most people to get good at them - and I find it semi-tragic that many highly intelligent people are wasting their time on that instead of on complex real-world problems (not to mention on learning things that would make them happier and more well-rounded).
Readit News
One of the key issues in recruiting is how to make hiring decisions quickly. Good candidates don't have the patience for your internal politics to resolve over a month or two to figure out whether they're getting an offer or not; they're just going to interview with other places in the meantime who will beat you to the punch.
Take a resume off the pile. Email or text to schedule a call. During the call, talk about relevant experience / past projects, what each side is looking for, etc. Make a decision during the call whether you want to invite the person for a two-hour interview, and schedule it during the call, first offering an evening time slot for the interview. In the first hour, talk about culture, problem-solving and teamwork approaches, engineering attitudes. In the second hour, pull out an actual current problem and solve it collaboratively. If all goes well, send an offer, and offer to set up a dinner to talk about / negotiate the offer.
It should not take more than a few hours to make a decision on someone; it should not require a candidate to take paid time off. Your hiring pipeline's average lead time should be measured in days.
What I've learned is that it is a very weak signal to talk to candidate about "their research" unless it's immediately and directly applicable to what we were doing as a company, so that we can talk about it as equal experts, which never really happened in our case. Previous research done by candidate was, like research almost always is in a narrow and specialize niche. Every candidate I was talking to was very good with talking about their research and from my perspective at the time of the interview it was impossible to validate and evaluate their claims. That conversation could convey the character of the candidate but I couldn't evaluate their ability to do new research.
Instead, presenting candidates with the same problem, that in our case took around 3 hours to solve, in our case proved a good way to compare candidates a bit more objectively and test how they approach solving problems they haven't seen before which is more applicable to what they'll be doing on the job.
The downside of that process is that during three hours you can only test so much, but hiring is a really hard thing to do right and we didn't want to take more of candidates time as we did value it highly and found that to be the sweet spot for us.
You had a lucky escape. I had the same from a company. Which was arguably a 2 full nights worth work if done right.
I took the challenge. Did it, it passed their functional test suite which they provided. Unit test cases, git repo, documentation etc etc all in place.
They came back saying it was not enough. So it's not like if you do the take home project, they let you in.
Also its a tad little irritating to make people work for so long, make them do everything. And even then with perfectly working solution turn them down.
What too many people choose to forget or ignore is employment itself (at least in CA) is a 2-way process. The employer is not doing you a favor by employing you, despite how many people have drank the kool-aid to the contrary. You and the company have entered into a mutually beneficial agreement. Unfortunately, many people don't have the means or desire to walk away when an employer veers into "you should be thankful we let you work here" territory.
Why are software engineers doing interviews at all? What makes software engineers qualified to interview people (especially when you consider the need to avoid hiring bias, problematic statements by interviewers that expose the company to legal suit, etc.)?
I'm not saying that companies should go to the other extreme and have all interviews conducted by HR - that presents its own set of problems in that HR doesn't know how to evaluate candidates for technical skill. But expecting hiring managers to open up a few evenings per week in the irregular and relatively uncommon circumstance (if it's common, then you have problems with churn on that team, and you should consider firing the manager) in which positions open up on their team to evaluate potential direct reports is not even remotely unreasonable.
there is only so many days i can take off. with two weeks holidays, i might spend all of my holiday budget for this year.
That being said, if a candidate just couldn't take off work for whatever reason, I am willing to do an evening interview.
Gotta be flexible, on both sides.
It seems to me this is an issue with your employer not the interview process.
I'm currently interviewing and so far I have had a 48 hour weekend project, a 3 hour code assessment, and I'm expected to endure 3 days of interviews. I'm fairly close to throwing in the towel and saying I can't be arsed with this.
This may not apply to all companies or situations, but for a Microsoft-sized organization this is the way it should be done.
Your hiring needs should reflect the size of your organization, and your hiring process should reflect your hiring needs. If you hire relatively infrequently then it makes sense to go over individual resumes and have infrequent evening interviews. If you need to hire hundreds of people this month just to negate churn then you have the resources to build out the kind of enforcement-process-oriented above since you surely need that kind of infrastructure anyway. Everything in between is just a matter of degree and where your company is on the process-vs-culture balance as a function of its size.
However TBH an in-person day with 4 or so interviews (perhaps post a phone screen) is absolutely standard for all sorts of professional positions and has been for many decades. I can only think of one case where I just had an informal in-person lunch instead--and that was a case where I knew the company's owner very well.
Any serious interviews will take a day extra traveling, your prep time etc - I agree about having 4-6 interviews in a day is just stupid.
This is me. Hard to get that across in an interview, but once people work with me, they're cool with me coming back an hour later in an email with some thoughts on the last meeting topic. They know and respect that that's the way I work[0].
[0] I'm not the only one on the team who does this, so that helps.
It's meant to characterize people who think about what they should have said/answered only when on their way out, in the staircases.
I think I work this way. Maybe it's a way to ignore my lack of wit, my slow paced brain, and to maintain some self esteem.
I tried to find some way around it, and the only way I see is to learn how to think loudly. The only way is to supress others from thinking by speaking aloud my thoughts as they come. Let others follow my thoughts, not the other way around. I figured it out while making some group work with ~20 yo students being a student myself. I'm older (I was 35-40 over that period) and therefore smarter and in the most cases I was able to do this group work myself a way better then they do.
I tried different approaches. I tried to take a leader role in a discussion and to talk everytime I feel like this. I tried to let someone else to become a leader and to follow her. I tried to just get out of the discussion, to sit quietly with my own thoughts and to come with my own solution to compare it with the group one. The conclusion is: I cannot think AND listen at the same time.
Given that many people report that their best thinking is done while walking or taking a jog, it might not be too wild to try out interviews which allow the candidate to take a walk before they give an answer.
I have done plenty of architecting. A large percentage of architectural problems that you are asked to solve have already been solved. After enough years at enough different companies and reading enough architectural books, it’s just pattern matching and figuring out the correct tools.
When the topic is something where I know some underlying complexity exists, or the stakes are high, I often want to spend a bit of time thinking it through by myself because that's how I think best about those sorts of problems.
I've encountered friction at times with those that aren't familiar with and/or care about the underlying complexities and just want to talk high-level and make decisions and move on. Sometimes it is important to make quick decisions to avoid analysis paralysis, but often it is worth giving those who do best thinking through things on their own a chunk of time to go off and do what they do best. I wish more people took that into consideration when conducting brainstorms.
they asked me to build something. I used dictionaries and wrote the code in ten minutes.
Both interviewers wouldn't believe that the code could solve the problem.
They later told me that dictionaries are not used in practical applications. Never understood why.
Are dicts not used in "real life"?
It is precisely because of my experience that my team is OK with me taking an hour to ponder.
Edit: Let me clarify one thing. I'm not talking about situations where, say, the PO comes and asks, "Hey, JustSomeNobody, why does the system do this under these conditions?" I know the answer to this. If they want logs for that specific instance, I can get them. What I'm talking about is, when the PO calls a meeting and says, "I just got off the phone with <Entity>, they want these changes to the system. What will it take?" I don't answer these questions, definitively, without thinking about them. That does NOBODY any good. I'll sometimes give a few initial thoughts for the sake of the meeting, but that's it.
It's probably not that these candidates can't think on their feet, it's just that they generally aren't willing to compromise so greatly on accuracy and quality of the answer.
I've worked with people that were reactive and would love to be "the fixer" when, in reality, 99% of the issues he fixed were caused by his ignorance/incompetence in the first place.
Specially for crucial 24/7 roles, you need people that think ahead and not reactive ones.
just had that happen. we had an overloaded server, and while we tried to figure out how to make the server faster, after some thinking we realized that the better long term solution was to change some of the architecture so that the server would get less load to begin with.
that was definitely not what came to mind first.
One data point... Senior engineer that founded the local java user group and wrote a published book on the java language. Couldn't code a simple reverse string algorithm, nor explain whether, after with help devising an answer, the method was thread-safe.
Another data point... A self-described expert in SQL with it all over their resume. Couldn't write a simple join, inner outer or other.
Weed-out questions exist because our industry is flooded with people who don't know, nor care to know, their craft. And there are plenty of employers who continue to hire these people, and promote them. I interviewed numerous "architects" (with development experience all over their resume) who couldn't come up with a naive solution, let alone the optimal, to a very simple problem. Want to fix the interview process in our industry? Figure out a way to weed out the liars? I'm all for it. I would rather not have to do basic algorithm questions, but that's the current environment.
Really, it's not fizz buzz or string reversal that causes people to re-study for their algorithms and data structures exam before an interview.
In many ways, I think this is similar to requiring that senior actuaries re-study integration by parts prior to every interview. They don't have to, because they have a proper exam that is widely accepted in their field. We don't. So instead, we are taken through full day whiteboard exams, under conditions of great secrecy, over and over, every time we interview.
Other hand, I'd suggest instead of "lying" that people are just myopic about their skill sets. I'd been writing JavaScript code for years, for example, but on an interview learned about a whole world of "modern" JavaScript that I didn't know existed. Clearly they thought I was a fraud. But I'd just been living on the 3rd floor without ever visiting the basement where all the pipes and boilers were.
You were interviewing senior engineers with Java experience and they couldn't come up with Assert.assertEquals("radar",StringUtils.reverse("radar"));
I suspect you were actually looking for the implementation of StringUtils.reverse() and if that's the case then you were focused on the wrong skills.
Deleted Comment
Dead Comment
They may exaggerate claims - yes, that is why you gave them a call in the first place. But then again you exaggerated your claims in the job posting of which skills are required for the job.
The one thing that I noticed is that only the US companies have this awkward parallel reality 'technical-interview' hiring process. In other countries like Germany or France the interview process is mostly focused on the person and the desire to work for the company and the willingness to learn the skills needed for the job. Just because you fail to answer immediately to a random question it does not mean you are unqualified for the job.
When I conduct an interview I try to only ask open questions like: * What is your favorite IDE? - Whatever the person answers, tells a lot about how he works, what tools he uses, and if he is aware of the common tools * What feature of the new Version of <Programming Language> do you like the most? - A) you may learn about some weird feature, b) you can discuss how that feature will improve code * Do you prefer JavaScript or TypeScript (dynamic vs typed language)?
There is no right answer. It is all about how the candidate argues his answer. You learn more about the person and if you can deal with him on a day to day basis (and if he has the brains to think about things).
http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz....
Deleted Comment
After about 5 minutes of this I literally said:
"Dude, have you seem my resume? I literally built a petabyte scale full text search engine and wrote 1.5M lines of code to do so and a HTTP framework that parses fetches more than 2PB of HTML per month. If there's some edge case that I might miss I can Google it in 30 seconds".
I no longer keep small tactical issues in my head. It's pointless. Now I try to understand the system as a whole.
Or they could just be asking gotcha trivia. It’s hard to tell.
It sucks, but this is what happens when the job you’re interviewing for has no license or certification requirements (but pays six figures).
1.5M lines is the output of a team of hundreds of developers over years. I don't believe anyone would write a million lines during a job.
2PB a month, is 125B documents at 16kB each, or 50k documents per second. That's decent. That's where it starts to be interesting in elasticsearch or hadoop.
;)
I would like to speak to this after interviewing at a few places recently, including Google. I don’t think a single interviewer looked at my resume. I’ve developed a sizable open source project that is used by real people for over a decade. If the company really wanted to know if I could write code they could just go and look at it. Instead they ask puzzle questions that I have no interest in and end up failing. These companies expect the candidate to spend weeks preparing for their hoop jumping, but can’t be bothered to actually read the CV. Multiple people didn’t seem to realize where I lived even when my address was at the top of the paper.
1. Make sure you actually know what’s on your resume
2. Make sure you have the strategic thinking and interpersonal skills to own the system you’ll be working on
3. Make sure you’re a good match for their fairly unique work culture
That's bad interviewing.
I can do the problem easily without someone looking over my shoulder, and I need more than 10 minutes just to think about the 8 or whatever rules he gave me.
He failed me, and I ended up working there anyway a year later after the guy quit (true story).
Fundamentally: Why are you having the "Can you code your way out of a wet paper bag" conversation instead of the "How much value can you add to this company" conversation?
Perhaps my experience as a consultant/freelancer is breaking my perspective. It's been several years since I last had a coding interview as an applicant.
Because people lie. It's that simple. Referrals lie, CVs lie, friends lie to get friends in a position. Do understand that there's a fundamentally different set of problem a 40.000 FTE company has to deal with in comparison to a 30 person startup.
Also is doing a full day interview really such a horrible sacrifice for most paid jobs in the world? Professions like laywers and doctors need to go through significantly longer hazing rituals to get to FAANG income... is a month of study to be essentially among the most paid people in the world REALLY such a horrible thing?
Since then every gig has come via an introduction by a former boss or co-worker, or by introducing myself to somebody with hiring authority and using a pile of impressive artifacts to skip the "can this guy do the work" questions.
The first several years of your career should be getting yourself into a position where nobody would ever dare ask you to sort things on a whiteboard.
Also, fizzbuzz isn't anything you really need to study for (certainly not for days or weeks). It's something you should be able to work out during the interview if you truly have developing experience. But then again, I've known people with amazing resumes who couldn't even understand the fizzbuzz question itself.
Because cheap stuff you can run away with on 1mil users suddenly falls apart when you need to serve more across continents and suddenly you kinda need to know what all those words in documentation actually mean.
Also, as for CV, everytime I've interviewed people I've gotten more than I few people that lied in their CVs. Persumed "senior developers" who couldn't do a fizzbuzz in their chosen language and tooling. CVs are utterly worthless from interviewers perspective - there will always be someone with exact same CV like you have but will be utterly incompetent.
The changes that I particularly like are: improvement in coordination between interviewers themselves, and stop sharing feedback between themselves until the very end. The latter really affects how next interviewers view the interviewee.
[1] https://blog.usejournal.com/rethinking-how-we-interview-in-m...
We've found we're a lot more accurate because we don't influence each other, as well as don't go into autopilot on an estimate assuming someone else has it under control, or someone just deferring to the more Sr. Dev when the more Sr. Dev made a mistake.
I could see how this would also work well with interviews.
LOVE IT.
I think best when I have a chance to actually think, not when I’m participating in some odd job interview game show.
> Our dev teams had taken to working with candidates to solve a bug or feature as part of the interview process. It was a collaborative effort with the candidate and the team working together to solve a real problem.
Sounds like a great idea.
Except it's so variable. What if you get lucky and you get a easy bug, whereas someone else gets something much harder. Let's standardize that by giving everyone the same task to work on.
And what if we want to hire people who aren't fluent in your project's programming language? That would be a big handicap for them. Let's deal with that by having minimally sized "projects" in all the languages, and allowing the candidate to choose their preferred language.
And we don't want the interviews to be biased against people who don't already have domain knowledge in a specific field. So let's make the task generic enough to be approachable by any good programmer.
Congrats, if you did all of the above, you've reinvented leetcode style interviews.
There's some cool stuff that MS has put into practice. I like the focus on reading/understanding existing code, instead of just writing code. I also like the relaxed pacing and "open book" approach.
But for the most part, I don't think this is really as revolutionary as people think it is.
This is unethical. How is this anything other than unpaid labor?
It’s like getting a free sample of food at the store, before committing to a full purchase
My thoughts on Redmond were that it rains all the time and the inside of the Microsoft buildings is pretty depressing.
I caveated I know networking better than most but I'm by no stretch a network engineer. They agreed and explicitly stated the position was on the systems side and it was fine.
The interview was only an hour and half. About 30 minutes into they ran out of questions and started to get bored. They tell they're all studying for their CCIE's and decided to start borrowing questions from the final exam.
They decided to play "lets stump the Cisco guy". And did they? YES! Did they stop? NO! This was 7+ years ago and I'll never forget how uncomfortable and bad that interview was. They managed to dredge up all these arcane, stupid questions, some of which bore no practical function.. and listened to me "I dont know" for a solid hour
I'm sure it left them with a bad impression of me, and it certainly left me not wanting to work there.