Readit News logoReadit News
wheaties · 9 years ago
When I had less than 5yrs experience I used to ask these types of questions, thinking they showed the ability of a developer to reason out critical software issues. Then I met a guy who had practically memorized everything I could ask. He spit out answers so fast it was appallingly clear he was regurgitating answers from a book. This proved nothing about him and everything about me.

Today, I recognize that good software development owes nothing to data structure knowledge or obscure algorithms. This knowledge can't hurt but very few jobs or languages require you build a skip list from scratch. Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.

LnxPrgr3 · 9 years ago
For functional purposes, I consider myself to know something if I could page in the details well enough to implement it in 5 minutes with Wikipedia or less.

How this plays in interviews is interesting for just how inconsistent it is. Sometimes I win the algorithm lottery and have had reason to implement the obvious solution myself at least once, and the interviewer learns my memory works. Sometimes I don't know what they're going for but come up with a reasonable answer anyway and the interviewer is thrilled I managed to have an independent thought. Sometimes I lose and the interviewer is clearly trying to guide me to the answer they want to see and I have no idea what they're getting at.

Though not quite as bad as a friend failing an interview for not using a hash table, even though the answer he described was a hash table—just with the identity function as the hash.

(Though maybe about as absurd as one interviewer digging for any formal training I might have had in C++. Like you can teach that language in a semester—please. There are maybe 4 proper experts on the planet.)

virmundi · 9 years ago
I think there might be a happy medium. Knowing how to implement a specific algorithm is not necessary. Knowing of algorithms and picking the right one is important. For example, a junior developer apparently forgot about hash sets. He used a list for lookup. Worked fine in his local tests. Fail miserably when we had to load millions of records. A more seasoned programmer should be able to explain why a hash would work well here.
pacaro · 9 years ago
Accidentally quadratic is good for a chuckle too

https://accidentallyquadratic.tumblr.com

Tade0 · 9 years ago
I used to do interviews in one company I worked for and we had this online test, which consisted of four tasks very similar to those in the list.

There was this one candidate who, by mistake, was given the test twice. Apparently he took offence to that. He found that his friend did the exact same tasks so he copied them.

The designers of that test tout that their system is able to detect plagiarism, but that was not the case here.

Anyway eventually we hired him because even though his first attempt wasn't particularly impressive and he cheated a face-to-face interview revealed that he was a skillful programmer.

One thing I learned from this experience is most of the time these test simply confirm that the candidate went to college - sometimes a particular one.

adamnemecek · 9 years ago
I agree 100%. I'm hoping that Disseminating these practice materials will eventually topple the whiteboard interview by exposing it's futility.
sumeno · 9 years ago
>Today, I recognize that good software development owes nothing to data structure knowledge or obscure algorithms. This knowledge can't hurt but very few jobs or languages require you build a skip list from scratch. Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.

While I don't think that the current state of interviewing is perfect, this statement is pretty out there. Good software development owes NOTHING to data structure knowledge? Choosing the right data structure is critical for software that performs well (or performs at all once you get to a certain scale), and demonstrating knowledge of various data structures shows at least that a candidate knows a bit about which tools they have available to them on the job.

Of course not many people are implementing skip lists from scratch in their everyday job, that's why in the list of 500 questions none of them are "Implement a skip list". Data structure questions rarely ask you to implement some data structure from scratch, they ask you to look at a problem and figure out which data structure best lends itself to the solution.

I won't argue that asking questions about obscure algorithms isn't a problem, but good interviewers aren't asking questions that need obscure algorithms, they are asking questions that need algorithms like DFS, basic variations of sorting, and basic DP problems.

hluska · 9 years ago
You're focusing too much on a particular phrase and not enough on the overall meaning. Read this again:

> Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.

Code review is the best time to talk about data structures and algorithms as they are a time to apply theoretical knowledge to real world problems. Good, constructive code reviews are about education and maintaining a quality code base.

Interview questions about data structures and algorithms rewards memorization and classroom knowledge. I've worked with way too many mediocre developers who are great at whiteboarding interview questions but who couldn't build a maintainable system if their lives depended on it. Consequently, I don't give a damn about a candidate's performance on these types of questions. Instead, I'd rather learn about some projects they've built, problems they've experienced and their experience with code reviews.

thearn4 · 9 years ago
Yeah, the strange "data structures and algorithms are either everything or nothing" polarization in the opinions on tech interviewing is strange.
skybrian · 9 years ago
Could you explain your reasoning? Someone doing really well doesn't seem like a reason to stop asking other people algorithms questions.
naasking · 9 years ago
Because he understood that this data point falsified the general theory that answering questions about algorithms and data structures implies an understanding of algorithms and data structures.
lefstathiou · 9 years ago
So that said how do you vet talent? Ask them to tell you stories about maintaining systems, debugging, etc?
rm999 · 9 years ago
Personally, yes - I like asking about high-level run-downs of a project on their resume. I then choose points to stop them (either randomly or if I know a lot about the subtopic) and dig deeper and see where their comfort level lies.

There are a lot of benefits to this approach:

1. Keeps the topic on something they are presumably comfortable with.

2. Gauges their seniority level: more senior folks can talk deeply about a wider variety of topics.

3. Tests their communication and explanation skills, again about a topic they should be comfortable with.

Some potential pitfalls:

1. Makes it harder to objectively compare candidates. I think this is impossible anyway, so I don't consider it a big deal.

2. Some candidates, especially junior ones, simply don't have many projects on their resume to talk about. In this case I'll often ask them to explain a skill or technology they are experienced with.

3. It's harder to conduct these interviews. Giving a problem and passively watching and helping as needed is a lot easier than keeping up an active dynamic conversation.

twic · 9 years ago
I've seen (from both sides) two things that seem to work.

Firstly, write actual code to solve a simple but real, or at least realistic, problem. That could be a take-home exercise, or it could be on-site pair programming, or some other format. Memorable examples: a stock exchange (submit orders, get trades); a Tetris engine (submit blocks, get rows filled); an auction house (submit bids, get winners); an uploader for the Azure blobstore (submit blocks, get files); an online metric summarizer (submit samples, get summaries). This tests someone's ability to actually write code.

Secondly, suggest an algorithm for a real, or realistic, problem that doesn't have a well-known good solution. Memorable examples: subset-sum, but on an ordered set, where the solution must be contiguous; placing primers for PCR [1]; online k-anonymisation [2]. This tests someone's ability to think on their feet, and synthesize facts they've learnt into something new.

I'm a bit dubious about the "tell me about something you worked on" question. Firstly, because it requires the candidate to have worked on something chewy, and there are lots of capable people out there who are unfortunate enough to have had crummy jobs where they don't get to do that. Secondly, because it attempts to evaluate the judgement of an individual by looking at the work of a team. Too often, as a candidate, i've had to answer "why did you do it that way?" with "because the client specifically asked for it" or "because my colleagues who were working on it at the time decided so", which isn't useful for me or the interviewer.

[1] http://wiki.c2.com/?StridingAcrossSteppingStones

[2] https://en.wikipedia.org/wiki/K-anonymity

jpiabrantes · 9 years ago
The best way to memorise hundreds of different algorithms (or anything else for that matter) is to understand the few underlying principles beneath them. I would be surprised if he knew them all and couldn't come up with intelligent ways of mixing them for achieving a new task.
vmasto · 9 years ago
The best way sure, not the only or even most popular way though.
tzs · 9 years ago
> Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.

What do you mean by "maintain a system"?

tzs · 9 years ago
I guess I need to elaborate, since I am getting down voted.

I have only heard of maintaining systems in contexts where it means keeping a system running. I don't see what that has to do with the ability to WRITE software (good or otherwise). Hence, my question.

nanospeck · 9 years ago
Does your company have current openings? I think I can prove myself better in your metrics.
wheaties · 9 years ago
We do, check out MediaMath. We will ask you design, debug and like the questions. Well also do a take hone problem.
maxaf · 9 years ago
Speak the truth. I'd hire you again, man. <3
wheaties · 9 years ago
Yeah and you'd be the first guy I'd think of in an early stage start up. You're a gets shit done kind of guy.
ziikutv · 9 years ago
Thank you!

Dead Comment

Deleted Comment

darioush · 9 years ago
- Programmers are increasingly more "API plumbers", not "pathfinders". The change being you have to more often create an "easy to maintain" glue layer between lots of existing code (most of which your company wrote a bunch of years ago, or is external to your code in a framework or library) instead of solving a problem from scratch using CS basics. Interview questions haven't shifted to "this library has such and such function calls, how would you expect it to report an error?" or "describe two distinct UI frameworks you have used and explain their differences". They're still very much in the era of OK we have two rectangles, how do we find the overlapping pixels.

- Software is increasingly built in teams. We don't ask about "how you handled a difficult person on your code review" or "how would you break this change in to two logical commits" or even "what's a good name for this function", all of which are more important than knowing the big-o for some obscure interval tree.

- CS 101 algorithms have very little to do with how they are implemented. e.g, in a recent discussion with a co-worker I discovered libstdc++ hashtable's iteration orders depend not only on hash code but also on insertion order. Looking at how libstdc++ and MSVC implemented hashtables was both very difficult and exhausting to follow and had many many more details than the typical CS 101 approach of just hash the key and direct lookup the bucket! Turns out writing code that is really fast needs more than knowing the big-o.

- Big O is such a garbage simplification of all the "fundamental" knowledge. Why not just ask some fundamental graph or combinatoric counting questions? Why nothing about what is computable and what is not, or which languages are regular? What about information theory? How come no information recall or database questions? or even these days with all the ML, why not some basic linear algebra?

NTDF9 · 9 years ago
Here's my gripe with this interview process. It encourages engineers to spend months and years into learning and remembering optimal solutions to textbook problems.

A REAL engineer would spend the same time doing something productive....things like:

- Writing a sql optimization for their current company that saves resources

- Creating shared memory interfaces for faster processing of logs

- Modifying filesystems or kernel to create their own log processing

- Modifying JVM to optimize critical code

- Rewriting parts of a server to set up security alerts

- Fast encryption and decryption of in-memory cache records

- Writing an interpreter for faster generation of low level code for high speed processing

- And on and on

My point is, today's interview process treats engineers with such complicated experience as shit because they spent time working on complex projects and couldn't memorize these questions. But then these same companies will hire some mediocre engineer who cannot think out of the box but is an expert at memorization.

Said mediocre engineers then hire other mediocre engineers using the same process, because these mediocre engineers do not even have the ability to grasp what kind of engineering would lead to above points.

To top it off, these same mediocre engineers sweep this problem under the rug claiming they are trying to reduce false negatives. Lol!

Sad state of the industry!

JDiculous · 9 years ago
Google style interviews don't screen for the best engineers, but they screen strongly for subservience. The fact that a candidate has to go out of their way to prepare for these interviews indicates that the candidate is likely willing to jump through whatever hoops they're told to jump through.

It's similar to how investment banking jobs place such a high premium on Ivy league degrees with high GPAs despite the actual job itself being so simple a high school student could do it. Someone who's willing to jump through all those hoops to get into a top school and graduate with a high GPA has a strong track record of following orders and is less likely to complain about 100 hours workweeks.

NTDF9 · 9 years ago
>> Google style interviews don't screen for the best engineers, but they screen strongly for subservience. The fact that a candidate has to go out of their way to prepare for these interviews indicates that the candidate is likely willing to jump through whatever hoops they're told to jump through.

Try saying this obvious observation to some Google engineers in real life. I've never seen such butthurt adult-kids!

eli_gottlieb · 9 years ago
I mean, that's all true, but what matters is whether you have some personal goal which requires you to go through time at Google.
qaq · 9 years ago
If this materially impacts the bottom lines of companies that are practicing this approach eventually companies that are not using this process will prevail.
inopinatus · 9 years ago
There are other equilibria, likely because the employment market does not have the characteristics necessary for analogies of the the Efficient Market Hypothesis to apply. (most glaringly: people are not fungible commodities / the agents involved do not have rational expectations)
NTDF9 · 9 years ago
>> If this materially impacts the bottom lines of companies that are practicing this approach eventually companies that are not using this process will prevail.

It would, if companies realize that they can save millions by improving their interview processes and not have to aqui-hire so much.

But this insight requires smart people (who they eliminated with this interview process) to be already hired in those companies.

BurningFrog · 9 years ago
Companies that hire the best engineers already use better processes.
pkaye · 9 years ago
So what approach do you suggest for the interview process? How has it been your experience hiring under this process?
eli_gottlieb · 9 years ago
Without spoilers, here's an outline of an interview problem I did last Friday:

1) Here is a very small VM (fewer registers and fewer instructions than you can count on your hands). Write an interpreter for it.

2) Add an I/O instruction.

3) Having your interpreter and your I/O instruction, you are given at least one example of a program with a specific property. Write a program analysis which will detect the property soundly and completely.

Managing to finish (3) relies on knowing about certain fundamental concepts from automaton theory and program analysis. The company does actually use this knowledge regularly, so it makes sense that they'll ask. However, even parts (1) and (2) actually manage to test basic programming in a reasonable way.

NTDF9 · 9 years ago
Step 1: Read the fucking resume

Step 2: Find somebody with relevant expertise to interview that candidate. Don't send the generic engineer who doesn't know anything beyond using hashmaps for all Java programs.

SamReidHughes · 9 years ago
> Here's my gripe with this interview process. It encourages engineers to spend months and years into learning and remembering optimal solutions to textbook problems.

Wrong. Questions aren't designed to be memorized. It shouldn't take months, or years, to prepare for this stuff if you already have a CS background unless you're really slow.

adamnemecek · 9 years ago
> Wrong. Questions aren't designed to be memorized.

Do you know that 'detect a cycle in a linked list' question? I can't imagine many people being able to answer that without having seen the question previously.

NTDF9 · 9 years ago
Who do you want to work with? A person who can regurgitate CS trivia or a person who has interest and track record in solving interesting problems?
khazhoux · 9 years ago
It's a tragedy of our industry that this super FUN list of basic algo problems, is connected in everyone's mind to lame-ass whiteboard interviews. These problems should be like playing scales: engineers should just work through these, practice them till they're no-brainers. But not to land that sweet entry-level SWE job at Google, but just to grease your own wheels in day-to-day coding and problem-solving.
acheron · 9 years ago
I've never been to an interview that would ask this kind of crap, and would be very surprised if a future interview did. So yeah, I'm just looking at it as a list of possibly interesting problems, which makes it a way better link than "interview help".
lkozma · 9 years ago
I agree -- in addition, such problems can also be seen as a springboard into research - tweak any of them or ask if you can prove that the solution is optimal, and you are quickly in the realm of open questions.

Deleted Comment

xaybey · 9 years ago
As a fun warm up-question, I always ask the candidate what editor and keyboard they use. The candidate who geeks out about their plugins and cherry switches inevitably passes all 5 rounds; the candidate who uses eclipse with maybe a vim plugin will go on to fail about 75% of the time. It's a good filter too; I'd say 1 out of every 20 candidates expresses strong convictions.

I'm seriously considering using it as a dog whistle and skipping the experience deep-dive (which tests if you are a good story teller), the systems design questions (which tests if you can sync to the interviewer's sense of judgement), and the algorithm hazing (which tests social anxiety). But I'm worried there is some kind of discriminatory correlation baked into that metric.

sbisker · 9 years ago
I worked at a company that tried that for a while and eventually decided to completely abandon it, because it was correlating so strongly to a particular set of experiences shared by the (mostly male, CS degreed) interviewers. Particularly with more junior engineers, where having a sexy setup is more likely to correlate who you hung out with in college than whether you actually coded enough to find a need for it yourself. So if you're going to play with that as a signal, I'd advise you to be very careful.
twic · 9 years ago
I have exactly the opposite bias. I think people who get really into keyswitches, alternative keyboard layouts, and .vimrcs are prone to getting distracted by unimportant details. Of course, we all want to use the most comfortable and productive tools, but a skilled person will attach more weight to things that have more impact.

Still, i'm glad i know people like that, because it's really easy to get a knowledgeable recommendation when i need a new keyboard!

edanm · 9 years ago
As someone who totally geeks out about my editor (vim, what else!?), I can understand that.

However, I'm pretty sure that out of the many, many developers I've worked with, including some brilliant ones, I'm one of barely a handful who gives much serious thought to editors. This could also reflect a bias of location - HN'ers are more likely to geek out over editors, but I think in general people in Silicon Valley tend to care more about this, as opposed to people in Israel, where I'm from.

I do sometimes ask similar questions about more technical topics that the candidate should be familiar with, e.g. which DB do you use (backend engineer), which JS framework you use, etc. This is something that the strong candidates must have brushed up against, and should have a strong opinion about.

poikniok · 9 years ago
Just out of curiosity to see how I do, how would you rate using stock Intellij IDEA and a Macbook? Additionally am interested because some top algorithm competitors (of which I am not one) have this very setup.
bshimmin · 9 years ago
I often ask people about their editor setup too - I tend to find that the level of enthusiasm they have about their editor is a moderately good indicator of their enthusiasm in general (though not, necessarily, their competence - I think that's stretching it too far). If they give me a look as if to say, "Why on earth is he asking this?" or they reply with something like, "Oh, whatever to get the job done... what do you use?" then it doesn't suggest to me that this is a person who gets much pleasure out of programming.

The other nice thing I find about asking this is that it isn't a question they've prepared for, so you tend to get an honest answer.

rachelbythebay · 9 years ago
nano and a $10 Logitech k120. What's your verdict?
mikekchar · 9 years ago
I really wanted to play this game, but after thinking about it, it really depends on why you use nano and the k120. The k120, in particular, is a very stiff keyboard with a mushy feel. It requires a fair amount of strength. I couldn't use it all day -- especially not with nano.

My initial thought was that you are a very careful programmer that tries not to make too many mistakes. I also thought that you probably try to write code in one pass and that you avoid TDD and other techniques that cause you to go back and revisit code again and again.

But, I cheated a bit and noticed in your profile that you have sysadmin experience, so now I wonder if the cheap keyboard is simply the keyboard that happens to be there. Nano is because you can't be bothered to spend your life learning vi and nano is the other editor that's guaranteed to be on whatever machine you happen to be logged in to. So that would make you pragmatic (as any good sysadmin is).

What's kind of interesting about this process for me is that I now have this utterly fictitious view of you based on ridiculous stereotypes. We could probably have a good conversation where you poke a million holes in my view, but without really challenging anything that I hold dear. Not really sure if it's a good idea, but it is definitely interesting to think about.

WWLink · 9 years ago
Sounds like someone that at least knows how to use a terminal and maybe some linux commandline tools like gdb.

I think the person we're replying to is looking for someone that wrote a few popular VIM/sublime plugins and uses an ergodex they built themselves.

I don't use an ergodox because I wanted an RGB keyboard, and I use sublime with a handful of simple plugins because that's what I like. I used to hate vim, but some of my coworkers love it, so I respect that and have tried to use it more often. :) The good news is the coworkers that live out of vim like sublime, too. They're just used to vim, nothing wrong with that.

It's not a terrible place to start though.

rfrey · 9 years ago
I'm probably just reading this into your comment, but the challenging tone makes it seem like you think OP is pretty misguided... like there's no information to be had here.

But isn't an interest in tools pretty common in most crafts/trades? If I were hiring a woodworker for my cabinet shop, I'd probably ask about favourite tools. I wouldn't actually care if they talked about how a bandsaw beats a tablesaw every time, or launched into a sermon about shopmade wooden handplanes versus metal monstrosities. It's the presence of an opinion at all that matters.

edit: I don't actually have a cabinet shop. :(

commentai · 9 years ago
I love the office keyboard. It's so fun!
pfarnsworth · 9 years ago
I have a friend who memorized all of these types of questions and answers. It worked because he got offers from Google, Facebook and Uber, amongst others. The key is to pretend like you're figuring it out on the spot and not just regurgitating it from memory. I plan on doing the exact same thing when I look for my next job. It's unfortunate, but the reality of the state of our interviews.
khazhoux · 9 years ago
I have two kinds of questions: (1) tell me about a problem you've solved, and (2) let me tell you about a problem I'm facing. I'll never ask candidates about made up problems. Why would I waste our time talking about fake stuff, when there's so many actual real problems in front of us that we could dive deep on?
alansammarone · 9 years ago
The best, most fair interviews I've been through follow this pattern. I remember one of them told me about a current problem they had, and told me "You can google anything you want, you have 30 minutes. After that, you should tell me how would you solve this problem, including any tradeoffs you've made".

This is by far the most realistic scenario employees of most companies face. It doesn't matter so much that one doesn't remember the specific details of tree traversal or how linked lists are implemented. It matters much more that one is able to find it out quickly, understand it and apply it. Those are the skills you're looking for.

inetknght · 9 years ago
If they solve it but you have other reasons not to hire them, would you implement their solution?
faragon · 9 years ago
In my opinion, it is great having people trying to "memorize" those questions. If they understand them, it would be like one half of formal computer science studies. What is a shame is people graduated from Computer Science university courses not remembering a clue about algorithms, because they "memorized" just to pass the exam. BTW, some of the interviews may be are too much hard, requiring being a Donald Knuth, having a "intelligence" index of 160, and being a nice team player (if you target very smart people you'll deal with crazy jerks most of the time -until those jerks become tamed, somehow-). Super-smart people should target working for the NASA or similar, not for writing APIs in Python 50 hours a week with just 3 week per year holidays.

Anyway, algorithms matter.