Readit News logoReadit News
smeeth · 2 years ago
People's intuitions around hiring aren't Bayesian enough. I think a good process reduces down to something like:

- Are they smart? (understands quickly + communicates effectively)

- Are they cool? (won't put poison in the keurig + pleasant to be around)

- Are they high energy? (initiative + action bias)

- Do they have the experience needed to be successful in the role?

Those are pretty strong priors for success. If you find someone with all the above, you've got an ~80% chance of a hit. No need to over-complicate.

In my experience, adding more boxes tends to index towards box-checkers who grew up wealthy. That's how you miss the hyper-smart/diligent state school kids who happened to spend their summers working instead of doing model UN.

CharlieDigital · 2 years ago
I have a 4-quardrant way of thinking about this that's similar.

- Y-axis is "drive"

- X-axis is "aptitude"

- low drive + low aptitude: never hire

- low drive + high aptitude: hire for targeted use cases where you need expertise

- high drive + low aptitude: hire, train, and foster aptitude growth

- high drive + high aptitude: hire on the spot

There's an indirect way of testing for this which is to test for curiosity and lack of ego. My experience has been that candidates with high curiosity tend to have low ego (they know what they don't know and are curious to learn). These candidates make great hires because you can teach them anything.

Wrote about this a little bit here (with a handy diagram): https://charliedigital.com/2020/01/15/effective-hiring-for-s...

smeeth · 2 years ago
I like this, but I do think it misses the social stuff. Just a few neurotic/unethical people can completely ruin a team and make all the best people quit. I find this to be extra true on diverse teams (all types of diversity).
walthamstow · 2 years ago
Aren't these quadrants from about 100 years ago? Some Prussian General von Whatshisname?

Edit: General von Hammerstein

I divide my officers into four classes as follows: the clever, the industrious, the lazy, and the stupid. Those who are clever and industrious I appoint to the General Staff. Use can under certain circumstances be made of those who are stupid and lazy. The man who is clever and lazy qualifies for the highest leadership posts. He has the requisite and the mental clarity for difficult decisions. But whoever is stupid and industrious must be got rid of, for he is too dangerous.

BeetleB · 2 years ago
> high drive + low aptitude: hire, train, and foster aptitude growth

This needs to be subdivided. There are plenty of people who've had a lot of opportunity for aptitude growth, but they didn't grow. And having high energy in those circumstances really messes up a lot of stuff (they're incompetent, but jump into and be involved in everything, and you have to politely tell them to get off your project).

brianmcc · 2 years ago
High on both sounds like Spolsky's "Smart and Gets Things Done", and not surprisingly your conclusions rather agree :-)

https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid...

gen220 · 2 years ago
This is The Way™! I've heard this alternatively described as "slope" (for drive, which incorporates the curiosity, lack of ego, excitability) and "intercept" for "aptitude".

Implication is that if you're high-slope, you'll eventually become high-intercept.

If you're a startup, you want to bias towards a mix of high slope, low intercept and positive slope, high intercept.

I think it's also worth pointing out that people can have negative slopes (due to social reasons, ego reasons), and that slope is not invariant over a person's career but often changes.

Ideally, you want to find people whose slope is positively-impacted by the environment of the company – a great hire is only a great hire if it's a great mutual-fit, in that way.

nerdponx · 2 years ago
But aptitude can be split into "experience" and "natural talent", which are different things and I think that's meant to be the whole point of the article.
astura · 2 years ago
>low drive + high aptitude: hire for targeted use cases where you need expertise

If you do choose to hire this person make sure to isolate them from other. I've seen a single low drive + high aptitude person single handly ruin teams. They are a cancer.

dboreham · 2 years ago
> Y-axis is "drive"

Someone I learned much from way back called this "wattage". As in: "he's too low wattage".

PeterStuer · 2 years ago
I'd go with the crazy/hot axis myself.

You meet minimum requirements on both, and if you do your contribution better exceed your high maintainance demands.

agumonkey · 2 years ago
out of curiosity, do you often run into low aptitude + low drive but highly politically skilled ? do they get caught early ?
intelVISA · 2 years ago
how do you keep 'high drive + high aptitude' from escaping to run their own venture?
powerset · 2 years ago
How do you test for curiosity?
CobrastanJorji · 2 years ago
How do you measure "are they cool" without just directly transforming all of your biases and stereotypes about the candidate into a numeric score?
Aurornis · 2 years ago
"Cool" isn't a good word to use, but it is important to filter for people who can get along with others.

Some candidates can't make it through an interview without being condescending to someone, making snide remarks, being arrogant, trying to start arguments about trivial things, or other negative behaviors. If they're doing this during the interview, you're only seeing the tip of the iceberg. It's going to be 100X worse when you have to deal with that person 5 days a week.

Other candidates behave well during interviews but have a history of causing social problems at other companies. It takes some work to uncover these (and, importantly, validate their veracity). Some of the most toxic people I ever worked with were very charming in interviews. They swung from company to company, leaving a trail of unhappy coworkers behind them. They could only get hired into new companies where nobody knew any of their past coworkers, because a simple reference check would reveal how difficult they were to work with.

Filtering this behavior out before someone joins the team is very important. Hiring a single socially toxic or subversive person into a team is like dropping a bomb on a healthy team dynamic. You may lose multiple good employees before you figure out what's going on and put together a case for firing the bad apple.

grdgyredb · 2 years ago
As long as you aren’t illegally discriminating, it’s more important that you avoid hiring shitty people than it is that you eliminate all bias.

You can’t single-handedly solve the systemic problem of unequal access to opportunity.

dzikimarian · 2 years ago
You put bias next to stereotype like it's the same and taint it with bad connotations. I might have team where talkative person can be good fit. Other teams may consider such trait distracting.

You're choosing companion for 8h a day. Nothing wrong with checking culture fit (because that's basically what it is).

th3byrdm4n · 2 years ago
I think "cool" is synonymous with, "would they be a good fit for your team/culture" ... not a specific type/race/ethnicity/political persuasion...

Are they someone that's going to disrupt your team's focus on product and execution. No? Cool.

946789987649 · 2 years ago
Have a diverse team and have them also score for cool?

Of course a bit of a chicken and egg with hiring the diverse team.

bigbillheck · 2 years ago
> People's intuitions around hiring aren't Bayesian enough.

People's intuitions around hiring are extremely Bayesian, which is why we have all sorts of laws and regulations and HR departments that try to counteract various prejudicial priors.

s0rce · 2 years ago
This is pretty much how I hire but also try to gauge "conscientiousness". Some people are smart, cool, high energy and decently experienced but just don't seem to care about the success of the team and aren't the best for a lot of roles. Its hard to judge because extraversion/cool/high energy can appear similar in an interview.
Shocka1 · 2 years ago
Breath of fresh air to see this at the top. The focus on leetcode always confused me, and this is coming from someone who took all the fancy graduate level algo courses.
babyshake · 2 years ago
Is "poison in the keurig " a euphemism for spreading bad morale?
mixermachine · 2 years ago
> who happened to spend their summers working

It is a super cool story to hear in what jobs somebody already has worked in and a good indicator. I worked as a dancing instructor assistant and learned so much about people. This job does not correlate with my current job as a Java/Kotlin/Android dev but was taken quite well by everyone the interviewed me 10 years back.

You just have very little time during a job interview and mistakes are costly.

> Are they smart? > Are they cool? ... Are the right questions that need to be answered, but not asked directly. The article talks especially about what to ask to get meaningful answers.

j7ake · 2 years ago
What you state do not seem like priors at all. They look like observed data.

A prior would tell you how much each of those attributes would change your viewpoint.

Having very strong priors actually means your decision won’t change unless you have very strong evidence on the contrary.

randomdata · 2 years ago
> adding more boxes tends to index towards box-checkers who grew up wealthy.

Yeah, but that's the point. Those who come from wealth bring connections. Connections are what make or break a business. A mediocre worker with rich parents is far more beneficial to the business than a standout worker that came from the slums.

throw18376 · 2 years ago
i think that in the United States, this is simply not the case up until the true super-rich.

the vast majority of rich people are not well-connected enough to drive any business your way. if you have a tech startup do you really care that you hired the son or daughter of a car dealership magnate from the other side of the country?

in the USA it's possible and in fact more common to become a very wealthy without becoming hugely influential.

they may be well-connected in their particular geographic region or within a certain business sector, but this would be a specific advantage which usually wouldn't benefit your business.

this may be different in some other countries, where nearly everyone wealthy is part of the same national elite group.

but generally, in America, you'd be better off hiring the better employee.

tptacek · 2 years ago
We built Fly.io resume-blind and without interviews, hiring people at every level of experience without having to make decisions based on that experience. We did it by throwing away all this stuff, ditching interviews, and replacing them with work-sample tests. Some of the best people on our team, the best I've worked with in my whole career, are working here in their first job in our field.

I'm a little grumpy about this "diamonds in the rough" shit. I'm more concerned about what's lurking in the diamond mines. If people can demonstrate that they can do the work, I don't much need to know if "they have a chip on their shoulders". More generally: I have zero faith in anyone's ability to learn much from psychological interviews.

swozey · 2 years ago
So you communicated via email and the only consideration was the quality of their sample?

Nobody at any point literally spoke to this person?

We've had a ton of issues at my corp of people having other people handle their interview questions, code, or having AI listen to the conversation and answer questions by text as if we prompted it. Then they show up online after being hired and they have webcam problems or never go on cam, talk as little as possible, basically never communicate to anyone really, miss a lot of meetings, etc.

We've had probably 5 instances of this last year we're absolutely super strict about things now. Our assumption is that they want to get in, potentially get any bonuses if there are any, get a few huge (for their call center) paychecks until they get caught. Do no or almost no work. And it's a call center you can sometimes hear tons of convos going on in the background so they're just doing this en masse.

This is happening ALL over since covid. I think some of them may have even filed unjust termination sort of things to the workforce commission but I may be wrong. Huge companies are dealing with this at a really rough scale.

tptacek · 2 years ago
Our process runs mostly through email until the last challenge, which is interactive (for every role in the company we simulate some kind of design or problem solving exercise on Slack with an random member of our team, which is then rated based on a rubric by a panel that reviews the transcript; it differs from an interview in a variety of ways, most notably that the team member delivering the exercise doesn't ever interrogate the candidate).

Most of our work sample stuff is design/architecture based. I'm not worried about how LLMs impact it.

This is the sole way we've hired engineers at Fly.io since 2020 and we have never once brought someone on who turned out to be 3 raccoons in a trenchcoat. Raising the bar further on ourselves: we hire globally, at SF scale salary worldwide.

strich · 2 years ago
In a world where chat gpt exists I think even work samples are untrustworthy sources of excellence unfortunately. It would seem one has to either watch the process or devise unique and difficult to game tests.
tptacek · 2 years ago
Work samples aren't all straight coding; in fact, only a small subset of ours are.

Later

I want to add: I fully believe LLMs can reliably knock out the code for most programming challenges, but a good coding work sample is as much about what you choose to code as it is about the code itself.

If you're asking people to invert binaries trees or whatever, that's not a work-sample test, it's a coding quiz. Work-sample tests mirror the actual work you're doing; that's what makes them predictive. Writing a function to do X, Y, or Z is usually the easy part about building software; the hard part is deciding which X, which Y, and which Z, and how they fit together.

We didn't deliberately set out to design work samples where the decisions we're grading are things LLMs don't just do for you; we do a lot of systems programming work and those kinds of decisions just dominate systems programming.

xena · 2 years ago
As a fly.io employee, I can tell. I can tell fairly quickly.
jefc1111 · 2 years ago
I don't know if I would hire like this, but I do think a lot about the current hiring landscape is pretty broken, so maybe I should! I expect an employer needs a certain amount of cachet before being able to do this or anything else similarly left-field.

Side note - I started down two paths of significant new learning early last year: Security & Elixir / BEAM. I love Security Cryptography Whatever and had no idea you crossed over into the Elixir world (fly.io). Cool!

tptacek · 2 years ago
I've been at Fly.io since 2020. I wouldn't describe us as an Elixir company so much as a company that appreciates Elixir. Some of our backend is in Elixir but the majority of it is Rust and Go.
i_h8_lc_grind · 2 years ago
This is good to know. I think a lot of people are tired of wasting their lifespan on grinding lc-medium/hards.

Is there a website with a list of companies that don't participate in the leetcode/FAANG interview process? I feel like there is a need for one.

em-bee · 2 years ago
i seem to remember an HN post about such a site, or at least one about a site that evaluates companies hiring processes. also related, about companies development practices.

try searching the HN post history. (and report back if you find something :-)

gwbas1c · 2 years ago
Just remember that all interview techniques have bias; what works well for one company / role may not work well for your company / role.

That being said, it sounds like your interview technique works well for you.

firstplacelast · 2 years ago
I started an interview at fly like 1.5 years ago.

Figured I’d give it a shot, they were very responsive, gave me the challenge and I spent more time than was suggested. They had a few core technologies I was not familiar with at all, but figured I could hack it…I could not.

Anyway, I really appreciated it. Walked away with nothing but positive feelings.

I have spent enough time applying/interviewing over the last decade that I am often in a dilemma when I get an offer that I don’t want bc the process is so fucky/I already have negative feelings. From HR setting expectations and then not meeting them, the laid-out interview process not lining up, remote/hybrid changing, inability to describe actual projects I would work on, not meeting people I would work under/with, made to feel like I lack intelligence by interviewers, etc.

I’m not a “rockstar” programmer and never have tons of offers, but I have turned down offers because of screwy interview processes/interviewers just to stay at the evil I know.

tptacek · 2 years ago
That may be, but I don't believe interviews work well for anybody.
BadHumans · 2 years ago
Is this how Fly still operates? If you're hiring Rust developers, are you open to someone who has never touched Rust in their life?
tptacek · 2 years ago
We wouldn't know if you've never touched it, only whether you can solve problems in it today.
mplanchard · 2 years ago
Do y’all have any blog posts on this? Would be curious to know more about the work sample tests, and about the process in general.
geor9e · 2 years ago
Plus you get a bunch of free work out of every applicant if you delegate real issues out as work-sample tests :P I'm pretty sure the new Audi A8 cup holders are from my mechanical design engineer third round volkswagen interview a couple years ago
gwbas1c · 2 years ago
It's nearly impossible to scope useful, shippable work that can be accomplished in a job interview. Otherwise, you're asking for 20+ hours of work, which means that only fools / suckers will do your interview.

Do you really want a company full of fools and suckers?

tptacek · 2 years ago
You do not in fact get free work.
noamchomsky1 · 2 years ago
> “Tell me about you. If your life was a book, give me the chapter titles from your birth till now.”

Utterly ridiculous.

Trasmatta · 2 years ago
> Once you’ve gotten the overview, dive into each “chapter” and plumb the depths for their real stories. Go back to their childhood!

Oh God. A job interview isn't a therapy session. Why would people ever feel this type of questioning is appropriate for a job interview? How would someone who's dealt with intense trauma during some of those "chapters" respond? Especially given the power imbalance in a job interview, and the pressure you feel to give an answer.

I'd like to believe that I would be willing to politely decline to answer questions like this, thank the interviewer and tell them there just seems to be a mismatch, and walk out of the door. But I don't know if I'd be able to, due to the stress of a job interview.

stefanos82 · 2 years ago
This reminds me of an interview I have had in the past that made me very uncomfortable.

They asked me the same question and I paused for a moment and replied "excuse me, but this question makes me feel uncomfortable; I'm here for an interview not for a date!".

They realized they overdid it and asked me actual questions.

Eventually decided I was the right candidate for them, but until they decided to reply back, I already had found job elsewhere.

Why do they ask such type of questions anyway?!

beretguy · 2 years ago
> Why do they ask such type of questions anyway?!

They are incompetent at doing their jobs.

reportgunner · 2 years ago
I'm getting red flags from this guide. "Tell me about your best X" is a very low effort question. Being excited about digging through person's life like it's an open book and even excited about them explaining how they cared of their dying parent is perverted at best.
SketchySeaBeast · 2 years ago
"Well, I like to call chapter 7 'The Fuck Pig'. It'll make sense in a minute."
kdazzle · 2 years ago
Dat you, David Cameron?
trgn · 2 years ago
Really condescending way of talking to a person.
karaterobot · 2 years ago
> “Tell me about you. If your life was a book, give me the chapter titles from your birth till now.” Once you’ve gotten the overview, dive into each “chapter” and plumb the depths for their real stories. Go back to their childhood! I learn a lot about their grit and commitment to excellence from their basketball obsession or maybe their experience caring for a dying parent.

God damn that sounds exhausting. How about let's skip to the current chapter, titled I'm Good At Computers And I Want A Job.

varjag · 2 years ago
She's clearly isn't hiring developers.
karaterobot · 2 years ago
Replace the subject matter with that of your own job, or pretend you are living inside the article and respond directly to the author if that feels right to you. Whatever gets you to seeing my point.
garciasn · 2 years ago
This is exactly what I do with my team. I staff the upper levels first with people I trust (people have worked with me for 15 years across 4 different companies now) and then start fleshing out the rest of the teams beneath them with high potential n00bs.

Why? Because that’s what I was when I started and now I’m paying it forward.

It’s been a very successful model and continues to be.

dojorno · 2 years ago
hiring?
dkasper · 2 years ago
Some of these questions may be borderline illegal. In my experience interviewing folks for Meta we are taught not to ask such questions because of the obvious bias. Even probing into what neighborhood someone lives in is dubious (oh you grew up on the wrong side of the tracks?). I don’t even look at résumés anymore, although recruiters have to screen them. Focus on the job, if you are conducting a coding interview asking coding questions, maybe ask about something they built in the past for fun/work/learning . If you’re doing system design ask them to architect something. If you’re doing behavioral interviews this is trickiest but focus on the on the job behaviors, even if their previous job was not tech or they only have educational experience you can see how someone works with others on a project. Personal questions like this are a really bad idea.
duxup · 2 years ago
I like that this isn't your typical skills demo.

I changed careers at age 40+, learned to code, now years later enjoying it. It just took someone who thought I could grow.

Interestingly I was hired along side some capable, albeit green, college grads. The difference in terms of understanding how a business works, speaking to customers, asking questions / follow through to get down to the problem we're solving was enormous. They could code circles around me, but they also just wanted to be told exactly what to code and had no interest beyond exactly that.

spaniard89277 · 2 years ago
I'm 36 And also trying to get my foot in. It's very hard, wish I could find someone who gives me an opportunity. I've been interviewed but it's always about the skillset you bring to the table.

Can I throw you against a codebase and be productive quickly? Do you already have experience with this obscure library? I want to build now a website using React + Function Calling + Puppeteer, IDK what else to do to impress people.

Same happening with my gf, who is a graphic designer + UI/UX. She has some experience but only as freelancer, she's never been hired by anyone because it seems never enough.

I teached her HTML + CSS and she's been able to do this by herself (https://sofialenti.com/) so IDK what more a designer can learn. The only thing that's left IMO is to learn JS and be able to code some awesome stuff, but at that point she would be almost an unicorn.

It seems the market is just too saturated, despite the frequent news about we need millions of people who can do X! It seems like a way to get cheap and disposable labor.

geraldhh · 2 years ago
> It seems the market is just too saturated, despite the frequent news about we need millions of people who can do X! It seems like a way to get cheap and disposable labor.

seems a good bet to stay clear of industrialized work environments if one doesn't want to be cheap and disposable labor.

chefandy · 2 years ago
Ambiguity and uncertainty are anathema to approaching problems from the classic engineering mindset. That's not a bad thing when used properly, and it's great in school where people hand you a stream of problems designed to be interesting in that mentality, but it's not the general-purpose tool for interfacing with all problems, as some imagine. I've worked both as a developer and designer, and with design tasks, you've only got a fuzzy sense of the wrong answers before you manage to craft a right one. Many, if not most engineers find it much easier to assume designers sit around all day moving things about as they fancy rather than imagining what it would take to solve ambiguous problems with no way to determine if you've got a really good solution until many people start interacting with it.
griffinkelly · 2 years ago
Having worked predominantly in startups, we've almost always been cash-constrained in hiring. Some of the best hires I've found are highly motivated fresh grads or recent grads. I always look for the desire to prove themselves and take charge and full responsibility of a project; a majority of the times, money is secondary to these folks to the ability to make a difference and prove you're capable of completing something difficult. And ultimately, that's always something a startup can offer a young employee. That said, I've often had to teach them foundational things, but the desire to learn and get moving quickly outweighs any cons.

When I've been at big companies, it's all about experience and grey hair, and people become more motivated by money and low risk. I find many times, the quality of the average person at a big company is lower than the average startup fresh grad.

rqtwteye · 2 years ago
"I find many times, the quality of the average person at a big company is lower than the average startup fresh grad."

I wouldn't blame it on the people but on the environment. I have worked at both big companies and startups. In a lot of big companies it's actually quite hard to make a difference. There are lots of people who can say "No", raise concerns, ask for plans, but not many people who say "Yes". So after a while people learn that it's not worth the effort.

faiD9Eet · 2 years ago
I would like to second that.

Big corp introduces a constant uphill battle and people get minted to avoid conflict (Why do you want to spend money on a subject that is not on your bosses boss roadmap? Does this new service obey our IT-compliance-rules? I know a virus scanner on Linux is a bad idea, but compliance demands it. I do not care about your threat model, have you installed one already? Can you spend 30,000 Currency Units, but have it billed in November, accepted in December, and paid out in January next year? Answer me until end-of-business!).

People want to have an impact on their environment and conflict is the wrong way to start with.