As an applicant, I've personally had a mixed experience with take-homes. We're designing the technical interviews right now, and I thought I'd ask for some stories on exciting take-home tasks. We're looking for something that resembles the actual job, so we'll allow any tool (including AI and debuggers).
Also curious to hear about any bad ones you've done.
The worst one was about 15 years ago. The task was to design and implement an ML system to solve a complex network optimization problem. It was so complex that I was given three months to do it. My estimation was that I could do it in three months if I worked on it full time, which was clearly a ridiculous ask.
The interviewer rejected it and said he wanted working and tested code, which I estimated would take nearly 100 hours to do.
I told him I wasn't interested in working for free, but I would gladly do it for my normal consulting fee that I was charging at the time.
Didn't hear back from him. Dodged a bullet there.
Lol! That's quite ridiculous.
I have probably done about 6 take home assignments in my 14 year career and 5 resulted in a job offer. In a recent company I worked for I was the hiring manager and we had about a 40% assignment completed to offer ratio. Which seems like a fairly decent conversion ratio, from the average candidates side that would be close to 1 offer for every 2 assignments.
Is there a correlation with people hating on them because the ratio is skewed much less favorably? Do I have too subjective a view? I know there are companies that give an assignment as practically the first step in the process; but I would never engage with that since the risk/reward is weighted entirely in the company's favor (which under the surface, is an even bigger red flag).
My favorite take homes, and almost always what I have come across, are the "build a mini api" ones. I will typically just draw from the structure I have in place in other applications, implement the custom functionality, and do a readme that delves into the philosophy (in a light touch way) of the choices I make. Then the conversation of going through the implementation in a technical interview is generally a relatively relaxed process.
If a potential employer wants to evaluate my ability and style for writing code I have a GitHub profile with numerous projects populated over numerous years. If that is too much trouble then why the fuck would I do a take home assignment for them?
- Recently: Given some basic intermediate output from LLM system, build out evaluation/quality tools, especially for dealing with hallucination - ~8 years ago: Given a database table of restaurants, identify near-duplicates to merge
On the hiring manager side of things, my candidates generally enjoyed doing a take-home to build a ML model for the Li & Roth question classification dataset. That involved a bit more creativity and it was very role-related.
After being on both sides of the interview process, my current feeling is that take-homes are a decent fit for junior roles but take-homes are not a good use of time (on either side) for senior roles.
It was up to you what you wanted to do / how / if you wanted to deploy. They only asked that you write how you would've liked to receive such service from another org.
Then in the day of the in person interview we reviewed my code / README / tests ; was very nice.
That speeds things up no end.
If it isn't worth four or eight or N hours of your staff's time to observe the candidate working on the problem, then it is probably not worth four or eight or N hours of the candidate's time.
Because what are you trying to measure?
How well the candidate lies about how long the project took?
How willing the candidate is to work many hours?
How much free time the candidate has to dedicate to their job search?
How well they GGS (Google and GPT and StackOverflow)?
How resourceful they are at hiring a freelancer?
Etc.
Sure those types of behaviors often resemble people's actual jobs, so maybe a take home coding task measures what your company is really looking for.
But realistically, inexperience and desperation are the most likely reasons someone will be excited about your take home challenge. The best skilled and most experienced candidates -- the kind people who know them want to work with -- probably won't be excited. Good luck.
> But realistically, inexperience and desperation are the most likely reasons someone will be excited about your take home challenge
Also, why they'll become more common - used by wolves in sheep's clothing. Selecting for those who are more junior or simply malleable.
Hence my fairly strong stance; we're interviewing 'together' (same time, remote/local/whatever)
At your stage (startup with 2 people), you are much better focusing with designing your product than designing an interview process. Hire someone you know or freelancers.
Architecture level tests are a sweet spot for nos. AI is not good at this, but senior candidates are often very good at them. You can give them template architecture to start with or work on, something that AI is known to trip over. The only way to know what AI is tripping on is to use it.
If the candidate is better at using AI to write code than the team, bring them in!
If a candidate uses AI as a help to solve their task, yet the output is still good, great! It's their job to solve the problem, with whatever tools they have.