I'm in the process of interviewing for senior and staff front end positions. I've done several multi-hour take home projects now and a couple were rejected with generic rejection messages. It's pretty insulting to be frank. Spending 2-4 hours on likely valueless work is a substantial amount of time relative to the week day. If I've spent the time making the project and they've spent the time recruiting me and reviewing the project, the recruiter and reviewer could spend 5 minutes sending constructive feedback.
Anyone else share in this frustrating experience? Have you successfully asked for feedback before?
As far as I remember, I was told it would take a few hours, maybe 4, but the assignment looked rather fun, so I thought, whatever, let’s do it.
Maybe I didn't understand the assignment fully or missed some cues; I don't know. But it took me roughly 10 hours spread across two days. After 5-6 hours, you don't feel like throwing the work away; you just want to finish. It was pretty frustrating.
After returning the assignment, I waited the two weeks they asked for and heard nothing. I sent one email, waited a week, got no reply, sent another, waited some more, and still received no reply. I ended up sending a handful of emails to various addresses I could find. I even sent a DM on Twitter to one of the founders to let them know. No reply anywhere.
Overall, it was a pretty bad experience.
The job listing promised mountains and that it will take few hours and they’ll take time in their feedback,etc.
except i just got a generic reply that “we’re not satisfied with the solution”. For something that took ~10 hours, i was at the least expecting a vague pointers on whys. I even sent an one pager explaining style decisions, caveats, etc. It was pretty insulting.
And when asked for a follow up, got a more generic bs about how the evaluation criteria is honed from their years of experience and that is not something they share outside.
One good thing that is it made me realise interviewing still sucks and i just stopped looking for jobs.
These should not take 10 hours, unless you're learning Rails from scratch or something (some people do this). We've tried different ways of saying "don't spend more than 2 hours on this unless you really want to" and it doesn't always come through.
Was it my issue waiting a short time to start it? Maybe, but you’d think they’d at least try to answer a question or two. Oh well, I just gave up because if they weren’t going to put in any time, why should I?
I doubt anyone will read frustrated comments and my responses and think "oh boy, I didn't want to apply there but that one guy responded and made me change my mind".
---
Unfortunately, we’re not going to be moving forward with your application for the backend role at this time.
The first thing we want you to know is that we got a lot of applications for these roles. Like, a lot a lot.
The way we evaluate candidate submissions is that we’ve built, over a year or so of running this challenge, a written rubric for what the strongest submissions look like. We started with something sane looking, and then iterated over time as we got a sense of what candidate submissions actually looked like.
We score submissions according to that rubric. Different members of our platform team score different applications; they’re just jumping in and looking at the code and making grading decisions. In the pool of candidates we’re evaluating right now, your code submission missed our cutoff.
A natural thing to want from us next is a copy of the rubric, or specific information about what your submission was missing. We’d want that too! But we can’t give that to you, for (at least) two reasons:
(1) The outcome on your code was a weighted combination of a bunch of different factors, so there isn’t a simple answer that doesn’t just dump our whole rubric to you.
(2) We want to preserve the opportunity for people to apply to these roles in the future, and spoiling the challenge would break that.
What we can tell you is that the most successful submissions had a combination of these attributes:
- Heavy user focus. Successful submissions explained how the system could do good things for users. - Straightforward database models that make the queries we care about fast to run. - Transactionally safe sync with Stripe. For example, the best submissions mentioned that Stripe API requests had to be idempotent to ensure the user is not charged multiple times.
We sincerely hope you stay in touch and re-apply in the future. In the meantime, we'd like to give you some Fly.io credits to play around with! If your email address here matches your Fly.io email, click here to get them. If not, please let us know which email you use for Fly.io and we'll set that right up.
Please feel free to reach back out to us about other roles, or about this role in the future. Thank you, again, for doing this.
Correction: It's not profitable.
They claimed it would take only a couple of hours but for the 15 or so bullet point requirements that was simply not realistic. Spent a couple of days on it and the only feedback I got was no thanks.
They had previously stated they pay for every stage regardless of outcome. After sending my details they never paid me.
Feels vastly hypocritical given their stated people first work culture.
Don't get me wrong if you had fun (and I enjoy them too) nothing wrong with that.
I also use them as starting points for the next take home assignment and even live interviews (I’ve been able to keep the project open in the IDE on a separate monitor and copy and paste code even when doing a camera-on + screenshare interview)
Here's another: https://github.com/fly-hiring/platform-challenge
I'll make sure we get you a response within the next week.
Feedback was something like "Assignment is private and you can't store code on GitHub, you should send zip archive"
The most fun task I had so far, the feedback was pretty good as well. Kudos to tptacek for that.
Maybe they had bad experience with sharing feedback directly?
If they said it would take 4 hours and you spent 10 hours on it, it’s likely your approach was ineffective compared to a more skilled engineer.
They promised this person if they completed it, they would get an actual human response back.
Not only did they not get a response back, they didn't get a response back from anybody, even when directly trying to follow-up.
And this was what was promised for spending 4 hours of your time on something.
At the very least, that seems unfair.
Long story short, the company passed on me, saying that my code was easy to understand and high quality, but I didn’t handle all of the corner cases. I listed most of the corner cases they mentioned in the readme as limitations from the time limit. It was clear that while they said to limit my time on the project, they didn’t really mean it. They wanted an exhaustively finished product in a ridiculous amount of time.
At this point in my career, I’ll just decline interviews that don’t respect my time.
I obviously noped out, but am still wondering if they are aware that all the people working there 100% cheated on their take homes.
Either they don't care or they see it and figure I must be some sort of whiz. I say that because the amount of people that don't know how git works or that file attributes are arbitrarily changeable is amazing.
And no not specifically to "cheat". Just on principle. If they require BS I give them BS. Else it's just fun. Sometimes the people actually hiring/doing the interview are also not the ones that care for this but HR/senior leadership only.
I wish I had saved a copy of the assignment. It was ridiculous.
The worst part was that after working there I found them to be terribly unproductive and it would have likely taken a team there a week to build what they were suggesting...
But yeah as you said, either ignore the time constraints if you really want the role or just politely decline due to the red flag / inconsiderate nature.
And I'm like.. an hour is not long enough to implement any reasonable webapp... I have no idea if other people went over or not, and it almost feels like a test of commitment (if I was serious I would take the actual 4 hours it would take and pretend I did it in 1 or something like that).
Given the diversity of FE it’s quite likely for a take home project to hit one of those criteria.
"we have a takehome, but don't worry! it's just a quick little thing that should just take an hour to throw together. we're not like those other companies that give you a huge task"
1. Other candidates were clearly spending 8-9 hours on the problem. I saw their commit logs. A commit every 1-2 hours for 9 hours. Several candidates public repos were like this. The assignment was like 2 pages of requirements supposed to be done in I believe 1 hour.
2. They told me (thru the recruiting firm; I never spoke with them) that it would have been resourceful to use the other candidates solutions.
> ... as I wasn’t paid for the interview and I didn’t feel it was fair to me to spend any more time on it than that.
I have been paid for an interview, a coding interview where I fixed a bug and added a feature while my pair (future colleague!) chilled .. I think he surfed and I'd ask him clarifying questions occasionally? Anyhow, I got paid like $400. Best interview ever.
This is my philosophy now. In my younger days when I had no experience I had a lot of my time wasted by recruiters and companies - never again. Now I ask up front and politely what the interview process will be like. If there are any home projects or mentions of whiteboarding I end up passing. No big deal - I know there are probably loads of developers willing to waste their time competing for the position, but not me.
It's worked very well for me and eliminated a lot of stress when I've been looking for other positions. The only negative I can see is the strategy wouldn't work great when in between positions and really needing something, which luckily I've never had to experience. I suppose in that case I would force myself to go through with the hazing process.
We don't fail take homes because of edge cases though. There are things we expect people to handle in the challenges that are important, but I don't think more/less time really influences those.
Those are standard enterprise CRUD salaries. I’m not trying to be demeaning. Those are the kinds of jobs I had most of my career until 2020. But they are a dime a dozen and I definitely wouldn’t do a take home test for one.
Honestly 2h from scratch is a rush to do anything? However simple it is, you're asking for an 'entire' greenfield project, of course I have skipped things and missed corner cases.
I’ve passed interviews at demanding companies like hedge funds, Google, Facebook, Snap. Not once have I “passed” a take home interview even when I know I got the solution correct. Sorry, but I refuse to do them now. Companies need to be able to show a token investment in time with an actual engineer interviewing me for me to be convinced they are actually considering me as a candidate and didn’t just send me a takehome link without any due diligence because it costs them nothing to do so.
A nice way to say this is, "I'd be happy to complete this takehome assessment, but it will take a significant amount of time and first I would like a short phone call to ask some questions and get a feel for the company, nothing too long, I just want to make sure we're all on the same page before I spend my time on this. When can we schedule a call?"
> I would like a short phone call to ask some questions and get a feel for the company
I've had some places do behavioral first and others do coding first. I think I'm more onboard with behavioral first because communication a lot of times is more important than raw technical skill. Everyone can save time if you can suss out early on that the person is a bad communicator or bad fit. And generally you can spend time in this interview asking questions about the company.
Unless you are the top company in your bracket, your completion rate will also not be so great for a take-home (people have limited time: they'll sort by how good your company is). Sure they'll do Google's take-home but local business that does local comp/prevailing wages? It's at the bottom of the pile. By the time they get to it, the best candidates will already have an offer from a better company.
That's why I prefer to bring people on-site as soon as possible, and do a little whiteboard (the point being that the candidate must... know how to code!). It's an artificial challenge that's really a pretext to have a technical discussion.
Only time I advise to use take-homes is when you are dealing with a massive number of applicants from institutions that have a poor signal to noise ratio. Bootcamps come to mind. But then you end up with candidates that have the homework done by someone else and it ends up not being a good filter either.
I don't know, perhaps it's because I've progressed every interview I've had the opportunity to do a take-home on (and I struggle with most leet-code style whiteboarding interviews) but I've always seen take-homes as an audition for your software development style -- what you deliver in a take-home assessment is what they will use to judge how you'll deliver on the job, plain and simple.
I've only done take homes for companies that I've already passed initial interviews for and generally like the people I've talked to and have favorable impressions of the culture/workplace thus far. I wouldn't do a take-home before that, but I'd do a take-home any day before a leet code grinding thing.
Few months later I got a call asking to interview again as they realized their standards were too high. Not a chance.
I wish people were less surprised when we tell them they deserve to be paid for their work, but companies have been advantaged in the hiring process for so long (excepting COVID) that disregard for people’s time is embedded in corporate culture.
However, to the point people have made about not getting comprehensive feedback: it’s just not worth it. Corporate counsel will basically refuse to let you tell a candidate why they were rejected for fear of any inadvertent lawsuit exposure. This is also deeply embedded in corporate culture and it’s hard to imagine it changing.
Twice I've aced these challenge with time to spare and was then rejected or ghosted.
It's the same idea; don't invest time into a company's interview loop unless you know they're investing something too.
I had this experience a few weeks ago with Wolverine Trading.
It seemed clear that they didn't respect my time. Perhaps I dodged a bullet.
I have had the exact same experience. They have always been a complete waste of time for me and I have since adopted a policy that any takehome challenges = automatic close the loop for me. It's not worth explaining why either. If this is your process and you find it appropriate to offer it as a first option, then it's just not a good fit.
Not joking. You hated interviews before? Get ready to rock.
As others have mentioned, I find the "limit yourself to 2-4 hours" BS. I can stand up a new http server in 2 hours to implement some API, sure. Adding unit tests, integration tests, a readme, build instructions/scripts, containerizing it, adding detailed comments, linting... generally not possible to do in 4 hours unless you have everything set up before then (I can't use my normal dev tools to do takehomes because they're through my employer). IRL you would generally scope this as a week-long task even for experienced employees.
Maybe if I were unemployed and desperate for a job I'd suffer it, but most places I'm interested in working at ask whiteboard/design problems anyway.
There is a risk here for something that even really strong intermediate programmers fall into where they take a very prescriptive approach to programming so they tend to not understand things built from first principles and also tend to overvalue things like future-proofing and abstraction. Since you don't mention any specifics it's easy for people to fill in the blanks here and assume that at least a good portion of the things you refer to are those kinds of things.
In my experience as an interviewer fishing for specific answers is what makes you miss the best engineers.
I used to do this during my peak Dunning-Kruger phase and gave relatively poor marks to an guy who later proved to surpass both me and the rest of the team in every aspect that mattered - even though originally he barely made it through the interview process.
Chiefly, he didn't do things the way we expected because he prioritized.
Deleted Comment
Got rejected a week after submission with no explanation. It was a really good problem and I had fun solving it and it satisfied all the requirements correctly. When, disappointed, I asked them for feedback, the recruiter simply said that they "felt that it fell under their expected level" and that "The team has a very high bar set for this role and have made it a point to make sure people who join the team can contribute immediately which unfortunately means tough decisions like this have to be made". They didn't share any specific feedback on my code despite my following up for more specifics. Really pissed me off.
As a wise man said, it is possible to make no mistakes and still lose. That is not a weakness, that is life.
Dead Comment
You probably dodged a stressful position. Companies or teams that can’t allocate time to onboard talent are likely poorly run and can’t afford to invest in people. “Contributing immediately” is a fair indicator of that problem.
There are so many hard problems in this industry. Onboarding is not one of them.
Typically what I have seen is that management is so busy striving for short-term deadlines etc. that they "can't spare" engineer-hours to help the new engineer properly. Heck, I've been one of the senior engineers that has not had sufficient time to spare for the new people - not by my choice. In other situations we've had new engineers inserted into our teams with zero notice -- so we've not even had the time to prepare some "day 1 wins" for the new person.
It seems incredibly obvious to me that great onboarding is a true force multiplier.
Last place I work paid a lot of fake lip service to the importance of onboarding. But the reality is that we were judged strictly by the numbers in terms of commits, Jira tickets closed, LOC, etc.
Word - I never considered this possibility. But now that you mention it, the hiring manager I spoke to had this smug swagger about him and a condescending sneer about my current company and open source in general. Not good vibes at all. As a member of his team, I might have placed myself in a position that is bad for my mental health. Thank you for sharing this perspective, really helped.
Believe it or not, I've also gotten takehomes which took multiple hours, involved some non-trivial data retrieval and management, and gotten rejected with less effort than it took them to prepare the ask.
Later I had a sinking suspicion that it was not a real position, what it was was three brogrammers who found a way to get unemployed rubes to write code for them, for free, so they could deliver it themselves without having the requisite talent.
Are you positive that didn't happen to you too?
The company I got the take home test from is a well known and established leader in its domain - not anything like the one you described above.
Got a "Thanks, you're not moving on" message and that was it. Man, the rejection is totally fine, but I really do wish they could've spent a few min just explaining some details. I've gotten more thoughtful rejections from just cover letters and resumes, not to mention public PRs in strangers' repos.
After spending so much time, it just feels like a betrayal of some unspoken developer ethos (vs say talking to a generic HR screener). If you're going to make someone code for you for hours and then dismiss their work, please at least tell them why in just a few sentences. You don't need to comment line by line in a code review, just general thoughts like "better code org" or "poor architecture and readability" or "better tests would've caught this major bug" or whatever.
In my case this was a small company I was super excited to work for, and waited more than a year to finally have a chance to apply for (once my current job ended). It was definitely disheartening and makes it hard to want to try again with them in the future.
But, you know, the other side of the coin is that maybe they're just getting swamped with so many applications they can't take the time to thoughtfully answer each one. I imagine I'm competing against a horde of more qualified ex FAANGers right now and maybe they're too busy trying to decide between the top 3 or 4 vs the long tail of hundreds of us who failed the take home. Who knows...
Maybe. At the point in the process where you're seeing someone's project, you've already spent time probably phone screening them, sending them the project, hopefully reviewing it. It takes relatively little time to write a quick sentence or two in review of the assessment. It would be nice even if they didn't look at the project and just said "sorry, we didn't have time to look at your project because we're moving forward with other candidates".
Why abuse? Well, let's say they have three solid candidates. They ask each to do an assignment. Would I do a 6-hour project for a 33% chance at a job? Maybe, especially if I really wanted that particular job.
Now let's say they have 30 candidates that they ask. Would I do 6 hours of throwaway work for a 3% chance at a job? No - not knowingly. If I'm an average candidate, I'm going to have to do that 30 times to land a job. That's 180 hours, or more than four full-time weeks of throwaway work. That's an abusive process.
Could I spend just as much time interviewing? I could, but there's a difference. If I'm interviewing with you, you're there talking to me. You can't waste my time without wasting your own. Whereas with a take-home assignment, you can waste my time but waste little or no of your own. As a result, interviewers (usually) pay some attention to not doing needless interviews, but pay less attention to not asking for needless take-home assignments.
Not just that: At every job where I interviewed candidates, HR expects some kind of feedback beyond just "hire"/"no hire". At least some bullet points of highs and lows, red flags, etc.
If nothing else, this helps protect the company from false allegations of illegal discrimination by having documentation for why a particular candidate was rejected. It also helps recruiters to know if they're finding candidates who are "close to what we want" or "not even in the ballpark".
Now, I know that some interviewers can be fairly blunt in their feedback, so you'd not necessarily want to just copy-paste it to the candidate, but the point is: that feedback already exists. The recruiter/HR just needs to (maybe) sand off some of the rough edges.
Well, it's a good lesson to have gone through though. If I'm ever in a situation where I am asked to evaluate others in a similar fashion, I'll be sure to leave detailed feedback (if they want it), now that I know what it feels like.
Speaking as someone who absolutely hates live-coding something during an interview (unless, maybe, by chance it's a practical exercise ... something like "build up this simple CRUD web service" ... not "solve this leetcode problem" ... but this is pretty uncommon in my experience, unfortunately), I rather like take-home assignments. As long as they don't take any more than 1-3 hours. I've been in situations where I've declined a take-home assignment that was given to me where I estimated it was going to take 10-20+ hours. No idea why any company would think that is reasonable.
For myself, what I hate is when the take-home assignment comes first. Like, before you talk with anyone at all, or maybe immediately after you did the 15 minute HR/recruiter screen. If I get a take-home assignment at that point, I decline.
There's no denying the fact that a take-home assignment is a large investment by the candidate on this random chance to get the job. I'm not willing to make that investment unless I've gotten a chance to talk to at least _some_ of the people I'd be working with at a company to better gauge what the situation is and whether I think it's a good fit for me. Even a 15-minute HR/recruiter screen is not going to do that. So yeah, if they just throw the take-home assignment at me right off the bat ... yeah, no thanks.
This all being said, yes, it is frustrating though to have to go through a bunch of these and get rejected with no feedback, or just ghosted, yeah.
If you ask the majority of job seekers here, the only correct way to hire is to hand out 300K/yr offers after a casual 30 minute chat.
My wife is a lawyer. They don’t send her a take home test, or ask her about her side project cases, or whiteboard bar exam questions. Somehow it manages to work.
I don't know why HN is so against algorithms problems. It is not something I have ever needed to "grind" because I understand ds&a well. While the more exotic ds&a you won't use on the job, and IMO shouldn't be tested for, IME the most complicated datastructures you generally need for both algorithms problems and on the job are hashmaps and hashsets.
which you have to pay them for
I didn't end up getting the job but it was easily the best and least anxiety inducing interview I've ever been through.
Well, that's the issue. It may be because I'm a bad programmer, but I can't think of a take home that took me less than 3 hours. The take home I did for my first job took some 20 hours over 4 days, and I exaggerated and said it took 12 (which didn't garner a response so I'm guessing that wasn't an unusual answer). I could do that while I'm a student, I can't do that again as a working professional without wasting an entire weekend after a week of full time work.
I still hate leetcode more, but at least there you have an explicit timer.
>For myself, what I hate is when the take-home assignment comes first. Like, before you talk with anyone at all, or maybe immediately after you did the 15 minute HR/recruiter screen.
I've never had a test come later. 15 minute interview call to make sure the high level details are correct (pay, location, physical office vs. Remote, etc), and then I am sent an interview test.
Granted, my last 2 jobs did not employ take home tests, so I know I'm not forced to do them. But given my experiences I understand why others would be opposed to them. It sounds like you would be opposed to my experiences as well.
Depends on the company. My personal experience has luckily been (so far) that more companies I've interviewed with who do take-homes at all have given them later on in the process.
I suspect with the current job market situation given all the recent layoffs that things will probably start to shift for those companies that do take-homes, where they'll probably start giving them to candidates at the beginning of the interview process, as they'll probably see it as an easy filter for themselves. Ugh.
I use to brag like you, about “not doing take home test” for 25 years. Then I landed at BigTech and saw that returning interns were making about what I made two years earlier at 45.
I’m not complaining, my goal had been to get into $BigTech in 2020 and relocate when my youngest (step)son graduated I did so without a coding interview and without relocating by pivoting to “cloud consulting/application modernization” (cloud + enterprise application architecture/development). But I tell my younger relatives to practice coding interviews and go for the most compensation possible.
Oh, but onto your questions. So yes it is frustrating. And as far as feedback. I stopped asking for feedback because it was always maligned. Nobody will really know why you didn't get hired, remember, these companies aren't really hiring unless you are young and cheap. In most cases my feedback sounded like it was almost for someone else. Like "needed more linux experience (I haven't touched a windows or mac OS since 2005). Or my favorite, after talking about a data warehouse I built to house 5TB of data and 15 billion rows, and all the different schemas I migrated through, their reason was they wanted someone with "more database experience".
Nah, I think there are more factors at work (Also, cheap is relative, therefore dependent on other factors, and therefore somewhat vague)-- its a multivariate situation where you, and the many factors you bring, flow into a process and the many factors it brings. If enough of those factors line up enough-- you get hired.
If it takes 18 months to get hired, I'd analyze the factors you bring. Such as: Web portfolio & Online presence. Network. Resume posted on multiple sites. Appearance & Demeanor during interviews. Skills/Years Exp/Qualifications relative to the role's stated requirements. Etc.
I'm guessing this is not an insignificant part of it.
I landed my first job at BigTech at 46.
I changed jobs 6x between the time I was 34 until I was 46.
I doubt that I’ve done more than 40 interviews over 25 years between 8 jobs.
If more engineers start to refuse these kinds of "dance, monkey, dance" processes, they'll start phasing them out.
Then the hiring manager came back, "all looks good! I need you to interview with the team, the CEO, the other team" and what not.
I declined and said that I'm not spending 10 hrs in a process you asked me for (as in I didn't apply, they reached out to me). I got an angry email saying "what do you mean? We have only met with you for two hours?". I pointed out that it takes time to do the tests, then nothing more than crickets from the company. Not even a thanks for your time.