I've also done this a lot, as has everybody. My experience is that there are two problems.
One, it turns out I never ever end up doing the actual steps as I planned to. After a few I have new insights, realize I forgot something, see an easier way, I don't know -- the plan is never followed.
The other is that I really hate working like this because it seems all the creative effort of thinking about how to make something is put in the first part. And then, the rest is still most of the work, but it's the really boring part. It's much more fun (and therefore faster, and resulting in better work) to spread the creativity and the boredom out more equally.
The two are probably related, and it's not inconceivable that I have ADHD.
Whenever you talk about breaking down tasks, estimations etc, those usually are for multi-person teams, and/or projects that have some specific budget or something.
Of course if you are exploring or building your own projects and/or don't have any other accountability, you wouldn't start planning that.. unless you are really into planning in and of itself.
But as soon as you have a boss and the boss asks "how long? who does what? where do we start?" - you have to have some framework.
I also have a very high aversion to "conforming to the system", esp. when these systems seem arbitrary or are overly rigid .. but systems in general need to be simple and flexible. Productivity systems are there to help people.
I don't think the author intended for this to be like a detailed recipe.. but more like an example of how he approaches these things, which can hopefully help others have their own ideas.
Holy... It still amuses me how much HN comments brings the reality to these kind of articles.
> the plan is never followed.
Sometimes it's even worst. As we add more tasks during execution, at the end, these tasks get twisted with the ones you created while planning. Then we end up with a messy list of undone tasks that aren't really required anymore (because they were created without having the full context, that only happens during the execution).
This is an accurate and insightful way of looking at work. It touches on something that I've been considering a lot lately: I love programming, but I hate doing it in a work environment. The fun thing about programming is that it is a flexible, flowing, and creative endeavor. You make things up as you go; it is an organic experience. In a work environment, this organic nature is removed mainly because there is a need for oversight and accountability.
I'm a person who hates lists and plans for personal tasks. But, am learning to love them. Because as I often realize, things get missed in the myriad of things to remember.
The plan to me is just a way of reminding my future self what I thought of as ideal output in the past. Instead of infinitely changing my plans and chasing fireflies
Another perspective is that planning is a breadth-first traversal of the solution space, and coming up with a path to the solution. When reality hits and the path is often wrong, one can switch to other paths quickly since the graph was created ahead of time. It's writing the table of contents for a book before fleshing it out.
Without planning, a depth first traversal is a high risk endeavor in the likelihood that the that path is wrong but backtracking and creating the graph is comparatively expensive and susceptible to sunk cost fallacy. Depth-first traversal is writing the book a chapter at a time without a table of contents in mind.
I make lists and break tasks in to smaller tasks, however I rarely reference these lists. They basically end up in the void. The act of making the list gets my juices flowing and reminds me of the side effects / bigger picture of my current challenge.
Realizing this also helps me avoid yak shaving over list making tools, since I could care less about even saving the lists! I keep a folder of text files in any project I'm working to hold my lists, but tbh it's unnecessary to even save them.
I also can't get myself to do it seriously for my own tasks, but when I had to do it for a jr colleague I found out... yeah, breaking it down its hard, as hard as coding - then you get most of the hard work already done, and coding the solution becomes trivial. Chances of false starts go from 80% to 20%.
(A few times there is prototyping involved but I won't share that with non-devs, since they don't seem to get it.)
I am somewhat like that myself. I do formulate a plan and break down tasks in advance, but I give myself an absolute right to change and modify it at will whenever new ideas come to light or just because I'm feeling like it.
Seriously asking : how? I can't cheat my brain into not automatically seeing the next steps after that and the cascading relation between the all. Within a second or less.
> Delivers change because, in a work context, a task only “matters” if something is different because of the work.
I think maintenance tasks may require more effort / more expansive qualification when it comes to the meaning(s) of "something is different".
I think the topic of "breaking down tasks" could very well be its own book genre or even podcast genre. My experience with self-help and organization titles has been that the activity of breaking down tasks is a sort of known primitive human operation that many authors assume their readers to have. However, my own personal experience and the accounts of others I have heard while participating in group sessions is that breaking down tasks can be very difficult and provoke emotions of avoidance and despair.
The most broadly relevant advice that I have come across when it comes to breaking down tasks is to keep breaking things down until I am 90% certain that I can successfully complete the task. The certainty here would vary on how much self-confidence an individual has and their risk tolerance, so some people may break down tasks to 70% certainty of success.
The problem I have with task breakdowns is you get way too much engineer overconfidence - i.e. there is literally no task which is less then an entire day's effort, in practice. There's about 6 usable hours in a day - I've watched people confidently bid they'll get something done in "half a day" but when you point out that's about 3 hours, suddenly they're way less sure about that number - or offended. Then 3 days later they're still working on it.
Sometimes it is not overconfidence but a side-effect of management interference & second-guessing the estimates.
In this case engineers give an estimate of what they think management wants to hear and not the actual time it takes. It happens if they are frequently asked to reduce their estimated time to within management expectations.
To deal with this, you need to bargain with them in the opposite direction to make sure the estimates are realistic.
Estimating time for a task is it's own skill. And, of course, the danger is in the task being insufficiently specified, leading to unknowns.
Cook lunch, estimate 30 minutes. Task starts, oh wait, cupboard is bare, need to go shopping.
Over a long career I've found that you do the best estimate you can with the information provided. Then multiply by 4. Sounds extreme, but you're still likely to be short a bit.
Equally, for estimates, I like to first estimate the number of data tables. That's a good factor to bring in to the rest. A system with 10 tables, 100 tables, 200 tables and so on "magnifies" the task list.
"Build reports" has a proportionate number. 10 tables might mean 5 reports. 100 tables might mean 50.
Building reports might be easy, but the number of them means time is needed.
But sometimes I see a task that would take about a day... and then people insist on breaking it down in four parts!
Then they take about a quarter day each, provided the same person does all four simultaneously. Or about a day each if you give them to different people.
Software development cannot be managed like this. This sort of task break downs come from classic management training. The problem most people are unaware of is software development is more a creative activity than anything else. Yes there are serious technical aspects to it but since the problem is virtual and not bound by any real world limitations, like how civil engineering would be, there isn't one optimal solution. Trying to define the solution before the problem has been properly examined would only limit the final output. Most of the exploration only happens when people actually start coding.
Not having a well defined final output and time is not very relevant for software, mostly because there is no per unit cost. Most managers do not realize this is they are not from a software engineering background. A versatile product can be sold to many different customers without additional development costs.
But since most companies are managed as factories, all the processors will ultimately create a very limited product targeted at a specific customer. What the big tech companies have avoided is this .
You'd be surprised how much creative tasks like 3D modeling and creating artwork can be broken down into tasks that are very well defined and can be estimated quite correctly.
I think it would fall down the same lines if you required a coherent world and high quality for each of the models you create.
That would mean redo models that don't fit the whole piece, endless tweak some models that are really difficult to get right, sometimes do swiping changing across all the parts because as a team you realize something is off etc.
The balance will be between delivering something within a time frame, and delivering the quality that was requested/expected.
The difference is that you break down software into smaller and smaller bits and then find a small bit that needs data from a far away small bit. I suspect this happens less with mechanical systems.
> Do you have more examples of companies that are not feature factories?
Actually, none of them are. They're just managed as if they were.
In a factory, you can literally have a machine that stamps out 1000 widget blanks an hour and if the operator does an hour of overtime, that's an extra 1000 widgets produced that day.
You've got your workers on 8-hour shifts and sales signs a contract to deliver 8,500 widgets per day? Then you know right away you're going to need 10-hour shifts, or regular overtime, or a second 8-hour shift, or a second machine, and you can figure out how much each of those would cost, and whether you've got enough car parking for the extra workers.
Whereas in software, your feature factory produces ???? per hour, sales promises customers ???? and an hour of overtime achieves ???.
Changing the topic from software development to R&D: the granular task oriented view is generally not good for greenfield research endeavours. It's a form of premature optimization. You're calcifying the search space and reducing flexibility to pivot immediately based on expert intuition and hunches. That being said, a coarse task orientation is necessary to keep people vaguely pointed in the right direction. The best resolution (time horizon) for task orientation depends on what you're doing.
Being a career long engineer, I'm not unaccustomed to breaking down huge projects into smaller things that can be parallelized, plotted in time, etc. We have to do it and get better at it.
But honestly, I think what holds most of us back is a better ability to just don't do that. Take that thing you want to make, and instead of planning out all the pieces, just make the tiniest possible thing that could possibly be of value. Using this example, just start with "Today" and the 4 buttons. Make a thing that shows your 4 buttons of exercise you will click on each day. You can probably make that today. No streaks or freezes or calendar view. Those are all great ideas, and you'll get to them. But you need momentum. And you'll probably decide you don't even like that calendar view in a few days. I think more projects need a kick in the pants of just do something small. Get it done today. See what the next task is from that point because it's probably not what you thought it ought to be during the planning phase.
This also isn't exactly easy either. It requires some thought and discipline and find the smallest thing and commit to shipping it without distraction. But it's also something good to practice at.
OMG, I love that you put Frozen 2 and princess Anna and a song onto this. Thank you for this coinage. I will absolutely be using this now. Please write a blog post on your coinage of this so that we all here can link to that and give you some credit. I'm serious. I looked up The Anna Principle and I wanted to see your blog on this.
"Do the next right thing" was a phrased used by Alcoholics Anonymous long before that 2019 movie came out, so I don't think calling it "The Anna Principle" is appropriate.
One thing that works for parallelization in these unsecure situations are things that you know you'll surely need. This even might just involve testing things. E.g. on member developes a comparison method for testing, another member goes for one approach, another for the other and a third for yet another. And after a set period you try to evaluate everything. Of course this approach only makes sense when there isn't already a favoured or clearly superior approach.
Similarily during actual development modularization can be a way to parallelize development. So one component per person or per team.
For my own stuff I also like to interleave the human-facing parts of my products with the technological guts, especially if I am still figuring things out — developing the technology without knowing how it is being used will force the way you can use it into a certain direction — designing the interface without knowing the technology has it's own traps as well.
Yes, definitely thought of that way most of the time. But I also think that might actually trivialize this concept some. Because one might say "ok, here's the crappy MVP version, and now that we're out of that one month phase everything after requires very rigorous and careful planning and meetings". I think most projects actually need to just stay in MVP mode. The entire project or product will be forever in a place of high uncertainty. Every feature should feel like it was the minimum we could do.
I'm actually not saying every project needs this of course, but I think maybe most might? Obviously there's huge things I've been apart of, and we needed to be thorough planning step by step months out. But that experience should probably be the exception and not the rule. Minimums the rule.
Absolutely. I mean, yes, there's those moments where you need to plan your retirement and you better think 40-50-60 years out. But most of the time, just do the next 5 minute thing and see what happens. I've been getting a lot better recently about just coming up with 3 "quests" I'm going to do today that probably take 5 minutes. And that little progress has kept building and building into something I didn't really see months ago when I started that first 5 minute step.
Breaking down work works great until the breakdown is unknowable. Things like research, that require creative experiments to POC things that are not known apriori. That’s when task breakdown breaks down.
Also when management sees what should be a POC or research as a deliverable they start promising to higher ups and other teams. I’ve developed the habit of doing most of my POC work behind closed doors and telling no one. If it works great, I can release it. If it doesn’t, I can kill it and move on without having to eat crow, or get told I need to figure out a way to make it work after I’ve seen it’s not wise to continue.
It sounds like a pretty small problem if you can build a PoC in 3 hours. Sometimes it takes 3 days to get to a point where you have a rough idea of what code changes might be needed.
This gets a lot of teachers - the task is so ingrained in them that the only part they consciously understand is the trivial bit that everyone already gets. Breaking down a task and estimation are almost equivalent - once the breakdown is done, assigning estimates to the chinks of work takes a few minutes as anyone with a few years experience has some standard estimates for small tasks. In my experience I break down a task by biting off chunks that I can estimate, then ask for opinions on anything I can't.
Although I don't even believe the real path to mastery is the task breakdown. Anyone can come up with a bad breakdown. The real point of mastery, which he didn't talk about here, is identifying when the evidence suggests a task breakdown is incorrect enough to cause problems and communicating that / doing a re-estimate [0]. And being comfortable that it will happen so not getting stressed up front putting out a schedule that is likely to change. Managers generally want an accurate roadmap up front. This is them asking for the impossible. IMO Good management is about flexibility and understanding that the expected nature of the work changes as time passes and developers learn.
He has a little example at the end. Ponder that if someone had come to him with an accurate estimate ("you can do this in a few evenings as a plane trip") he'd have rejected that as too risky. That illustrates that estimating isn't even about accuracy, there are many unspoken factors here around risk management, expectation management, task familiarity, etc, etc.
A lot of the "never taskers" in this comment section seemed to have never worked with the juniors I have.
These are good, new software folks who genuinely don't know how to stand up basic functionality because the space is new to them. In my experience they want tasks they can learn from and excel at, leading to growth.
I think the thing that you absolutely must keep in mind is process overhead is a continuum you tune to your team. No one in the NBA is going into a huddle and getting instruction on how to throw a ball mid game, but the kids in third grade are. Both styles of planning and coaching are appropriate to the team at hand.
I'm curious: if you hired a contractor to, I dunno, paint your walls, would you accept "I don't know" as a time frame or price quote? If not, what's different about software development that makes "I don't know" a reasonable answer in our profession?
Software development isn't (universally) a painting type task. In painting a room, the painter can estimate (by measuring) the amount of paint needed, they know from experience the time to acquire a particular color (if it needs to be mixed or can be bought readymade) in a particular volume, they know how long it takes to paint (they've done it many times before), how long to dry, and how many coats are needed for the type of surface.
Now, ask them to paint a mural instead. The estimates will change, they'll probably give you a broader range instead of being able to estimate almost down to the minute (and being off by no more than an hour or two) like someone just painting walls (with a known environment, surface type, and paint material).
Does the mural need to be designed? Are the desired qualities of the mural well-specified or will there be a series of back and forths? Maybe a set of prototypes (sketches) so that you can refine your requirements. At that point, the estimate for the total task (design and paint a mural, or design and develop a software application) becomes far less clear.
There are certainly programming tasks which are closer to the paint-a-wall task which are much easier to estimate reliably, but they're far from the only thing people in this field work on.
> what's different about software development that makes "I don't know" a reasonable answer in our profession?
The seemingly infinite amount of variations in software tasks and the ambiguity of the requirements? If my job involved only putting up or modifying API endpoints that involved querying a single SQL database, I'd get quite good at that and be able to tell you with very good accuracy how long an endpoint would take to put up.
Just as if I painted many walls before, I could reasonably tell you how long painting a wall would take based on the surface area, height, and amount of non-wall obstructions. Also, contractors tend to be their own salesmen, so just because they speak confidently, doesn't mean they are. It's not like contractors are universally famous for being on-time and underbudget.
Contractor horror stories are actually very common. They may not actually say to you "I don't know", but big delays in contractor jobs happen all the time. I would argue that software engineers are at least trying to be more honest about the unknowns of the work unlike some sleazy contractors.
My answer to this is that software engineering is a lot more like creating a blueprint than creating an artifact based on that blueprint. Or put another way, it's usually more of a design process than a manufacturing process. (I've switched analogies on you, but "painting walls" in your analogy is akin to "manufacturing artifacts" in mine.)
The "manufacturing the artifact" step in software is done automatically by the computer, when given the "blueprint" (code).
I guess to try to go back to your painting walls analogy. In my view, creating software is more like if someone asked you to create a wall-painting machine that will work for any room that fits a certain specification. They could contract you to paint just one room in this way, but that would certainly be harder than just painting the room! But more likely, they have a million rooms they want you to paint. Either way, the hard part is creating the room-painting machine. And it would indeed be quite difficult to give an accurate estimate of how long it will take to do that. But once that exists, you can easily estimate how long it will take to paint any individual room.
And real engineers, indeed, have this same issue with the design phase of projects, for the same reasons. Just like us, they try to break apart and estimate how long it will take to do that part, but just like us, my impression is that it is known to be fraught to accurately estimate that part of a project.
But also just like us, this is a continuum based on how novel the thing they are engineering is to them. On the one end of the spectrum, there is the equivalent of off-the-shelf software, like using a blueprint for a standard single-family home that's been built a million times already. And on the other end of the spectrum are things like designing the motor for the first model of a new electric vehicle manufacturer, where it's all brand new. But then there's a whole spectrum between those two, where it is fairly easy to break down and estimate the design process for things you've done a bunch of times, and nearly impossible for things you've never done.
This is a very long-winded way of saying that the difference comes down to how novel the project you're working on is! This is why freelance / contractor shops do best when they find a particular niche of a kind of thing to build, and then find clients who want them to build essentially the same thing over and over again. It really is possible to get very good at estimating this kind of nearly-cookie-cutter work. (I did this a bit for awhile back in the day, with multi-page Rails CRUD apps, and it was indeed easy to break things up and estimate.) But this is also why it can be frustrating to work with freelance / contract shops, because it behooves them to figure out how to fit your project into their cookie cutter, and that can end up being a worse outcome than building something bespoke, iteratively, without a detailed plan and estimate.
Agree, the entire point of a task list is to block yourself from doing other things. That makes sense until it doesn't. "The map is not the territory".
Readit News
One, it turns out I never ever end up doing the actual steps as I planned to. After a few I have new insights, realize I forgot something, see an easier way, I don't know -- the plan is never followed.
The other is that I really hate working like this because it seems all the creative effort of thinking about how to make something is put in the first part. And then, the rest is still most of the work, but it's the really boring part. It's much more fun (and therefore faster, and resulting in better work) to spread the creativity and the boredom out more equally.
The two are probably related, and it's not inconceivable that I have ADHD.
Of course if you are exploring or building your own projects and/or don't have any other accountability, you wouldn't start planning that.. unless you are really into planning in and of itself.
But as soon as you have a boss and the boss asks "how long? who does what? where do we start?" - you have to have some framework.
I also have a very high aversion to "conforming to the system", esp. when these systems seem arbitrary or are overly rigid .. but systems in general need to be simple and flexible. Productivity systems are there to help people.
I don't think the author intended for this to be like a detailed recipe.. but more like an example of how he approaches these things, which can hopefully help others have their own ideas.
Why?
Does it work? Is it productive? Is it better than not doing that?
> the plan is never followed.
Sometimes it's even worst. As we add more tasks during execution, at the end, these tasks get twisted with the ones you created while planning. Then we end up with a messy list of undone tasks that aren't really required anymore (because they were created without having the full context, that only happens during the execution).
The plan not being perfect and prescient is part of the plan.
Next time you plan for same or similar activity, your future plan will be better.
The Project Managers even have a mantra: "fail to plan is plan to fail."
But for me it's very easy to overdo it. Give me interesting chunks of about a week or two, with a tight deadline, and I'll do my best work.
Remember, Linus wrote Git in two weeks. And there was no project manager in sight.
The plan to me is just a way of reminding my future self what I thought of as ideal output in the past. Instead of infinitely changing my plans and chasing fireflies
Without planning, a depth first traversal is a high risk endeavor in the likelihood that the that path is wrong but backtracking and creating the graph is comparatively expensive and susceptible to sunk cost fallacy. Depth-first traversal is writing the book a chapter at a time without a table of contents in mind.
You can plan all you want and it is essential... however you'll quickly find out that most of your plans are in your newborns' hands.
Realizing this also helps me avoid yak shaving over list making tools, since I could care less about even saving the lists! I keep a folder of text files in any project I'm working to hold my lists, but tbh it's unnecessary to even save them.
(A few times there is prototyping involved but I won't share that with non-devs, since they don't seem to get it.)
I do the GTD thing of only thinking about the _next_ action, not trying to think of every action.
I think maintenance tasks may require more effort / more expansive qualification when it comes to the meaning(s) of "something is different".
I think the topic of "breaking down tasks" could very well be its own book genre or even podcast genre. My experience with self-help and organization titles has been that the activity of breaking down tasks is a sort of known primitive human operation that many authors assume their readers to have. However, my own personal experience and the accounts of others I have heard while participating in group sessions is that breaking down tasks can be very difficult and provoke emotions of avoidance and despair.
The most broadly relevant advice that I have come across when it comes to breaking down tasks is to keep breaking things down until I am 90% certain that I can successfully complete the task. The certainty here would vary on how much self-confidence an individual has and their risk tolerance, so some people may break down tasks to 70% certainty of success.
In this case engineers give an estimate of what they think management wants to hear and not the actual time it takes. It happens if they are frequently asked to reduce their estimated time to within management expectations.
To deal with this, you need to bargain with them in the opposite direction to make sure the estimates are realistic.
Cook lunch, estimate 30 minutes. Task starts, oh wait, cupboard is bare, need to go shopping.
Over a long career I've found that you do the best estimate you can with the information provided. Then multiply by 4. Sounds extreme, but you're still likely to be short a bit.
Equally, for estimates, I like to first estimate the number of data tables. That's a good factor to bring in to the rest. A system with 10 tables, 100 tables, 200 tables and so on "magnifies" the task list.
"Build reports" has a proportionate number. 10 tables might mean 5 reports. 100 tables might mean 50.
Building reports might be easy, but the number of them means time is needed.
Then they take about a quarter day each, provided the same person does all four simultaneously. Or about a day each if you give them to different people.
This has been amazingly accurate for time estimates - in aggregate... Individually, they are wildly inaccurate.
So... they are probability estimates. Over many events, they even out.
Not having a well defined final output and time is not very relevant for software, mostly because there is no per unit cost. Most managers do not realize this is they are not from a software engineering background. A versatile product can be sold to many different customers without additional development costs.
But since most companies are managed as factories, all the processors will ultimately create a very limited product targeted at a specific customer. What the big tech companies have avoided is this .
That would mean redo models that don't fit the whole piece, endless tweak some models that are really difficult to get right, sometimes do swiping changing across all the parts because as a team you realize something is off etc.
The balance will be between delivering something within a time frame, and delivering the quality that was requested/expected.
Do you have more examples of companies that are not feature factories? It seems the entire industry has adopted this pattern.
Actually, none of them are. They're just managed as if they were.
In a factory, you can literally have a machine that stamps out 1000 widget blanks an hour and if the operator does an hour of overtime, that's an extra 1000 widgets produced that day.
You've got your workers on 8-hour shifts and sales signs a contract to deliver 8,500 widgets per day? Then you know right away you're going to need 10-hour shifts, or regular overtime, or a second 8-hour shift, or a second machine, and you can figure out how much each of those would cost, and whether you've got enough car parking for the extra workers.
Whereas in software, your feature factory produces ???? per hour, sales promises customers ???? and an hour of overtime achieves ???.
Changing the topic from software development to R&D: the granular task oriented view is generally not good for greenfield research endeavours. It's a form of premature optimization. You're calcifying the search space and reducing flexibility to pivot immediately based on expert intuition and hunches. That being said, a coarse task orientation is necessary to keep people vaguely pointed in the right direction. The best resolution (time horizon) for task orientation depends on what you're doing.
But honestly, I think what holds most of us back is a better ability to just don't do that. Take that thing you want to make, and instead of planning out all the pieces, just make the tiniest possible thing that could possibly be of value. Using this example, just start with "Today" and the 4 buttons. Make a thing that shows your 4 buttons of exercise you will click on each day. You can probably make that today. No streaks or freezes or calendar view. Those are all great ideas, and you'll get to them. But you need momentum. And you'll probably decide you don't even like that calendar view in a few days. I think more projects need a kick in the pants of just do something small. Get it done today. See what the next task is from that point because it's probably not what you thought it ought to be during the planning phase.
This also isn't exactly easy either. It requires some thought and discipline and find the smallest thing and commit to shipping it without distraction. But it's also something good to practice at.
> when faced with uncertainty, one must simply focus on doing "The Next Right Thing." [0]
This is not to trivialize it! Figuring out the next right thing to do is hard, but also the most valuable part of planning, in my view.
0: https://en.wikipedia.org/wiki/The_Next_Right_Thing#Synopsis
Similarily during actual development modularization can be a way to parallelize development. So one component per person or per team.
For my own stuff I also like to interleave the human-facing parts of my products with the technological guts, especially if I am still figuring things out — developing the technology without knowing how it is being used will force the way you can use it into a certain direction — designing the interface without knowing the technology has it's own traps as well.
I'm actually not saying every project needs this of course, but I think maybe most might? Obviously there's huge things I've been apart of, and we needed to be thorough planning step by step months out. But that experience should probably be the exception and not the rule. Minimums the rule.
"spend 3 hours on investigating X, 3 on Y, 3 on Z"; and then have a planning session about what to do next.
You have 4 outcomes - the first one solves it, the second, the third or none.
If it is solvable with mild effort, you have a 50% chance of solving it by attempt 2/6 hours.
And at some point, you just gotta do it.
Although I don't even believe the real path to mastery is the task breakdown. Anyone can come up with a bad breakdown. The real point of mastery, which he didn't talk about here, is identifying when the evidence suggests a task breakdown is incorrect enough to cause problems and communicating that / doing a re-estimate [0]. And being comfortable that it will happen so not getting stressed up front putting out a schedule that is likely to change. Managers generally want an accurate roadmap up front. This is them asking for the impossible. IMO Good management is about flexibility and understanding that the expected nature of the work changes as time passes and developers learn.
He has a little example at the end. Ponder that if someone had come to him with an accurate estimate ("you can do this in a few evenings as a plane trip") he'd have rejected that as too risky. That illustrates that estimating isn't even about accuracy, there are many unspoken factors here around risk management, expectation management, task familiarity, etc, etc.
[0] https://jacobian.org/2021/jun/8/incorrect-estimates/ - he has a post on the topic
These are good, new software folks who genuinely don't know how to stand up basic functionality because the space is new to them. In my experience they want tasks they can learn from and excel at, leading to growth.
I think the thing that you absolutely must keep in mind is process overhead is a continuum you tune to your team. No one in the NBA is going into a huddle and getting instruction on how to throw a ball mid game, but the kids in third grade are. Both styles of planning and coaching are appropriate to the team at hand.
I think that’s the main truth about software engineering. Maybe an open source version of that Streak app exists and OP is re-inventing the wheel.
I think the brain doesn’t like task list. It may be pleasant to do, but most of entrepreneurship or coding is exploring.
And a task list blocks you from exploring.
Now, ask them to paint a mural instead. The estimates will change, they'll probably give you a broader range instead of being able to estimate almost down to the minute (and being off by no more than an hour or two) like someone just painting walls (with a known environment, surface type, and paint material).
Does the mural need to be designed? Are the desired qualities of the mural well-specified or will there be a series of back and forths? Maybe a set of prototypes (sketches) so that you can refine your requirements. At that point, the estimate for the total task (design and paint a mural, or design and develop a software application) becomes far less clear.
There are certainly programming tasks which are closer to the paint-a-wall task which are much easier to estimate reliably, but they're far from the only thing people in this field work on.
The seemingly infinite amount of variations in software tasks and the ambiguity of the requirements? If my job involved only putting up or modifying API endpoints that involved querying a single SQL database, I'd get quite good at that and be able to tell you with very good accuracy how long an endpoint would take to put up.
Just as if I painted many walls before, I could reasonably tell you how long painting a wall would take based on the surface area, height, and amount of non-wall obstructions. Also, contractors tend to be their own salesmen, so just because they speak confidently, doesn't mean they are. It's not like contractors are universally famous for being on-time and underbudget.
The "manufacturing the artifact" step in software is done automatically by the computer, when given the "blueprint" (code).
I guess to try to go back to your painting walls analogy. In my view, creating software is more like if someone asked you to create a wall-painting machine that will work for any room that fits a certain specification. They could contract you to paint just one room in this way, but that would certainly be harder than just painting the room! But more likely, they have a million rooms they want you to paint. Either way, the hard part is creating the room-painting machine. And it would indeed be quite difficult to give an accurate estimate of how long it will take to do that. But once that exists, you can easily estimate how long it will take to paint any individual room.
And real engineers, indeed, have this same issue with the design phase of projects, for the same reasons. Just like us, they try to break apart and estimate how long it will take to do that part, but just like us, my impression is that it is known to be fraught to accurately estimate that part of a project.
But also just like us, this is a continuum based on how novel the thing they are engineering is to them. On the one end of the spectrum, there is the equivalent of off-the-shelf software, like using a blueprint for a standard single-family home that's been built a million times already. And on the other end of the spectrum are things like designing the motor for the first model of a new electric vehicle manufacturer, where it's all brand new. But then there's a whole spectrum between those two, where it is fairly easy to break down and estimate the design process for things you've done a bunch of times, and nearly impossible for things you've never done.
This is a very long-winded way of saying that the difference comes down to how novel the project you're working on is! This is why freelance / contractor shops do best when they find a particular niche of a kind of thing to build, and then find clients who want them to build essentially the same thing over and over again. It really is possible to get very good at estimating this kind of nearly-cookie-cutter work. (I did this a bit for awhile back in the day, with multi-page Rails CRUD apps, and it was indeed easy to break things up and estimate.) But this is also why it can be frustrating to work with freelance / contract shops, because it behooves them to figure out how to fit your project into their cookie cutter, and that can end up being a worse outcome than building something bespoke, iteratively, without a detailed plan and estimate.