A couple make flags that are useful and probably not very well known:
Output synchronization which makes `make` print stdout/stderr only once a target finishes. Otherwise it's typically interleaved and hard to follow:
make --output-sync=recurse -j10
On busy / multi-user systems, the `-j` flag for jobs may not be best. Instead you can also limit parallelism based on load average:
make -j10 --load-average=10
Randomizing the order in which targets are scheduled. This is useful for your CI to harden your Makefiles and see if you're missing dependencies between targets:
Maybe the make authors could compile a list of options somewhere and ship it with their program, so users could read them? Something like a text file or using some typesetting language. This would make that knowledge much more accessible.
If "make -j" successfully drowns a machine, I can argue that the machine has no serious bottlenecks for the job. Because, make is generally I/O bound when run with high parallelism, and if you can't saturate your I/O bandwidth, that's a good thing in general.
However, if "make -j" is saturates a machine, and this is unintentional, I'd assume PEBKAC, or "holding it wrong", in general.
I will not restrict myself to an arcane subset of Make just because you refuse to type 'gmake' instead of 'make'. Parallel execution, pattern rules, order-only prerequisites, includes, not to mention the dozens of useful function like (not)dir, (pat)subst, info... There's a reason why most POSIX Makefiles nowadays are generated. It's not GNU's fault that POSIX is stale.
EDIT: There's one exception, and that would be using Guile as an extension language, as that is often not available. However, thanks to conditionals (also not in POSIX, of course), it can be used optionally. I once sped up a Windows build by an order of magnitude by implementing certain things in Guile instead of calling shell (which is notoriously slow on Windows).
Not every project has to be a multi-platform, multi-os, multi-language monster. It is perfectly fine to target a specific set of architecture, os, etc. And I find insulting and silly calling it a “toy project”
Agreed if you're looking at it through the lens of portable software that you plan to distribute. Automake generates portable Makefiles for a reason.
But there's another huge category: people who are automating something that's not open-source. Maybe it stays within the walls of their company, where it's totally fine to say "build machines will always be Ubuntu" or whatever other environment their company prefers.
GNU Make has a ton of powerful features, and it makes sense to take advantage of them if you know that GNU Make will always be the one you use.
Way back in the dark ages of 1985, I encountered a guy at the Boston University Graphics lab that was using Makefiles to drive the generation of a 3D renderer for animation. He was a Lisp guy, doing early procedural generation and 3D actor systems. His Makefile was extremely elegant, about 10 lines total. It generated hundreds of animations, all based on the simple file date dependency. He had Lisp generating the 3d form for each frame, and them Make would generate the frames. This being '85, pre pretty much everything we take for granted with 3D and animation, the guy was blowing everyone's mind. He went on to write the 3D renderer for Iron Giant, and was key in Caroline too, I seem to remember. Brian Gardner.
Make is one of those things that I'm really glad I learnt at the beginning of my career. Not because I use it much any more, but because it showed me the power of a declarative system over an imperative one.
I also realised at one point how naturally the idea extends to other tasks that I do. Going by the picture at the top of this site, it seems the author realised a similar thing to me: you can understand food recipes better if you think about them declaratively like makefiles, rather than imperatively like scripts, which is how recipes are traditionally written down.
I always scribble down recipes in a way that I can read like a Makefile and take that into the kitchen with me. I'm curious if anyone has tried typesetting or displaying recipes in this way as I feel like it would save a lot of time when reading new recipes as I wouldn't have to convert from a script to a makefile myself.
A nice thing about this approach is that it passes more control to the user who is essentially now responsible for resolving the dependency graph themselves and “be” the executor. Taking your cooking example, the declarative nature better exposes where there are open choices in what to do next, which affords the user more freedom to take into account other externalities and constraints not formally specified in the makefile (like specific orderings that make washing up easier).
Of course the tradeoff is that you have to resolve the dependency graph yourself. That’s more work on you when you just want a set of pre-serialised, sequential steps to follow.
The article says most people don’t mark recipes as .PHONY and seems to use that as a reason to not bother in the tutorial. I think that is a weak excuse and we should teach the right way to use a tool.
My teammates gave me a hard time for adding and maintaining .PHONY on all our recipes since we use make as a task runner.
Clark Grubb has a great page explaining a style guide for make files:
I just gave that a read. Good doc overall. There are a few items I disagree with:
- Cargo-culted use of -o pipefail. Pipefail has its uses, but it breaks one of the most common things people do in a pipeline: filter the output with grep. Add it on a per-recipe basis instead.
- Marking non-file targets as .PHONY. This is strictly correct, but it's usually not necessary. I think it adds unneeded verbosity to the Makefile, especially if you have a lot of targets. Better to add it on an as-needed basis IMO.
- Recipes with multiple output files. Use of dummyfiles/flagfiles used to be the standard if a pattern-rule wasn't the right fit. But as of GNU Make 4.3 (shipping in Ubuntu 22.04 LTS), there is native support for grouped targets. Check it out here: https://www.gnu.org/software/make/manual/html_node/Multiple-...
Another thing that's interesting lately is that CMake has decided that Makefiles are unfit for projects that use C++20 modules, and ninja is the way to go. [1]
Basically it's considered too hard if not impossible to statically define the target's dependencies. This is now done dynamically with tools like `clang-scan-deps` [2]
Correct me if I'm wrong, but I think this limitation wrt Makefiles is entirely a choice by CMake, or at least a lack of volunteers for adding support to the Makefile generator. Ninja itself doesn't have any support for C++ modules (see https://github.com/ninja-build/ninja/issues/2457). In fact, Ninja has fewer features in that regard than plain Make as Ninja requires all dependencies to be statically defined.
People keep writing and using other alternatives (like just), which provide a very slight improvement on pure shell at the cost of installing yet another tool everywhere.
I stick with bash, write every task as a separate function, and multiplex between them with a case statement (which supports globs et al. and is very readable).
It replaces the "list of short shell-scripts" aspect of Make, but it doesn't replace the "only execute rules that need to be re-executed" part, which is the actually useful bit.
This is the most frustrating bit of this weird recursive ecosystem of build tools. No one really uses all of make, so they only clone the bits they need, so their tool is simple and clean and beautiful to a subset of the community that has their same problem. But it can't replace make, so seven months later someone with a slightly different problem shows up with a make replacement, and the circle of life continues.
And you see this on the other side of the problem area too, where large and ugly tools like cmake are trying to do what older large and ugly software like autotools did, and trying to replace make. And they suck too.
I continue to believe the GNU make in the late 80's was and remains a better generic tool than everything in the modern world in all ways but syntax (and in many cases, again c.f. cmake, it had better syntax too). Had the original v7 syntax used something other than tabs, and understood that variable names longer than 1 byte were a good thing, we might never have found ourselves in this mess.
they do place themselves as an alternative to make, but imho they're entirely different and not at all comparable. make is centered around creating artefacts and not rebuilding what is already built. just is a command runner.
The main benefit I see with using Make as a command runner is that it's a standard tool that's installed "everywhere". Even though these replacements seem nicer to use, I never felt like they bring enough to the table to warrant having to install an extra tool.
I also use just as a command runner, but I gotta agree with the others here that it should be described accurately as a command runner, while make is a build system.
There are some uses of make, especially by people who have never used it to build C/C++ projects, which makes more sense to replace with just. It doesn't have the baggage that make does, and they're not using it to actually make files. They also quite likely don't know the conventions (e.g. what a lot of us expect "make install" to do), and I support them in not learning the conventions of make—as long as they use something else. :)
Other uses of make will need other modern replacements, e.g. Cmake or Bazel.
It is possible that Kids These Days can say "no thanks" when someone tries to teach them make, and that the future of make is more along the lines of something us greybeards complain about. Back in _my_ day, etc.
Makefiles are great but do try not to get carried away. Years ago I tried to create a pure GNU Make framework, only to realize I was effectively reinventing autoconf. That was the moment I finally understood what the GNU autotools had been made for.
Makefiles are eerily lisplike turing tarpits. GNU Make even has metaprogramming capabilities. Resisting the urge to metaprogram some unholy system inside the makefile can be difficult. The ubiquitousness of GNU Make makes it quite tempting.
Readit News
Output synchronization which makes `make` print stdout/stderr only once a target finishes. Otherwise it's typically interleaved and hard to follow:
On busy / multi-user systems, the `-j` flag for jobs may not be best. Instead you can also limit parallelism based on load average: Randomizing the order in which targets are scheduled. This is useful for your CI to harden your Makefiles and see if you're missing dependencies between targets:Maybe the make authors could compile a list of options somewhere and ship it with their program, so users could read them? Something like a text file or using some typesetting language. This would make that knowledge much more accessible.
(`make --help` will only print the most common options)
Will give you the command line options. And GNU make has decent documentation online for everything else:
https://www.gnu.org/software/make/manual/html_node/index.htm...
However, if "make -j" is saturates a machine, and this is unintentional, I'd assume PEBKAC, or "holding it wrong", in general.
Can’t the OS scheduler handle it?
But not portable. Please don't use them outside of your own non-distributable toy projects.
EDIT: There's one exception, and that would be using Guile as an extension language, as that is often not available. However, thanks to conditionals (also not in POSIX, of course), it can be used optionally. I once sped up a Windows build by an order of magnitude by implementing certain things in Guile instead of calling shell (which is notoriously slow on Windows).
GNU Make is feature rich and is itself portable. It's also free software, as in freedom. Just use it.
But there's another huge category: people who are automating something that's not open-source. Maybe it stays within the walls of their company, where it's totally fine to say "build machines will always be Ubuntu" or whatever other environment their company prefers.
GNU Make has a ton of powerful features, and it makes sense to take advantage of them if you know that GNU Make will always be the one you use.
Deleted Comment
I also realised at one point how naturally the idea extends to other tasks that I do. Going by the picture at the top of this site, it seems the author realised a similar thing to me: you can understand food recipes better if you think about them declaratively like makefiles, rather than imperatively like scripts, which is how recipes are traditionally written down.
I wrote about it here: https://blog.gpkb.org/posts/cooking-with-make/
I always scribble down recipes in a way that I can read like a Makefile and take that into the kitchen with me. I'm curious if anyone has tried typesetting or displaying recipes in this way as I feel like it would save a lot of time when reading new recipes as I wouldn't have to convert from a script to a makefile myself.
Of course the tradeoff is that you have to resolve the dependency graph yourself. That’s more work on you when you just want a set of pre-serialised, sequential steps to follow.
My teammates gave me a hard time for adding and maintaining .PHONY on all our recipes since we use make as a task runner.
Clark Grubb has a great page explaining a style guide for make files:
https://clarkgrubb.com/makefile-style-guide
Does anyone else use this style guide? Or for phony recipes marking phony at the recipe declaration vs a giant list at the top of the file?
I would love to have a linter that enforced this…
Deleted Comment
- Cargo-culted use of -o pipefail. Pipefail has its uses, but it breaks one of the most common things people do in a pipeline: filter the output with grep. Add it on a per-recipe basis instead.
- Marking non-file targets as .PHONY. This is strictly correct, but it's usually not necessary. I think it adds unneeded verbosity to the Makefile, especially if you have a lot of targets. Better to add it on an as-needed basis IMO.
- Recipes with multiple output files. Use of dummyfiles/flagfiles used to be the standard if a pattern-rule wasn't the right fit. But as of GNU Make 4.3 (shipping in Ubuntu 22.04 LTS), there is native support for grouped targets. Check it out here: https://www.gnu.org/software/make/manual/html_node/Multiple-...
Basically it's considered too hard if not impossible to statically define the target's dependencies. This is now done dynamically with tools like `clang-scan-deps` [2]
[1] https://cmake.org/cmake/help/latest/manual/cmake-cxxmodules....
[2] https://llvm.org/devmtg/2019-04/slides/TechTalk-Lorenz-clang...
People sometimes treat it as a generic “project specific job runner”, which it’s not a good fit for. Even simple conditionals are difficult.
I’ve seen several well-intentioned attempts at wrapping Terraform with it, for example, which have ended terribly.
This is no longer true imo.
More robust and well-defined build systems have been created in the last 2 decades. Time to update.
Edit: Sorry, it looks like I totally misunderstood what you meant by "job runner".
People keep writing and using other alternatives (like just), which provide a very slight improvement on pure shell at the cost of installing yet another tool everywhere.
I stick with bash, write every task as a separate function, and multiplex between them with a case statement (which supports globs et al. and is very readable).
And you see this on the other side of the problem area too, where large and ugly tools like cmake are trying to do what older large and ugly software like autotools did, and trying to replace make. And they suck too.
I continue to believe the GNU make in the late 80's was and remains a better generic tool than everything in the modern world in all ways but syntax (and in many cases, again c.f. cmake, it had better syntax too). Had the original v7 syntax used something other than tabs, and understood that variable names longer than 1 byte were a good thing, we might never have found ourselves in this mess.
- Task (Go): https://github.com/go-task/task
- Cake (C#): https://github.com/cake-build/cake
- Rake (Ruby): https://github.com/ruby/rake
Or an entirely different concept: Makedown, as discussed on HN 8 months ago: https://news.ycombinator.com/item?id=41825344
- SCons
https://scons.org/
https://en.m.wikipedia.org/wiki/SCons
*https://taskfile.dev/
There are some uses of make, especially by people who have never used it to build C/C++ projects, which makes more sense to replace with just. It doesn't have the baggage that make does, and they're not using it to actually make files. They also quite likely don't know the conventions (e.g. what a lot of us expect "make install" to do), and I support them in not learning the conventions of make—as long as they use something else. :)
Other uses of make will need other modern replacements, e.g. Cmake or Bazel.
It is possible that Kids These Days can say "no thanks" when someone tries to teach them make, and that the future of make is more along the lines of something us greybeards complain about. Back in _my_ day, etc.
Makefiles are eerily lisplike turing tarpits. GNU Make even has metaprogramming capabilities. Resisting the urge to metaprogram some unholy system inside the makefile can be difficult. The ubiquitousness of GNU Make makes it quite tempting.