递归 `make` 和 `-j`
Recursive `make` and `-j`

原始链接: https://quuxplusone.github.io/blog/2026/09/30/recursive-make-j/

文章探讨了递归调用 `make` 时,GNU Make 3.81 至 4.3 如何处理并行执行。如果配方中直接运行 `make`,父进程的作业服务器(jobserver)可能不会传递给子进程,从而出现类似 `jobserver unavailable: using -j1` 的警告,导致并行能力丧失,甚至可能调用 `$PATH` 中另一个 `make` 可执行文件。 在配方前加上 `+`(例如 `+make -C subdir`),或使用惯用的 `$(MAKE)` 变量,可以解决这些问题,并确保子进程继承父进程的作业服务器设置。 显式指定 `make -j2` 或 `make -j4` 等选项会覆盖继承的作业数限制,并禁用或重置作业服务器模式;不同 GNU Make 版本产生的警告措辞可能不同。单独使用 `make -j` 会请求无限并行,但在未标记的递归配方中,其行为因版本而异,因此不可移植。总体而言,递归配方应使用 `$(MAKE)`,除非有意覆盖父进程的作业服务器配置,否则应避免指定 `-j`。

一场 Hacker News 讨论探讨了:在并行构建中,递归调用 `make` 是否仍然有用。GNU Make 的顶层进程负责管理作业服务器;递归调用会向它申请作业槽位,而 `-j` 应在顶层命令中指定。评论者指出,GNU Make 4.4 引入的基于 FIFO 的作业服务器,不再需要仅为了保留作业服务器访问权限而使用 `+` 前缀或 `$(MAKE)`,不过在处理其他构建工具时仍可能需要特殊方式。`MAKEFLAGS=-j8` 可以将并行度传递给子进程,但手动传递可能不够可靠。 这场争论不只涉及语法。“递归 Make 有害”一文认为,嵌套构建会模糊依赖关系、重复执行工作,并且妨碍并行化。另一些人则反驳说,如果项目构成清晰的 DAG,每个子项目的范围都经过精心划分,并且叶子 makefile 直接生成最终产物,那么递归 make 仍然可以很好地工作。即便如此,按目录进行递归仍可能造成重复的子构建或过于粗粒度的调度,因此,使用统一的依赖图或现代构建工具可能更加简洁。
相关文章

原文

Today I looked into “recursive” invocations of make; that is, cases in which a Makefile recipe itself invokes make. The official documentation is quite good, but since I’m working with a range of different GNU Make versions, I looked at how the behavior has changed over time, from GNU Make 3.81 to 4.3.

Basically, the issue is that if you have a recipe in your Makefile like this:

and then you run make -j4 ser, recent versions of GNU Make will give a warning (jobserver unavailable: using -j1). And you’ll lose the parallelism you wanted, because the “jobserver” (basically, the -j option) isn’t forwarded to the child make. And, incidentally, it won’t necessarily use the same make binary that you invoked; it’ll run whatever make happens to be in your path.

You can fix all three of those issues by prefixing + to the recipe line, or (since that feels uncomfortably magical) replacing make with $(MAKE). More on this at the end of the post. Now that we’re forwarding the “jobserver” properly, I wondered, how exactly do the various -j options interact, when you specify one degree of parallelism on the command line and another in the recursive recipe?

Here’s the pair of Makefiles I used for my test:

detab -E -4 - >Makefile <<EOF
ser:
    make -C subdir
par2:
    make -j2 -C subdir
par4:
    make -j4 -C subdir
par:
    make -j -C subdir
serplus:
    +make -C subdir
par2plus:
    +make -j2 -C subdir
par4plus:
    +make -j4 -C subdir
parplus:
    +make -j -C subdir
sermake:
    $(MAKE) -C subdir
par2make:
    $(MAKE) -j2 -C subdir
par4make:
    $(MAKE) -j4 -C subdir
parmake:
    $(MAKE) -j -C subdir
EOF
mkdir subdir
detab -E -4 - >subdir/Makefile <<EOF
all: one two three four five
one:
    echo 'one' && sleep 1 && echo 'ONE'
two:
    echo 'two' && sleep 1 && echo 'TWO'
three:
    echo 'thr' && sleep 1 && echo 'THR'
four:
    echo 'fou' && sleep 1 && echo 'FOU'
five:
    echo 'fiv' && sleep 1 && echo 'FIV'
EOF

Then I tested running make ser, make -j2 ser, etc., on various versions of Make available to me.

Here’s GNU Make 3.81 (on OS X). The number in each cell indicates the number of recipes running in parallel. The four rows marked with a dagger † differ among the versions of Make I tested; those idioms are “non-portable” and IMHO should be avoided in practice.

make…   -j2 -j4 -j
ser† 1 2 4 ∞
serplus 1 2 4 ∞
sermake 1 2 4 ∞
par2 2 2b 2b 2
par2plus 2 2b 2b 2
par2make 2 2b 2b 2
par4 4 4b 4b 4
par4plus 4 4b 4b 4
par4make 4 4b 4b 4
par† ∞ 2 4 ∞
parplus† ∞ 2 4 ∞
parmake† ∞ 2 4 ∞

In the table above, “b” indicates this warning:

$ make -j2 par2
make[1]: warning: -jN forced in submake: disabling jobserver mode.

GNU Make 4.0 (on Debian 8.11):

make…   -j2 -j4 -j
ser† 1 1a 1a ∞
serplus 1 2 4 ∞
sermake 1 2 4 ∞
par2 2 2b 2b 2
par2plus 2 2b 2b 2
par2make 2 2b 2b 2
par4 4 4b 4b 4
par4plus 4 4b 4b 4
par4make 4 4b 4b 4
par† ∞ 1a 1a ∞
parplus† ∞ 2 4 ∞
parmake† ∞ 2 4 ∞

In the table above, “a” and “b” indicate these two warnings, respectively:

$ make -j2 ser
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

$ make -j2 par2
make[1]: warning: -jN forced in submake: disabling jobserver mode.

GNU Make 4.2.1 (on Rocky 8.10) and GNU Make 4.3 (on Red Hat 9 or Ubuntu 22.04) tweak the latter error message to include the value of N:

$ make -j2 par
make[1]: warning: -j0 forced in submake: resetting jobserver mode.
$ make -j2 par2
make[1]: warning: -j2 forced in submake: resetting jobserver mode.

and also change the behavior of rows par, parplus, and parmake:

make…   -j2 -j4 -j
ser† 1 1a 1a ∞
serplus 1 2 4 ∞
sermake 1 2 4 ∞
par2 2 2b 2b 2
par2plus 2 2b 2b 2
par2make 2 2b 2b 2
par4 4 4b 4b 4
par4plus 4 4b 4b 4
par4make 4 4b 4b 4
par† ∞ ∞b ∞b ∞
parplus† ∞ ∞b ∞b ∞
parmake† ∞ ∞b ∞b ∞

Warning “a,” which explicitly tells you to “Add '+' to parent make rule,” will indeed always vanish if you prefix the make-containing line with +, like this:

I find that solution as uncomfortably magical as prefix @ or prefix -. (Personally, I’d rather see “|| true” at the end of a recipe line than a magic “-” at the beginning.) A solution that’s less magic-looking (although equally magic under the hood, apparently) is to use $(MAKE), like this:

sermake:
    $(MAKE) -C subdir

Neither method silences warning “b,” about “resetting jobserver mode,” though.

What we really want is a row that would look like fantasy below. But at the moment, it seems we have to choose our preferred tradeoff from one of the quoted rows, instead.

make…   -j2 -j4 -j
fantasy nprocs 2 4 ∞
sermake 1 2 4 ∞
par4make 4 4b 4b 4
联系我们 contact @ memedata.com