site/src/guides/recipes/dry-run-capture.md
A dry run is only worth parsing if it prints every compiler command, so ask make for all of them at once:
LC_ALL=C make -Bnwk | bear parse-sh
Each flag closes off one way the text comes up short. -n prints
recipes instead of running them. -B prints them even in a tree that is
already built. -k carries on past an error rather than stopping at the
first one. -w makes the top-level make announce its directory, which
sub-makes already do. LC_ALL=C keeps those announcements in the
English wording Bear matches. Three things no flag fixes: a recursion
that never descends, a build tool whose dry run prints progress instead
of commands, and a log that was already captured without them.
Under -n make prints a recipe line instead of running it, and that
applies to a recipe that starts a make in a subdirectory too: the line
prints, the sub-make never runs, and nothing from that subdirectory
reaches the text. The exception is a line make recognizes as recursion,
which it does by the text, $(MAKE) or ${MAKE} somewhere in the line,
or a leading +. A Makefile that spells the recursion out literally:
all:
make -C sub
gives a dry run that stops at the top level:
$ make -nw
make: Entering directory '/home/you/project'
make -C sub
make: Leaving directory '/home/you/project'
bear parse-sh writes [] from that and has nothing to report:
make -C sub is a valid command, it is just not a compiler. Writing the
same rule as $(MAKE) -C sub makes the sub-make run under -n as well
and print its own recipes, Entering directory markers included.
When the Makefile is not yours to change, dry-run each subdirectory
separately and combine the results with --append (see Command-line
options):
make -nw | bear parse-sh
make -nw -C sub | bear parse-sh --append
make -C announces the directory it changed into, so the appended
entries carry the right directory whichever directory you run the
command from.
An entry whose file is *.c comes from a recipe that never had its
wildcard expanded. -n prints the recipe line as written, and the shell
that would have expanded the glob never ran, so the glob reaches Bear
intact and is recorded as the source it claims to be:
lib.a : *.c
$(CC) -c $(CFLAGS) *.c
$ make -Bnw
cc -c -Wall *.c
$ make -Bnw | bear parse-sh -o -
[
{
"file": "*.c",
"arguments": [ "cc", "-c", "-Wall", "*.c" ],
"directory": "/home/you/project"
}
]
Have make expand the wildcard instead of the shell. $(wildcard) runs
when make reads the Makefile, before any recipe is printed, so the dry
run carries the file names:
SRCS = $(wildcard *.c)
lib.a : $(SRCS)
$(CC) -c $(CFLAGS) $(SRCS)
$ make -Bnw
cc -c -Wall a.c b.c
$ make -Bnw | bear parse-sh -o - | grep '"file"'
"file": "a.c",
"file": "b.c",
The two recipes build the same thing; only the expansion moves. When the
Makefile is not yours to change, run the build under interception
instead: the shell expands the glob before the compiler starts, so the
exec() Bear observes already names both sources, and the same two
entries come out.
Neither case is loud: everything Bear does get is valid text, so no line is skipped and the run exits 0, however little reached the database. A tree that is already built has no recipe left to print:
$ make -nw
make: Entering directory '/home/you/project'
make: Nothing to be done for 'all'.
make: Leaving directory '/home/you/project'
That middle line is a valid command that is not a compiler, so there is
no skip to report. -B (--always-make) treats every target as out of
date and prints every recipe.
An error stops the dry run where it would stop a real build. The
compiles make had already reached are printed, the rest never are, and
-k (--keep-going) is what gets the unaffected targets printed
anyway:
$ make -nw
make: Entering directory '/home/you/project'
make: *** No rule to make target 'gone.c', needed by 'gone.o'. Stop.
make: Leaving directory '/home/you/project'
$ make -nwk
make: Entering directory '/home/you/project'
make: *** No rule to make target 'gone.c', needed by 'gone.o'.
gcc -c -o a.o a.c
gcc -c -o b.o b.c
make: Leaving directory '/home/you/project'
Sub-makes announce their directory on their own; the top-level make
announces its own only under -w. Without it, every command printed
before the first Entering directory line lands in whatever directory
parse-sh itself was run in, which is the build root only by accident.
The markers are matched in English. A make running under a translated
locale announces its directories in words Bear does not match, so no
directory change is tracked at all and every entry silently carries the
starting directory. LC_ALL=C on the capture is what keeps both of
these right:
LC_ALL=C make -nw | bear parse-sh
ninja -n prints each edge's description rather than the command line
behind it:
$ ninja -n
ninja: Entering directory `build'
[1/3] Building C object CMakeFiles/demo.dir/src/a.c.o
[2/3] Building C object CMakeFiles/demo.dir/src/b.c.o
[3/3] Linking C static library libdemo.a
There is no command in that text, so every line is skipped and the run exits non-zero:
bear: warning: line 2: skipped (glob in executable)
bear: warning: line 3: skipped (glob in executable)
bear: warning: line 4: skipped (glob in executable)
bear: warning: parse-sh: 0 command(s) parsed, 3 line(s) skipped
bear: error: Event production failed: every non-empty line was skipped; no commands parsed (see warnings above)
ninja -t commands prints the real command lines. It prints no
directory markers with them, so name the build directory with
--directory:
ninja -C build -t commands | bear parse-sh -C "$PWD/build"
Make needs no equivalent: -n prints a recipe line even when a leading
@ would have silenced it in a real build, so automake's silent rules
and CMake's Makefile generator both put the full compiler command in the
dry run without V=1 or VERBOSE=1.
Nothing recovers a command the text never contained, so a log missing
whole subdirectories has to be captured again, or the build run under
Bear instead. What is still fixable after the fact is the directory the
parse starts in: --directory sets it, for a log that came from another
machine or another checkout:
bear parse-sh -i build.log -C /home/you/project
Recover compile_commands.json from a build log walks a handed-over log end to end.
A line using shell syntax outside the supported subset is skipped, and each skip is reported on standard error with its line number and reason, followed by a summary:
$ make -nw | bear parse-sh
bear: warning: line 3: skipped (shell keyword)
bear: warning: line 4: skipped (subshell)
bear: warning: parse-sh: 1 command(s) parsed, 2 line(s) skipped
Both lines there hid a compile: a for loop over sources, and a
(cd sub && gcc ...) subshell. A skip costs an entry only when the
skipped line was a compilation, and the run still exits 0 as long as one
line parsed, so read the summary rather than the exit code. The line
numbers are the input's, so capture to a file when you want to look them
up:
make -nw > build.log
bear parse-sh -i build.log
bear parse-sh in the bear(1) man page lists the shell
constructs the subset covers.
Every limit above is the dry run's, not the parser's, and interception
has none of them: it observes the exec() calls the build really makes,
so recursion, shell loops, silenced recipes, and the locale all stop
mattering.
bear -- make
Keep parse-sh for the build that cannot be run again.
Generate compile_commands.json for a Makefile project for running the build under Bear instead.
Recover compile_commands.json from a build log for the end-to-end walkthrough of parsing a saved log.
Bear produces an empty compile_commands.json when the build did run under Bear and still produced nothing.
Command-line options
for --input, --output, --append, and --directory.
How Bear works for what interception observes that text cannot carry.
Recipes for the other tasks.