Interaction of error and -- param produces different results in output of 3c

[kyleheadley]

When running 3c from the top directory of the clang project, I get the usual message about a missing compile_commands, and otherwise the following produces the expected output build/bin/3c clang/test/3C/allocator.c > outfile

However, running the equivalent command from the build directory, ie bin/3c ../clang/test/3C/allocator.c > outfile produces an non-rewritten "foo" function along with some errors starting with:

warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
/Users/kyleheadley/Documents/Projects/3C/checkedc-testing/build/../clang/test/3C/allocator.c:13:1: error: 
      unknown type name '_Itype_for_any'
_Itype_for_any(T) void *malloc(size_t size) : itype(_Array_ptr<T>) byte_...
^

This could be a configuration issue on my machine, or could be a side affect of the compile_commands file in that directory.

The strange part is that when adding the dashes, i.e. bin/3c ../clang/test/3C/allocator.c -- > outfile all console messages are suppressed and the output is the correctly rewritten result.

So -- is either incorrectly suppressing an error (an undefined malloc should be marked WILD), or is selecting a different project configuration (choosing a malloc that is defined). Or something else. Either way this seems to be undesirable.

[kyleheadley]

After a few more encounters with this bug, it seems that my system is running 3c in C++ mode, as mentioned in the error. The -- prevents this. I'm guessing that it's triggered by code that appears to be C++, like the additional features of checkedc over c. It shows up on code that's been at least partially converted.

[mattmccutchen-cci]

I think I figured out what is going on here. A little-known feature of LibTooling (on which 3c is based) is that if the command line includes a --, then no compilation database is loaded and any arguments after the -- are taken as compiler options for every source file. If there is no --, then LibTooling tries to find a compilation database in an ancestor of the -p directory, or if -p is not given, an ancestor directory of the first source file.

LibTooling computes the absolute path of the first source file by prepending the working directory. In the first case, this is /PATH/TO/YOUR/WORKING/TREE/clang/test/3C/allocator.c, and none of the ancestor directories contains a compile_commands.json file. In the second case, the path is /PATH/TO/YOUR/WORKING/TREE/build/../clang/test/3C/allocator.c, so LibTooling walks up to the "ancestor" directory /PATH/TO/YOUR/WORKING/TREE/build (arguably a bug, like #327) and finds /PATH/TO/YOUR/WORKING/TREE/build/compile_commands.json. Since the input file ../clang/test/3C/allocator.c is not listed in the compilation database, LibTooling chooses a file in the compilation database according to some heuristics and uses its compiler options. Apparently it is choosing a C++ file (no surprise because most of Clang is written in C++) and getting compiler options that cause the 3C input file to be treated as C++. The Checked C parser extensions are not active in C++ mode, so we get parse errors.

My conclusion: If we want to run 3c without a compilation database, we should pass --. This will ensure that LibTooling doesn't automatically detect a compilation database we don't want to use and, as a bonus, suppress the error about not finding a compilation database. Also, anything that is currently running 3c -extra-arg=-foo ... without a compilation database should be able to run 3c ... -- -foo instead. Even though the .. bug that affected Kyle's particular case might be fixed in a newer version of Clang, it's good to get in the habit of using -- whenever we don't want to use a compilation database in order to avoid surprises. For example, I am converting the 3C regression tests that currently use -output-postfix to write to the lit temporary directory instead (#412), and that directory is under the build directory, so when the output files are passed back into 3C for idempotence tests, I get the same problem unless I use --.

Are we OK with keeping LibTooling's "standard" behavior (but documenting it better for our users in the 3c -help output), or do we want to try to override the command line processing in 3c?

[mattmccutchen-cci]

We decided to keep the standard use of -- and document it in the 3c -help output. I will do that in #412.