Add recursive macro expansion for SConsCPPConditionalScanner #4656
Comments
relfock
changed the title
|
The intent of this post is to document some of the existing issues and limitations of the CConditionalScanner, and to a lesser extent, the CScanner. Please report any erroneous assumptions, interpretations, and conclusions. Table of Contents:
1. IntroductionThe content presented is based on insights and results derived from a series of examples and tests. Section 2 provides repository links for the source code and scripts used for the examples and tests. Section 3 documents some of the issues identified with the SCons C scanners. Section 4 documents some of the limitations of the SCons C scanners. Section 5 contains a discussion of results from the custom compiler preprocessor scanners. Section 6 contains documentation of the file inclusion search paths from the c and c++ standards and the MSVC and GNU compilers. Section 7 contains a discussion of the research presented. Addendum A contains additional known issues for the SCons C scanners. 2. Research RepositorySubsection map: The research repository is available at: https://github.com/jcbrill/scons-prototypes/tree/main/scons/scons-issue-4656. Almost all of the test examples and scripts assume that:
The non-windows scripts were run in WSL on Windows. The author has limited experience on non-windows platforms so the bash scripts are likely cringe-inducing for non-windows users. The primary test environments employed for this document are:
2.1 External LibraryThe external library is a limited subset of the author's research and reference library for msvc detection. Folder layout: site-scons
├─ mswindev
├─ __init__.py
├─ pyargs.py
└─ pyinfo.pypyinfo.py is a minimal shim in order for pyargs.py to work in isolation for the research repository. 2.1.1 Platform Split/Quote Command-Line ArgumentsSource file: site-scons/mswindev/pyargs.py pyargs.py exports the following functions:
The three (3) platforms supported are:
There is a known issue with the SCons splitting of command-line arguments containing quoted strings with embedded spaces. 2.2 Custom ScannersThree custom scanners were developed as part of the research effort. site-scons
├─ ppscanner
├─ __init__.py
├─ _common.py
├─ _preprocess.py
├─ gcc.py
├─ msvc.py
└─ scons.py2.2.1 GCCPreProcessorScannerSource files: Disclaimers:
The GCCPreProcessorScanner parses the output produced by the gcc compiler preprocessor for include dependencies. System include file detection is based on the optional flags present in the line directive output. Errors are possible based on the author's understanding of the meaning attached to the line directive flag values. 2.2.2 MSVCPreProcessorScannerSource files: Disclaimers:
The MSVCPreProcessorScanner parses the output produced by the msvc compiler preprocessor for include dependencies. A path classification heuristic is employed for system include file detection when processing the msvc preprocessor output. The path classification heuristic is incomplete and should be revisited. 2.2.3 CConditionalModScannerSource files: The CConditionalModScanner is implemented by subclassing SConsCPPConditionalScanner and SConsCPPConditionalScannerWrapper and overriding a limited set of methods to address some, but not all, of the issues presented in Section 3 with the CConditionalScanner. 3. SCons Scanner IssuesSubsection map:
3.1 CConditionalScanner RecursionFor recursive scanners (e.g., CScanner), the dependencies list produced when invoking the scanner scan function directly and the dependencies list produced by SCons execution via get_implicit_dependencies may not be the same. Lines 978 to 1011 in 144af4a Line 1009 calls the Lines 283 to 316 in 144af4a Lines 312-313 contain the However, for scanner implementations that automatically recurse through discovered dependencies (e.g., CConditionalScanner), the Folder layout: RecurseTest
├─ include
│ ├─ include
│ │ └─ recurse2.h
│ └─ recurse1.h
├─ SConstruct
└─ main.cSConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)main.c: #define SKIP_INCLUDE2
#include "include/recurse1.h"
int main(void);
int main(void) {}include/recurse1.h: #ifndef RECURSE1_H
#define RECURSE1_H
// #pragma message("recurse1.h")
#ifndef SKIP_INCLUDE2
#include "include/recurse2.h"
#endif
#endifinclude/include/recurse2.h: #ifndef RECURSE2_H
#define RECURSE2_H
// #pragma message("recurse2.h")
#endifCConditionalScanner:
The SCons produced tree using CConditionalScanner is: The SConsCPPConditionalScannerWrapper __call__ method (Lines 297-310) creates a new SConsCPPConditionalScanner object for each source file scanned. When file include/recurse1.h is scanned again due to being returned in the recurse_nodes list, all caller definitions produced in the source code (e.g., As the CConditionalSCanner implementation is already recursive, the wrapper implementation of def recurse_nodes(self, nodes):
return []Scanner dependencies:
3.2 PreProcessor File InclusionThere are issues with the include processing in the Lines 583 to 587 in 144af4a Include file handling issues:
Folder layout: MultipleTest
├─ include
│ ├─ include1.h
│ ├─ include2.h
│ ├─ include3.h
│ └─ include4.h
├─ SConstruct
└─ main.cSConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)main.c: #include "include/include1.h"
int main(void);
int main(void) {}include/include1.h: include/include2.h: include/include3.h: include/include4.h: File include/heade2.h intentionally does not have include guards and is expected to be included more than once. Graphical representation of the source tree: graph LR;
main.c-->include1.h;
include1.h-->include2.h;
include1.h-->include2.h;
include2.h-->include3.h;
include2.h-->include4.h;
Expected include processing trails from the root file to all leaf files: Scanner dependencies:
CConditionalScannerMod and CConditionalScanner result list sequence: 3.3 CConditionalScanner Alternate Search PathsInclude file search path notes:
Code fragments:
Issues:
The Folder layout: KnownIncl
└─ include
├─ include1.h
├─ include2.h
├─ include3.h
└─ include4.h
KnownTest
├─ include
│ ├─ include2.h
│ └─ include4.h
├─ SConstruct
└─ main.cKnownTest/SConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
CPPPATH = ['./include', '../KnownIncl/include'],
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)KnownTest/main.c: #include <include1.h>
#include <include2.h>
#include "include3.h"
#include "include4.h"
int main(void);
int main(void) {}Note: the header files (e.g., include1.h) are not shown as they effectively contain include guards only. Scanner dependencies:
3.4 Angle Bracket Include Search PathInclude file search path notes:
Modern msvc, mingw-w64, and gnu compilers do not appear to include the directory of the source file in the angle bracket search path by default. Code fragments:
Scanner inclusion of the source file directory in the angle bracket search path:
Folder layout: AngleBracketTest
├─ SConstruct
├─ anglebracket.h
├─ limits.h
└─ main.cSConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)main.c: #include <limits.h>
#include <anglebracket.h>
int main(void);
int main(void) {}Notes:
Scanner dependencies:
3.4.1 CConditionalScanner Extended ExampleFolder layout: IncSysPathTest
├─ include
│ └─ include.h
├─ SConstruct
├─ anglebracket.h
├─ limits.h
└─ main.cSConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)main.c: #include "include/include.h"
int main(void);
int main(void) {}include/include.h: #ifndef INCLUDE_H
#define INCLUDE_H
// #pragma message("include.h")
#include <limits.h>
#include <anglebracket.h>
#endifNotes:
Scanner dependencies:
3.5 Text Transformations Prior to ScanningThe CConditionalScanner and CScanner scanners do not perform all text transformations as described in the c and c++ standards on source file contents prior to scanning for preprocessor directives. Notably, comments are not stripped from the source file contents prior to scanning for preprocessor directives. The SCons implementations remove extraneous text from preprocessor directives after detecting the directives. Due to the current scanner implementations:
A simple text transformation implementation to remove comments can be found here: https://github.com/jcbrill/scons-prototypes/blob/main/scons/scons-issue-4656/site-scons/ppscanner/_preprocess.py The simple text transformation code linked above does not handle c++ raw strings. The implementation was complete prior to discovering the existence of raw strings in c++. Based on the language in the c++ standard, handling raw strings makes implementation significantly more challenging. Note that regular expressions cannot be employed to detect and remove comments globally in the source file contents due to the possibility of comment delimiters appearing in literal strings (e.g., Folder layout: CommentTest
├─ active
│ ├─ comment3.h
│ └─ comment4.h
├─ inactive
│ ├─ comment1.h
│ └─ comment2.h
├─ SConstruct
└─ main.cSConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)main.c: /* #include "inactive/comment1.h" */
/*
#include "inactive/comment2.h"
*/
/* */ #include "active/comment3.h" /* */
/*
*/ #include "active/comment4.h"
int main(void);
int main(void) {}Scanner dependencies:
CConditionalScanner and CScanner:
For the CConditionalScanner, processing commented-out directives (.e.g., 4. SCons Scanner LimitationsSubsection map: 4.1 #pragma once Is Not SupportedSee Section 3.2 for discussion concerning If the issues in Section 3.2 are resolved, it might be beneficial to investigate adding limited support for 4.2 Compiler Defined SymbolsBy default, the SCons c and c++ scanners do not include any of the compiler-specific pre-defined symbols that c and c++ source files may employ in conditional logic. Folder layout: MacroTest
├─ SConstruct
├─ arch_x64.h
├─ arch_x86.h
├─ compiler_gcc.h
├─ compiler_mingw.h
├─ compiler_msvc.h
└─ main.cSConstruct: from sconstruct_common import scanner_configuration
DefaultEnvironment(tools=[])
scanner_cfg = scanner_configuration()
env = Environment(
tools = scanner_cfg.tools,
)
sourcefile = "main.c"
if scanner_cfg.scanner_dependencies:
scanner_cfg.dependencies(sourcefile, env)
else:
env.Program(sourcefile)
#if defined(__MINGW32__) || defined(__MINGW64__)
#include "compiler_mingw.h"
#elif defined (__GNUC__) || defined(__GNUG__)
#include "compiler_gcc.h"
#elif defined(_MSC_VER)
#include "compiler_msvc.h"
#else
#error "Unsupported compiler!"
#endif
#if defined(__amd64__) || defined(__amd64) || defined(__x86_64__) || defined(__x86_64) || defined(_M_X64) || defined(_M_AMD64)
#include "arch_x64.h"
#elif defined(i386) || defined(__i386) || defined(__i386__) || defined(_M_IX86) || defined(_X86_)
#include "arch_x86.h"
#else
#error "Unsupported architecture!"
#endif
int main(void);
int main(void) {}Scanner dependencies:
5. Custom Compiler Preprocessor ScannersSee Section 2.2.1 and Section 2.2.2 for discussion of the custom compiler preprocessor scanner disclaimers and limitations. The file main.c referenced in the top post of Issue 4656 was modified to support MSVC compilers (versions 6.0 and later) and is implemented as ScannerTest/test-03.c which is shown below. The results produced by the MSVCPreProcessorScanner and the GCCPreProcessorScanner for six (6) different configurations of the FEATURE_A_ENABLED value are shown below. Both custom compiler preprocessor scanners produce the expected results for all 6 test configurations. ScannerTest/test-03.c: #if defined(_MSC_VER) && _MSVC_VER < 1400
/* MSVC 6.0 - 7.1 */
#define IS_ENABLED(config_macro) _IS_ENABLED1(config_macro,0)
#define _IS_ENABLED1(val) _IS_ENABLED2(_XXXX ## val)
#define _XXXX1 _YYYY,
#define _IS_ENABLED2(one_or_two_args) _IS_ENABLED3((one_or_two_args 1, 0))
#define _IS_ENABLED3(arglist) _IS_ENABLED4 arglist
#define _IS_ENABLED4(ignore_this, val) val
#elif defined(_MSC_VER)
#define IS_ENABLED(config_macro,...) _IS_ENABLED1(config_macro)
#define _IS_ENABLED1(...) _IS_ENABLED2(_XXXX ## __VA_ARGS__)
#define _XXXX1 _YYYY,
#define _IS_ENABLED2(one_or_two_args) _IS_ENABLED3((one_or_two_args 1, 0))
#define _IS_ENABLED3(arglist) _IS_ENABLED4 arglist
#define _IS_ENABLED4(ignore_this, val, ...) val
#else
/**
* @brief Macro for checking if the specified identifier is defined and it has
* a non-zero value.
*
* Normally, preprocessors treat all undefined identifiers as having the value
* zero. However, some tools, like static code analyzers, can issue a warning
* when such identifier is evaluated. This macro gives the possibility to suppress
* such warnings only in places where this macro is used for evaluation, not in
* the whole analyzed code.
*/
#define IS_ENABLED(config_macro) _IS_ENABLED1(config_macro)
/* IS_ENABLED() helpers */
/* This is called from IS_ENABLED(), and sticks on a "_XXXX" prefix,
* it will now be "_XXXX1" if config_macro is "1", or just "_XXXX" if it's
* undefined.
* ENABLED: IS_ENABLED2(_XXXX1)
* DISABLED IS_ENABLED2(_XXXX)
*/
#define _IS_ENABLED1(config_macro) _IS_ENABLED2(_XXXX##config_macro)
/* Here's the core trick, we map "_XXXX1" to "_YYYY," (i.e. a string
* with a trailing comma), so it has the effect of making this a
* two-argument tuple to the preprocessor only in the case where the
* value is defined to "1"
* ENABLED: _YYYY, <--- note comma!
* DISABLED: _XXXX
*/
#define _XXXX1 _YYYY,
/* Then we append an extra argument to fool the gcc preprocessor into
* accepting it as a varargs macro.
* arg1 arg2 arg3
* ENABLED: IS_ENABLED3(_YYYY, 1, 0)
* DISABLED IS_ENABLED3(_XXXX 1, 0)
*/
#define _IS_ENABLED2(one_or_two_args) _IS_ENABLED3(one_or_two_args 1, 0)
/* And our second argument is thus now cooked to be 1 in the case
* where the value is defined to 1, and 0 if not:
*/
#define _IS_ENABLED3(ignore_this, val, ...) val
#endif
#if IS_ENABLED(FEATURE_A_ENABLED)
#include "if-true.h"
#else
#include "if-false.h"
#endif
int main(void);
int main(void) {}ScannerTest/SConstruct test configurations for ScannerTest/test-03.c: test_config(
sourcefile="test-scons-03.c",
expected=["if-false.h"],
),
test_config(
sourcefile="test-scons-03.c",
cppdefines=[("FEATURE_A_ENABLED", "")],
expected=["if-false.h"],
),
test_config(
sourcefile="test-scons-03.c",
cppdefines=[("FEATURE_A_ENABLED", None)],
expected=["if-true.h"],
),
test_config(
sourcefile="test-scons-03.c",
cppdefines=[("FEATURE_A_ENABLED", "0")],
expected=["if-false.h"],
),
test_config(
sourcefile="test-scons-03.c",
cppdefines=[("FEATURE_A_ENABLED", "1")],
expected=["if-true.h"]),
test_config(
sourcefile="test-scons-03.c",
cppdefines=[("FEATURE_A_ENABLED", "2")],
expected=["if-false.h"],
),Output fragments (output/win-output-scanner-tests.txt): test-scons-03.c MSVC C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.EXE /nologo /E /w S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c MSVC ['if-false.h']
test-scons-03.c MSVC pass
test-scons-03.c GCC C:\mingw-w64\x86_64-1120-win32-seh-rt_v9-rev0\mingw64\bin\gcc.EXE -fsyntax-only -E -dI -w S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c GCC ['if-false.h']
test-scons-03.c GCC pass
...
test-scons-03.c MSVC C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.EXE /nologo /E /w /DFEATURE_A_ENABLED= S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c MSVC /DFEATURE_A_ENABLED=
test-scons-03.c MSVC ['if-false.h']
test-scons-03.c MSVC pass
test-scons-03.c GCC C:\mingw-w64\x86_64-1120-win32-seh-rt_v9-rev0\mingw64\bin\gcc.EXE -fsyntax-only -E -dI -w -DFEATURE_A_ENABLED= S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c GCC -DFEATURE_A_ENABLED=
test-scons-03.c GCC ['if-false.h']
test-scons-03.c GCC pass
...
test-scons-03.c MSVC C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.EXE /nologo /E /w /DFEATURE_A_ENABLED S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c MSVC /DFEATURE_A_ENABLED
test-scons-03.c MSVC ['if-true.h']
test-scons-03.c MSVC pass
test-scons-03.c GCC C:\mingw-w64\x86_64-1120-win32-seh-rt_v9-rev0\mingw64\bin\gcc.EXE -fsyntax-only -E -dI -w -DFEATURE_A_ENABLED S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c GCC -DFEATURE_A_ENABLED
test-scons-03.c GCC ['if-true.h']
test-scons-03.c GCC pass
...
test-scons-03.c MSVC C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.EXE /nologo /E /w /DFEATURE_A_ENABLED=0 S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c MSVC /DFEATURE_A_ENABLED=0
test-scons-03.c MSVC ['if-false.h']
test-scons-03.c MSVC pass
test-scons-03.c GCC C:\mingw-w64\x86_64-1120-win32-seh-rt_v9-rev0\mingw64\bin\gcc.EXE -fsyntax-only -E -dI -w -DFEATURE_A_ENABLED=0 S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c GCC -DFEATURE_A_ENABLED=0
test-scons-03.c GCC ['if-false.h']
test-scons-03.c GCC pass
...
test-scons-03.c MSVC C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.EXE /nologo /E /w /DFEATURE_A_ENABLED=1 S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c MSVC /DFEATURE_A_ENABLED=1
test-scons-03.c MSVC ['if-true.h']
test-scons-03.c MSVC pass
test-scons-03.c GCC C:\mingw-w64\x86_64-1120-win32-seh-rt_v9-rev0\mingw64\bin\gcc.EXE -fsyntax-only -E -dI -w -DFEATURE_A_ENABLED=1 S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c GCC -DFEATURE_A_ENABLED=1
test-scons-03.c GCC ['if-true.h']
test-scons-03.c GCC pass
...
test-scons-03.c MSVC C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.EXE /nologo /E /w /DFEATURE_A_ENABLED=2 S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c MSVC /DFEATURE_A_ENABLED=2
test-scons-03.c MSVC ['if-false.h']
test-scons-03.c MSVC pass
test-scons-03.c GCC C:\mingw-w64\x86_64-1120-win32-seh-rt_v9-rev0\mingw64\bin\gcc.EXE -fsyntax-only -E -dI -w -DFEATURE_A_ENABLED=2 S:\SCons\Test-4656\tests\ScannerTest\test-scons-03.c
test-scons-03.c GCC -DFEATURE_A_ENABLED=2
test-scons-03.c GCC ['if-false.h']
test-scons-03.c GCC pass
...6. File Inclusion Search PathsSubsection map: The include file search paths for quote, angle bracket, and preprocessing token forms of include directives are implementation dependent. 6.1 C and C++ StandardsConditional file inclusion documentation is not included. Refer to the appropriate standards documents for more information. 6.1.1 C23 StandardReference: C23 (ISO/IEC 9899:2024 (en) — N3220 working draft).
6.1.2 C++23 StandardReference: C++23 (ISO/IEC N4950 working draft).
6.2 Compilers6.2.1 MSVC CompilerReference: https://learn.microsoft.com/en-us/cpp/preprocessor/hash-include-directive-c-cpp?view=msvc-170.
6.2.2 GCC CompilerReference: https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html
There are a number of command-line options you can use to add additional directories to the search path. The most commonly-used option is -Idir, which causes dir to be searched after the current directory (for the quote form of the directive) and ahead of the standard system directories. You can specify multiple -I options on the command line, in which case the directories are searched in left-to-right order. If you need separate control over the search paths for the quote and angle-bracket forms of the ‘#include’ directive, you can use the -iquote and/or -isystem options instead of -I. See Invocation, for a detailed description of these options, as well as others that are less generally useful. If you specify other options on the command line, such as -I, that affect where the preprocessor searches for header files, the directory list printed by the -v option reflects the actual search path used by the preprocessor. Note that you can also prevent the preprocessor from searching any of the default system header directories with the -nostdinc option. This is useful when you are compiling an operating system kernel or some other program that does not use the standard C library facilities, or the standard C library itself. 7. DiscussionThe estimated effort/complexity to address the issues described in Section 3 is summarized in the following table.
Notes:
Refer to the individual linked sections for a more comprehensive discussion and examples of each of the issues. As described in Section 6.2, the gcc and MSVC compilers have different search path implementations for quoted include files. This could possibly lead to the CConditionalScanner producing erroneous dependencies if the compiler search path rules were relied on when writing the source code for a specific compiler. The issues discussed in this document coupled with the issues documented earlier (reproduced in Section A.1 below) will take a significant development effort to resolve. The author has doubts about the efficacy of a compiler agnostic conditional scanner. File inclusion search paths are implementation defined. Compatibility with the c and c++ standards needs to continuously evolve. Correctly evaluating preprocessing directives is very difficult and is predicated on the correct discovery and inclusion of dependent files. For the gcc and MSVC compilers, it may be less involved and more maintainable to use the compiler preprocessors directly to detect implicit dependencies when compared to fixing the known issues with the CConditionalScanner. However, there are a number of challenges involved with using the compiler preprocessors directly as well. Basically, one set of challenges is traded for a different set of challenges. A. Known Issues AddendumSubsection map: A.1 SCons/cpp.pyReference: #4629 (comment) (edited locally) Known issues, potential issues and research items for
A.2 Test Suite |
|
I will refrain from too much commentary while it's Under Construction... the choice of how to implement the C scanner (including C++; pundits are demanding never to use the C/C++ term as they're two completely different languages - really, guys?) was absolutely a tradeoff. Several people over the years have asked to just run a "real" preprocessor instead, but the demands of finding deps are pretty low, it was concluded, and having to deal with a host of different compilers that need to be asked in different ways, and might report dep information in different ways was - presumably - just too much of a complication (I wasn't there, guessing).. The CConditionalScanner - many years later - tried to mitigate some of the downsides of the tradeoff by restoring missing features, but comes at additional cost. I understand the original did not make a distinction between |
|
@mwichmann Your feedback is appreciated at all times. I'm counting on you to find all erroneous assertions. No pressure.
References above to c/c++ were changed to c and c++. In some instances the reference may have been to the preprocessor. In some compilers there can be a single preprocessor that handles both the c and c++ languages.
See Section 2.2.1 GCCPreProcessorScanner and 2.2.2 MSVCPreProcessorScanner above for links to quick-and-dirty-likely-error-riddled implementations of SCons Scanners that call the compiler preprocessor and parse the results for gcc and msvc. Using the compiler preprocessor trades one set of problems for a different set of problems. The biggest issue with using the compiler preprocessor is resolving how to classify the absolute include path. The three mechanisms in play: via CPPPATH includes, system, or via relative links in the include statement. Due to relative includes, the files can be arbitrarily anywhere (e.g., out-of-tree). The human readable output sent to stderr for gcc and msvc contains absolute paths. For msvc, the absolute path is preceded by "Note: including file:" which is language/locale specific (i.e., "English") and would have to be internationalized (i.e., i18n). The preprocessor output sent to stdout is more voluminous but is somewhat straightforward. The gcc preprocessor includes flags on the line directives that indicate if the included file is a system file. No such luck for msvc, and a quick-and-dirty-heuristic of known system paths is used. Note that allowing a user to import from the os environment can make the problem harder. Both scanners can be configured to "cross-check" the stdout processing against the stderr output. This is disabled by default. A missing include file causes a compiler preprocessor error and the dependencies list might be truncated. There is no attempt to detect error messages in stderr. The current state of the repository displays the command-line used to invoke the compiler preprocessor.
If a missing file is needed by the compiler preprocessor, the build is going to fail anyway. Tangential Topic: See Section 2.1.1 Platform Split/Quote Command-Line Arguments for the link to pyargs.py. Included is code to split/quote/join arguments for Windows. It is part of my own research library which is undergoing yet another rewrite and has not been committed yet. This particular code could land in it's own repository for illustration. I'm hoping to cover some of this ground in Sections 2 and possible 4 above. |
|
@mwichmann Off-topic: inline LaTeX math-mode color definitions As you know, much to your annoyance, I am a fan of using inline LaTeX math-mode to change the foreground color text. As you have rightly pointed out, the quality of the color definition contrast is dependent on the background color (i.e., colors that look good on a light background may not look good on a dark background). A while ago I wrote a brute-force script that attempted to find common color definitions that are reasonable for all of the GitHub appearance themes. Using color contrast calculations and some funky ranking and filtering, a number of color definition tables were produced using a measure of perceptual difference for a set of independent hue ranges. I typically use red, yellow, and green to highlight text. It turns out that decent red and yellow hues are more difficult to find when considering light and dark backgrounds. The WIP post above is using the red and yellow definitions from Column 0 of the color definition tables. The color definition tables can be found at: https://github.com/jcbrill/sandbox/tree/main/github/github-colors-bg At your convenience, any feedback from viewing with your light and dark backgrounds would be appreciated. |
The macro expansion for #if (and possibly other statements) is not recursive. Only the first expansion is performed. In this case, evaluation results in a non empty string (the expanded string containing another macro) which is True causing the inclusion of the header. Ref: #4623 (comment)
SConstruct file:
main.c file:
output:
More info can be found in the bug #4623
The text was updated successfully, but these errors were encountered: