preprocessor: C++ module-directives

C++20 modules introduces a new kind of preprocessor directive -- a
module directive.  These are directives but without the leading '#'.
We have to detect them by sniffing the start of a logical line.  When
detected we replace the initial identifiers with unspellable tokens
and pass them through to the language parser the same way deferred
pragmas are.  There's a PRAGMA_EOL at the logical end of line too.

One additional complication is that we have to do header-name lexing
after the initial tokens, and that requires changes in the macro-aware
piece of the preprocessor.  The above sniffer sets a counter in the
lexer state, and that triggers at the appropriate point.  We then do
the same header-name lexing that occurs on a #include directive or
has_include pseudo-macro.  Except that the header name ends up in the
token stream.

A couple of token emitters need to deal with the new token possibility.

	gcc/c-family/
	* c-lex.c (c_lex_with_flags): CPP_HEADER_NAMEs can now be seen.
	libcpp/
	* include/cpplib.h (struct cpp_options): Add module_directives
	option.
	(NODE_MODULE): New node flag.
	(struct cpp_hashnode): Make rid-code a bitfield, increase bits in
	flags and swap with type field.
	* init.c (post_options): Create module-directive identifier nodes.
	* internal.h (struct lexer_state): Add directive_file_token &
	n_modules fields.  Add module node enumerator.
	* lex.c (cpp_maybe_module_directive): New.
	(_cpp_lex_token): Call it.
	(cpp_output_token): Add '"' around CPP_HEADER_NAME token.
	(do_peek_ident, do_peek_module): New.
	(cpp_directives_only): Detect module-directive lines.
	* macro.c (cpp_get_token_1): Deal with directive_file_token
	triggering.
This commit is contained in:
Nathan Sidwell
2020-11-18 10:24:12 -08:00
parent 7ceb899e93
commit c9c3d5f28a
6 changed files with 514 additions and 4 deletions
+7 -3
View File
@@ -487,6 +487,9 @@ struct cpp_options
/* Nonzero for the '::' token. */
unsigned char scope;
/* Nonzero means tokenize C++20 module directives. */
unsigned char module_directives;
/* Holds the name of the target (execution) character set. */
const char *narrow_charset;
@@ -842,6 +845,7 @@ struct GTY(()) cpp_macro {
#define NODE_USED (1 << 5) /* Dumped with -dU. */
#define NODE_CONDITIONAL (1 << 6) /* Conditional macro */
#define NODE_WARN_OPERATOR (1 << 7) /* Warn about C++ named operator. */
#define NODE_MODULE (1 << 8) /* C++-20 module-related name. */
/* Different flavors of hash node. */
enum node_type
@@ -900,11 +904,11 @@ struct GTY(()) cpp_hashnode {
unsigned int directive_index : 7; /* If is_directive,
then index into directive table.
Otherwise, a NODE_OPERATOR. */
unsigned char rid_code; /* Rid code - for front ends. */
unsigned int rid_code : 8; /* Rid code - for front ends. */
unsigned int flags : 9; /* CPP flags. */
ENUM_BITFIELD(node_type) type : 2; /* CPP node type. */
unsigned int flags : 8; /* CPP flags. */
/* 6 bits spare (plus another 32 on 64-bit hosts). */
/* 5 bits spare (plus another 32 on 64-bit hosts). */
union _cpp_hashnode_value GTY ((desc ("%1.type"))) value;
};