This is an unofficial snapshot of the ISO/IEC JTC1 SC22 WG21 Core Issues List revision 115d. See http://www.open-std.org/jtc1/sc22/wg21/ for the official list.
2024-10-26
The syntactic nonterminal punctuator appears in the grammar for token in 5.6 [lex.token], but it is nowhere defined. It should be merged with operator and given an appropriate list of tokens as a definition for the merged term.
Proposed resolution (October, 2017):
Change 5.5 [lex.digraph] paragraph 2 as follows
In all respects of the language except in an attribute-token (9.12.1 [dcl.attr.grammar]), each alternative token behaves the same, respectively, as its primary token, except for its spelling.18 The set of alternative tokens...
Change the grammar in 5.6 [lex.token] as follows:
{ } [ ] ( ) ; : ... ? :: . .* + - * / % ^ & | ~ ! = < > += -= *= /= %= ^= &= |= << >> <<= >>= == != <= >= && || ++ -- , ->* ->
Change 5.6 [lex.token] paragraph 1 as follows:
There are
fivefour kinds of tokens: identifiers, keywords, literals,19operators, and other separatorsand symbols.Blanks, horizontal and vertical tabs, newlines, formfeeds, and comments (collectively, “white space”), as described below, are ignored except as they serve to separate tokens. [Note: Some white space is required to separate otherwise adjacent identifiers, keywords, numeric literals, and alternative tokens containing alphabetic characters. —end note]Each preprocessing-token resulting from translation phase 6 is converted into the corresponding token as follows:
If the preprocessing-token is an identifier or is one of the preprocessing-op-or-punc tokens new or delete, the resulting token is a keyword if it is listed in Table 5, and otherwise is an identifier.
Otherwise, if the preprocessing-token is a pp-number with the lexical form of an integer-literal or floating-literal, or is a character-literal or string-literal, the resulting token is of the corresponding form.
Otherwise, if the preprocessing-token is a pp-number with the lexical form of a user-defined-integer-literal or user-defined-floating-literal or is a user-defined-character-literal or user-defined-string-literal, the resulting token is a user-defined-literal.
Otherwise, if the preprocessing-token is a preprocessing-op-or-punc, and there is a corresponding symbol (after converting alternative token representations to their primary tokens), the resulting token is that symbol.
Otherwise, the program is ill-formed.
[Note: Within an attribute-token (9.12.1 [dcl.attr.grammar]), a token formed from a preprocessing-token that satisfies the syntactic requirements of an identifier is considered to be an identifier with the spelling of the preprocessing-token. —end note]
Delete the final sentence of 5.12 [lex.operators] paragraph 1.
Each preprocessing-op-or-punc is converted to a single token in translation phase 7 (5.2 [lex.phases]).