Tempting paths for an implementor
Can a lexer be split up into a number of simpler passes?
- Can nested comments be removed first?
- If so, the rest could be specified using regular expressions, and perhaps
implemented easily using a standard tool.
- Can qualified identifiers be recognized in a separate pass?
- If so, the DFA
produced by a lexer generator could be much smaller.
- Can keywords and reserved operators be recognized in a separate pass?
- If so, the DFA
produced by a lexer generator could be much smaller.