The tasks of a lexical analyzer for Haskell
The main tasks of the lexical analyzer is grouping characters into lexemes
and throwing away white space.
This sounds simple enough, but there
are many non-trivial subtasks:
- Removing nested comments,
- Preserving position information,
- Interacting with the parser to implement the layout processing.
- Recognizing string literals,
- Recognizing simple identifiers,
- Recognizing qualified identifiers,
- Recognizing keywords and reserved operators.
- ...
This is why the monolithic code is so complex...