In computer science, lexical analysis is the process of converting a sequence of characters into a sequence of tokens, i.e. meaningful character strings. A program or function that performs lexical analysis is called a lexical analyzer, lexer, tokenizer, or scanner, though "scanner" is also used for the first stage of a lexer. A lexer is generally combined with a parser, which together analyze the syntax of computer languages, such as in compilers for programming languages, but also HTML parsers in web browsers, among other examples.
Strictly speaking, a lexer is itself a kind of parser – the syntax...