overlookmotel 0e32618664 refactor(parser): combine token kinds for skipped tokens (#2072) ... Small optimization to the lexer. Whitespace, line breaks, and comments are all skipped by `read_next_token()`. At present there's a different `Kind` for each, and `read_next_token()` decides whether to skip with `matches!(kind, Kind::WhiteSpace | Kind::NewLine | Kind::Comment | Kind::MultiLineComment)`. These `Kind`s are used for no other purpose, so there seems little reason to differentiate them. This PR combines them all into `Kind::Skip`, so then the test of whether to skip is reduced to `kind == Kind::Skip`. Only produces ~0.3% performance bump on parser benchmarks. But, why not?...		2024-01-18 21:14:12 +08:00
..
examples	chore(clippy): enable undocumented_unsafe_blocks	2023-10-16 15:18:14 +08:00
src	refactor(parser): combine token kinds for skipped tokens (#2072)	2024-01-18 21:14:12 +08:00
Cargo.toml	perf(parser): lexer byte handlers consume ASCII chars faster (#2046)	2024-01-16 12:31:45 +08:00