oxc/crates/oxc_parser/src
Boshen 4706765d2a
refactor(parser): reduce Token size from 32 to 16 bytes (#1962)
Part of #1880

`Token` size is reduced from 32 to 16 bytes by changing the previous
token value `Option<&'a str>` to a u32 index handle.

It would be nice if this handle is eliminated entirely because
the normal case for a string is always
`&source_text[token.span.start.token.span.end]`

Unfortunately, JavaScript allows escaped characters to appear in
identifiers, strings and templates. These strings need to be unescaped
for equality checks, i.e. `"\a"  === "a"`.

This leads us to adding a `escaped_strings[]` vec for storing these
unescaped and allocated
strings.

Performance regression for adding this vec should be minimal because
escaped strings are rare.

Background Reading:

* https://floooh.github.io/2018/06/17/handles-vs-pointers.html
2024-01-09 15:17:02 +08:00
..
js refactor(parser): reduce Token size from 32 to 16 bytes (#1962) 2024-01-09 15:17:02 +08:00
jsx refactor(parser): reduce Token size from 32 to 16 bytes (#1962) 2024-01-09 15:17:02 +08:00
lexer refactor(parser): reduce Token size from 32 to 16 bytes (#1962) 2024-01-09 15:17:02 +08:00
ts refactor(parser): reduce Token size from 32 to 16 bytes (#1962) 2024-01-09 15:17:02 +08:00
context.rs fix(parser): false postive for "Missing initializer in const declaration" in declare + namespace (#1724) 2023-12-18 17:03:42 +08:00
cursor.rs refactor(parser): reduce Token size from 32 to 16 bytes (#1962) 2024-01-09 15:17:02 +08:00
diagnostics.rs fix(parser): error on source larger than 4 GiB (#1860) 2024-01-02 11:05:28 +08:00
lib.rs refactor(parser): reduce Token size from 32 to 16 bytes (#1962) 2024-01-09 15:17:02 +08:00
list.rs fix(parser): fix crashing on empty ParenthesizedExpression with comments (#263) 2023-04-06 17:16:15 +08:00
state.rs refactor(ast): change Option<Vec> to Vec for decorators (#84) 2023-03-02 15:52:46 +08:00