## What This PR Does
- perf(lexer): use bit shifting when parsing hex, octal, and binary
integers instead of `mul_add`-ing on `f64`s. Check out the difference in
assembly generated [here](https://godbolt.org/z/zMEKaeYzh)
- perf(lexer): skip redundant utf8 check when parsing BigInts
- refactor(lexer): remove `unsafe` usage (as per @overlookmotel's
request
[here](https://github.com/oxc-project/oxc/pull/3283#issuecomment-2111814598))
- test(lexer): add numeric parsing unit tests
I don't expect this PR to have a large performance improvement, since
the most common case (`Kind::Decimal`) is not affected. We could do
this, however, by splitting `Kind::Decimal` into `Kind::DecimalFloat`
and `Kind::DecimalInt` when the lexer encounters a `.`
Allow mutable access to scopes tree and symbol table.
Closes#3189.
This completes the v1 scopes-in-traverse implementation, and provides all the primitives required to implement the missing APIs listed in https://github.com/oxc-project/oxc/discussions/3251.
Performance is abysmal, as noted in #3304, but we can fix that later on by taking `Semantic` out of the picture, or optimizing it.
Pass `&mut TraverseCtx` to `Traverse::enter_*` and `Traverse::exit_*` visitor methods. Previously was immutable `&TraverseCtx`.
This is a step towards exposing mutable scope tree + symbol table to visitors.
Move the ancestry stack into it's own type `TraverseAncestry`, and expose it via `ctx.ancestry` "namespace".
The "namespaced" way of getting parent etc (`ctx.ancestry.parent()` rather than just `ctx.parent()`) will come in useful once we introduce methods which require a `&mut TraverseCtx`.
`Traverse` use `Semantic` to construct scopes tree and expose it to visitors via `TraverseCtx`.
Currently scopes tree is immutable. Will expose it as a mutable in a follow-on.
This is extremely inefficient. Semantic does all kinds of stuff (control flow graph etc) which `Traverse` doesn't need, and `Traverse` just throws away all that work after semantic has done it. Intent here is to get a working implementation first, and then to do another pass later on to improve performance.
`oxc_semantic` populate `scope_id` fields in AST nodes as it walks the tree.
This does produce some duplication - scope IDs are stored both in the AST itself, and in `AstNode`. Will clean this up later on.
Move `BabelOptions` to Transformer. The `output.json` is a standard babel configuration. We can reuse BabelOptions to read [babel.config.json](https://babeljs.io/docs/configuration#babelconfigjson) or our configuration(maybe oxc.config.json)
The current `from_babel_options` implementation is copied from the `transform_options` in `test_case.rs`, which I'll completely reimplement next
## What This PR Does
Updates numeric literal token lexing to record when separator characters
(`_`) are found in a new `Token` flag. This then gets passed to
`parse_int` and `parse_float`, removing the need for a second `_` check
in those two functions.
When run locally, I see no change to lexer benchmarks and minor
improvements to codegen benchmarks. For some reason, semantic and source
map benches seem to be doing slightly worse.
Note that I attempted to implement this with `bitflags!` (making
`escaped` and `is_on_newline` flags as well) and this caused performance
degradation. My best guess is that it turned reads on these flags from a
`mov` to a `mov` + a binary and.
---------
Co-authored-by: Boshen <boshenc@gmail.com>
If submodules are outdated, it'll panic with the following message
```
Repository is outdated, please run `just submodules` to update it.
```
For us maintainers, we'll need the env `UPDATE_SNAPSHOT` to force an update.
