Minifier benchmark currently includes the time to run parser and semantic as well as the minifier itself. Similar to transformer benchmark, only measure the minifier itself in the benchmark.
This will give us a clearer picture of performance changes, and should also reduce variance in the benchmarks.
We no longer need to build semantic data inside the transformer.
The caller should be responsible for handling semantic data and its
errors.
The best way to achieve this in via `CompilerInterface`.
closes#3565
After studying google closure compiler, I'm leaning towards a multi-ast-pass infrastructure for the minifier.
This is one of the few places where we are going to trade maintainability over performance, given the goal of the minifier is compression size not performance.
All of the terminologies and separation of concerns are aligned with google closure compiler.
Infrastructure of `terser` and `esbuild` are not suitable for us to study nor pursuit. Their code are so tightly coupled - I failed to comprehend any of them every time I try to walk through a piece of optmization. Google closure compiler despite being written in Java, it's actually the most readable minifier out there.
To improve performance between ast passes, I envision a change detection system over a portion of the code.
The benchmark will demonstrate the performance regression of running 5 ast passes instead of 2.
To complete this PR, I need to figure out "fix-point" and order of these ast passes.
This PR builds on #3201 to further speed up the benchmarks and reduce CI
time.
* Build and run each benchmark as separate job (like before).
* But now each bench is only built with the dependencies it needs.
* For linter benchmarks, build benchmark in 1 job (like #3201 does).
* Run each linter fixture in a separate job as they're slow.
This reduces total time to complete benchmarks from between 6m-7m to
~4m40s.
All the individual jobs complete in under 1m30s, except for building
linter benchmark which takes 2m30s. So there won't be the problem of
blocking the CI queue that there was before.
NB: I did try this before, and didn't see a benefit. But I realized
today what I was doing wrong - it only works once the caches are
populated by a previous run on main branch.
So the CI times in this PR won't look good, but once it's merged to
main, it will take effect. Here it is running on main branch of my fork:
https://github.com/overlookmotel/oxc/actions/runs/9030511348
I also added a step to delete the temp artefacts which aren't needed
once the run has completed.
The sourcemap implement port from
[rust-sourcemap](https://github.com/getsentry/rust-sourcemap), but has
some different with it.
- Encode sourcemap at parallel, including quote `sourceContent` and
encode token to `vlq` mappings.
- Avoid `Sourcemap` some methods overhead, like `SourceMap::tokens()`
caused extra overhead at common cases. Here using `SourceViewToken` to
instead of it.
The `parser_napi` pseudo-benchmark does not depend on any Oxc crates, so
remove all dependencies for this benchmark. This reduces build time of
this benchmark on CI by ~1 minute.
After #2780, NAPI parser benchmark sometimes took the longest of all the
benchmarks, so making it faster reduces time to complete the benchmarks
overall.
Add NodeJS parser to benchmarks.
Previous attempt #2724 did not work due CodSpeed producing very
inaccurate results (https://github.com/CodSpeedHQ/action/issues/96).
This version runs the actual benchmarks without CodSpeed's
instrumentation. Then another faux-benchmark runs within Codspeed's
instrumented action and just performs meaningless calculations in a loop
for as long as is required to take same amount of time as the original
uninstrumented benchmarks took.
It's unfortunate that we therefore don't get flame graphs on CodSpeed,
but this seems to be the best we can do for now.
Introduce invariant that only a single `lexer::Source` can exist on a thread at one time.
This is a preparatory step for #2341.
2 notes:
Restriction is only 1 x `ParserImpl` / `Lexer` / `Source` on 1 *thread* at a time, not globally. So this does not prevent parsing multiple files simultaneously on different threads.
Restriction does not apply to public type `Parser`, only `ParserImpl`. `ParserImpl`s are not created in created in `Parser::new`, but instead in `Parser::parse`, where they're created and then immediately consumed. So the end user is also free to create multiple `Parser` instances (if they want to for some reason) on the same thread.
This PR adds benchmarks for the lexer. I'm doing some work on optimizing
the lexer and I thought it'd be useful to see the effects of changes in
isolation, separate from the parser.
These benchmarks may not be ideal to keep long-term, but for now it'd be
useful.
In order to do so, it's necessary for `oxc_parser` crate to expose the
lexer, but have done that without adding it to the docs, and using an
alias `__lexer`.
closes#273closes#814
HIR is removed from this PR, with the minifier being commented out.
HIR is a wonderful idea for compiling to lower languages, but after
sitting on it for a few months I found that it only adds confusion and
uncertainties to both myself and future contributors.
It also adds too much burden to maintainers if we plan to support more
downstream tools.
1 AST is the only way.
