This PR builds on #3201 to further speed up the benchmarks and reduce CI
time.
* Build and run each benchmark as separate job (like before).
* But now each bench is only built with the dependencies it needs.
* For linter benchmarks, build benchmark in 1 job (like #3201 does).
* Run each linter fixture in a separate job as they're slow.
This reduces total time to complete benchmarks from between 6m-7m to
~4m40s.
All the individual jobs complete in under 1m30s, except for building
linter benchmark which takes 2m30s. So there won't be the problem of
blocking the CI queue that there was before.
NB: I did try this before, and didn't see a benefit. But I realized
today what I was doing wrong - it only works once the caches are
populated by a previous run on main branch.
So the CI times in this PR won't look good, but once it's merged to
main, it will take effect. Here it is running on main branch of my fork:
https://github.com/overlookmotel/oxc/actions/runs/9030511348
I also added a step to delete the temp artefacts which aren't needed
once the run has completed.
Follow-on from #2751. Further shards linter benchmarks so each fixture runs in its own job.
This reduces total time to run benchmarks by another ~75 secs. So approx 2.5 mins shaved off in total.
Following #2297, this adds another benchmark.
This one is from radix-ui website. I've chosen this particular file
because it differs from the other benchmark sources in 3 ways:
1. JSX not TSX (despite the file extension).
2. Contains no logic, only JSX component hierarchy, and content text.
3. Very small (60 LOC).
The last is particularly important, I think. Often developers will be
working on small files (single component per file convention). And some
possible directions for the parser (SIMD etc) involve optimizing chewing
through chunks of text, with a de-opt at the end to process the final
batch of bytes. If that imposes a penalty on short files, this benchmark
will surface it.
---------
Co-authored-by: Boshen <boshenc@gmail.com>
