[](https://renovatebot.com)
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [CodSpeedHQ/action](https://togithub.com/CodSpeedHQ/action) | action |
major | `v2` -> `v3` |
---
### Release Notes
<details>
<summary>CodSpeedHQ/action (CodSpeedHQ/action)</summary>
### [`v3`](https://togithub.com/CodSpeedHQ/action/compare/v2...v3)
[Compare Source](https://togithub.com/CodSpeedHQ/action/compare/v2...v3)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 10am on monday" in timezone
Asia/Shanghai, Automerge - At any time (no schedule defined).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View the
[repository job log](https://developer.mend.io/github/oxc-project/oxc).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Upgrading to Rust 1.80.0 (#4474) regressed some benchmarks (#4478). We
didn't notice because benchmarks didn't run on the PR. Make benchmarks
run on changes to `rust-toolchain.toml` so we'll be alerted if this
happens again on any future updates.
#3776 does seem to have stabilized the benchmark results. Re-enable
semantic benchmarks which we disabled because they were showing wild
variance, and see if they still do.
This PR builds on #3201 to further speed up the benchmarks and reduce CI
time.
* Build and run each benchmark as separate job (like before).
* But now each bench is only built with the dependencies it needs.
* For linter benchmarks, build benchmark in 1 job (like #3201 does).
* Run each linter fixture in a separate job as they're slow.
This reduces total time to complete benchmarks from between 6m-7m to
~4m40s.
All the individual jobs complete in under 1m30s, except for building
linter benchmark which takes 2m30s. So there won't be the problem of
blocking the CI queue that there was before.
NB: I did try this before, and didn't see a benefit. But I realized
today what I was doing wrong - it only works once the caches are
populated by a previous run on main branch.
So the CI times in this PR won't look good, but once it's merged to
main, it will take effect. Here it is running on main branch of my fork:
https://github.com/overlookmotel/oxc/actions/runs/9030511348
I also added a step to delete the temp artefacts which aren't needed
once the run has completed.
The flakiness and all the noise produced by it is starting to hurt
maintenance because I can no longer tell if main or PR is failing
properly :-(
codspeed doesn't have the feature "let it run but don't report as error"
or "different thresholds on different benchmarks"
The sourcemap implement port from
[rust-sourcemap](https://github.com/getsentry/rust-sourcemap), but has
some different with it.
- Encode sourcemap at parallel, including quote `sourceContent` and
encode token to `vlq` mappings.
- Avoid `Sourcemap` some methods overhead, like `SourceMap::tokens()`
caused extra overhead at common cases. Here using `SourceViewToken` to
instead of it.
The `parser_napi` pseudo-benchmark does not depend on any Oxc crates, so
remove all dependencies for this benchmark. This reduces build time of
this benchmark on CI by ~1 minute.
After #2780, NAPI parser benchmark sometimes took the longest of all the
benchmarks, so making it faster reduces time to complete the benchmarks
overall.
Not a substantive change. This just prevents the other benchmark jobs
showing greyed-out "Build NAPI benchmark" steps, and makes the workflow
file easier to follow.
CI tasks were stuck at "Waiting for status to be reported", and unable
to merge until they passed. Problem was that CI task doesn't run if only
`.yml` files are altered. So have added a dummy comment to a random Rust
file just to get it to run. Will make another PR after to remove the
comment again!
Add NodeJS parser to benchmarks.
Previous attempt #2724 did not work due CodSpeed producing very
inaccurate results (https://github.com/CodSpeedHQ/action/issues/96).
This version runs the actual benchmarks without CodSpeed's
instrumentation. Then another faux-benchmark runs within Codspeed's
instrumented action and just performs meaningless calculations in a loop
for as long as is required to take same amount of time as the original
uninstrumented benchmarks took.
It's unfortunate that we therefore don't get flame graphs on CodSpeed,
but this seems to be the best we can do for now.
#2751 contained a mistake, which was pointed out by Adrian @ CodSpeed on
Discord.
For PRs from forks, `CODSPEED_TOKEN` is not provided, and the submission
to CodSpeed is "tokenless". #2751 wrongly assumed all runs are submitted
with a token. This PR fixes that.
Follow-on from #2751. Further shards linter benchmarks so each fixture runs in its own job.
This reduces total time to run benchmarks by another ~75 secs. So approx 2.5 mins shaved off in total.
This PR shards benchmarks when running on CI. Each benchmark (parser, minifier etc) runs as a separate job, and then a final job combines the results and uploads to Codspeed.
A bit of a hacky implementation. Uses a small NodeJS HTTP server to intercept the results from `codspeed-runner`, and then another NodeJS script to combine them all together, and upload to CodSpeed.
I will submit PRs on Codspeed's runner + action to do it properly, but as I imagine it'll be a slow process getting that merged upstream, I wanted to see if it worked first. We can replace this once it's supported upstream.
Sharding only reduces total time to run the benchmarks by about 70 secs at present, because linter benchmark takes 6 mins alone and holds up the whole process (all the rest are done in ~2 mins). If we can split up the linter benchmark, we can likely get total run time down to around 3 mins. I'll try that in a follow-on PR.
I guess the other upside is we can now add as many benchmarks as we like with impunity - they'll run in parallel, and so won't slow things down overall.
#2724 added CodSpeed benchmarks for NodeJS `oxc-parser`.
Unfortunately it turns out CodSpeed's results are wildly inaccurate.
Unclear why, but have raised an issue with CodSpeed
(https://github.com/CodSpeedHQ/action/issues/96). In meantime it seems
best to remove the benchmarks as they're not useful at present.
Closes#2616.
Adds benchmarks for NodeJS NAPI build. Measurement includes `JSON.parse`
of the AST on JS side, since that's how it'll be used 99% of the time.
Benchmarks run against same files as Rust parser benchmarks, so we can
see the overhead of transferring AST to JS.
I've recently switched over from hacking on a fork to working directly
on upstream, in order to be able to use Graphite.
One thing I miss now is ability to create branches and run CodSpeed on
them while developing. Compared to running benches locally, CodSpeed has
the advantage of giving you a visual history, showing how each commit
you push moves the dials.
This PR makes the benchmarks run for any branch named `bench-*`, in
order to enable this workflow on upstream, without having to open a
draft PR for every experiment.
Bumps [CodSpeedHQ/action](https://github.com/codspeedhq/action) from 1
to 2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/codspeedhq/action/releases">CodSpeedHQ/action's
releases</a>.</em></p>
<blockquote>
<h2>v2.0.0</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/CodSpeedHQ/action/compare/v1...v2.0.0">https://github.com/CodSpeedHQ/action/compare/v1...v2.0.0</a></p>
<p>This major update greatly simplifies the action and moves all the
core logic to the <a
href="https://github.com/CodSpeedHQ/runner">https://github.com/CodSpeedHQ/runner</a>
repo.</p>
<p>Updating is as easy as changing <code>CodSpeedHQ/action@v1</code> to
<code>CodSpeedHQ/action@v2</code>.</p>
<h2>What's changed</h2>
<ul>
<li>feat: integrate the native runner with a composite action by <a
href="https://github.com/art049"><code>@art049</code></a> and <a
href="https://github.com/adriencaccia"><code>@adriencaccia</code></a>
in <a
href="https://redirect.github.com/CodSpeedHQ/action/pull/86">CodSpeedHQ/action#86</a></li>
</ul>
<h3>Breaking changes</h3>
<ul>
<li><code>upload_url</code> input is now <code>upload-url</code></li>
<li><code>pytest-codspeed</code> is no longer installed automatically by
this action</li>
</ul>
<h2>v1.8.1</h2>
<h2>What's Changed</h2>
<ul>
<li>fix: update apt registry before installing valgrind by <a
href="https://github.com/adriencaccia"><code>@adriencaccia</code></a>
in <a
href="https://redirect.github.com/CodSpeedHQ/action/pull/80">CodSpeedHQ/action#80</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/CodSpeedHQ/action/compare/v1.8.0...v1.8.1">https://github.com/CodSpeedHQ/action/compare/v1.8.0...v1.8.1</a></p>
<h2>v1.8.0</h2>
<h2>What's Changed</h2>
<h3>Added</h3>
<ul>
<li>Add the <code>working-directory</code> input parameter by <a
href="https://github.com/art049"><code>@art049</code></a> in <a
href="https://redirect.github.com/CodSpeedHQ/action/pull/77">CodSpeedHQ/action#77</a></li>
</ul>
<h3>Internals</h3>
<ul>
<li>Update documentation examples by <a
href="https://github.com/adriencaccia"><code>@adriencaccia</code></a>
in <a
href="https://redirect.github.com/CodSpeedHQ/action/pull/76">CodSpeedHQ/action#76</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/CodSpeedHQ/action/compare/v1.7.1...v1.8.0">https://github.com/CodSpeedHQ/action/compare/v1.7.1...v1.8.0</a></p>
<h2>v1.7.1</h2>
<h2>🎉 What's Changed</h2>
<ul>
<li>Prepare for release on GitHub Action Marketplace</li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/CodSpeedHQ/action/compare/v1.7.0...v1.7.1">https://github.com/CodSpeedHQ/action/compare/v1.7.0...v1.7.1</a></p>
<h2>v1.7.0</h2>
<h2>🎉 What's Changed</h2>
<ul>
<li>
<p>A <a href="https://github.com/CodSpeedHQ/valgrind-codspeed">fork of
valgrind</a> is now used, allowing us to <a
href="https://docs.codspeed.io/features/trace-generation/">generate
execution traces</a> with Node and Python.</p>
</li>
<li>
<p>V8 flags previously set as an environment variable
<code>CODSPEED_V8_FLAGS</code> are now handled directly by the
integration library and injected to Nodejs through our <a
href="https://github.com/CodSpeedHQ/action/blob/main/dist/bin/node">custom
node executable</a>.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/CodSpeedHQ/action/blob/main/CHANGELOG.md">CodSpeedHQ/action's
changelog</a>.</em></p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3cb272459e"><code>3cb2724</code></a>
Release v2.0.1 🚀</li>
<li><a
href="6790ba286f"><code>6790ba2</code></a>
feat: simplify action output</li>
<li><a
href="0c6eabe26d"><code>0c6eabe</code></a>
fix: quiet the codspeed-runner installation</li>
<li><a
href="4f1d7b4ec1"><code>4f1d7b4</code></a>
Release v2.0.0 🚀</li>
<li><a
href="de30d2b7e6"><code>de30d2b</code></a>
chore: use full version in script and stop relying on pnpm</li>
<li><a
href="5a53587953"><code>5a53587</code></a>
chore(ci): add multiline check</li>
<li><a
href="e2bcafd32c"><code>e2bcafd</code></a>
feat: integrate the native runner with a composite action</li>
<li><a
href="e8372ebb5f"><code>e8372eb</code></a>
chore: update readme with vitest</li>
<li>See full diff in <a
href="https://github.com/codspeedhq/action/compare/v1...v2">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
benchmark execution time reduced from 15 mins to 12 mins. Time reduction
should be more noticeable for large code changes which require longer
compile time.
