#2751 contained a mistake, which was pointed out by Adrian @ CodSpeed on
Discord.
For PRs from forks, `CODSPEED_TOKEN` is not provided, and the submission
to CodSpeed is "tokenless". #2751 wrongly assumed all runs are submitted
with a token. This PR fixes that.
This PR shards benchmarks when running on CI. Each benchmark (parser, minifier etc) runs as a separate job, and then a final job combines the results and uploads to Codspeed.
A bit of a hacky implementation. Uses a small NodeJS HTTP server to intercept the results from `codspeed-runner`, and then another NodeJS script to combine them all together, and upload to CodSpeed.
I will submit PRs on Codspeed's runner + action to do it properly, but as I imagine it'll be a slow process getting that merged upstream, I wanted to see if it worked first. We can replace this once it's supported upstream.
Sharding only reduces total time to run the benchmarks by about 70 secs at present, because linter benchmark takes 6 mins alone and holds up the whole process (all the rest are done in ~2 mins). If we can split up the linter benchmark, we can likely get total run time down to around 3 mins. I'll try that in a follow-on PR.
I guess the other upside is we can now add as many benchmarks as we like with impunity - they'll run in parallel, and so won't slow things down overall.
