Follow-on after #6404.
`ArrayExpressionElement` and `Elision` are not used in the TS types, because `ArrayExpression` has an override for the field it uses.
002289b4b1/crates/oxc_ast/src/ast/js.rs (L293-L302)
Prevent these TS type defs being emitted by introducing a new `#[estree(custom_ts_def)]` attr, to go with `#[estree(custom_serialize)]`.
Beginning of #6347. Instead of using serde-derive, we generate
`Serialize` impls manually.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: overlookmotel <theoverlookmotel@gmail.com>
Improves how config docs are rendered for the website. If a config property is an array of primitive values, we now skip subsection rendering and render a better type, e.g. `string[]` instead of `array`.
## What This PR Does
1. Recover on, and provide a better message for, invalid `export` modifier on constructor parameters. Before, an `unexpected token` error would be produced and the parser would panic. Now, the parser recovers and produces a message saying `'export' modifier cannot appear on a parameter.`
```ts
class Foo {
constructor(export x: number) {}
}
```
2. Recover on, and provide a better message for, invalid modifiers on index signatures. Same recovery/message characteristics as above.
```ts
class Foo {
public [x: string]: string;
}
```
## What This PR Does
Provide better error messages when a reserved word is used as a `BindingIdentifier`
```ts
var const = 1;
export enum const {}
const if = 1;
```
Our convention is that AST type fields are ordered in order they appear in source. Move `type_parameters` fields in TS function declaration types to before `this_param` and formal parameters.
> Closes#5046
This PR migrates the linter crate and oxlint app to use the new `LinterBuilder` API. This PR has the following effects:
1. `plugins` in oxlint config files are now supported
2. irons out weirdness when using CLI args and config files together. CLI args are now always applied on top of config file settings, overriding them.
# Breaking Changes
Before, config files would override rules set in CLI arguments. For example, running this command:
```sh
oxlint -A correctness -c oxlintrc.json
```
With this config file
```jsonc
// oxlintrc.json
{
"rules": {
"no-const-assign": "error"
}
}
```
Would result in a single rule, `no-const-assign` being turned on at an error level with all other rules disabled (i.e. set to "allow").
Now, **CLI arguments will override config files**. That same command with the same config file will result with **all rules being disabled**.
## Details
For a more in-depth explanation, assume we are running the below command using the `oxlintrc.json` file above:
```sh
oxlint -A all -W correctness -A all -W suspicious -c oxlintrc.json
```
### Before
> Note: GitHub doesn't seem to like deeply nested lists. Apologies for the formatting.
Here was the config resolution process _before_ this PR:
<details><summary>Before Steps</summary>
1. Start with a default set of filters (["correctness", "warn"]) if no filters were passed to the CLI. Since some were, the filter list starts out empty.
2. Apply each filter taken from the CLI from left to right. When a filter allows a rule or category, it clears the configured set of rules. So applying those filters looks like this
a. start with an empty list `[]`
b. `("all", "allow")` -> `[]`
c. `("correctness", "warn")` -> `[ <correctness rules> ]`
d. `("all", "allow")` -> `[]`
e. `("suspicious", "warn")` -> `[ <suspicious rules> ]`. This is the final rule set for this step
3. Apply overrides from `oxlintrc.json`. This is where things get a little funky, as mentioned in point 2 of what this PR does. At this point, all rules in the rules list are only from the CLI.
a. If a rule is only set in the CLI and is not present in the config file, there's no effect
b. If a rule is in the config file but not the CLI, it gets inserted into the list.
c. If a rule is already in the list and in the config file
i. If the rule is only present once (e.g. `"no-loss-of-precision": "error"`), unconditionally override whatever was in the CLI with what was set in the config file
ii. If the rule is present twice (e.g. `"no-loss-of-precision": "off", "@typescript-eslint/no-loss-of-precision": "error"`),
a. if all rules in the config file are set to `allow`, then turn the rule off
b. If one of them is `warn` or `deny`, then update the currently-set rule's config object, but _leave its severity alone_.
So, for our example, the final rule set would be `[<all suspicious rules: "warn">, no-const-assign: "error"]`
</details>
### After
Rule filters were completely re-worked in a previous PR. Now, lint filters aren't kept on hand-only the rule list is.
<details><summary>After Steps</summary>
1. Start with the default rule set, which is all correctness rules for all enabled plugins (`[<all correctness rules: "warn">]`)
2. Apply configuration from `oxlintrc.json` _first_.
a. If the rule is warn/deny exists in the list already, update its severity and config object. If it's not in the list, add it.
b. If the rule is set to allow, remove it from the list.
The list is now `[<all correctness rules except no-const-assign: "warn">, no-const-assign: "error"]`.
3. Apply each filter taken from the CLI from left to right. This works the same way as before. So, after they're applied, the list is now `[<suspicous rules: "warn">]`
</details>
> closes#5454
Adds a `categories` property to config files, where each key is a `RuleCategory` and each value is `"allow"/"off"`, `"warn"`, or `"deny"/"error"`.
Note that this change won't come into effect until after #6088 is merged.
Use `BindingIdentifier` instead of `IdentifierName` so that AST visitors can access the bound symbol id for the namespace's name. This makes namespace consistent with other named declarations, such as `Class`, `Function`, and `TSInterfaceDeclaration`.
We should consider modifying `TSModuleDeclarationName::StringLiteral(...)` to also store a `symbol_id`, since one gets bound in semantic for it as well.
Quality-of-life improvement for invalid `const` modifiers on class elements. Instead of panicking, the parser will eat `const` and report it as an error.
```ts
class C {
static const H = 1;
}
```
This will fold expressions like `new Object()` to `{}`, and `new Array()` to `[]`. Based on the closure compiler tests: b7e380b632/test/com/google/javascript/jscomp/PeepholeSubstituteAlternateSyntaxTest.java (L78).
This is outside my usual area, so feedback is welcome.
NOTE: this was previously a full stack of PRs, but Graphite decided to stop working completely for some reason and only gave me this error when I submitted a PR:
```
ERROR: Failed to submit PR for 10-02-feat_minifier_fold_single_arg_new_array_expressions:
{}
```
so I decided to just completely remake this stack and submit as 1 PR.
Make naming of temp vars follow Babel. It wasn't apparent that our version of this transform was behaving differently from Babel because the Babel plugin has very few tests. The added tests replicate Babel's output.
Fix exponentiation operator transform to bail out early if a private class property.
We can't transform this:
```js
class C {
#p;
method() {
this.#p **= 2;
}
}
```
But we should at least leave it alone. Previously `get_obj_ref` called `ast.move_expression(expr)` on the member expression's object before bailing out, so example above was transformed to:
```js
class C {
#p;
method() {
null.#p **= 2; // <-- `null`
}
}
```
This PR makes it spot this case early and bail out *before* calling `ast.move_expression(expr)`.
Closes#5020.
`TraverseCtx::generate_uid` and associated methods return a `BoundIdentifier`, containing both symbol ID and name. This reduces boilerplate code for the caller and avoids and unnecessary lookup to get the name.
Also add a couple of helper methods to `BoundIdentifier` to reduce boilerplate further.
Move post-transform checker into a `tasks` crate. It doesn't feel like it belongs in `oxc_semantic`. It also feels like too heavy a lump of code to put in `tasks/common`.
This PR makes 2 changes to improve the existing API that are not very useful.
- Remove `(Literal)Parser` and `FlagsParser` and their ASTs
- Add `with_flags(flags_text)` helper to `ParserOptions`
Here are the details.
> Remove `(Literal)Parser` and `FlagsParser` and their ASTs
Previously, the `oxc_regular_expression` crate exposed 3 parsers.
- `(Literal)Parser`: assumes `/pattern/flags` format
- `PatternParser`: assumes `pattern` part only
- `FlagsParser`: assumes `flags` part only
However, it turns out that in actual usecases, only the `PatternParser` is actually sufficient, as the pattern and flags are validated and sliced in advance on the `oxc_parser` side.
The current usecase for `(Literal)Parser` is mostly for internal testing.
There were also some misuses of `(Literal)Parser` that restore `format!("/{pattern}/{flags}")` back and use `(Literal)Parser`.
Therefore, only `PatternParser` is now published, and unnecessary ASTs have been removed.
(This also obsoletes #5592 .)
> Added `with_flags(flags_text)` helper to `ParserOptions`
Strictly speaking, there was a subtle difference between the "flag" strings that users were aware of and the "mode" recognised by the parser.
Therefore, it was a common mistake to forget to enable `unicode_mode` when using the `v` flag.
With this helper, crate users no longer need to distinguish between flags and modes.
Minifier benchmark currently includes the time to run parser and semantic as well as the minifier itself. Similar to transformer benchmark, only measure the minifier itself in the benchmark.
This will give us a clearer picture of performance changes, and should also reduce variance in the benchmarks.
An alternative version of #6177.
Convert `ModuleImports` into a common transform. Works much as before, but it inserts `import` / `require` statements by passing them to `TopLevelStatements` common transform, so they get inserted in one go with any other inserted top-level statements. This avoids shuffling up the `Vec<Statement>` multiple times, which can be slow with large files.
`VarDeclarations` also inserts any declarations via `TopLevelStatements` but runs after `ModuleImports`, so can control whether a `var` statement is inserted before or after `import` statements by inserting it via `VarDeclarations` (to appear after `import` statements) or directly into `TopLevelStatements` (to appear before `import` statements). Insertion order is not actually important, but allows us to match Babel's output and pass its tests.
When writing new rules, it is always inevitable to add `diagnostic`.
Perhaps we can preset it in the `template`.
In addition, guide users to view relevant documents.
First step towards #5049.
Various transforms need to add a `var` statement at top of enclosing statement block.
e.g.:
```js
// Input
a ??= b;
```
```js
// Output
var _a;
(_a = a) !== null && _a !== void 0 ? _a : (a = b);
```
Each of these transforms previously maintained it's own stack and added `var` statements individually.
Share this functionality in a "common" utility transform which maintains a single stack to serve them all.
## What This PR does
Adds a new `oxc-security/api-key` rule that scans for hard-coded API keys.
It is broken up into "secret rules", where each one is responsible for finding a different kind of key. It is architecturally identical to how lint rules themselves. This PR also includes the first of these rules, for AWS access key IDs.
Logic and rules are based on [keyhunter](https://github.com/Donisaac/keyhunter). I've licensed that repo under GNU GPLv3, but it's my code and I can do what I want with it 😈 (read: I'm fine with it being MIT for oxc).
This PR is a complete feature in its own right, but does not represent the end of this work. See https://github.com/oxc-project/backlog/issues/116 to track overall progress.
This will replace `OxlintOptions` in an upstream PR. This also adds `plugins` to
`Oxlintrc`. This field gets respected by the builder, but not by
`OxlintOptions`.
Fix JSX source transform when run alone without main JSX transform.
The added test case is same as one of Babel's exec test cases, but the exec test fails due to `transformAsync` not being present.
Look at first: #5864
the only merge conflict for this PR should then be the ts snap file.
If you want, you can also skip the other PR and merge this one directly
:)
`SymbolFlags::ArrowFunction` is an oddity, as whether a symbol is an arrow function is not statically knowable. In the following cases, `f` symbol did not have `ArrowFunction` flag set:
```js
const {f} = {f: () => {}};
```
```js
let f = 123;
f = () => {};
```
`SymbolFlags::ArrowFunction` is therefore not particularly useful, and possibly misleading. Having it complicates the transformer, and it's not used anywhere in Oxc.
This PR removes it.
The last failed test is only because of the comments. the oxc prettier
output is:
```
enum Direction {
Up = 1,
Down,
Left,
Right,
}
enum FileAccess {
None,
Read = // constant members
1 << 1,
Write = 1 << 2,
ReadWrite = Read | Write,
G = // computed member
"123".length,
}
enum Empty {}
const enum Enum {
A = 1,
B = A * 2,
}
```
Expected output:
aa3853b776/tests/format/typescript/enum/__snapshots__/format.test.js.snap (L91-L99)
Hope you can tell more why this happens :)
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Add `SemanticBuilder::with_excess_capacity` method to request that `SemanticBuilder` over-allocate space in `Semantic`'s `Vec`s.
Use this method to reserve 200% extra capacity for transformer to create more scopes, symbols and references.
200% is an unscientific guess of how much extra capacity is required. Obviously it depends on what transforms are enabled and content of the source code.
### Difference
In `enter_expression`: Recursive transform JSX
In `exit_expression`: Deep first transform
After the change, `transform_jsx` still has a lot of room for improvement.
I expect small performance regression.
But managed to improve the following case from react.developmement.js
```
oxc main ❯ diff before.js after.js
670c670
< if (!(dispatcher !== null)) throw Error("Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:\n1. You might have mismatching versions of React and the renderer (such as React DOM)\n2. You might be breaking the Rules of Hooks\n3. You might have more than one copy of React in the same app\nSee https://reactjs.org/link/invalid-hook-call for tips about how to debug and fix this problem.");
---
> if (dispatcher === null) throw Error("Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:\n1. You might have mismatching versions of React and the renderer (such as React DOM)\n2. You might be breaking the Rules of Hooks\n3. You might have more than one copy of React in the same app\nSee https://reactjs.org/link/invalid-hook-call for tips about how to debug and fix this problem.");
```
`just ast` panic due to
553262842c,
change the filename to makesure `ast_tool` still working
1. crates/oxc_span/src/source_type/types.rs is removed in
553262842c
See https://babel.dev/docs/options#misc-options for background on `unambiguous`
Once `SourceType::Unambiguous` is parsed, it will correctly set the returned `Program::source_type` to either `module` or `script`.
I was so confused as to why the `AST Changes` step wasn't running at times. Turns out we weren't watching it at all😅
This means for a duration of time we weren't running the `AST Changes` step on PRs that edited only `ast_tools` that didn't update any side-effects.
This also removes the confusing `SourceType::always_strict` field.
I hacked it with `SourceType::always_strict`, but what we actually want is add `'use strict'`.
This is technically a breaking change but I don't expect anyone using this outside of oxc.
The snapshot has a large diff due to every single line shifting by 1 row :-/
Realized we can get the source type from the AST.
The next PR will introduce `unambiguous` to `SourceType` and directly set `Program::source_type` to either `script` or `module`.
Currently whether conformance fixture files have trailing line breaks is inconsistent - some have them, some don't. Make all of them have a trailing line break.
Rename transform conformance RegExp test fixtures folder from "esbuild-tests" to "regexp", to reflect that not all these tests are copied from ESBuild.
related: #4754
The implementation port from [esbuild](332727499e/internal/js_parser/js_parser.go (L12820-L12840)). And cover all babel's regexp plugins
---
## The following description was generated by `Graphite` 😋
### TL;DR
Added support for transforming various RegExp features to ensure compatibility with older JavaScript environments.
### What changed?
- Implemented a new `RegExp` transformer to handle unsupported RegExp literal features
- Added options to control different RegExp transformations (e.g., sticky flag, unicode flag, dot-all flag, etc.)
- Updated the transformer to convert unsupported RegExp literals into `new RegExp()` constructor calls
- Added test cases for different RegExp transformations
- Integrated the new RegExp transformer into the existing transformation pipeline
### How to test?
1. Run the existing test suite to ensure no regressions
2. Execute the new RegExp-specific tests in the `tasks/transform_conformance/tests/esbuild-tests/test/fixtures/regexp/` directory
3. Try transforming code with various RegExp features using different target environments to verify correct transformations
As of now if we remove the implementation of a trait for a type and implement the method on that type directly it wouldn't break while it isn't the original trait anymore so that method might do something entirely different.
This change is more explicit on trait calls so we hit compile errors on these kinds of changes.
close: #5435
The behavior of `IdentifierReference` in `TSPropertySignature` is the same as in `TSTypeQuery`, both allow only reference value bindings and type-only import bindings.
I still use `ReferenceFlags::TSTypeQuery` here because I want to avoid producing many changes unrelated to the bug in this PR. I will refactor it in the follow-up PR soon
This should be causing more negative tests to fail than it actually is. This is
because our typescript coverage tests use the "declarations" option to change
the source type to `.d.ts`, which is incorrect. This setting enables `.d.ts`
emits, it does not mean that input files should be treated as `.d.ts`
themselves.
In an effort toward the implementation of #5256, this PR allows us to have a separately generated "derive" file for each crate.
This also eliminates a bunch of boilerplate when writing new "derive" generators and generally makes it more approachable.
#5223 added a new variant `JSXElementName::IdentifierReference` for JSX identifiers which are treated as references (e.g. `<Foo>`) as opposed to `JSXElementName::Identifier` which is for lowercase (e.g. `<div>`).
But we were incorrectly categorizing identifiers beginning with `_` or `$` - these should also be treated as references.
(as discussed in https://github.com/oxc-project/oxc/pull/5339#issuecomment-2321037779)
Follow-on after #5223.
JSX identifiers which start with a capital letter are now `JSXElementName::IdentifierReference`s, so no need to check for capitalized `JSXElementName::Identifier`s.
Preparation for upstack PRs. Graphical reporter diagnostics look exactly the same. This does, however, change `to_string()` to only show diagnostic messages, and not error codes + messages.
Fixing code is an important part of linter logic. We want to make sure fixers for each rule, and the code responsible for applying those fixes, are included in benchmarks.
As it currently stands, fixer closures are applied regardless of whether the user wants fixers to be applied. However, this is an implementation detail and is subject to change. I also want to bench the performance of `Fixer`.
Transform checker don't output spans in errors. They're inaccurate because (a) spans refer to the source text pre-transform, and (b) most errors relate to scopes/symbols/references newly created in transformer, which have no span.
Transformer benchmark measure only the transformer itself. Parse, generate `Semantic`, and drop `Semantic` outside of the measured section.
This should:
1. Give us greater visibility of how changes to transformer affect its performance.
2. Reduce variance in transformer benchmarks, since they will no longer include the variance introduced by `SemanticBuilder`.
Not ready to merge yet. We should first add an "end to end" benchmark testing the entire compilation process (parse - semantic - transform - minify - codegen).
This PR depends on https://github.com/Boshen/criterion2.rs/pull/49. This PR currently makes Oxc's dependency on `criterion2` a git dependency on that PR's branch. That can be changed once the upstream PR is merged.
Include dropping `Semantic` in semantic benchmark measure.
If you create a `Semantic`, you have to drop it, so the time it takes to
drop is part of the cost of using this API, and we should be working to
reduce it. Therefore I think it should be included in the benchmark.
It'll be interesting to see what effect a PR like #5232 which removes a
bunch of `Vec`s from `Semantic` has on the drop time.
### Before
```
x Flag u is mentioned twice in regular expression literal
,-[1:20]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
x Unexpected flag x in regular expression literal
,-[1:21]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
```
### After
```
x Flag u is mentioned twice in regular expression literal
,-[1:19]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
x Unexpected flag x in regular expression literal
,-[1:20]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
```
Closes#5177
While making this, I noticed an uncaught parse error for accessors: accessors cannot be optional. I'll add a fix for this in an up-stack PR.
Include "Output mismatch" in transform conformance snapshots when a test also fails due to scopes/symbols mismatches.
At present, many of the tests are failing on scopes/symbols mismatches, so it's hard to see from snapshots which also have an output mismatch. In particular, when working to fix scope mismatches, it's hard to notice if changes you make cause the output to be wrong. These extra messages make that visible.
Snapshots generated by transform conformance were missing a few errors (e.g. `declarations/const-enum/input.ts` at bottom of diff). Make sure all errors fail the test.
This reverts the previous changes to handling labels so that all tests can pass.
This passes all false postivies found in `monitor-oxc` (node_modules/flow-parser/flow_parser.js)
As it turns out this requires less code and produces better diagnostics.
Blocked by #5008
In the SemanticBuilder, we insert all CatchClause parameters in the BlockStatement scope, a child-scope of CatchClause
858f510d59/crates/oxc_semantic/src/builder.rs (L709-L718)
So we should do the same thing in this plugin, but because CatchClause has no parameters, SemanticBuilder doesn't create scope for `CatchClause`. This cause we cannot find the `BlockStatement` scope_id.
This PR has changed to the correct logic. Just to wait #5008 solved
Part of #5008. Make scopes for `ForStatement`, `ForInStatement` and `ForOfStatement` unconditional. i.e. always create a scope, even if there is no lexical binding (e.g. `for (i of a) {}`).
Transform checker check root unresolved references.
The transform checker is now checking pretty much everything it can.
Only fields of `ScopeTree` and `SymbolTable` that it's *not* checking are those which contain `AstNodeId`s, because transformer does not create node IDs at present:
* `ScopeTree::node_ids`
* `SymbolTable::declarations`
* `Reference::node_id`
Checking also only proceeds in "from AST" direction.
i.e. for each `SymbolId` which appears in the AST, we check everything about that symbol. But we *don't* go through all the "rows" in `SymbolTable` and check if there are any extra symbols in the table which aren't in the AST.
Presumably transformer will leave a lot of old symbols lying around in `SymbolTable` (ditto scopes and references). We'd need to add `ScopeFlags::Deleted`, `SymbolFlags::Deleted` and `ReferenceFlags::Deleted` for the transformer to be able to "delete" existing symbols.
Previously was checking that references point to symbols with same name, but not necessarily the same symbol (there could be 2 different symbols with same name).
We no longer need to build semantic data inside the transformer.
The caller should be responsible for handling semantic data and its
errors.
The best way to achieve this in via `CompilerInterface`.
closes#3565
Fix semantic error caused by `clone_in` causing `reference_id` to not exist.
In this PR, I try to use `move_expression` as much as possible instead of `expr.clone_in`. But in some cases, we need to reuse the same expression in multiple places. I have added a `clone_expression` to workaround it
I felt a bit painful we missing a [clone_in_scope](https://github.com/oxc-project/oxc/issues/4804) API
Pure refactor of transform checker. Store all scope data in `PostTransformChecker` so it doesn't need to be passed around. `SemanticData` contains a full set of all semantic data for semantic pass, so we have 2 instances of it for 1. after transform and 2. rebuilt semantic.
Transformer checker use `SemanticIds` to store collected IDs. `SemanticIds` only contains the IDs, without the `Vec` of errors which is only used temporarily.
