### Before
```
x Flag u is mentioned twice in regular expression literal
,-[1:20]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
x Unexpected flag x in regular expression literal
,-[1:21]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
```
### After
```
x Flag u is mentioned twice in regular expression literal
,-[1:19]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
x Unexpected flag x in regular expression literal
,-[1:20]
1 | const a = /\2(.)/uuxig;
: ^
2 | debugger;
`----
```
Closes#5177
While making this, I noticed an uncaught parse error for accessors: accessors cannot be optional. I'll add a fix for this in an up-stack PR.
This reverts the previous changes to handling labels so that all tests can pass.
This passes all false postivies found in `monitor-oxc` (node_modules/flow-parser/flow_parser.js)
As it turns out this requires less code and produces better diagnostics.
Part of #5008. Make scopes for `ForStatement`, `ForInStatement` and `ForOfStatement` unconditional. i.e. always create a scope, even if there is no lexical binding (e.g. `for (i of a) {}`).
Transform checker check root unresolved references.
The transform checker is now checking pretty much everything it can.
Only fields of `ScopeTree` and `SymbolTable` that it's *not* checking are those which contain `AstNodeId`s, because transformer does not create node IDs at present:
* `ScopeTree::node_ids`
* `SymbolTable::declarations`
* `Reference::node_id`
Checking also only proceeds in "from AST" direction.
i.e. for each `SymbolId` which appears in the AST, we check everything about that symbol. But we *don't* go through all the "rows" in `SymbolTable` and check if there are any extra symbols in the table which aren't in the AST.
Presumably transformer will leave a lot of old symbols lying around in `SymbolTable` (ditto scopes and references). We'd need to add `ScopeFlags::Deleted`, `SymbolFlags::Deleted` and `ReferenceFlags::Deleted` for the transformer to be able to "delete" existing symbols.
Previously was checking that references point to symbols with same name, but not necessarily the same symbol (there could be 2 different symbols with same name).
Pure refactor of transform checker. Store all scope data in `PostTransformChecker` so it doesn't need to be passed around. `SemanticData` contains a full set of all semantic data for semantic pass, so we have 2 instances of it for 1. after transform and 2. rebuilt semantic.
Transformer checker use `SemanticIds` to store collected IDs. `SemanticIds` only contains the IDs, without the `Vec` of errors which is only used temporarily.
closes https://github.com/oxc-project/oxc/issues/4790
@overlookmotel enjoy ... take a look at the snapshots and probably nothing else.
The snapshots are minimal right now, but it's already showing symbols from import specifiers are not being removed. We can iterate on the snapshot representation to aid debugging later.
I'll extend this to `transformer_conformance` and `oxc-monitor` in an up coming PR.
close: #4870
I added the `move_identifier_reference` and `move_member_expression` methods used to take ownership in `ast_builder_impl`. This way can let us get rid of `ast.copy`.
Another possible approach is to add `get_expression_owner` to `SimpleAssignmentTarget` and a `get_inner_expression_owner` method to `Expression`. And add an `into_xxxxx` method for `inherit_variants` macro
The implementation looks like this
```rs
let Some(expression) = self.get_expression_owner() else { return; }
match expr.get_inner_expression_owner() {
Expression::Identifier(ident) => {
*target = self.ctx.ast.simple_assignment_target_from_identifier_reference(ident);
}
inner_expr @ match_member_expression!(Expression) => {
*target = SimpleAssignmentTarget::from(
inner_expr.into_member_expression()
);
}
_ => (),
}
```
After studying google closure compiler, I'm leaning towards a multi-ast-pass infrastructure for the minifier.
This is one of the few places where we are going to trade maintainability over performance, given the goal of the minifier is compression size not performance.
All of the terminologies and separation of concerns are aligned with google closure compiler.
Infrastructure of `terser` and `esbuild` are not suitable for us to study nor pursuit. Their code are so tightly coupled - I failed to comprehend any of them every time I try to walk through a piece of optmization. Google closure compiler despite being written in Java, it's actually the most readable minifier out there.
To improve performance between ast passes, I envision a change detection system over a portion of the code.
The benchmark will demonstrate the performance regression of running 5 ast passes instead of 2.
To complete this PR, I need to figure out "fix-point" and order of these ast passes.
close: #3969close: #2023
We need to add scope according to [this](d8086f14b6/src/compiler/binder.ts (L3883-L3962)). There are still some ASTs that need to be added to the scope.
---
Context from @Boshen:
Before this whole journey of fixing symbols and scopes I asked @Dunqing to debug through binder.ts via a debugger to fully understand how it does resolution.
We then agreed to align the implementation so that when a problem occurs, we can debug through both implementations and find where our problem is.
tsc doesn't have a specification, so we need to align with the reference implementation instead.
related: https://github.com/oxc-project/oxc/issues/4142#issuecomment-2219125356
Although we called `enter_node(Class)`, that doesn't mean we're in the `class` scope. It causes our must to visit decorators before `enter_node`.
Let's look at this case. It causes a syntax error if we don't visit decorators before `enter_node`
```js
// This file was procedurally generated from the following sources:
// - src/decorator/decorator-call-expr-identifier-reference-yield.case
// - src/decorator/syntax/valid/cls-expr-decorators-valid-syntax.template
/*---
description: Decorator @ DecoratorCallExpression (Valid syntax for decorator on class expression)
esid: prod-ClassExpression
features: [class, decorators]
flags: [generated, noStrict]
info: |
ClassExpression[Yield, Await] :
DecoratorList[?Yield, ?Await]opt class BindingIdentifier[?Yield, ?Await]opt ClassTail[?Yield, ?Await]
DecoratorList[Yield, Await] :
DecoratorList[?Yield, ?Await]opt Decorator[?Yield, ?Await]
Decorator[Yield, Await] :
@ DecoratorMemberExpression[?Yield, ?Await]
@ DecoratorParenthesizedExpression[?Yield, ?Await]
@ DecoratorCallExpression[?Yield, ?Await]
...
DecoratorCallExpression[Yield, Await] :
DecoratorMemberExpression[?Yield, ?Await] Arguments[?Yield, ?Await]
DecoratorMemberExpression[Yield, Await] :
IdentifierReference[?Yield, ?Await]
DecoratorMemberExpression[?Yield, ?Await] . IdentifierName
DecoratorMemberExpression[?Yield, ?Await] . PrivateIdentifier
IdentifierReference[Yield, Await] :
[~Yield] yield
...
---*/
function decorator() {
return () => {};
}
var yield = decorator;
var C = @yield() class {};
```
Errors:
```shell
× The keyword 'yield' is reserved
╭─[language/statements/class/decorator/syntax/valid/decorator-call-expr-identifier-reference-yield.js:45:2]
44 │
45 │ @yield() class C {}
· ─────
╰────
```
The changed code makes more sense. Only if we call `enter_scope` for class, the flags will contain `StrictMode`. Also, we can get the exact `flags` of the `scope` in the `class` at the transformer
For example:
```
class A {
B() {
// Before: flags is `Function`
// After: flags is `Function | StrictMode`
}
}
```
The regression tests will be fixed in follow-up PRs
```ts
class Klass <T> extends Root <R> {}
// ^^^^^ ^^^ ^^^^ ^^^ ^^
// id type_paramter super_class super_type_parameters body
```
I reorder fields according to the order above
The class scope is not defined in the spec. But we need to create a scope for `class` to store `TypeParamters`
I'm going to be AFK today(till about 9 PM UTC). Meanwhile, I Didn't want to be a blocker so here we go.
It would fix the #4200 merge if you guys find it in the correct order otherwise feel free to close it.
close: #4186
CatchClause has two scopes. The first one is `CatchClause`, which will add a `CatchParameter` to it. The second one is `Block`, which will add binding that declares in the current block scope.
The spec has a syntax error about `CatchParameter`
- It is a Syntax Error if any element of the BoundNames of CatchParameter also occurs in the LexicallyDeclaredNames of Block.
Adds checks to `TSImportEqualsDeclaration` for invalid use of `import type` modifier.
```ts
import { Foo } from './foo'
namespace Bar {
export class Baz {}
}
import type A = Foo.Baz; // not allowed
import type B = Bar.Baz; // not allowed
import type C = require('./c'); // allowed
```
This PR introduces two type alias to avoid the confusing const generic `pub struct Codegen<'a, const MINIFY: bool>`
* CodeGenerator - Code generator without whitespace removal.
* WhitespaceRemover - Code generator with whitespace removal.
Usage is changed to a builder pattern:
```rust
CodeGenerator::new()
.enable_comment(...)
.enable_sourcemap(...)
.build(&program);
```
feat(transformer-dts): report error for async function and generator
fix(transformer-dts): correct span for type containing private name error
fix(transformer-dts): should add reference for export specifiers
The typescript transform pass is now required to strip typescript syntax
for codegen to print things properly.
Codegen will now print whatever is in the AST.
This PR aims to provide a more accurate error/finalization flow, I've nuked the old error path approach and rewrote it with more versatility in mind.
We used to visit the finalizer block twice and create 2 sets of AstNodes/Basic Blocks for them, This was there to differentiate between the error path finalizer and success path one. There is no longer a need for having 2 separate sets of nodes to do this differentiation.
As for the error path now we have 2 kinds of them, Everything is attached to an error block - even if it is not in a try-catch statement - this results in a lot of extra edges to keep track of these "Implicit" error blocks but I believe in future it can help us to track cross block error paths, For example, we can dynamically attach and cache the implicit error block of a function to its call site error path (either implicit or explicit).
Currently, we lack a test to check if the TS AST has been completely deleted. I have thought of a way to test it. Let's have our idempotency test print the TypeScript code the first time and the second time print the JavaScript code only. If the two results do not match, it means that there are still undeleted TS ASTs or other bugs. Since ideally the TS ASTs are completely deleted, the two results should be the same.
We have a conclusion that codegen will print whatever is in the AST,
instead of having an option to enable printing TypeScript syntax. I plan
to remove codegen's `enable_typescript` option after we strip out all
typescript AST in the transformer typescript plugin.
---------
Co-authored-by: Boshen <boshenc@gmail.com>
The `@jsx: react` is a typescript option. The `Babel` typescript plugin handles @jsx as well, but this is different. `@jsx` in babel is a [pragma](https://babeljs.io/docs/babel-plugin-transform-react-jsx#pragma) option. So we should use code without these meta options to avoid Babel parsing `@jsx` incorrectly
We should not print typescript code as javascript code. Forcing to print as JavaScript code may result in syntax errors. If we truly want javascript code, we can use the `oxc_transformer`.
Move `BabelOptions` to Transformer. The `output.json` is a standard babel configuration. We can reuse BabelOptions to read [babel.config.json](https://babeljs.io/docs/configuration#babelconfigjson) or our configuration(maybe oxc.config.json)
The current `from_babel_options` implementation is copied from the `transform_options` in `test_case.rs`, which I'll completely reimplement next
If submodules are outdated, it'll panic with the following message
```
Repository is outdated, please run `just submodules` to update it.
```
For us maintainers, we'll need the env `UPDATE_SNAPSHOT` to force an update.
part of #3213
We should only have one diagnostic struct instead 353 copies of them, so we don't end up choking LLVM with 50k lines of the same code due to monomorphization.
If the proposed approach is good, then I'll start writing a codemod to turn all the existing structs to plain functions.
---
Background:
Using `--timings`, we see `oxc_linter` is slow on codegen (the purple part).
The crate currently contains 353 miette errors. [cargo-llvm-lines](https://github.com/dtolnay/cargo-llvm-lines) displays
```
cargo llvm-lines -p oxc_linter --lib --release
Lines Copies Function name
----- ------ -------------
830350 33438 (TOTAL)
29252 (3.5%, 3.5%) 808 (2.4%, 2.4%) <alloc::boxed::Box<T,A> as core::ops::drop::Drop>::drop
23298 (2.8%, 6.3%) 353 (1.1%, 3.5%) miette::eyreish::error::object_downcast
19062 (2.3%, 8.6%) 706 (2.1%, 5.6%) core::error::Error::type_id
12610 (1.5%, 10.1%) 65 (0.2%, 5.8%) alloc::raw_vec::RawVec<T,A>::grow_amortized
12002 (1.4%, 11.6%) 706 (2.1%, 7.9%) miette::eyreish::ptr::Own<T>::boxed
9215 (1.1%, 12.7%) 115 (0.3%, 8.2%) core::iter::traits::iterator::Iterator::try_fold
9150 (1.1%, 13.8%) 1 (0.0%, 8.2%) oxc_linter::rules::RuleEnum::read_json
8825 (1.1%, 14.9%) 353 (1.1%, 9.3%) <miette::eyreish::error::ErrorImpl<E> as core::error::Error>::source
8822 (1.1%, 15.9%) 353 (1.1%, 10.3%) miette::eyreish::error::<impl miette::eyreish::Report>::construct
8119 (1.0%, 16.9%) 353 (1.1%, 11.4%) miette::eyreish::error::object_ref
8119 (1.0%, 17.9%) 353 (1.1%, 12.5%) miette::eyreish::error::object_ref_stderr
7413 (0.9%, 18.8%) 353 (1.1%, 13.5%) <miette::eyreish::error::ErrorImpl<E> as core::fmt::Display>::fmt
7413 (0.9%, 19.7%) 353 (1.1%, 14.6%) miette::eyreish::ptr::Own<T>::new
6669 (0.8%, 20.5%) 39 (0.1%, 14.7%) alloc::raw_vec::RawVec<T,A>::try_allocate_in
6173 (0.7%, 21.2%) 353 (1.1%, 15.7%) miette::eyreish::error::<impl miette::eyreish::Report>::from_std
6027 (0.7%, 21.9%) 70 (0.2%, 16.0%) <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
6001 (0.7%, 22.7%) 353 (1.1%, 17.0%) miette::eyreish::error::object_drop
6001 (0.7%, 23.4%) 353 (1.1%, 18.1%) miette::eyreish::error::object_drop_front
5648 (0.7%, 24.1%) 353 (1.1%, 19.1%) <miette::eyreish::error::ErrorImpl<E> as core::fmt::Debug>::fmt
```
It's totalling more than 50k llvm lines, and is putting pressure on rustc codegen (the purple part on `oxc_linter` in the image above.
---
It's pretty obvious by looking at https://github.com/zkat/miette/blob/main/src/eyreish/error.rs, the generics can expand out to lots of code.
It seems like we need to rebuild the scopes and symbols while
traversing. We can't utilize the scopes and symbols built by semantic
because they are immutable.
The sourcemap implement port from
[rust-sourcemap](https://github.com/getsentry/rust-sourcemap), but has
some different with it.
- Encode sourcemap at parallel, including quote `sourceContent` and
encode token to `vlq` mappings.
- Avoid `Sourcemap` some methods overhead, like `SourceMap::tokens()`
caused extra overhead at common cases. Here using `SourceViewToken` to
instead of it.
