-
-
Notifications
You must be signed in to change notification settings - Fork 18
feat: migrate SlowBuffer to Buffer.allocUnsafeSlow() #193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
b04a60b
to
d2d54d6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WOW, thank you and congratulations on your first contribution.
Here is the first wave of reviews. Overall, it's good. You've covered a lot of cases except for dynamic imports. For that, you have the getNodeImportCalls
function, which allows you to query all nodes that are dynamic imports.
As mentioned in one of the comments, you're using a lot of regex. It's better to base it on the AST for greater stability. Also, as mentioned in a message, you have tools to visualise things, and for query examples, you have utility functions that are quite complex.
2. Direct `SlowBuffer` calls to `Buffer.allocUnsafeSlow()` | ||
3. Import/require statements to use `Buffer` instead of `SlowBuffer` | ||
|
||
## Examples |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
example on rest of codemods use this structure :
## Examples
**Before:**
<!-- code snippet -->
**After:**
<!-- code snippet -->
I think your aproach is better in terms of semantics.
@nodejs/userland-migrations what should we do modify this pr or update other codemod ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it’s fine to just update this PR for now. In the future, it might be a good idea to add a script or system at the project root that, when run with
npm run new:codemon --name='slow-buffer-to-buffer-alloc-unsafe-slow'
,
generates a template project for implementing a new codemon. New contributors and existing ones would benefit from it to have all the basic down.
If this template project would have a different structure, it would be then a good opportunity to correct existing codemons.
Let me know if you find this idea interesting and I can expand on it, or we can discuss how it could be implemented.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh interesting point.
- Codemod CLI ofter a
codemod init
but it's don't really cover our code style/architecture. - We can have a script that generate the basic of a codemod.
- Simple way copy paste a codemod (what I do now)
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
* For ESM imports: { SlowBuffer as SB } -> { Buffer as SB } | ||
* Preserves original formatting as much as possible | ||
*/ | ||
function transformDestructuringPattern(originalText: string): string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here it's use too much regex, we need to relie on AST for reliability. If you have difficult to visualize the ast you can use Codemod Studio, ast-grep playground or any Tree-sitter sitter visualization tools.
/** | ||
* Process comments mentioning SlowBuffer and update them | ||
*/ | ||
function processSlowBufferComments(rootNode: SgNode, edits: Edit[]): boolean { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea but in practice we doesn't want to update the user comment. But you can console.warn
+ file://<path>:<line>
where a comment mention this api.
for the red ci just run |
a68a702
to
48798a1
Compare
@AugustinMauroy thank you very much for the review 💪. I have commited the suggestions and marked the addressed comments with a thums up emoji. The comments that have a requested change, do not have any emoji. I will aim to commit this over the weekend. |
Wow super! also any pressure we are open source. Also for your information you have a button with "resolve conversation" which is collapsing the review 😁 |
if ( | ||
originalText.includes("SlowBuffer") && | ||
!originalText.includes(",") && | ||
!originalText.includes(":") && | ||
!originalText.includes(" as ") | ||
) { | ||
return originalText.replace(/SlowBuffer/g, "Buffer"); | ||
} | ||
|
||
let newText = originalText; | ||
|
||
// Handle aliased patterns (both CommonJS : and ESM as syntax) | ||
// { SlowBuffer: SB } -> { Buffer: SB } | ||
newText = newText.replace(/SlowBuffer(\s*:\s*\w+)/g, "Buffer$1"); | ||
// { SlowBuffer as SB } -> { Buffer as SB } | ||
newText = newText.replace(/SlowBuffer(\s+as\s+\w+)/g, "Buffer$1"); | ||
|
||
// If Buffer is already present in this specific pattern, just remove SlowBuffer | ||
if (originalText.includes("Buffer") && originalText.includes("SlowBuffer")) { | ||
// Remove non-aliased SlowBuffer references very carefully to preserve spacing | ||
newText = newText | ||
.replace(/,\s*SlowBuffer(?!\s*[:as])/g, "") // Remove SlowBuffer with leading comma | ||
.replace(/SlowBuffer\s*,\s*/g, "") // Remove SlowBuffer with trailing comma and space | ||
.replace(/SlowBuffer(?!\s*[:as])/g, ""); // Remove standalone SlowBuffer (not followed by : or as) | ||
|
||
// Clean up any double spaces after opening brace | ||
newText = newText.replace(/{\s{2,}/g, "{ "); | ||
// Clean up any double commas but preserve spacing | ||
newText = newText.replace(/,\s*,/g, ","); | ||
} | ||
// If Buffer is not present, replace first SlowBuffer with Buffer, remove others | ||
else if (originalText.includes("SlowBuffer")) { | ||
// Replace the first non-aliased SlowBuffer with Buffer | ||
newText = newText.replace(/SlowBuffer(?!\s*[:as])/, "Buffer"); | ||
// Remove any remaining non-aliased SlowBuffer references | ||
newText = newText | ||
.replace(/,\s*SlowBuffer\s*(?![\s:as])/g, "") | ||
.replace(/SlowBuffer\s*,\s*/g, "") | ||
.replace(/SlowBuffer\s*(?![\s:as])/g, ""); | ||
// Clean up commas | ||
newText = newText.replace(/,\s*,/g, ","); | ||
} | ||
|
||
return newText; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have a utility function that resolves these scenarios
please check utils/ast-grep/resolve-binding-path.ts
// Find all named_imports that include SlowBuffer aliases (ESM imports) | ||
const namedImports = rootNode.findAll({ rule: { kind: "named_imports" } }); | ||
for (const importNode of namedImports) { | ||
const text = importNode.text(); | ||
// Match patterns like { SlowBuffer as SB } to extract alias "SB" | ||
const aliasMatches = text.matchAll(/SlowBuffer\s+as\s+(\w+)/g); | ||
for (const match of aliasMatches) { | ||
slowBufferAliases.add(match[1]); | ||
} | ||
} | ||
|
||
// Handle constructor patterns: new SlowBuffer(size) and new SB(size) for aliases | ||
const constructorCalls = rootNode.findAll({ | ||
rule: { | ||
kind: "new_expression", | ||
has: { | ||
field: "constructor", | ||
kind: "identifier", | ||
regex: "^SlowBuffer$", | ||
}, | ||
}, | ||
}); | ||
|
||
for (const constructorCall of constructorCalls) { | ||
const args = constructorCall.field("arguments"); | ||
if (args) { | ||
// Extract the arguments text (without parentheses) | ||
const argsText = args.text().slice(1, -1); // Remove ( and ) | ||
edits.push(constructorCall.replace(`Buffer.allocUnsafeSlow(${argsText})`)); | ||
hasChanges = true; | ||
} | ||
} | ||
|
||
// Handle constructor calls with aliases | ||
for (const alias of slowBufferAliases) { | ||
const aliasConstructorCalls = rootNode.findAll({ | ||
rule: { | ||
kind: "new_expression", | ||
has: { | ||
field: "constructor", | ||
kind: "identifier", | ||
regex: `^${alias}$`, | ||
}, | ||
}, | ||
}); | ||
|
||
for (const constructorCall of aliasConstructorCalls) { | ||
const args = constructorCall.field("arguments"); | ||
if (args) { | ||
const argsText = args.text().slice(1, -1); | ||
edits.push(constructorCall.replace(`${alias}.allocUnsafeSlow(${argsText})`)); | ||
hasChanges = true; | ||
} | ||
} | ||
} | ||
|
||
// Handle direct function calls: SlowBuffer(size) | ||
const directCalls = rootNode.findAll({ | ||
rule: { | ||
kind: "call_expression", | ||
has: { | ||
field: "function", | ||
kind: "identifier", | ||
regex: "^SlowBuffer$", | ||
}, | ||
}, | ||
}); | ||
|
||
for (const directCall of directCalls) { | ||
const args = directCall.field("arguments"); | ||
if (args) { | ||
// Extract the arguments text (without parentheses) | ||
const argsText = args.text().slice(1, -1); // Remove ( and ) | ||
edits.push(directCall.replace(`Buffer.allocUnsafeSlow(${argsText})`)); | ||
hasChanges = true; | ||
} | ||
} | ||
|
||
// Handle direct function calls with aliases | ||
for (const alias of slowBufferAliases) { | ||
const aliasDirectCalls = rootNode.findAll({ | ||
rule: { | ||
kind: "call_expression", | ||
has: { | ||
field: "function", | ||
kind: "identifier", | ||
regex: `^${alias}$`, | ||
}, | ||
}, | ||
}); | ||
|
||
for (const directCall of aliasDirectCalls) { | ||
const args = directCall.field("arguments"); | ||
if (args) { | ||
const argsText = args.text().slice(1, -1); | ||
edits.push(directCall.replace(`${alias}.allocUnsafeSlow(${argsText})`)); | ||
hasChanges = true; | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This scenario can be simplified using utils/resolve-binding-path
, and improving findAll from ast-grep with any and multiple matches.
const objectPatterns = rootNode.findAll({ rule: { kind: "object_pattern" } }); | |
for (const pattern of objectPatterns) { | |
const text = pattern.text(); | |
// Match patterns like { SlowBuffer: SB } to extract alias "SB" | |
const aliasMatches = text.matchAll(/SlowBuffer\s*:\s*(\w+)/g); | |
for (const match of aliasMatches) { | |
slowBufferAliases.add(match[1]); | |
} | |
} | |
// Find all named_imports that include SlowBuffer aliases (ESM imports) | |
const namedImports = rootNode.findAll({ rule: { kind: "named_imports" } }); | |
for (const importNode of namedImports) { | |
const text = importNode.text(); | |
// Match patterns like { SlowBuffer as SB } to extract alias "SB" | |
const aliasMatches = text.matchAll(/SlowBuffer\s+as\s+(\w+)/g); | |
for (const match of aliasMatches) { | |
slowBufferAliases.add(match[1]); | |
} | |
} | |
// Handle constructor patterns: new SlowBuffer(size) and new SB(size) for aliases | |
const constructorCalls = rootNode.findAll({ | |
rule: { | |
kind: "new_expression", | |
has: { | |
field: "constructor", | |
kind: "identifier", | |
regex: "^SlowBuffer$", | |
}, | |
}, | |
}); | |
for (const constructorCall of constructorCalls) { | |
const args = constructorCall.field("arguments"); | |
if (args) { | |
// Extract the arguments text (without parentheses) | |
const argsText = args.text().slice(1, -1); // Remove ( and ) | |
edits.push(constructorCall.replace(`Buffer.allocUnsafeSlow(${argsText})`)); | |
hasChanges = true; | |
} | |
} | |
// Handle constructor calls with aliases | |
for (const alias of slowBufferAliases) { | |
const aliasConstructorCalls = rootNode.findAll({ | |
rule: { | |
kind: "new_expression", | |
has: { | |
field: "constructor", | |
kind: "identifier", | |
regex: `^${alias}$`, | |
}, | |
}, | |
}); | |
for (const constructorCall of aliasConstructorCalls) { | |
const args = constructorCall.field("arguments"); | |
if (args) { | |
const argsText = args.text().slice(1, -1); | |
edits.push(constructorCall.replace(`${alias}.allocUnsafeSlow(${argsText})`)); | |
hasChanges = true; | |
} | |
} | |
} | |
// Handle direct function calls: SlowBuffer(size) | |
const directCalls = rootNode.findAll({ | |
rule: { | |
kind: "call_expression", | |
has: { | |
field: "function", | |
kind: "identifier", | |
regex: "^SlowBuffer$", | |
}, | |
}, | |
}); | |
for (const directCall of directCalls) { | |
const args = directCall.field("arguments"); | |
if (args) { | |
// Extract the arguments text (without parentheses) | |
const argsText = args.text().slice(1, -1); // Remove ( and ) | |
edits.push(directCall.replace(`Buffer.allocUnsafeSlow(${argsText})`)); | |
hasChanges = true; | |
} | |
} | |
// Handle direct function calls with aliases | |
for (const alias of slowBufferAliases) { | |
const aliasDirectCalls = rootNode.findAll({ | |
rule: { | |
kind: "call_expression", | |
has: { | |
field: "function", | |
kind: "identifier", | |
regex: `^${alias}$`, | |
}, | |
}, | |
}); | |
for (const directCall of aliasDirectCalls) { | |
const args = directCall.field("arguments"); | |
if (args) { | |
const argsText = args.text().slice(1, -1); | |
edits.push(directCall.replace(`${alias}.allocUnsafeSlow(${argsText})`)); | |
hasChanges = true; | |
} | |
} | |
} | |
var ImportNode // importNode can be fetch with getNodeImportStatement or getNodeRequireCalls | |
const binding = resolveBindPath(importN, "$.SlowBuffer") //it will resolve the name of the SlowBuffer in this file, independent of the way it was imported. | |
const aliasConstructorCalls = rootNode.findAll({ | |
rule: { | |
any: [ | |
{ | |
kind: "new_expression", | |
pattern: `new ${binding}($$$ARGS)`, | |
}, | |
{ | |
kind: "call_expression", | |
pattern: `${binding}($$$ARGS)`, | |
}, | |
], | |
}, | |
}); | |
for(const match of matches) { | |
if(match.kind() === 'new_expression') { // each scenario can be handled by kind here if needed | |
const args = match.getMultipleMatches() //get args | |
edits.push(match.replace(/* handle replace */) | |
} | |
} | |
if ( | ||
originalText.includes("SlowBuffer") && | ||
!originalText.includes(",") && | ||
!originalText.includes(":") && | ||
!originalText.includes(" as ") | ||
) { | ||
return originalText.replace(/SlowBuffer/g, "Buffer"); | ||
} | ||
|
||
let newText = originalText; | ||
|
||
// Handle aliased patterns (both CommonJS : and ESM as syntax) | ||
// { SlowBuffer: SB } -> { Buffer: SB } | ||
newText = newText.replace(/SlowBuffer(\s*:\s*\w+)/g, "Buffer$1"); | ||
// { SlowBuffer as SB } -> { Buffer as SB } | ||
newText = newText.replace(/SlowBuffer(\s+as\s+\w+)/g, "Buffer$1"); | ||
|
||
// If Buffer is already present in this specific pattern, just remove SlowBuffer | ||
if (originalText.includes("Buffer") && originalText.includes("SlowBuffer")) { | ||
// Remove non-aliased SlowBuffer references very carefully to preserve spacing | ||
newText = newText | ||
.replace(/,\s*SlowBuffer(?!\s*[:as])/g, "") // Remove SlowBuffer with leading comma | ||
.replace(/SlowBuffer\s*,\s*/g, "") // Remove SlowBuffer with trailing comma and space | ||
.replace(/SlowBuffer(?!\s*[:as])/g, ""); // Remove standalone SlowBuffer (not followed by : or as) | ||
|
||
// Clean up any double spaces after opening brace | ||
newText = newText.replace(/{\s{2,}/g, "{ "); | ||
// Clean up any double commas but preserve spacing | ||
newText = newText.replace(/,\s*,/g, ","); | ||
} | ||
// If Buffer is not present, replace first SlowBuffer with Buffer, remove others | ||
else if (originalText.includes("SlowBuffer")) { | ||
// Replace the first non-aliased SlowBuffer with Buffer | ||
newText = newText.replace(/SlowBuffer(?!\s*[:as])/, "Buffer"); | ||
// Remove any remaining non-aliased SlowBuffer references | ||
newText = newText | ||
.replace(/,\s*SlowBuffer\s*(?![\s:as])/g, "") | ||
.replace(/SlowBuffer\s*,\s*/g, "") | ||
.replace(/SlowBuffer\s*(?![\s:as])/g, ""); | ||
// Clean up commas | ||
newText = newText.replace(/,\s*,/g, ","); | ||
} | ||
|
||
return newText; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it can be handled with updateBinding
utility function #182 , but it hasn’t been merged yet.
@AugustinMauroy Could we check with someone to follow up on the #182 merge?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AugustinMauroy Could we check with someone to follow up on the #182 merge? yess
Hey guys! Based on your and suggestiongs I have pushed some changes. I have included some test for dynamic imports and I decided to remove the warning for the comments, with all the suggestions it was increasing the complexity and I was loosing track of what was what. In addition to this I am not sure if updating this branch with a commit merge from main is fine or if you would prefer to rebase it. I will leave it down to you |
It's look good I am still on holiday I am going to review it when I'll be back ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGMT !
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
recipes/slow-buffer-to-buffer-alloc-unsafe-slow/src/workflow.ts
Outdated
Show resolved
Hide resolved
Add codemod to handle deprecated SlowBuffer API migration covering CommonJS, ES6 imports, and usage patterns Closes: nodejs#125 (comment)
@AugustinMauroy @brunocroh Thank you very much for the reviews! I managed to learn a lot. |
Add codemod to handle deprecated SlowBuffer API migration covering CommonJS, ES6 imports, and usage patterns
Closes: #125 (comment)