Skip to content

Commit 61dad1f

Browse files
authored
buildifier: improve Windows runner performance (#1404)
* buildifier: improve Windows runner performance This change aims to improve the `buildidier`'s runner performance on Windows where it can sometimes take minutes to complete. It consists in replacing the slow COM `Scripting.FileSystemObject` with a "native" PowerShell `Get-ChildItem -Recurse` for file discovery, and batch `buildifier` invocations (100 files at a time) instead of invoking once per file. This improves performance on large codebases (like from about 2 minutes to less than 3 seconds). Notes: - cross-process COM calls add latency, whereas `Get-ChildItem` is a compiled "cmdlet". - by default, `Get-ChildItem` doesn't recurse into symbolic links to directories, which is consistent with the current implementation that explicitly avoids entering `ReparsePoint`s, - batching files passed to `buildifier` reduces process creation overhead, with a limit to account for Windows command line length limitations. References: - https://stackoverflow.com/questions/73893997/comments-in-a-long-line-powershell-code-in-a-batch-script - https://stackoverflow.com/questions/3205027/maximum-length-of-command-line-string - https://devblogs.microsoft.com/oldnewthing/20031210-00/?p=41553
1 parent 82ab633 commit 61dad1f

File tree

1 file changed

+10
-18
lines changed

1 file changed

+10
-18
lines changed

buildifier/runner.bat.template

Lines changed: 10 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,8 @@ rem Get the absolute path to the buildifier executable
1212
for /f "tokens=2" %%i in ('findstr /r "\<buildifier\.exe\>" MANIFEST') do (set buildifier_abs_path=%%i)
1313

1414
powershell ^
15-
function Buildify($Root)^
16-
{^
17-
$Folder = (New-Object -Com Scripting.FileSystemObject).GetFolder($Root);^
18-
$Files = $Folder.Files ^| Where-Object {^
15+
$Files = Get-ChildItem -LiteralPath '%BUILD_WORKSPACE_DIRECTORY:/=\%' -Recurse -File -ErrorAction SilentlyContinue ^|^
16+
Where-Object {^
1917
$_.Name -eq 'BUILD.bazel' `^
2018
-or $_.Name -eq 'BUILD' `^
2119
-or $_.Name -eq 'WORKSPACE' `^
@@ -29,17 +27,11 @@ function Buildify($Root)^
2927
-or $_.Name -clike 'WORKSPACE.*.bazel' `^
3028
-or $_.Name -clike 'WORKSPACE.*.oss'^
3129
};^
32-
foreach ($File in $Files)^
33-
{^
34-
^& '%buildifier_abs_path%' %stripped_args% $File.Path;^
35-
};^
36-
foreach ($SubFolder in $Folder.Subfolders)^
37-
{^
38-
$CurrentItem = Get-Item $SubFolder.Path -ErrorAction SilentlyContinue;^
39-
if ($CurrentItem -and !$CurrentItem.Attributes.ToString().Contains('ReparsePoint'))^
40-
{^
41-
Buildify($SubFolder.Path);^
42-
};^
43-
};^
44-
};^
45-
Buildify('%BUILD_WORKSPACE_DIRECTORY%');
30+
^<# Process files in batches of 100- to avoid exceeding CreateProcess' maximum length of 32,767 characters #^> ^
31+
$i = 0;^
32+
while ($i -lt $Files.Count)^
33+
{^
34+
$Batch = $Files[$i..($i + 99)];^
35+
^& '%buildifier_abs_path%' %stripped_args% $Batch.FullName;^
36+
$i += $Batch.Count;^
37+
};

0 commit comments

Comments
 (0)