ls: Fix excessive ls -al syscalls for ACL metadata #8660

kimono-koans · 2025-09-18T20:15:47Z

Reduces syscalls by searching for the specific ACL xattr key instead of requesting a list of all keys and values.

➜  ~ strace -c ./program/coreutils/target/release/ls -al
...
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 20.88    0.001559          11       130           listxattr
 16.02    0.001196          35        34           getdents64
 14.43    0.001077          13        79         9 statx
 10.89    0.000813         813         1           execve
  6.47    0.000483          17        28         1 openat
  4.65    0.000347          11        31           close
  4.21    0.000314          24        13           mmap
  4.07    0.000304          23        13           read
  3.99    0.000298          11        27           fstat
  3.03    0.000226          16        14         6 readlink
  1.81    0.000135          33         4         4 connect
  1.62    0.000121          30         4           socket
  1.25    0.000093          93         1           write
  1.11    0.000083          13         6           newfstatat
  0.90    0.000067          11         6           brk
  0.64    0.000048           6         7           rt_sigaction
  0.60    0.000045           9         5           mprotect
  0.58    0.000043           8         5           lseek
  0.58    0.000043           8         5           ioctl
  0.50    0.000037          18         2           munmap
  0.46    0.000034          17         2           pread64
  0.31    0.000023           7         3           sigaltstack
  0.25    0.000019          19         1         1 access
  0.15    0.000011           5         2           getrandom
  0.13    0.000010          10         1           arch_prctl
  0.11    0.000008           8         1           sched_getaffinity
  0.11    0.000008           8         1           set_tid_address
  0.11    0.000008           8         1           set_robust_list
  0.11    0.000008           8         1           rseq
  0.07    0.000005           2         2           prlimit64
  0.00    0.000000           0         1           poll
------ ----------- ----------- --------- --------- ------------------
100.00    0.007466          17       431        21 total

➜  ~ strace -c ./program/coreutils/target/release/ls -al
...
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 19.08    0.000962          28        34           getdents64
 17.12    0.000863          10        79         9 statx
 16.20    0.000817         817         1           execve
  8.67    0.000437          15        28         1 openat
  6.41    0.000323          24        13           mmap
  5.28    0.000266           4        66        66 getxattr
  4.20    0.000212           7        27           fstat
  4.17    0.000210          16        13           read
  3.91    0.000197           6        31           close
  3.35    0.000169          12        14         6 readlink
  2.42    0.000122          24         5           mprotect
  1.94    0.000098          16         6           brk
  1.21    0.000061           8         7           rt_sigaction
  1.03    0.000052          10         5           ioctl
  0.87    0.000044          22         2           munmap
  0.65    0.000033          16         2           pread64
  0.46    0.000023          11         2           getrandom
  0.42    0.000021           3         6           newfstatat
  0.38    0.000019           9         2           prlimit64
  0.36    0.000018          18         1         1 access
  0.36    0.000018           6         3           sigaltstack
  0.34    0.000017          17         1           poll
  0.34    0.000017           3         5           lseek
  0.22    0.000011          11         1           sched_getaffinity
  0.18    0.000009           9         1           arch_prctl
  0.16    0.000008           8         1           set_tid_address
  0.16    0.000008           8         1           rseq
  0.14    0.000007           7         1           set_robust_list
  0.00    0.000000           0         1           write
  0.00    0.000000           0         4           socket
  0.00    0.000000           0         4         4 connect
------ ----------- ----------- --------- --------- ------------------
100.00    0.005042          13       367        87 total

…s/coreutils into reduce_syscalls_acls

github-actions · 2025-09-18T21:21:12Z

GNU testsuite comparison:

GNU test failed: tests/ls/acl. tests/ls/acl is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/ls/capability. tests/ls/capability is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/misc/stdbuf (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-09-18T21:45:06Z

GNU testsuite comparison:

GNU test failed: tests/ls/capability. tests/ls/capability is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/misc/stdbuf (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-09-18T22:40:21Z

GNU testsuite comparison:

GNU test failed: tests/ls/capability. tests/ls/capability is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-09-19T04:53:32Z

GNU testsuite comparison:

GNU test failed: tests/ls/capability. tests/ls/capability is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/misc/stdbuf (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-09-19T23:17:29Z

GNU testsuite comparison:

GNU test failed: tests/ls/capability. tests/ls/capability is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

sylvestre · 2025-09-20T11:18:49Z

excellent, happy to merge this when it is ready :)

codspeed-hq · 2025-09-20T17:21:57Z

CodSpeed Performance Report

Merging #8660 will degrade performances by 2.18%

_{Comparing kimono-koans:reduce_syscalls_acls (1699244) with main (b4f8eac)}

🎉 Hooray! `codspeed-rust` just leveled up to 4.0.0!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

❌ 1 regression
✅ 64 untouched
⏩ 73 skipped¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
❌	`ls_recursive_long_all_deep_tree[(100, 4)]`	3.1 ms	3.2 ms	-2.18%

73 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2025-09-20T17:29:38Z

GNU testsuite comparison:

GNU test failed: tests/ls/capability. tests/ls/capability is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/misc/stdbuf (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

sylvestre · 2025-09-24T07:39:28Z

I guess it has a positive impact on performance, did you run benchmarking (hyperfine) on this ? thanks

kimono-koans · 2025-09-24T07:58:36Z

I guess it has a positive impact on performance, did you run benchmarking (hyperfine) on this ? thanks

Right now any performance differences are modest (worth 1 or 2 ms in hyperfine?). I have a feeling this will have a big impact when I/others merge a few more PRs.

After spending some time with it, my guess is that, oddly, any performance difference is not down to syscalls. These seem to happen pretty fast.

Much more likely it is silly allocation behavior. Like create a bunch of small Strings instead of creating one String, etc. I have another PR where we were previously doing an insert and a remove on a HashMap for each directory we entered, needlessly. Stuff like that.

github-actions · 2025-09-25T16:48:44Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-09-27T01:16:11Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

This reverts commit 8adf358.

github-actions · 2025-09-28T05:29:52Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

…s/coreutils into reduce_syscalls_acls

github-actions · 2025-10-02T17:19:49Z

GNU testsuite comparison:

Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-10-02T18:54:43Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-10-02T20:41:36Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

This reverts commit 37b6b96.

github-actions · 2025-10-02T22:25:24Z

GNU testsuite comparison:

Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

github-actions · 2025-10-03T17:16:41Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/ls/no-cap is now being skipped but was previously passing.

kimono-koans · 2025-10-03T19:05:37Z

@sylvestre these failing checks seem to have nothing to do with this PR. Should I close and resubmit?

kimono-koans added 6 commits September 18, 2025 15:11

Initial commit

44efeaa

Merge branch 'uutils:main' into reduce_syscalls_acls

41fb8f9

Add default ACL

18ee2b8

Merge branch 'reduce_syscalls_acls' of https://github.com/kimono-koan…

95f5c05

…s/coreutils into reduce_syscalls_acls

Revert, do not count a default acl key as an ACL because GNU does not

80df9ee

Wrapped Result<Option<Vec<_>>> must be non-empty to indicate an ACL

6616412

Add back default ACL

d68298d

kimono-koans added 3 commits September 18, 2025 16:59

Cleanup

65f2b1f

Avoid 2nd syscall for non-directories

c5be794

Cleanup

22cf205

kimono-koans changed the title ~~Fix excessive ls -al syscalls~~ ls: Fix excessive ls -al syscalls for ACL metadata Sep 19, 2025

Fix buggy tests

9818dc0

Merge branch 'main' into reduce_syscalls_acls

cb75dac

kimono-koans added 7 commits September 20, 2025 11:21

Lazily obtain ACL, only really useful in LongFormat

ff6748e

Cleanup

59198d3

Cleanup

d80eb15

Cleanup

a97d3af

Fix broken test

d7271bf

Cleanup

c9a71e0

Cleanup

35e9ff8

Cleanup

59410bd

This comment was marked as outdated.

Sign in to view

Merge branch 'main' into reduce_syscalls_acls

3d0fd7d

Merge branch 'main' into reduce_syscalls_acls

c867d6a

kimono-koans added 3 commits September 27, 2025 23:55

Fewer read_link calls

8adf358

Revert "Fewer read_link calls"

b4b8321

This reverts commit 8adf358.

Fewer read_link calls

759f1e9

kimono-koans mentioned this pull request Sep 28, 2025

ls: uutils locales support causing excess of statx and readlink syscalls #8761

Open

kimono-koans added 7 commits September 29, 2025 20:13

Reminder: Don't merge until we update a new version of xattr

a0011ea

Merge branch 'main' into reduce_syscalls_acls

15e8bd4

Bump xattr as 1.6.1 has updated logic which allows for fewer syscalls

77fc04e

Merge branch 'reduce_syscalls_acls' of https://github.com/kimono-koan…

d06d25f

…s/coreutils into reduce_syscalls_acls

Fix merge conflicts

82c83a6

Fix lints

c8bcda0

Cleanup

6de3265

kimono-koans added 2 commits October 2, 2025 13:17

Fix has_capability

d715b02

Fix Cargo.lock

9693255

Quicker has_acl processing

d893dc2

kimono-koans added 2 commits October 2, 2025 16:42

Fix broken deny.toml

37b6b96

Revert "Fix broken deny.toml"

6fb1f64

This reverts commit 37b6b96.

Merge branch 'main' into reduce_syscalls_acls

1699244

Uh oh!

ls: Fix excessive ls -al syscalls for ACL metadata #8660

Are you sure you want to change the base?

ls: Fix excessive ls -al syscalls for ACL metadata #8660

Uh oh!

Conversation

kimono-koans commented Sep 18, 2025

Uh oh!

github-actions bot commented Sep 18, 2025

Uh oh!

github-actions bot commented Sep 18, 2025

Uh oh!

github-actions bot commented Sep 18, 2025

Uh oh!

github-actions bot commented Sep 19, 2025

Uh oh!

github-actions bot commented Sep 19, 2025

Uh oh!

sylvestre commented Sep 20, 2025

Uh oh!

codspeed-hq bot commented Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #8660 will degrade performances by 2.18%

🎉 Hooray! codspeed-rust just leveled up to 4.0.0!

Summary

Benchmarks breakdown

Footnotes

Uh oh!

github-actions bot commented Sep 20, 2025

Uh oh!

This comment was marked as outdated.

sylvestre commented Sep 24, 2025

Uh oh!

kimono-koans commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 27, 2025

Uh oh!

github-actions bot commented Sep 28, 2025

Uh oh!

github-actions bot commented Oct 2, 2025

Uh oh!

github-actions bot commented Oct 2, 2025

Uh oh!

github-actions bot commented Oct 2, 2025

Uh oh!

github-actions bot commented Oct 2, 2025

Uh oh!

github-actions bot commented Oct 3, 2025

Uh oh!

kimono-koans commented Oct 3, 2025

Uh oh!

Uh oh!

codspeed-hq bot commented Sep 20, 2025 •

edited

Loading

🎉 Hooray! `codspeed-rust` just leveled up to 4.0.0!

kimono-koans commented Sep 24, 2025 •

edited

Loading