Filesystem: speed up get pids #2084
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With force_umount=safe, we "manually" scan the /proc/ file system.
We look for symlinks pointing into the path we are interested in.
Specifically, we are interested in
/proc/<pid>/{root,exe,cwd}
/proc/<pid>/fd/<fd>
We also look for relevant memory mappings in /proc/<pid>/maps
All these are per process, not per "task" or "thread".
see procfs(5) and pthreads(7).
Still, we currently also scan /proc/<pid>/task/<tid>/
for all the same things.
With a large system with many heavily threaded processes,
this can significantly slow down this scanning,
without gaining new information.
Adding -maxdepth to the find command line avoids this useless work,
potentially reducing the scanning time by orders of magnitute
on systems with many heavily threaded processes.
We could also write a dedicated helper in C to do the very same thing,
with the option to "short circuit" and proceed with the next pid
as soon as the first "match" is found for the currently inspected pid.
That could further reduce the scanning time
by about an additional factor of 10.
If we have /proc/<pid>/map_files/* symlinks, we don't need to additionally grep /proc/<pid>/maps. Also don't first collect output of commands into variables just to pipe them to sort -u later, just pipe the output of the commands through sort -u directly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For
force_umount=safe, we can significantly speed up get_pids() by limiting the search for significant symlinksto the process and skipping the tasks (or threads).
We are interested in
/proc/<pid>/{cwd,root,exe}and/proc/<pid>/fd/<fd>as well as memory mappings.All of these are per process, not per thread. We can save us a lot of time and effort by not scanning
/proc/<pid>/taks/<tid>/*, we'd only to find identical information there.Even on a mostly idle system with just a few "heavily threaded" processes,
this can speed up the scanning by a factor of 10.
With "modern" linux kernels, we can also drop the "grep" in
maps, we already found the symlinks inmap_files/.