Skip to content

fix: set multiprocessing start method to 'fork' in main function(python3.14 defaults to forkserver) #5019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: devel
Choose a base branch
from
Open

fix: set multiprocessing start method to 'fork' in main function(python3.14 defaults to forkserver) #5019

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
27e58b3
fix: set multiprocessing start method to 'fork' in main function(pyth…
OutisLi Oct 20, 2025
a4262d6
refactor and exclude windows
OutisLi Oct 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions deepmd/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -984,6 +984,19 @@ def main(args: Optional[list[str]] = None) -> None:
RuntimeError
if no command was input
"""
try:
import multiprocessing
import sys

# Force fork multiprocessing start method (not available on Windows)
if sys.platform != "win32":
multiprocessing.set_start_method("fork", force=True)
logging.debug("Successfully set multiprocessing start method to 'fork'.")
else:
logging.debug("Skipping fork start method on Windows (not supported).")
except (RuntimeError, ValueError) as e:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of try...except?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python3.14 by default use forkserver as the start mode of multiprocessing. As a result, with this mode, the NUM_WORKERS will be set to 0. So the setting of this mode is required at the beginning of the program. The try and except is for the compatibility of windows since it does not support this behaviour.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why NUM_WORKERS is set to 0. @caic99 Do you know details?

This module is the command line module. It seems you should move to the PyTorch module.

It seems that line 996 is for Windows. Will it touch line 997?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why NUM_WORKERS is set to 0. @caic99 Do you know details?

The current implementation of deepmd dataloader requires either using fork as the worker launching method, or setting the number of workers to 0, or an error will be raised. I've tried it in earlier versions of Python.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why NUM_WORKERS is set to 0. @caic99 Do you know details?

The current implementation of deepmd dataloader requires either using fork as the worker launching method, or setting the number of workers to 0, or an error will be raised. I've tried it in earlier versions of Python.

Is there any suggestion of how to set this mode in a better way?

logging.warning(f"Could not set multiprocessing start method: {e}")

args = parse_args(args=args)

if args.backend not in BACKEND_TABLE:
Expand Down