(torchx/workspace) Support multi-project/directory workspace #1114
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
meta-cla
bot
added
the
CLA Signed
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
2683a61 to
1a366c7
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 12, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
1a366c7 to
0846364
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 12, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
0846364 to
096868a
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
kiukchung
added a commit
that referenced
this pull request
Sep 12, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
force-pushed
the
export-D82169554
branch
from
096868a to
a7c3daf
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 12, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
a7c3daf to
4e97e31
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 12, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
4e97e31 to
4581fed
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
highker
approved these changes
Sep 12, 2025
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 12, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
4581fed to
9c3fb4c
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 15, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
9c3fb4c to
9593260
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
kiukchung
added a commit
that referenced
this pull request
Sep 15, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
added a commit
that referenced
this pull request
Sep 15, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
added a commit
that referenced
this pull request
Sep 15, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
force-pushed
the
export-D82169554
branch
from
9593260 to
33ea153
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 15, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
added a commit
that referenced
this pull request
Sep 16, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
force-pushed
the
export-D82169554
branch
from
33ea153 to
ec62c8e
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 16, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
ec62c8e to
18cb679
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
kiukchung
added a commit
that referenced
this pull request
Sep 16, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
kiukchung
force-pushed
the
export-D82169554
branch
from
18cb679 to
14e1e0d
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 16, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
14e1e0d to
8fb543a
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Sep 16, 2025
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
8fb543a to
663483c
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
Summary: Usage: In your `.torchxconfig` file specify multi-project workspace as follows (notice the indent after `workspace =`, this uses INI file's multiline strings and reads the multiline contents as YAML - at some point we should make .torchxconfig a toml or YAML but that's a BC breaking change): ``` [cli:run] workspace = /home/$USER/github/myproj: /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl/.torchxconfig ``` Then the workspace is built as: 1. `/home/$USER/github/myproj/**` -> `$REMOTE_ROOT/**` 1. `/home/$USER/github/verl/**` -> `$REMOTE_ROOT/verl/**` 1. `/home/$USER/.torchxconfig` -> `$REMOTE_ROOT/verl/.torchxconfig` Notes: * The mappings are basically like BUCK `python_library`'s `resoruce` attribute. Mapping to an empty string, or simply leaving the target blank (as we did for `myproj` in the example above) copies the contents of the src dir directly into the remote workspace root * To copy the project dir contents directly into the remote root, map it to `""` (empty string) or leave the target blank ``` [cli:run] workspace = /home/$USER/github/myproj: # same as /home/$USER/github/myproj: "" # same as /home/$USER/github/myproj: '' ``` * Files can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example: ``` [cli:run] workspace = /home/$USER/github/verl: verl /home/$USER/.torchxconfig: verl ``` copies `.torchxconfig` into `$REMOTE_ROOT/verl/.torchxconfig`. * However switching the order of the projects won't work: ``` [cli:run] workspace = # copies .torchxconfig -> verl (verl is a file) /home/$USER/.torchxconfig: verl # ERROR trying to copy github/verl/** into verl/** (since verl is a file) /home/$USER/github/verl: verl ``` Reviewed By: highker Differential Revision: D82169554
facebook-github-bot
force-pushed
the
export-D82169554
branch
from
663483c to
e8181b6
Compare
|
@kiukchung has exported this pull request. If you are a Meta employee, you can view the originating diff in D82169554. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Usage:
In your
.torchxconfigfile specify multi-project workspace as follows:Then the workspace is built as:
/home/$USER/github/myproj/**->$REMOTE_ROOT/**/home/$USER/github/verl/**->$REMOTE_ROOT/verl/**/home/$USER/.torchxconfig->$REMOTE_ROOT/verl/.torchxconfigNotes:
The mappings are basically like BUCK
python_library'sresoruceattribute. Mapping to an empty string, or simply leaving the target blank (as we did formyprojin the example above) copies the contents of the src dir directly into the remote workspace rootFiles can be specified as src. The target of a file can be a directory, and if so, the file will be copied into the directory. For example:
copies
.torchxconfiginto$REMOTE_ROOT/verl/.torchxconfig. However if we switching the order of the projects won't work:This would copy
.torchxconfigto the file calledverlin$REMOTE_ROOT/verl, thengithub/verlwould fail to copy.Differential Revision: D82169554