[ray]: serialize address field to flyteidl for cluster selector mode#18
Open
jld-adriano wants to merge 1 commit intomasterfrom
Open
[ray]: serialize address field to flyteidl for cluster selector mode#18jld-adriano wants to merge 1 commit intomasterfrom
jld-adriano wants to merge 1 commit intomasterfrom
Conversation
Add address field serialization to RayJob model so it flows from flytekit to flytepropeller. When address is set, flytepropeller will use ClusterSelector mode instead of creating a new RayCluster. Changes: - Add address parameter to RayJob.__init__ - Add address property to RayJob - Serialize address in to_flyte_idl() - Deserialize address in from_flyte_idl() - Pass address from RayJobConfig to RayJob in task.py Author: Devin AI (claude-sonnet-4-20250514) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
This PR enables long-lived Ray clusters in Flyte by serializing the
addressfield fromRayJobConfigto flyteidl. Whenaddressis set, flytepropeller will useClusterSelectormode instead of creating a new ephemeral RayCluster, allowing faster iteration during development.Previously, the
addressfield inRayJobConfigwas only used forray.init()inside the task container and was not serialized to flyteidl, so flytepropeller had no way to know about it.What changes were proposed in this pull request?
addressparameter toRayJob.__init__in models.pyaddressproperty toRayJobclassaddressinto_flyte_idl()methodaddressinfrom_flyte_idl()methodaddressfromRayJobConfigtoRayJobin task.py'sget_custom()methodHow was this patch tested?
Manual review of serialization logic.
Important: This PR requires the companion PR in exa-labs/flyte that adds the
addressfield toflyteidl/plugins/ray.protoand implements the cluster selector logic in flytepropeller.Check all the applicable boxes
Related PRs
addressfield to ray.proto and implements cluster selector logic in flytepropeller)Human Review Checklist
address if self.address else "") is correct for protobuf string fieldsLink to Devin run: https://app.devin.ai/sessions/91929162dbc241d8a83ef3c18aeb9aba
Requested by: adriano@exa.ai (@jld-adriano)