Skip to content

Conversation

@chi-yelp
Copy link
Contributor

@chi-yelp chi-yelp commented Jun 3, 2025

Problem

paasta CLI commands such as paasta validate, paasta mark-for-deployment are blocked by the jira ticket check since they call the get_spark_config while trying the build the Tron configs.

More details in: https://github.yelpcorp.com/sysgit/srv-configs/pull/44869

Solution

Use the user explicitly passed from get_spark_conf parameters for jira ticketing check, instead of reading from environment variables.

Test

  • Unit tests
  • Hard-coding jira_ticket.enabled to true and build a whl and install to local paasta repo to test the following commands:
./.tox/py38-linux/bin/paasta validate -y ~/projects/sysgit/yelpsoa-configs -s spark
./tox/py38-linux/bin/paasta spark-run --aws-profile=dev --cmd bash

@chi-yelp chi-yelp requested review from nemacysts and sidsatyelp June 3, 2025 10:52
@chi-yelp chi-yelp marked this pull request as ready for review June 3, 2025 10:52
Comment on lines 80 to 82
USER_BATCH = 'batch' # used by batch servers
USER_TRON = 'TRON' # used by Tron jobs, or other paasta CLI commands such as `paasta validate/mark-for-deployment`
USER_UNSPECIFIED = 'UNSPECIFIED'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TICKET_NOT_REQUIRED_USERS = {
    'batch',  # non-human spark-run from batch boxes
    'TRON',  # tronjobs that run commands like paasta mark-for-deployment
    None,  # placeholder for being unable to determine user
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, I think this makes my next question a lot clearer: do we need to add UNSPECIFIED to this set?

Copy link
Contributor Author

@chi-yelp chi-yelp Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's for keeping the user pod label spark.yelp.com/user for the metrics / pod search, specific to _get_k8s_spark_env(). I changed the constant name to USER_LABEL_UNSPECIFIED for clarification

@sidsatyelp
Copy link
Member

How do we verify that this fixes the issue for paasta mark-for-deployment ?
Afaik, that paasta subcommand has no dry-run option.

@chi-yelp chi-yelp force-pushed the u/chi/fix_jira_user_check branch from bdad6c6 to 5f0095a Compare June 4, 2025 10:34
@chi-yelp chi-yelp requested review from nemacysts and sidsatyelp June 4, 2025 10:35
@chi-yelp
Copy link
Contributor Author

chi-yelp commented Jun 4, 2025

How do we verify that this fixes the issue for paasta mark-for-deployment ? Afaik, that paasta subcommand has no dry-run option.

Perhaps @nemacysts will have a better idea. On the other hand, I think if paasta validate passes there shouldn't be a problem for paasta mark-for-deployment, as they construct tron configs in the same way

@chi-yelp chi-yelp force-pushed the u/chi/fix_jira_user_check branch from 64ec8b6 to add8d63 Compare June 4, 2025 15:58
@chi-yelp chi-yelp merged commit ff171ef into master Jun 4, 2025
1 check passed
@chi-yelp chi-yelp deleted the u/chi/fix_jira_user_check branch June 4, 2025 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants