fix: Make auth tokens resolved dynamically per request. #108
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mgiannakopoulos
requested review from
hbindu-ss,
kesmit13,
nunogoncalves03,
ricardoasmarques and
srinathnarayanan
…ccepting http_client parameter.
ricardoasmarques
approved these changes
Nov 27, 2025
kesmit13
reviewed
Dec 1, 2025
Comment on lines
+48
to
+109
| if callable(api_key): | ||
| api_key_getter_fn = api_key | ||
| else: | ||
| def api_key_getter_fn() -> Optional[str]: | ||
| if api_key is None: | ||
| return os.environ.get('SINGLESTOREDB_USER_TOKEN') | ||
| return api_key | ||
|
|
||
| if obo_token_getter is not None: | ||
| obo_token_getter_fn = obo_token_getter | ||
| else: | ||
| if callable(obo_token): | ||
| obo_token_getter_fn = obo_token | ||
| else: | ||
| def obo_token_getter_fn() -> Optional[str]: | ||
| return obo_token | ||
|
|
||
| # handle model info | ||
| if base_url is None: | ||
| base_url = os.environ.get('SINGLESTOREDB_INFERENCE_API_BASE_URL') | ||
| if hosting_platform is None: | ||
| hosting_platform = os.environ.get('SINGLESTOREDB_INFERENCE_API_HOSTING_PLATFORM') | ||
| if base_url is None or hosting_platform is None: | ||
| inference_api_manager = ( | ||
| manage_workspaces().organizations.current.inference_apis | ||
| ) | ||
| info = inference_api_manager.get(model_name=model_name) | ||
| else: | ||
| info = InferenceAPIInfo( | ||
| service_id='', | ||
| model_name=model_name, | ||
| name='', | ||
| connection_url=base_url, | ||
| project_id='', | ||
| hosting_platform=hosting_platform, | ||
| ) | ||
| if base_url is not None: | ||
| info.connection_url = base_url | ||
| if hosting_platform is not None: | ||
| info.hosting_platform = hosting_platform | ||
|
|
||
| # Extract timeouts from http_client if provided | ||
| t = http_client.timeout if http_client is not None else None | ||
| connect_timeout = None | ||
| read_timeout = None | ||
| if t is not None: | ||
| if isinstance(t, httpx.Timeout): | ||
| if t.connect is not None: | ||
| connect_timeout = float(t.connect) | ||
| if t.read is not None: | ||
| read_timeout = float(t.read) | ||
| if connect_timeout is None and read_timeout is not None: | ||
| connect_timeout = read_timeout | ||
| if read_timeout is None and connect_timeout is not None: | ||
| read_timeout = connect_timeout | ||
| elif isinstance(t, (int, float)): | ||
| connect_timeout = float(t) | ||
| read_timeout = float(t) | ||
| if timeout is not None: | ||
| connect_timeout = timeout | ||
| read_timeout = timeout | ||
| t = httpx.Timeout(timeout) |
Collaborator
There was a problem hiding this comment.
This code looks nearly identical to the code in SingleStoreChatFactory. Is there a reason they can't be combined into a single function that is called to do these operations instead of duplicating code?
Contributor
Author
There was a problem hiding this comment.
I will do this refactoring as part of cleaning up the code along with the deprecated parameters.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
WIth this change we:
hosting_platformand theendpoint_urlof the provided InferenceAPI model.http_clientparameter in favor oftimeout. In the next release we are going to remove it.obo_token_getterparameter in favor ofobo_token. In the next release we are going to remove it.JIRA Issues
MCDB-86327
MCDB-86334