Skip to content

Conversation

@MrCrayon
Copy link
Contributor

@MrCrayon MrCrayon commented Jul 6, 2025

Description

This PR refactors Text handlers to extend TextHandler abstract class, once standarized many methods where the same between different providers.
I removed parameters that were being passed between calls like $request and $response and use instead class properties.
This was initially part of #413 but I decided to separate it, once this and a refactor to Stream handlers is done it's going to be easier to finish that PR.

Changes

  • Pass $request in constructor not as handle parameter.
  • Create tempResponse so we have something consistent to pass to events feat: add events #413
  • Build request parameters with a static method buildHttpRequestPayload, it can be useful to build the request and use it in batched requests Request pooling? #417.

Differences to clarify

There were some small differences in some providers I think my changes are working but it would be great to have that reviewed.

OpenAI ToolMap

In OpenAI Text handler in the handle method toolCalls to be passed to AssitantMessage are evaluated considering reasoning

array_filter(data_get($data, 'output', []), fn (array $output): bool => $output['type'] === 'reasoning'),

while in addStep method reasoning is ignored, I uniformed it to include reasoning also in tempResponse that is then used to build Step in addStep.

Ollama finish reason

In Ollama Text handler, since there is no different finish reason for tool calls, it's always

 "done_reason":"stop"

there was a check before the finishReason match, to keep it consistent with other providers I moved that check to mapFinishReason and I'm setting FinishReason::ToolCalls when recognized:

protected function mapFinishReason(array $data): FinishReason
{
   if (! empty(data_get($data, 'message.tool_calls'))) {
       return FinishReason::ToolCalls;
   }

   return FinishReasonMap::map(data_get($data, 'done_reason', ''));
}

FinishReason::Length

Some providers have FinishReason::Length some don't, I left the match for all managed by handleStop.

return match ($finishReason) {
    FinishReason::Stop, FinishReason::Length => $this->handleStop(),

Mistral inifinite steps

In Mistral maxSteps set to 0 is taken as no limit, I don't see any reference in documentation, personally I'd rather set an high number but I would never risk an infinite loop.

if ($this->request->maxSteps() === 0) {
    return true;
}

Since in this PR all providers are using the same check that has been removed.

Other classes changed

I had to do small changes in other handler classes because I am not passing any parameter to validateResponse, not very elegant but I hope that can be fixed in a future refactor for the other handlers.

@ChrisB-TL I would appreciate, if you have time, if you can have a look.

@ChrisB-TL
Copy link
Contributor

ChrisB-TL commented Jul 7, 2025

I think I like this. @sixlive and I talked about doing it when we did the last major refactor but I think we decided to leave it until enough edge cases came out.

I think we are probably there now, and this tidies things up. If / when we do have a provider that needs something slightly different, we can always override methods.

I do like the tempResponse, and used it in Anthropic. My only slight worry about having it in a provider wide abstract is whether, when implementing a new provider without knowledge of the internals, its purpose is clear enough without diving into source (okay trade off with Anthropic, as if there you are already in source). I am probably overthinking it though - we can just cover in the Custom Provider docs.

Re: validateResponse(), I would:

  • Re-name it handleResponseError()
  • Leave it abstract in TextHandler and implement the empty method in the providers that don't use it (helpful reminder when implementing a new provider)

@kinsta
Copy link

kinsta bot commented Jul 7, 2025

Preview deployments for prism ⚡️

Status Branch preview Commit preview
❌ Failed to deploy N/A N/A

Commit: 5f53879d1f18be74cfe96e92390adb9aef8bd02b

Deployment ID: 485cd4cf-5586-4b8a-a1fb-eaae2154b561

Static site name: prism-97nz9

@MrCrayon MrCrayon force-pushed the refactor-provider-handlers branch from bfdad67 to dee8b85 Compare July 8, 2025 03:15
@MrCrayon
Copy link
Contributor Author

MrCrayon commented Jul 8, 2025

@ChrisB-TL thanks for feedback.
I rebased with last commits, renamed ValidateReponse to HandleResponseError and made method abstract.

MrCrayon added 2 commits July 12, 2025 10:05
Refactor `Text` handlers to extend `TextHandler` abstract class
 - Use class properties instead of moving around parameters in methods.
- Pass `$request` in constructor not as `handle` parameter.
- Always create `tempResponse` so we have something consistent to pass to events
- Always build request parameters with a static method `buildHttpRequestPayload`, it can be useful to build the request and use it in batched requests.
@MrCrayon MrCrayon force-pushed the refactor-provider-handlers branch from dee8b85 to bac7aea Compare July 12, 2025 08:05
@MrCrayon
Copy link
Contributor Author

Formatting is failing because php-cs-fixer was updated and it's now enforcing visibility keyword to be first.
I fixed that in #488

@sixlive what do you think about this PR? Anything you want to be changed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants