feat: add events

[MrCrayon]

Description

This PR introduces the following events:

• PrismRequestStarted
• PrismRequestCompleted
• HttpRequestStarted
• HttpRequestCompleted
• ToolCallStarted
• ToolCallCompleted

Each event is fired with some data in relative to the operation, all events have a trace property composed by:

[
    'traceId' => Str::uuid()->toString(),
    'parentTraceId' => self::getParentId(),
    'traceName' => $operation,
    'startTime' => microtime(true),
    'endTime' => null,
];

traceId is maintained across operation and passed as parent for nested operations, as example:

• PrismRequestStarted traceId 1
  • HttpRequestStarted parentTraceId 1, traceId 2
  • HttpRequestCompleted parentTraceId 1, traceId 2
• PrismRequestCompleted traceId 1

Implementation

This PR seats on top of #386.

• To track operations stack Laravel Context is used through Trace class.
• Http requests are tracked with middlewares in the unified client method, to get stream content without distrupting streaming StreamWrapper is used.
• All operations from all providers are tracked in the shared PendingRequest classes, except when tool are invoked.
  • When tool are invoked PrismRequestCompleted is fired inside Text/Stream provider Handler class and PrismRequestStarted is fired if another operation starts based on max steps allowed.
  • Tool calls are traced CallsTools if called or directly in Text/Stream provider Handler class if it's managed there.

Anthropic

I made some specific changes in Anthropic handler:

• Removed "'Maximum tool call" Exception in Stream Handler and implemented shouldContinue check that does not raise an exception.
• Added $step property in Stream Handler so that we are not passing $depth everywhere.
• Remove isToolUseFinish in handleMessageStopfor Stream Handler.

Other provideres

Currently all other providers are tracked only in PendingRequest, if tools are used for this providers they are going to be nested in the current PrismRequest not separate.
This is just because I used Anthropic as test, there should be no problem changing all other Text/Stream Handlers.

Tests

Added Content-Type headers for Gemini and OpenAI stream tests, Content-type is used in client middleware to recognize if it's a streamed response or normal.

Use

A Listener should be implemented like:

<?php

declare(strict_types=1);

namespace App\Listeners;

use Prism\Prism\Events\HttpRequestCompleted;
use Prism\Prism\Events\HttpRequestStarted;
use Prism\Prism\Events\PrismRequestCompleted;
use Prism\Prism\Events\PrismRequestStarted;
use Prism\Prism\Events\ToolCallCompleted;
use Prism\Prism\Events\ToolCallStarted;

class TraceEventListener
{
    public function handlePrismRequestStarted(PrismRequestStarted $event): void
    {
        logger('PrismRequestStarted: '.$event->trace['traceName'], ['event' => $event]);
    }

    public function handlePrismRequestCompleted(PrismRequestCompleted $event): void
    {
        logger('PrismRequestCompleted: '.$event->trace['traceName'], ['event' => $event]);
    }

    public function handleHttpRequestStarted(HttpRequestStarted $event): void
    {
        logger('HttpRequestStarted: '.$event->trace['traceName'], ['event' => $event]);
    }

    public function handleHttpRequestCompleted(HttpRequestCompleted $event): void
    {
        logger('HttpRequestCompleted: '.$event->trace['traceName']);
    }

    public function handleToolCallStarted(ToolCallStarted $event): void
    {
        logger('ToolCallStarted: '.$event->trace['traceName'], ['event' => $event]);
    }

    public function handleToolCallCompleted(ToolCallCompleted $event): void
    {
        logger('ToolCallCompleted '.$event->trace['traceName'], ['event' => $event]);
    }
}

A trace that includes all run can be created before calling prism like:

// Optional global trace
Trace::begin(
    operation: 'run #1',
    callback: null, // optional callback to fire event or do something,
);

Prism::text()
    ->using('anthropic', 'claude-3-5-sonnet-20240620')
    ->withTools($tools)
    ->withMaxSteps(3)
    ->withPrompt('What time is the tigers game today and should I wear a coat?')
    ->asText();

Trace::end();

TODO

• Add documentation
• Implement tool calls separation for other providers
• Add / Refactor tests

[sixlive]

What do you think of main...feat/evented-telemetry

[MrCrayon]

@sixlive sorry for the long comment 😃 TLDR I like this solution better for several reason.

List of reasons:

1. Keep firing events and telemetry separate

Once you have events in place telemetry can be a separate package, so users can decide what to take from events and how to implement telemetry.
For instance in a project I'm working on I want to keep track of the cost of each call rather than the time.

2. Less verbose implementation

Everything about tracing is implemented in Trace class.
Instead of this

$spanId = Str::uuid()->toString();
$parentSpanId = Context::get('prism.telemetry.current_span_id');
$rootSpanId = Context::get('prism.telemetry.root_span_id') ?? $spanId;

Context::add('prism.telemetry.current_span_id', $spanId);
Context::add('prism.telemetry.parent_span_id', $parentSpanId);

Event::dispatch(new ToolCallStarted(...));

this

Trace::start('tool', () => event(new ToolCallStarted(...));

3. Nesting

I like to keep each call separate, in your implementation, in case tools are used, the first TextGenerationStarted ends when all tools and subsequent text generations are called, so you don't really know how long it took for that text generation or any text generation made in the loop.

4. Level of nesting

You are manually managing only two levels root_span_id and current_span_id, while my class Trace can manage any level of nesting using a LIFO stack.
This is needed if a tool uses other Prism operations like in a RAG flow:

• ToolCallStarted
  • PrismRequestStarted // Embeddings
    • HttpRequestStarted
    • HttpRequestCompleted
  • PrismRequestCompleted
• ToolCallCompleted

5. Data

I tried to collect all important data, and some things still need to be adjusted, to make it available in the fired events so that it's up to the Listener to decide what to use.
In your implementation there is no raw response because your primary goal, as far as I can tell, was to track the time for each call but that way we are missing a big opportunity.

[kinsta]

Preview deployments for prism ⚡️

Status	Branch preview	Commit preview
❌ Failed to deploy	N/A	N/A

Commit: 5f53879d1f18be74cfe96e92390adb9aef8bd02b

Deployment ID: 485cd4cf-5586-4b8a-a1fb-eaae2154b561

Static site name: prism-97nz9

[TonsiTT]

I have been following this for a while, and yeah, my primary case would be tracking costs and responses as well(as would most AI users do id assume and there is big lack of such functionality in php land ), so I also prefer @MrCrayon approach

[MrCrayon]

@sixlive let me know what you think about this PR.

Tests are failing for Laravel 11 lowest because they are running on v11.22.0 while Context pop was introduced in v11.31.0 we could not use popHidden or change requirement in composer.lock.

[sixlive]

I dont know if I like the configuration method approach for this. I think I'd rather like to just return a base client with the middleware defined and then let the providers can customize the client as needed.

[MrCrayon]

I submitted a PR #427 just for the client trait using a baseClient where we can add middlewares and the rest of configuration happens in provider client. Is that what you meant?

I separated in another PR because I refactored exceptions that were scattered in all handlers and that added a lot of noise that I'd like to keep separate from this PR.

// Provider class
protected function client(array $options = [], array $retry = [], ?string $baseUrl = null): PendingRequest
{
    return $this->baseClient()
        // configuration for that provider
        ->baseUrl($baseUrl ?? $this->url);
}

[sixlive]

Yeah man, I think I dig this

[sixlive]

@sixlive let me know what you think about this PR.

Tests are failing for Laravel 11 lowest because they are running on v11.22.0 while Context pop was introduced in v11.31.0 we could not use popHidden or change requirement in composer.lock.

I think we can just bump the minimum version in composer.json

[MrCrayon]

@sixlive is it ok if I rebase?
It's cleaner than merging if you are not using this branch.

[sixlive]

Yeah, go for it

[JYoo1]

Thanks for the work on this! 🙌
This would be super helpful for something we're working on. Does this still need dev work? Happy to step in and assist!

[TonsiTT]

Thanks for the work on this! 🙌 This would be super helpful for something we're working on. Does this still need dev work? Happy to step in and assist!

I'd be happy to assist as well. Would be a major feature, perhaps appropriate for a v1 release 👀

[benbjurstrom]

Would love to see this added. Until then is there any way to know what's happening in the tool call loop? In the example below I'd like some observability into what's being search, what the results are, what the LLM thought about the results, whether a new query is being submitted, etc.

I don't need streaming, but I also don't want to wait for the entire task to complete before getting an update.

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withMaxSteps(5)
    ->withPrompt('How do I create a controller in Laravel?')
    ->withTools([$docSearchTool])
    ->asText();