Repository navigation
fix(server-utils): Record AI SDK tool spans for tools run outside its tool loop - #25820
Merged
Merged
Conversation
Contributor
size-limit report 📦Compared: 2fca194 (PR test merge using develop at 445c2f5) Against: 1ffbccd (develop) · base run #63950
|
Contributor
Optional E2E testsSome optional checks were not successful. They do not block the merge. 2 not successful, 60 successful. See the run.
Successful checks
|
… tool loop eve 0.84+ runs tools in its own loop instead of the AI SDK's, so the `ai:telemetry` channel never published `executeTool` and no `gen_ai.execute_tool` spans or tool errors were recorded. eve still calls the AI SDK's registered telemetry integrations, so the Vercel AI integration now registers one that publishes those tool executions to the channel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mydea
force-pushed
the
fix/eve-tool-spans
branch
from
October 28, 2026 13:55
bf1f2b5 to
b12e123
Compare
mydea
marked this pull request as ready for review
October 28, 2026 14:12
mydea
requested review from
andreiborza and
isaacs
and removed request for
a team
October 28, 2026 14:12
andreiborza
approved these changes
Oct 28, 2026
mydea
added a commit
that referenced
this pull request
Oct 28, 2026
…#25809) **Why:** eve 0.84 moved to its v27 event format, which replaced the `step.attempt.*` instrumentation events with `model.started`/`model.retried`/`model.settled`. `eveInstrumentation` sets the conversation id from `turn.started` and `step.attempt.started`, so on eve 0.84+ AI spans from resumed steps lost `gen_ai.conversation.id`. It now also handles `model.started` (same `scope.sessionId` shape) and keeps `step.attempt.started` for older eve. The same release replaced the session stream's `session.waiting`/`turn.failed` events with `delivery.settled`, so the node-eve e2e helper never saw the turn end and aborted after 25s. That's why `node-eve (latest)` has been failing on `develop`. The helper now also stops on `delivery.settled`, keeping the old names for the pinned `^0.63.0` variant. **note**: With only this change, node-eve (latest) is still failing. There is another thing that needs to be fixed for this, which should be taken care of in #25820 stacked on top of this. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
51 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why: eve 0.84+ runs tools in its own loop instead of the AI SDK's, so
ai:telemetrynever publishesexecuteToolfor them. eve apps lose theirgen_ai.execute_toolspans, and tool errors aren't captured (this is whynode-eve (latest)fails). eve still calls the AI SDK's globally registered telemetry integrations around each tool call, so the Vercel AI integration now registers one that publishes those executions to the channel. The existing subscriber then builds the span, keeps it active while the tool runs (manual spans nest under it) and captures errors. The span ends inonToolExecutionEnd, because that is the only place eve passes the tool output.This isn't eve-specific, but it only changes anything for code that runs tools outside the AI SDK's loop. The AI SDK's own loop also calls the integration, but always inside its own tool span. The integration detects that by tool call id and just runs the tool, which the existing "captures error in tool" tests (exactly 3 spans, ai v6 and v7) cover.
How to review:
vercel-ai-tool-telemetry.tsis the whole change. The fact thatnode-eve (latest)E2E now passes is what shows that this works as intended.Unsure about: We write to
globalThis.AI_SDK_TELEMETRY_INTEGRATIONSdirectly, which is whatai'sregisterTelemetrydoes, because the SDK can't importai. eve's owncall.requested/call.settledevents would avoid that, but the span wouldn't be active during the tool.Stacked on #25809.
🤖 Generated with Claude Code