Scores and signals

A score is any evaluation value attached to a trace or a session. A success signal is a score the platform aggregates into a Success Metric. @withgateway/sdk/scores sends both. The shape matches the Python SDK’s Run.score() and Run.success() exactly, so signals from either client aggregate together without change. This subpath needs @opentelemetry/api installed, because it reads the active span to find the trace a score belongs to.

Create a client

import { ScoresClient } from "@withgateway/sdk/scores";

const scores = ScoresClient.fromEnv();

fromEnv(config?) reads GATEWAY_HOST, GATEWAY_PUBLIC_KEY and GATEWAY_SECRET_KEY, and throws ScoreError naming what is missing. Any field passed in config wins over the environment, and timeoutSeconds defaults to 15.

Post a score

await scores.score("answer_relevance", 0.82, {
  sessionId,
  comment: "graded by the nightly judge",
});

Option	Type	Meaning
`sessionId`	`string`	The session to attach to; the preferred target when present
`traceId`	`string`	The trace to attach to; falls back to the active span’s trace
`datasetRunId`	`string`	A dataset-run score
`observationId`	`string`	A span-level score, which requires `traceId`
`dataType`	`"NUMERIC" \| "CATEGORICAL" \| "BOOLEAN"`	Defaults to `NUMERIC`
`comment`	`string`	Stored alongside the score
`metadata`	`Record<string, unknown>`	Merged into the score’s metadata
`environment`	`string`	Defaults to the project’s `default` environment

A score targets exactly one of a session, a dataset run, or a trace, in that order of preference. With no target given and no active span, the call is a no-op rather than an error.

score() and success() are fail-soft. A broken transport warns and resolves rather than throwing into your agent. Use postScore() with the same arguments when you want the failure.

Emit a success signal

success(name, value?, options?) records a score tagged as an aggregatable signal. A boolean value records a pass or fail, and a number records a measurement.

await scores.success("resolved_ticket", true, {
  agentId: "support-triage",
  sessionId,
});

await scores.success("handle_time_s", 42, {
  agentId: "support-triage",
  sessionId,
});

value defaults to true. success takes every score option plus two of its own.

Option	Type	Meaning
`agentId`	`string`	The agent the signal belongs to, so the platform can roll it up without a trace join
`metricName`	`string`	The Success Metric to feed; defaults to `name`

The signal keys are written last, so your own metadata cannot overwrite them and drop the signal from aggregation. Emit one whenever a real unit of value happens: a ticket resolved, an eval run, a human accepting an edit.

Score the trace you are already in

With tracing initialized and a span open, omit the target and the active trace is used.

import { observe } from "@withgateway/sdk/tracing";
import { ScoresClient, currentTraceId } from "@withgateway/sdk/scores";

const scores = ScoresClient.fromEnv();

await observe("support-agent.run", async () => {
  const answer = await handleTicket(ticket);
  console.log(currentTraceId());        // the trace this span belongs to
  await scores.success("resolved_ticket", answer.resolved);
  return answer.text;
}, { input: ticket });

currentTraceId() returns the active trace id, or undefined when there is no valid active span.

Score into an isolated project

A handle from tracing.initIsolated() carries a scores client bound to that handle’s keys. Its signals land in the same project the handle’s spans export to, with no credentials threaded through your own code.

import { initIsolated } from "@withgateway/sdk/tracing";

const support = initIsolated({ serviceName: "support-agent" });

await support.scores.success("resolved_ticket", true, {
  agentId: "support-triage",
  sessionId,
});

See Tracing and experiments for what an isolated handle is.

Where to go next

Evaluation and Replay for how scores show up on the platform.
Runs and Scoring for the Python equivalents.
Tracing and experiments for the traces these scores attach to.

CLI

Python SDK

TypeScript SDK

SDK Reference

REST API

MCP Server

Create a client

Post a score

Emit a success signal

Score the trace you are already in

Score into an isolated project

Where to go next

​Create a client

​Post a score

​Emit a success signal

​Score the trace you are already in

​Score into an isolated project

​Where to go next

Create a client

Post a score

Emit a success signal

Score the trace you are already in

Score into an isolated project

Where to go next