# obss evals and insights - Obss

CLI
obss evals and insights
Preview, start, list, and page Evals and Insights attempts, and compare treatments and agents, from the terminal.
Evals
score each finished run against its task’s rubric, and
Insights
read each run’s trajectory to find frictions. Each time either runs on an experiment, Obss records an attempt (
eval_
or
insights_
), and earlier attempts stay in the history.
Statuses
An attempt is
queued
,
running
,
completed
,
completed_with_errors
,
failed
, or
not_run
. Each of its runs has a
status
:
Status
Meaning
queued
,
running
No result yet. The run itself can still be going.
completed
The result is final.
skipped
,
failed
This attempt gives the run no result.
For when a pending result becomes final, see
Pending verdicts
and
Pending frictions
.
obss evals preview
Check, without changing anything, which runs Evals can score, which rubrics changed, and whether the
Evals model
is ready.
obss
evals
preview
<
experiment-i
d
>
[flags]
Plugin (MCP) tool:
preview_evals
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
Takes only the
global flags
.
Examples
Preview Evals:
obss
evals
preview
exp_8f2c41d0
--output
json
Result fields
Field
Meaning
ready
Whether Evals can start now.
modelsReady
and
userProviderReady
say what is missing.
setupUrl
The dashboard page that fixes a missing model or provider, or
null
eligibleRunCount
Runs with a usable rubric and output
runsWithoutRubricsCount
Runs with no rubric
runsWithInvalidRubricsCount
Runs whose saved rubric settings must be reviewed and saved
runsNotJudgeableCount
Runs that were blocked or haven’t finished
rubricChanges
Each task whose rubric changed, with
fromVersion
and
toVersion
failedRunIds
Runs whose latest Evals failed, to pass to
--run-ids
mode
The modes this experiment can use, as below
mode
has these fields:
Field
Meaning
experimentMode
The mode the experiment launched with. A trigger without
--mode
uses it.
latestMode
The mode of the latest Evals attempt that ran
agentAvailable
Whether your organization can use Agent mode
estimate
Only while your organization can use Agent mode: the
runs
the evals agent would review, and the
model
it uses
Notes
While your organization can use
Agent mode
, the preview prints what Agent mode would use, for example “Agent mode: Obss’s evals agent reads the whole run and checks the agent’s claims before it scores. Slower, and costs more. Up to 56 agent reviews on Claude Opus 5.5.” For an experiment that launched in Agent mode, it also prints “This experiment launched in Agent mode. Evals use it unless you add —mode standard.”
obss evals trigger
Score an experiment’s finished runs, for the first time or again, optionally with another model, instructions, mode, or set of runs. The agent doesn’t run again.
obss
evals
trigger
<
experiment-i
d
>
[flags]
Plugin (MCP) tool:
trigger_evals
The CLI doesn’t ask for confirmation. In Ask Obss, you approve each call.
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
Flags
Flag
Type
Default
Description
--mode
standard
,
agent
The experiment’s launch mode
How this attempt scores each run. See
Notes
.
--run-ids
string
Every eligible run
Comma-separated
run_
IDs to score, such as the preview’s
failedRunIds
. One ineligible ID fails the whole request.
--project-ai-selections
string
The organization’s
Evals model
A JSON array with one
{"role":"judge","providerConnectionId":"aic_…","providerModelId":"aim_…","effortLevel":"…"}
.
effortLevel
is optional.
--project-ai-instruction-overrides
string
The organization setting
A JSON array with one
{"role":"judge","customInstructions":"…"}
.
--attempt-id
string
A new ID
An ID you choose, so that a retry doesn’t repeat the change. It can be up to 100 characters.
Also takes the
global flags
.
Examples
Score an experiment again after you fix a rubric:
obss
evals
trigger
exp_8f2c41d0
Score only the runs whose Evals failed last time:
obss
evals
trigger
exp_8f2c41d0
--run-ids
run_6b31e09f,run_c27d4a58
Score in Agent mode:
obss
evals
trigger
exp_8f2c41d0
--mode
agent
--output
json
Result fields
Field
Meaning
evalsId
The new attempt’s
eval_
ID. Read its results with
obss evals get
.
status
,
mode
The attempt’s status, and the mode it uses: the one you chose, or the experiment’s launch mode
pollAfterMs
How long to wait before you read the results: 60 seconds in Agent mode, 10 seconds in Standard mode, or
null
when the attempt has already finished
replayed
true
when a repeated
--attempt-id
returned the earlier attempt
Notes
Agent mode for Evals and Insights is in private beta. Email
support@obss.app
to enable it for your organization.
Both modes use your
Evals model
and score against the same rubric.
standard
scores each run in one pass with your Evals model. It’s the fastest and lowest-cost mode.
agent
has Obss’s evals agent read the whole run and the changed files and check the agent’s claims before it scores. It costs more and takes a few minutes per run. Use it for long or complex runs, or when a score depends on checking what the agent did.
Without
--mode
, the attempt uses the experiment’s launch mode, not the latest attempt’s mode as the dashboard does. To repeat the latest mode, pass the preview’s
mode.latestMode
.
If your organization can’t use Agent mode, a request for it is refused with
agent_mode_unavailable
: “Agent mode isn’t available for your organization. Use Standard mode instead.” If you left out
--mode
and the experiment launched in Agent mode, the message is “This experiment launched Evals in Agent mode, which your organization can’t use now. Rerun with —mode standard.” If the
Evals model
can’t run Agent mode, the request is refused with
agent_mode_model_unsupported
and a
setupUrl
to
Obss AI features
, for example “Agent mode can’t use Claude Pro / Max models. Choose an Evals model from an API-key provider, or use Standard mode.”
obss evals list
List an experiment’s Evals attempts, with each one’s status, mode, and models.
obss
evals
list
<
experiment-i
d
>
[flags]
Plugin (MCP) tool:
list_evals
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
Flags
Flag
Type
Default
Description
--limit
integer
25
Results per page, up to 100 (default 25).
--cursor
string
The
nextCursor
from the previous page.
Also takes the
global flags
.
Examples
List an experiment’s Evals attempts:
obss
evals
list
exp_8f2c41d0
Result fields
Each attempt has its
eval_
ID,
status
,
mode
(
standard
or
agent
), and
models
. A failed attempt has
error
(
code
and
message
). An Agent mode attempt also has
agentUsage
once its agent has reviewed runs: the
inputTokens
,
cacheReadTokens
(the part your provider read from its prompt cache), and
outputTokens
that the evals agent has used so far. The terminal prints the mode after the status, as in
agent mode · 2.4M input tokens on Claude Opus 5.5 (300K cached)
, and one line for each model, with its provider, model, and effort.
Notes
Each attempt records the models it started with, so later changes under
Obss AI features
don’t affect it. An attempt from before Obss recorded models shows
Not recorded
.
obss evals get
Page the per-run results of one Evals attempt, or read one run’s full verdict.
obss
evals
get
<
experiment-i
d
>
[evals-id] [flags]
Plugin (MCP) tool:
get_evals_results
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
[evals-id]
The attempt’s
eval_
ID. Omit it to read the latest attempt, except with
--offset
above 0.
Flags
Flag
Type
Default
Description
--view
summary
,
trial_evidence
summary
summary
: one row per run.
trial_evidence
: one run’s full verdict, with
--trial-id
.
--limit
integer
50
Results per page, up to 200 (default 50). Only in the
summary
view.
--cursor
string
The
nextCursor
from the previous page.
--trial-id
string
The
run_
ID to read in the
trial_evidence
view.
--offset
integer
0
The byte to start from in the
trial_evidence
view.
--limit-bytes
integer
12288
Bytes per
trial_evidence
page, at most 12,288.
--wait-ms
integer
Wait for a change before you return, up to 25,000 ms.
Also takes the
global flags
.
Examples
Read the latest attempt:
obss
evals
get
exp_8f2c41d0
--output
json
Read the next page of the same attempt:
obss
evals
get
exp_8f2c41d0
--cursor
<
next-curso
r
>
--output
json
Read the next page of one run’s verdict:
obss
evals
get
exp_8f2c41d0
eval_5f1a9c
--view
trial_evidence
--trial-id
run_6b31e09f
--offset
12288
Result fields
Field
Meaning
attempt.evalsId
The attempt this page reads
attempt.mode
,
attempt.agentUsage
The attempt’s mode, and in Agent mode what its agent has used, as in
obss evals list
status
Each row’s
status
error
For a failed row,
code
and
message
. The message starts with “Evals could not evaluate this run, so it has no Pass or Fail.” and gives the cause and the next step.
outcome
The run’s verdict, with an explanation
evidenceCoverage
Which steps the
Evals model
read in full, shortened, or left out to fit its context window, and where the changed files came from.
null
means the read wasn’t recorded, which doesn’t prove a full read. See
What Evals read
.
hasMore
,
nextCursor
Whether more rows remain, and the cursor for the next page
Notes
A cursor stays on the attempt it came from, even after a newer attempt starts. A page can hold fewer rows than
--limit
because each page also has a size limit. When something was left out of a run’s read, the CLI prints a note under its row.
obss evals compare
See which differences between an experiment’s treatments and agents hold up, task by task, from the Evals verdicts on the Matrix. The output starts as
Comparisons
does: one sentence that answers the experiment’s question, then a block for each difference with each side’s pass rate, how the tasks went, the result, and the next step. Every comparison follows.
obss
evals
compare
<
experiment-i
d
>
[evals-id] [flags]
Plugin (MCP) tool:
get_evals_comparisons
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
[evals-id]
The attempt’s
eval_
ID. Omit it to compare the latest attempt.
Flags
Flag
Type
Default
Description
--kind
treatment
,
agent
,
interaction
Every kind
List only this kind. See
Notes
.
--label
clear
,
likely
,
unclear
Every label
List only this label. See
Notes
.
--limit
integer
200
Results per page, up to 200 (default 200).
--cursor
string
The
nextCursor
from the previous page. Send it with the same
--kind
and
--label
.
Also takes the
global flags
.
Examples
Compare the latest Evals of a seven-task experiment with two treatments and two agents:
obss
evals
compare
exp_8f2c41d0
Stripe CLI did better than Raw Agent on all 7 tasks.
Stripe CLI and Raw Agent  cmp_4a0e2c9d
Stripe CLI   93%   26 of 28 runs
Raw Agent     0%    0 of 28 runs
Better on all 7 tasks.
Clear difference
Both agents did better with Stripe CLI.
All 6 comparisons (Eval 1 (latest))
Treatments across all agents
cmp_4a0e2c9d  Clear difference
Stripe CLI   93%   26 of 28
Raw Agent     0%    0 of 28
Stripe CLI better on all 7
Treatments with Claude Code · claude-haiku-5-5 · default
cmp_7b1d3e5f  Likely difference · Run again to be sure · 42 runs: obss experiments clone-plan exp_8f2c41d0 --comparison cmp_7b1d3e5f
Stripe CLI   86%   12 of 14
Raw Agent     0%    0 of 14
Stripe CLI better on all 7
Treatments with Claude Code · claude-opus-5-5 · default
cmp_2c8e0a4b  Likely difference · Run again to be sure · 42 runs: obss experiments clone-plan exp_8f2c41d0 --comparison cmp_2c8e0a4b
Stripe CLI   100%   14 of 14
Raw Agent      0%    0 of 14
Stripe CLI better on all 7
Agents with Raw Agent
cmp_5e0b8c73  No clear difference
Claude Code · claude-haiku-5-5 · default   0%   0 of 14
Claude Code · claude-opus-5-5 · default    0%   0 of 14
Same on all 7
Agents with Stripe CLI
cmp_9f3a6d21  No clear difference
Claude Code · claude-haiku-5-5 · default    86%   12 of 14
Claude Code · claude-opus-5-5 · default    100%   14 of 14
Claude Code · claude-opus-5-5 · default better on 2, same on 5
Does a treatment help one agent more?
cmp_1d6f4a98  No clear difference
Stripe CLI helped Claude Code · claude-opus-5-5 · default more on 2 tasks, the same on 5.
Notes
With 7 tasks, a result for one agent alone or between agents can be a likely difference at most.
In 2 of 28 Matrix cells, repeated runs got different results.
The sentence comes first, then the block for the one difference, with its
cmp_
ID. Then
All 6 comparisons
lists each comparison under its group, with its
cmp_
ID. A likely difference prints, on its result line, the command that plans the runs to
run it again
.
Notes
come last.
Read the comparisons as JSON:
obss
evals
compare
exp_8f2c41d0
--output
json
Result fields
Field
Meaning
state
ready
, or the first reason that applies:
nothing_to_compare
,
too_many_agents
(one treatment with more than 6 agents),
evals_off
, or
no_verdicts
message
The
title
and
description
for the state, or
null
when the state is
ready
. For
no_verdicts
, it says whether more verdicts can still come.
summary
The sentence that answers the experiment’s question over its main comparisons:
text
, and
parts
, where each treatment or agent it names carries its
treatmentKey
or
agentKey
.
null
when the experiment has no main comparison. The same on every page and with every
--kind
and
--label
.
differences
One block for each clear or likely difference, in the order to read them, as below: the main comparisons first, then any other comparison with a clear difference, up to 5. A comparison with no clear difference is never in it. The same on every page and with every
--kind
and
--label
.
restLine
One sentence on what
summary
and
differences
leave out, such as “7 other comparisons show a likely difference. See All 40 comparisons.”
null
when they leave nothing out.
notes
One sentence each on what limits the comparisons, such as too few tasks. With an empty state, only the too-few-tasks note stays.
evals
The attempt:
evalsId
,
sequenceNumber
,
isLatest
, and
status
running
true
while the experiment still has runs to finish. Labels can change until the experiment and its Evals attempt finish.
comparisonCount
Every comparison in the experiment, whatever
--kind
and
--label
list
comparisons
The listed comparisons, as below
treatments
The experiment’s treatments, each with
treatmentKey
,
name
, and
isReference
for the baseline
agents
The agents that the listed comparisons name, each with
agentKey
and
label
. Here an agent is one agent with one model and effort.
taskCount
,
runsEach
The experiment’s tasks, and the trials of each combination
scoredRuns
,
unscoredRuns
Runs with an Evals verdict, and runs without one
noiseFloor
The
cases
where a task ran more than once with the same agent and treatment, and how many of them were
disagreeing
.
null
when no task ran more than once.
ceilingTaskCount
,
floorTaskCount
Tasks that passed in every run, and tasks that failed in every run. These tasks can’t show a difference.
agentPairsOmitted
true
when the experiment ran more than 6 agents. Agents are then not compared with each other, and there are no
interaction
comparisons.
hasMore
,
nextCursor
Whether more comparisons remain, and the cursor for the next read
Each block in
differences
has these fields:
Field
Meaning
comparisonId
The
cmp_
ID of the comparison
label
clear
or
likely
name
The two sides, the side that did better first, and what the comparison holds fixed when it isn’t a main comparison, for example “Stripe CLI and Raw Agent, with Claude Code · claude-haiku-5-5 · default”
sides
Both sides, the side that did better first:
key
,
label
,
passes
,
runs
, and
percent
(a whole number that is 100 only when every run passed, and 0 only when none did). Empty for an
interaction
.
taskLine
How the tasks went for the side that did better, for example “Better on 14 tasks, worse on 2, same on 26.” For an
interaction
, its task sentence.
headline
The difference as one sentence, for example “Stripe CLI did better than Raw Agent on all 7 tasks.” A likely one says “likely did better than”.
verdict
The result: “Clear difference”, “Likely difference”, “Likely difference · Needs more tasks to be sure”, or “No clear difference”. Show it on one line with
nextStep
, joined by ” · ”.
nextStep
kind
run_again
with its
text
(“Run again to be sure · 42 runs”) and
runs
, or
kind
more_tasks
with its
text
(“Launch again with more tasks”).
null
when there is nothing to do, or while the results can still change.
byAgent
How the agents split, for example “Both agents did better with Stripe CLI.”
null
unless the block compares treatments for all agents together and the experiment ran two or more agents.
Each comparison has these fields:
Field
Meaning
comparisonId
The
cmp_
ID. It stays the same when Evals run again.
kind
,
label
What it compares, and its result
family
primary
for a main comparison, or
breakdown
. See
Main comparisons and breakdowns
.
title
,
scope
The two sides, and what they share:
All agents
, an agent, or a treatment
headline
,
sides
,
verdict
,
nextStep
As in a block
sentence
The headline, both sides’ runs, and the verdict in one line
groupLabel
The group that
All N comparisons
lists it under, for example “Treatments across all agents”, “Treatments with Claude Code · claude-haiku-5-5 · default”, “Agents with Raw Agent”, or “Does a treatment help one agent more?”
taskSplit
How the tasks went, naming the side that did better, for example “Stripe CLI better on all 7”. For an
interaction
, a sentence.
treatmentKey
,
referenceTreatmentKey
The two treatments of a
treatment
or
interaction
comparison, or the treatment of an
agent
comparison (
treatmentKey
only)
agentKey
,
referenceAgentKey
The two agents of an
agent
comparison, or the agent of a
treatment
comparison (
agentKey
, which is
null
for all agents together)
arm
,
reference
The
passes
and
runs
that each side compared. Each task compares the same number of runs on each side, so these counts can be lower than the
Matrix
. An
interaction
has no top-level
arm
and
reference
.
first
,
second
For an
interaction
, the two agents, each with
agentKey
,
arm
, and
reference
tasks
The tasks where
arm
did better (
higher
), worse (
lower
), or the same (
same
). For an
interaction
,
higher
counts the tasks where the treatment helped
first
more than
second
.
pValue
,
adjustedPValue
The exact two-sided sign test on the tasks that differed, and the same after the Holm correction within the comparison’s
family
runsLeftOut
Runs of the two sides that the comparison doesn’t use: runs with no verdict, runs on a task where the other side has no verdict, and a side’s runs beyond the other side’s count
confirmation
For a likely difference, the runs that check it again:
taskCount
,
treatmentKeys
,
agentKeys
,
runsEach
(runs of each combination), and
runs
(the total).
null
for other labels.
confirmationUnavailable
Why a likely difference has no
confirmation
, as below
confirmationUnavailable
Meaning
earlier_evals
The read is of an earlier Evals attempt. A likely difference is run again from the latest Evals.
in_progress
The experiment or its Evals is still running.
cannot_settle
Running these tasks again can’t settle the difference: too few of them can still show a difference, or it would take more than 10 runs each. Launch the experiment again with more tasks.
Notes
The sentence and the blocks come first. Then the most decisive comparisons come first: a clear difference, then a likely one, then the rest, and within a label the main comparisons before the breakdowns.
--kind
and
--label
only filter which comparisons are listed: labels don’t change, and
summary
,
differences
, and
comparisonCount
stay the same.
--kind
Compares
treatment
A treatment against the baseline, for all agents together and for each agent. Without a baseline, each pair of treatments. With one agent,
agentKey
names it.
agent
Two agents with the same treatment
interaction
Whether a treatment helps one agent more than another (
Does a treatment help one agent more?
)
--label
is
clear
(
Clear difference
),
likely
(
Likely difference
), or
unclear
(
No clear difference
).
unclear
means that these runs can’t tell the two sides apart, not that the sides are the same.
The main comparisons are checked together, and the breakdowns are checked together and more strictly, because there are more of them. So the same split can be clear for all agents together and likely for one agent. See
How results are decided
.
One read lists up to 200 comparisons, and only as many as fit in 20 KiB. Page with
--cursor
while
hasMore
is true. If the comparisons changed since the first read, for example because Evals ran again, the cursor fails with
invalid_cursor
, and you start again without it.
When a likely difference is run again, a
treatment
comparison keeps its agent, or every agent for all agents together. An
agent
comparison keeps its treatment, and an
interaction
keeps both treatments and both agents. Plan the runs with
obss experiments clone-plan --comparison
.
A difference needs at least 6 tasks where one side did better than the other. Tasks where both did the same don’t count. With fewer than 6 tasks, no comparison can show a difference, and
notes
says so.
An experiment with more than 10,000 Matrix cells fails with
experiment_too_large
.
obss insights preview
Check whether the
Insights model
and
Experiment insights model
are ready for an experiment. This command changes nothing.
obss
insights
preview
<
experiment-i
d
>
[flags]
Plugin (MCP) tool:
preview_insights
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
Takes only the
global flags
.
Examples
Preview Insights:
obss
insights
preview
exp_8f2c41d0
Result fields
Field
Meaning
ready
Whether Insights can start now.
modelsReady
and
userProviderReady
say what is missing.
setupUrl
The dashboard page that fixes a missing model or provider, or
null
failedRunIds
Runs whose latest Insights failed, to pass to
obss insights trigger --run-ids
mode
The modes this experiment can use, as below
mode
has these fields:
Field
Meaning
experimentMode
The mode the experiment launched with. A trigger without
--mode
uses it.
latestMode
The mode of the latest Insights attempt that ran
agentAvailable
Whether your organization can use Agent mode
estimate
Only while your organization can use Agent mode: the
runs
the insights agent would review, the
model
it reviews them with, and
experimentModel
when the
Experiment insights model
differs from the
Insights model
Notes
While your organization can use
Agent mode
, the preview prints what Agent mode would use, for example “Agent mode: Obss’s insights agent reads the whole run and checks each friction against what the tools returned. Slower, and costs more. Up to 56 agent reviews on Claude Opus 5.5.” For an experiment that launched in Agent mode, it also prints “This experiment launched in Agent mode. Insights use it unless you add —mode standard.”
obss insights trigger
Read an experiment’s finished runs for frictions, for the first time or again, optionally with other models, instructions, mode, or runs. Evals verdicts don’t change.
obss
insights
trigger
<
experiment-i
d
>
[flags]
Plugin (MCP) tool:
trigger_insights
The CLI doesn’t ask for confirmation. In Ask Obss, you approve each call.
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
Flags
Flag
Type
Default
Description
--mode
standard
,
agent
The experiment’s launch mode
How this attempt reviews each run. See
Notes
.
--run-ids
string
Every eligible run
Comma-separated
run_
IDs to read, such as the preview’s
failedRunIds
. One ineligible ID fails the whole request.
--project-ai-selections
string
The organization’s models
A JSON array of
{"role":"…","providerConnectionId":"aic_…","providerModelId":"aim_…","effortLevel":"…"}
.
role
is
sweep
for the
Insights model
or
consolidation
for the
Experiment insights model
.
--project-ai-instruction-overrides
string
The organization setting
A JSON array of
{"role":"…","customInstructions":"…"}
, with
role
sweep
or
consolidation
. A role you omit uses the organization setting.
--attempt-id
string
A new ID
An ID you choose, so that a retry doesn’t repeat the change. It can be up to 100 characters.
Also takes the
global flags
.
Examples
Find frictions again on an experiment:
obss
insights
trigger
exp_8f2c41d0
Review in Agent mode:
obss
insights
trigger
exp_8f2c41d0
--mode
agent
--output
json
Result fields
Field
Meaning
insightsId
The new attempt’s
insights_
ID
status
,
mode
The attempt’s status, and the mode it uses: the one you chose, or the experiment’s launch mode
pollAfterMs
How long to wait before you read the results: 60 seconds in Agent mode, 10 seconds in Standard mode, or
null
when the attempt has already finished
replayed
true
when a repeated
--attempt-id
returned the earlier attempt
Notes
When Insights finish, Obss groups the findings into the project’s frictions. Read them with
obss frictions list
.
Both modes use your
Insights model
and
Experiment insights model
, and report frictions in the same way.
Agent mode
is in private beta.
standard
reviews each run in one pass with your Insights model. It’s the fastest and lowest-cost mode.
agent
has Obss’s insights agent read the whole run and check each friction against what the tools returned. It costs more and takes a few minutes per run. Use it for long or complex runs, or when one pass misses frictions.
In Agent mode, the insights agent also does the experiment’s grouping of frictions, which Standard mode does in one pass. For each task of each difference in the Brief, it says what a failed run and a passed run did differently (
obss experiments brief
). Without
--mode
, the attempt uses the experiment’s launch mode, not the latest attempt’s mode as the dashboard does. To repeat the latest mode, pass the preview’s
mode.latestMode
.
If your organization can’t use Agent mode, a request for it is refused with
agent_mode_unavailable
: “Agent mode isn’t available for your organization. Use Standard mode instead.” If you left out
--mode
and the experiment launched in Agent mode, the message is “This experiment launched Insights in Agent mode, which your organization can’t use now. Rerun with —mode standard.” If the
Insights model
or the
Experiment insights model
can’t run Agent mode, the request is refused with
agent_mode_model_unsupported
and a
setupUrl
to
Obss AI features
. The message names the model, for example “Agent mode can’t use Claude Pro / Max models. Choose an Insights model from an API-key provider, or use Standard mode.”
obss insights list
List an experiment’s Insights attempts, with each one’s status, mode, and models.
obss
insights
list
<
experiment-i
d
>
[flags]
Plugin (MCP) tool:
list_insights
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
Flags
Flag
Type
Default
Description
--limit
integer
25
Results per page, up to 100 (default 25).
--cursor
string
The
nextCursor
from the previous page.
Also takes the
global flags
.
Examples
List an experiment’s Insights attempts:
obss
insights
list
exp_8f2c41d0
Result fields
Each attempt has its
insights_
ID,
status
,
mode
(
standard
or
agent
), and
models
: the
Insights model
and
Experiment insights model
it started with.
A failed attempt has
error
(
code
and
message
). In Agent mode, when the insights agent couldn’t group the frictions,
error.message
starts with “Insights could not group this experiment’s frictions.” and gives the cause and the next step.
An Agent mode attempt also has
agentUsage
once its agent has reviewed runs: the
inputTokens
,
cacheReadTokens
(the part your provider read from its prompt cache), and
outputTokens
that the insights agent has used so far. It includes grouping the frictions and the pairs of runs in the Brief (
obss experiments brief
).
The terminal prints the mode after the status, as in
agent mode · 2.4M input tokens on Claude Opus 5.5 (300K cached)
, and one line for each model. The usage line names a model only when the Insights model and the Experiment insights model are the same.
obss insights get
Page the per-run results of one Insights attempt, read its report, or read one run’s full findings.
obss
insights
get
<
experiment-i
d
>
[insights-id] [flags]
Plugin (MCP) tool:
get_insights_results
Arguments
Argument
Description
<experiment-id>
Required. The experiment’s
exp_
ID.
[insights-id]
The attempt’s
insights_
ID. Omit it to read the latest attempt, except with
--offset
above 0.
Flags
Flag
Type
Default
Description
--view
summary
,
attempt_report
,
trial_evidence
summary
summary
: one row per run.
attempt_report
: the attempt’s report across the experiment.
trial_evidence
: one run’s findings, with
--trial-id
.
--limit
integer
50
Results per page, up to 200 (default 50). Only in the
summary
view.
--cursor
string
The
nextCursor
from the previous page.
--trial-id
string
The
run_
ID to read in the
trial_evidence
view.
--offset
integer
0
The byte to start from in the
attempt_report
and
trial_evidence
views.
--limit-bytes
integer
12288
Bytes per page in the
attempt_report
and
trial_evidence
views, at most 12,288.
--wait-ms
integer
Wait for a change before you return, up to 25,000 ms.
Also takes the
global flags
.
Examples
Read the latest attempt:
obss
insights
get
exp_8f2c41d0
--output
json
Read the next page of an attempt report:
obss
insights
get
exp_8f2c41d0
insights_9e04b2
--view
attempt_report
--offset
12288
Result fields
Field
Meaning
attempt.insightsId
The attempt this page reads
attempt.mode
,
attempt.agentUsage
,
attempt.error
The attempt’s mode, what its agent has used in Agent mode, and its error, as in
obss insights list
status
Each row’s
status
error
For a failed row,
code
and
message
. The message starts with “Insights could not analyze this run.” and gives the cause and the next step.
evidenceCoverage
How much of the trajectory the
Insights model
read.
null
means the read wasn’t recorded. See
What Insights read
.
hasMore
,
nextCursor
Whether more rows remain, and the cursor for the next page
Notes
A cursor stays on the attempt it came from, even after a newer attempt starts. The
attempt_report
and
trial_evidence
views page with
--offset
, which doesn’t name the attempt, so pages after the first need the
insights-id
argument.
Next steps
Evals
What Evals read, Agent mode, and pending verdicts.
Insights
What Insights read, the modes, and pending frictions.
obss frictions
Read the frictions that Insights found.
Comparisons
Read the labels and run a likely difference again.
obss runs and results
Count pending and inconclusive runs with results analyze.
Obss AI features
Choose the Evals and Insights models.
frictions
tags
Powered by
This documentation is built and hosted on Mintlify, a developer documentation platform