Skip to main content
All 80 tools on the Obss MCP server (https://mcp.obss.app/mcp), grouped by area. Each entry starts with the title your host shows in bold, then its badge, matching CLI command, and parameters. Sign-in, approvals, paging, retries, and errors are on the overview. Areas: Identity and projects · Catalog, resources, and uploads · Experiments · Schedules · Runs and results · Frictions · Evals and Insights · Tags · Data · Agents, models, and setup · Quota and notifications · Ask Obss
  • Badges are explained under Approvals: Read-only, Changes data, Destructive (your host asks first), Irreversible, and You approve each call (spends runs, creates a standing rule, or makes results public).
  • project_id is optional on most tools. Without it, the server picks a project for that call only (Projects).
  • expected_organization_id is an optional string on every tool that changes data, left out of the tables. Send organization.id from get_identity (Organization check).
  • limit above its maximum is capped, not refused.

Identity and projects

get_identity

Get the signed-in user and organization. Returns your Obss user and signed-in organization. Send organization.id as expected_organization_id on every write. Read-only · CLI: obss whoami This tool takes no parameters.

get_project_context

Get the organization and its projects. Returns the organization and its projects, with the values manage_project needs to rename one. Read-only · CLI: obss projects list, obss projects current, obss projects use, obss projects inspect

manage_project

Create or rename a project. It follows the same rules as the dashboard (Projects). Changes data · CLI: obss projects create, obss projects update

preview_delete_project

Preview project deletion. Counts what deleting a project removes by kind, the running experiments it stops, and the blockers. Show the result to the person before delete_project. Read-only · CLI: obss projects delete --dry-run

delete_project

Permanently delete a project. Deletes a project and everything in it, from tasks, tools, and treatments to experiments, schedules, runs, Evals, and Insights. Running experiments stop first; the result says how many. Destructive · Irreversible · CLI: obss projects delete With blockers, it returns project_deletion_blocked and deletes nothing. An already deleted project returns project_not_found.

Catalog, resources, and uploads

Find, create, change, upload, and delete the project’s tasks, files and machines, tools, and treatments.

inspect_catalog

Search the project catalog. Lists the project’s tasks, files, machines, tools, treatments, and agents, in a fixed order, each with its tags. total and kindCounts cover every match, not just the page. Read-only · CLI: obss catalog list Result fields items, total, kindCounts, and page (requestedLimit, returnedCount, hasMore, and nextCursor). Each item lists its tags. An agent item’s id is its agm_ ID, which a launch plan’s agentSelections names. With projection: "full", an MCP server’s facts.toolCount is the number of tools its latest version listed at its last check, or null until a check lists them.

browse_github_source

Browse public GitHub repositories. Finds a public github.com repository, ref, and path to import as a file or skill, and saves nothing. Pass the result to author_resource as source.repository, source.ref, and source.path. Read-only · CLI: obss github browse When truncated is true, narrow query. A URL result also returns its ref and path.

inspect_resource

Get a resource and its versions. Returns one record’s details, direct dependencies, and a page of version history. Read-only · CLI: obss resources inspect A machine’s definition view adds variableOverrides: machine variables that a CLI or SDK in the project also sets, with those tools and a note. In that tool’s treatments, the tool’s value wins (When a tool sets the same name).

inspect_resource_impact

Check what depends on a resource. Shows the tasks and treatments that use a record directly, and whether it is managed or pinned. Call it before a change. Read-only · CLI: obss resources impact

author_resource

Create or update a resource. Creates or updates a task, file, machine, tool, or treatment, with the dashboard’s rules. The tool’s schema has the exact definition fields and limits. Changes data · CLI: obss resources apply Fields of operation:
  • Rubric: a rubric has schemaVersion: 1 and up to 100 criteria, each with index and text. Criteria use Yes / no by default, and Advanced mode adds fields such as type, negate, and aggregation (Tasks and rubrics).
  • Shape: follow the branch for your action and kind exactly. Don’t copy catalog-only fields such as id, key, version, or versionId into a branch that doesn’t take them. An exact retry returns the first result, and a stale expectedUpdatedAt returns resource_revision_conflict.
  • GitHub source: a GitHub source saves a snapshot at the resolved commit. Later GitHub changes apply only when you call manage_resource with sync_github. Omit ref for the default branch and path for the whole repository. A whole-repository skill import finds every SKILL.md at any depth and names each skill by the name in its SKILL.md (Skills). For local archives and task imports, use create_upload.
  • Secrets: a definition names the environment variables and headers it needs, never their values. A value in an argument returns secure_input_requires_web, and the saved record returns the dashboard link where a person enters each value.
Result fields Every project already has the Raw Agent treatment for a baseline.

bulk_author_resources

Create or update many resources. Applies up to 20 author_resource operations in order, each with its own mutationAttemptId and revision. Changes data · CLI: obss resources bulk-apply On a failure, the result keeps the completed entries and gives failedIndex, failedMutationAttemptId, the error, and notStartedCount. Fix the failed operation and resend the whole batch: completed attempt IDs return their first result. For archives, use create_upload.

manage_resource

Set official versions and defaults, or sync from GitHub. Sets or clears a tool’s official version or the default machine, refreshes a file or skill from GitHub, or moves a treatment to its tools’ current versions. Changes data · CLI: obss resources lifecycle sync_github saves a new snapshot only when the selected content changed. Past runs keep their snapshots.

download_resource_archive

Download a file or skill archive. Returns the ZIP as base64, with its size and SHA-256. Archives over 8 MB download from the record’s dashboard page. Read-only · CLI: obss resources archive

create_upload

Start an archive upload. Creates or replaces a file, creates or versions skills, or imports tasks from an archive. Never put archive bytes or base64 in arguments. Changes data · CLI: obss uploads create Choose one path:
  1. You can read the archive. Send file, then PUT the raw bytes to transfer.url with curl -X PUT --upload-file <archive> and one -H per entry in transfer.headers. Never send Obss credentials there. Then call complete_upload.
  2. A person picks the archive. Omit file. The person opens upload.uploadUrl where they’re signed in (hosts that support URL requests show it), and that page completes the upload. Poll get_upload.
Pass upload.projectId as project_id to the other upload tools. Each upload expires at upload.expiresAt, and an abandoned upload creates nothing. A stale expectedUpdatedAt fails at once. A task import creates at most 50 tasks.

get_upload

Get an archive upload. Returns an upload’s status, declared file, uploadUrl, lastError, and, once completed, its records in result. Only its creator can read it. Read-only · CLI: obss uploads get Poll while status is awaiting_upload. If it has lastError, upload a corrected archive. completed, failed, cancelled, and expired are final.

complete_upload

Complete an archive upload. Checks the archive against the declared size and SHA-256, validates it, and applies the operation once, including any file replacement or official skill version change. Destructive · CLI: obss uploads complete The call returns upload_not_ready until the bytes arrive. An archive error keeps the upload awaiting_upload with lastError. An operation error, such as a stale revision or a deleted target, ends the upload as failed.

cancel_upload

Cancel an archive upload. Cancels an unfinished upload and deletes its archive. Destructive · CLI: obss uploads cancel Cancelling again returns the cancelled upload. A completed upload can’t be cancelled, so change or delete its records instead.

preview_delete_resources

Preview resource deletion. Shows what deleting the records removes or cancels (related experiments, runs, Evals, and stored files) and what blocks it. Read-only · CLI: obss resources delete --dry-run resourceType is task, task_version, skill, skill_version, file, machine, mcp_server, mcp_server_version, cli, cli_version, sdk, sdk_version, treatment, or experiment. Result fields Show the plan to the person, including each schedule, before you call delete_resources.

delete_resources

Permanently delete resources. Deletes experiments, tasks, files, machines, tools, or treatments (or one version of a task or tool), with the experiments, runs, Evals, and Insights that depend on them, after cancelling active runs. Schedules of a deleted experiment are deleted; schedules whose experiment uses what it deletes are paused. Destructive · Irreversible · CLI: obss resources delete Blocked items return resource_deletion_blocked. Otherwise the deletion starts in this call, and you poll get_resource_deletion until its status is final.

get_resource_deletion

Get resource deletion status. Returns the status of a deletion from delete_resources. Read-only · CLI: obss deletions status Poll while the status is preview_ready, queued, or deleting. These statuses are final: completed, failed, failed_to_start, needs_attention, and cancelled_before_start. Any other status reads unknown.

Experiments

An experiment runs tasks × treatments × agents × trials. These tools check, launch, watch, cancel, and share experiments. A typical launch calls inspect_catalog, preflight_experiment, launch_experiment, get_launch_status, get_experiment_status, and then list_runs.

preflight_experiment

Check an experiment plan before launch. Checks the exact plan, its tools’ and machines’ setup, your provider access, the revisions, and your run balance, without reserving runs. Read-only · CLI: obss experiments preflight The call succeeds even when the launch is blocked, so check ready. A plan with evalsMode or insightsMode set to agent gets the blocker agent_mode_unavailable when your organization can’t use that Agent mode, or agent_mode_model_unsupported, with a setupUrl to Obss AI features, when a model can’t run it. Result fields These are the same fields as in obss experiments preflight.

launch_experiment

Launch an experiment. Launches an exact plan that names catalog versions by ID, and returns a launchId (adm_…) for get_launch_status. Changes data · You approve each call · CLI: obss experiments launch Fields of plan: Result fields A refused launch returns one of these codes:

get_launch_status

Get launch status. Returns a launch’s progress by the launchId from launch_experiment, rerun_experiment, or test_custom_agent, before it has an exp_ ID, with the same phase, terminal, and counts as get_experiment_status. Read-only · CLI: obss experiments status, obss experiments watch Once experiment.experimentId is set, use it with the other experiment tools (Polling).

list_experiments

List experiments. Lists the project’s experiments with their tags and lineage. Experiments launched again or by a schedule share their source’s name, so tell them apart by lineage. Read-only · CLI: obss experiments list An experiment is listed when one of its runs matches every scope field. schedule_id also lists a launched experiment that has no runs yet. An unknown ID returns scope_not_found. The result has no scope. Result fields Each experiment has its tags, infraBlockedCount (Progress counts), and lineage as in get_experiment. hasMore and nextCursor page the list.

get_experiment

Get an experiment. Returns an experiment’s details, progress counts, launcher, runs used, and lineage. Read-only · CLI: obss experiments get Result fields lineage has these parts:
  • launchedVia: new, from_experiment, schedule, or schedule_launch_now.
  • basedOn: the parent experiment, the one it was launched from, with deleted: true once it’s gone.
  • schedule: the sch_ schedule that launched it, and its slot.
  • children: its child experiments (list them with list_experiments and based_on_experiment_id), scheduleCount, and the 10 newest of those schedules (all of them with list_schedules and experiment_id).

get_experiment_status

Get experiment status. Returns the phase, terminal, and progress counts. Read-only · CLI: obss experiments status, obss experiments watch While terminal is false, call again with wait_ms or after pollAfterMs. When terminal is true, read the final runs with list_runs.

get_experiment_plan

Get an experiment’s launch plan. Builds a launchable plan from an earlier experiment, at its task versions or the latest ones. Edit it, then pass it to launch_experiment. Read-only · CLI: obss experiments clone-plan With comparison_id from get_evals_comparisons, the plan runs that likely difference again: the tasks that can still show a difference, the two sides (both treatments and agents for an interaction), and enough runs each to likely settle it. Launching the plan spends the person’s runs, so say the run count and ask first. version_mode then defaults to experiment, treatments use their current tools as with Launch again, and Evals uses the latest Evals attempt’s settings.

rerun_experiment

Launch an experiment again. Relaunches a finished experiment with the same tasks, treatments, agents, trials, Evals, and Insights, or launches a schedule now with schedule_id. Evals and Insights run in the modes the experiment launched with, including Agent mode. If your organization can no longer use that Agent mode, the launch is refused with agent_mode_unavailable; use get_experiment_plan and set evalsMode or insightsMode to standard. Changes data · You approve each call · CLI: obss experiments rerun, obss schedules launch With schedule_id, it launches the schedule’s experiment at the latest task versions, under the schedule’s name, without moving the next launch. It refuses with experiment_schedule_run_active while the last one still runs. Poll the launchId with get_launch_status. To change the plan first, use get_experiment_plan, then launch_experiment.

cancel_experiment

Cancel an experiment or run. Cancels a whole experiment, or every still-active trial of one task, treatment, and agent combination. Completed work stays available. Destructive · CLI: obss experiments cancel A finished or stopping target returns its status. Poll get_experiment_status until the cancellation finishes.

publish_experiment

Publish an experiment. Creates a public link that anyone can view without an account, with runs, traces, outputs, Evals, Insights, and text files up to 256 KiB. Larger or binary files and private machine settings aren’t shared. Changes data · You approve each call · CLI: obss experiments publish Later results go live only through update_experiment_publication. To share privately, invite the person to your organization (Share results).

list_experiment_publications

List published experiments. Lists your organization’s public links. Read-only · CLI: obss experiments publications list

get_experiment_publication

Get an experiment’s public link. Returns a link’s status, current version, whether an update is available, and the lockVersion a change needs. Read-only · CLI: obss experiments publications get

update_experiment_publication

Update a published experiment. Replaces the results at the same link with the latest ones. The earlier results stay until the update is ready. Destructive · You approve each call · CLI: obss experiments publications update

unpublish_experiment

Unpublish an experiment. Stops sharing an experiment, and the public link stops working at once. Destructive · CLI: obss experiments publications unpublish Unpublishing again returns the unpublished experiment.

get_public_experiment

Get results from a public link. Reads the results at a public experiment link, one section at a time. Read-only · CLI: obss gallery get A large Frictions result arrives in parts (part.index of part.count). Read every page, then append each part’s clusters, family clusters, and outcome badge audits in order. A repeated clusterKey continues the previous cluster’s occurrences.

Schedules

A schedule launches one finished experiment again on a cron cadence in a time zone.

list_schedules

List schedules. Lists a project’s schedules, newest first, with nextRunAt, the person each runs as, the last result, and needsAttention with pauseReason when Obss paused it. Read-only · CLI: obss schedules list For a schedule’s experiments, call list_experiments with schedule_id.

get_schedule

Get a schedule. Returns its cadence in plain words, nextRunAt, experiment, run use, last result, and the lock_version a change needs. Read-only · CLI: obss schedules get Result fields runsPerLaunch is the runs each launch uses. runsPer30Days is the runs in the next 30 days, not a monthly average. It’s 0 while the schedule is paused, or when no launch falls in the next 30 days.

preview_schedule

Preview a schedule. Shows what a new schedule or a change would do. Call it before manage_schedule and tell the person the runs it uses. Read-only · CLI: obss schedules preview Result fields The preview has the cadence in plain words, the time zone it stores, the next 3 launches, runsPerLaunch, and runsPer30Days. A preview of a paused schedule counts it as resumed, and schedule.status says whether a changed schedule stays paused.
  • shortfall is a warning, not a refusal. It’s null while your runs cover the launches, or while the schedule is paused. Otherwise it gives at (the first uncovered launch), runsShort, availableRuns, launchesCovered, runsToAdd (to cover every launch until until), until, untilReset (true when until is your next run reset), a message, and an action linking Usage & billing.
  • blockers lists everything that would stop a launch now, each with a code, message, and action with its fix link, so the person can fix them all at once. While it isn’t empty, a create, a resume, or a change to the experiment, cron, or time zone refuses with schedule_launch_blocked. blocker (the first one) is for older CLI releases.

manage_schedule

Create or change a schedule. Creates, changes, pauses, or resumes a schedule. Each launch uses the project’s tasks, treatments, and agents as they are then. Changes data · You approve each call · CLI: obss schedules create, obss schedules update, obss schedules pause, obss schedules resume
  • A create, a resume, or a change to the experiment, cron, or time zone checks the launch as you, and makes you the person the schedule runs as. If blocked, it refuses with schedule_launch_blocked, with details.blockers shaped as in preview_schedule; the message is the blocker’s own, or a count when there are several.
  • update changes only the fields you send. A schedule still named after its experiment takes the new experiment’s name.
  • Obss pauses a schedule when a launch needs a fix, or after 3 scheduled launches in a row fail. A checked change clears the reason, but the schedule stays paused until you resume it.
  • After the person fixes the cause, resume checks the launch and resumes. If something still blocks it, give the person every fix link in details.blockers, then resume again. Pause and resume are safe to repeat, and resume never launches at once.
  • A create or resume past 50 active schedules in a project returns schedule_limit_reached.
  • To launch a schedule now, call rerun_experiment with schedule_id.

delete_schedule

Delete a schedule. Deletes a schedule. The experiments it launched stay. Destructive · Irreversible · CLI: obss schedules delete An already deleted schedule returns schedule_not_found.

Runs and results

A run is one execution of a task with a treatment and an agent. These tools read results and run details.

list_runs

List run results. Returns a page of runs with stable IDs, typed errors, and a plain explanation of each result in criteriaSummary. Read-only · CLI: obss runs list Result fields Each run has its stable IDs, typed errors, and criteriaSummary. It can also have:
  • treatmentUse: each tool’s state and calls, as in get_run_profile. It’s null when the run has no usable profile.
  • blockedReason: on a blocked run, its cause and a one-line message.
  • setupFailure: present when a setup command stopped the run before its agent started. It has the owner step, commandNumber when the owner has several commands of that kind (null for a package install, where manager and packageOwners name the package manager and who declared the packages), the command, exitCode (null on a timeout, with timeoutSeconds), the last output lines, and outputTruncated. See troubleshooting.
The scope fields, experiment_id to friction_id, name the records whose runs to read; a run must match them all. list_frictions and get_friction take the same ones, and list_experiments all but experiment_id. An unknown ID returns scope_not_found, not an empty list. scope in the result names each record with its dashboard page.

get_run_profile

Get a run profile. Returns the same profile as the dashboard’s Run profile rail: treatment use, tool calls, what stood out, tokens, cost, and time. Read-only · CLI: obss runs profile Result fields Step ordinals refer to get_run_trajectory, and the per-turn series is the runTurns dataset of query_data.

get_run_trajectory

Get a run trajectory. Returns a page of a run’s steps in order, each with its model turn and tool, plus the run’s input, cached-input, and output token totals (or null). Read-only · CLI: obss runs trajectory Continue with nextAfterOrdinal while truncated is true. Result fields steps (each with its ordinal, model turn, and tool; tool steps include their measured duration when the agent reported it), the run’s token totals, truncated, nextAfterOrdinal, and coverage, which gives the returned range, the total steps, and whether this page holds the whole trajectory. For a custom agent, the run also has the same stopReason, reported model, and agentError as get_run_profile.

get_run_transcript

Get a run’s raw transcript. Returns a page of the original transcript: UTF-8 text in contentText, other bytes as base64 in contentBase64. sha256 covers the whole transcript. Read-only · CLI: obss runs transcript

get_run_output

Get a run’s output and files. Returns a page of the final message, the unified diff, the file manifest, one kept file, or the output archive. Read-only · CLI: obss runs output Text pages are UTF-8 in contentText, and other pages are base64 in contentBase64. sourceTruncated marks a preview that was capped.

analyze_results

Analyze experiment results. Runs one ranked analysis over experiment and run results. Read-only · CLI: obss results analyze scope takes these fields. A run counts only when it matches every field you send. Ranked analysis is never paged: limit is an upper bound, and the response size limit can return fewer items. Result fields When page.truncated is true, these fields say why and where to read the rest: The analysis is in result:
  • summarize_scope: result.totals (runs, passedRuns, failedRuns, evalsFailedRuns, infraBlockedRuns, inconclusiveRuns, pendingRuns) and result.treatments[] (treatmentKey, runCount, passedRuns, failedRuns).
  • compare_treatments: result.treatments[] with the same counts plus failureRate, isReferenceTreatment, and deltaFromReference.failureRate, and result.referenceTreatmentKey.
The pass rate is passedRuns / (passedRuns + failedRuns), or 1 - failureRate. pendingRuns counts runs with no verdict that Evals can still judge. They stay out of the pass and fail counts and out of inconclusiveRuns, so wait until pendingRuns is 0. summarize_scope and search_frictions return insightsCoverage: analyzedRuns, pendingRuns, and unanalyzedRuns (Insights off, skipped, failed, or never run). An empty friction list is final when pendingRuns is 0.

Frictions

A friction is a problem that agents hit, tracked across runs under a stable frc_ ID.

list_frictions

List a project’s frictions. Finds what keeps going wrong for agents, in the project or any scope, and what to fix first. By default, the most runs the agent never got past come first. Read-only · CLI: obss frictions list A friction is listed when one of its runs matches every filter. Result fields Each field is explained under obss frictions list.

get_friction

Read one friction with its evidence. Returns its summary, impact, suggested fix, what the agent did, where it happens most, and up to ten recent occurrences with run, recovery, cited steps, and quotes. Read-only · CLI: obss frictions get Result fields The friction has the fields of list_frictions, plus these: To list all the friction’s runs, call list_runs with friction_id and the same scope.

Evals and Insights

Evals score each run against its task’s rubric, and Insights read each run’s trajectory to find frictions. Preview, trigger, then poll the results with wait_ms.

preview_evals

Preview Evals for an experiment. Shows which runs Evals can score, rubrics that need attention or changed, model readiness, and the modes available. Read-only · CLI: obss evals preview Result fields ready, modelsReady, userProviderReady, setupUrl, eligibleRunCount, runsWithoutRubricsCount, runsWithInvalidRubricsCount, runsNotJudgeableCount, rubricChanges, failedRunIds, and mode: experimentMode (the mode the experiment launched with), latestMode, agentAvailable, and, only while your organization can use Agent mode, estimate (runs and model). Their meanings are under obss evals preview.

trigger_evals

Start Evals for an experiment. Starts an Evals attempt, or reruns Evals on the same recorded work after a rubric change. Changes data · You approve each call · CLI: obss evals trigger Standard scores each run in one pass with your Evals model and condenses very long runs to fit its context window. It’s the fastest and lowest-cost mode. Agent has Obss’s evals agent read the full run and the changed files and check the agent’s claims before it scores. It makes many calls to your Evals model, so it costs more and takes a few minutes per run. Both modes score against the same rubric (how the modes differ). A request for Agent mode, stated or inherited, is refused with agent_mode_unavailable when your organization can’t use it: “Agent mode isn’t available for your organization. Use Standard mode instead.” Without mode, the refusal has details.inheritedMode set to agent and says “This experiment launched Evals in Agent mode, which your organization can’t use now. Rerun with mode: “standard”.” When the Evals model can’t run Agent mode, the refusal is agent_mode_model_unsupported, with a setupUrl to Obss AI features. Agent mode is in private beta. Result fields evalsId (the new eval_ attempt), status, mode (the mode the attempt uses), pollAfterMs (60 seconds in Agent mode, 10 seconds in Standard mode, or null when the attempt has already finished), and replayed (true when a repeated trigger_attempt_id returned the earlier attempt).

list_evals

List Evals history. Returns the experiment’s Evals attempts, newest first, without reports, each with its models and mode. A failed attempt has error (code and message). An Agent mode attempt also has agentUsage once its agent has reviewed runs: the inputTokens, cacheReadTokens, and outputTokens that the evals agent has used so far. Read-only · CLI: obss evals list

get_evals_results

Get Evals results. Returns a summary for each run of an attempt, or the exact rubric and verdict JSON for one trial. Read-only · CLI: obss evals get Result fields

get_evals_comparisons

Compare treatments and agents. Compares an experiment’s treatments and agents task by task, from the Matrix’s Evals verdicts. It returns the sentence that answers the experiment’s question and a block for each difference first, as at the top of Comparisons, then every comparison with its result. Read-only · CLI: obss evals compare kind is treatment (a treatment against the baseline, for all agents together and for each agent, or each pair of treatments without a baseline), agent (two agents with the same treatment), or interaction (Does a treatment help one agent more?). label is clear (Clear difference), likely (Likely difference), or unclear (No clear difference). unclear means that the runs can’t tell the two sides apart, not that the sides are the same. The most decisive comparisons come first, the main comparisons before the breakdowns within a label. kind and label only filter the list: labels don’t change, and summary, differences, restLine, and comparisonCount stay the same. A read returns up to 200 comparisons that fit in the response. A cursor fails with invalid_cursor once the comparisons change. More than 10,000 Matrix cells fails with experiment_too_large. Result fields Each comparison has comparisonId (cmp_), kind, label, family (primary for a main comparison, or breakdown), title, scope, headline, sides, verdict, nextStep, sentence, groupLabel, taskSplit, treatmentKey, referenceTreatmentKey, agentKey, referenceAgentKey, arm and reference (or first and second for an interaction), tasks (higher, lower, same), pValue (an exact two-sided sign test), adjustedPValue (the Holm correction within its family), runsLeftOut, confirmation, and confirmationUnavailable (earlier_evals, in_progress, or cannot_settle). confirmation holds the runs that check a likely difference again. cannot_settle means that running these tasks again can’t settle it, and the experiment needs more tasks. To plan them, pass its comparisonId to get_experiment_plan as comparison_id, with a new launch_attempt_id. Say the result in the words of summary and each verdict, which every surface shows, with verdict and nextStep on one line. nextStep is run_again (with text and runs) or more_tasks (with text), or null. Each field is explained under obss evals compare.

get_experiment_brief

Get an experiment’s Brief. Reads an experiment’s Brief: its clear and likely differences, worded as in get_evals_comparisons, with one written sentence and the frictions seen with the side that did worse. Obss counts every number from the experiment’s Evals verdicts and frictions. A model writes only each sentence and, in Agent mode, each pair’s line. Read-only · CLI: obss experiments brief state says how complete the Brief is. ready, failed, not_written, and no_insights are final for the Brief. While state is writing, the differences and counts are already in the read: call again after waitMs milliseconds until state changes. In Agent mode, also call again after waitMs while any finding’s pendingTaskCount is above 0. The tool takes no wait_ms. Result fields Each finding has the fields of its block in differences (comparisonId, label, name, sides, taskLine, verdict, nextStep, byAgent, headline), and kind, title, scope, arm and reference, tasks, pValue, adjustedPValue, worseSide, worseTaskCount, notReviewed, sentence, confirmation, confirmationUnavailable, frictions, afterFix, and issueMarkdown. Each friction has frictionId (frc_), title, summary (what goes wrong, in one line), fixIn, tasks, dashboardUrl, failedRuns (how many of the worse side’s failed runs had it, out of runs), passedRuns (the same for the other side’s passed runs), and, for arm and reference, runsWithFriction out of runsReviewed, which is how the friction was picked. The counts use only the worseTaskCount tasks and the runs that Insights reviewed. afterFix is set for a clear difference with a friction to fix in the product: its text, taskCount, and runs. issueMarkdown is the difference as an issue in Markdown, the text that Copy as issue copies. When the experiment’s latest Insights ran in Agent mode, each finding also has these fields: Each pair has taskVersionId (tskver_), taskName, dashboardUrl, and state: pending while the agent reads it, completed when its lines are written, or failed when the agent gave no answer that passed Obss’s checks, which shows only the two runs. Its worse and better runs each have these fields: Show the pairs in that order, as the dashboard and issueMarkdown do: the example with “Like N other tasks” (from similarTaskCount), each task that went differently with “Went differently”, then the rest. A pair says where two runs went different ways, not why the difference happened. What the agent used to read the pairs counts in the Insights attempt’s agentUsage (list_insights). When you report a friction, give both sides’ counts (“Raw Agent 24 of 28 failed runs · Stripe CLI 0 of 26 passed runs”) and say it was seen with the side that did worse. Don’t call it the cause. To run a likely difference again, pass its comparisonId to get_experiment_plan as comparison_id. To run a clear difference again after a fix, add after_fix set to true. Each field is explained under obss experiments brief.

preview_insights

Preview Insights for an experiment. Shows whether the Insights models and your provider access are ready, with a dashboard link for any secure setup, and the modes this experiment can use. Read-only · CLI: obss insights preview Result fields ready, modelsReady, userProviderReady, setupUrl, failedRunIds, and mode: experimentMode (the mode the experiment launched with), latestMode, agentAvailable, and, only while your organization can use Agent mode, estimate (runs, model, and experimentModel when the Experiment insights model is a different model). Their meanings are under obss insights preview.

trigger_insights

Start Insights for an experiment. Starts an Insights attempt, or reruns Insights on the same recorded work. Changes data · You approve each call · CLI: obss insights trigger Standard reviews each run in one pass with your Insights model. It’s the fastest and lowest-cost mode. Agent has Obss’s insights agent read the whole run and check each friction against what the tools returned. It costs more and takes a few minutes per run. It also does the experiment’s grouping of frictions, which Standard mode does in one pass. For each task of each difference in the Brief, it says what a failed run and a passed run did differently (get_experiment_brief). Both modes report frictions in the same way. Without mode, the attempt uses the experiment’s launch mode, not the latest attempt’s mode as the dashboard does. A request for Agent mode, stated or inherited, is refused with agent_mode_unavailable when your organization can’t use it: “Agent mode isn’t available for your organization. Use Standard mode instead.” Without mode, the refusal has details.inheritedMode set to agent and says “This experiment launched Insights in Agent mode, which your organization can’t use now. Rerun with mode: “standard”.” When the Insights model or the Experiment insights model can’t run Agent mode, the refusal is agent_mode_model_unsupported, with a setupUrl to Obss AI features. Agent mode is in private beta. Result fields insightsId (the new insights_ attempt), status, mode (the mode the attempt uses), pollAfterMs (60 seconds in Agent mode, 10 seconds in Standard mode, or null when the attempt has already finished), and replayed (true when a repeated trigger_attempt_id returned the earlier attempt).

list_insights

List Insights history. Returns the experiment’s Insights attempts, newest first, without reports, each with its models and mode. A failed attempt has error (code and message); when the insights agent couldn’t group the frictions, error.message starts with “Insights could not group this experiment’s frictions.” An Agent mode attempt also has agentUsage once its agent has reviewed runs: the inputTokens, cacheReadTokens, and outputTokens that the insights agent has used so far, including grouping the frictions and the pairs of runs in the Brief. Read-only · CLI: obss insights list

get_insights_results

Get Insights results. Returns a summary for each run of an attempt, the attempt’s public report, or the exact Insights JSON for one trial. Read-only · CLI: obss insights get Result fields attempt (with its insightsId, models, mode, error, and, in Agent mode, agentUsage), and for each row its status, its error (a failed row’s message starts with “Insights could not analyze this run.” and gives the cause), which follows the same rules as in get_evals_results, and evidenceCoverage (null for older runs). hasMore and nextCursor page the summary view.

Tags

Tags are your own labels on tasks, files, machines, tools, treatments, and experiments. Limits are under obss tags.

list_tags

List a project’s tags. Lists every tag with its tag_ ID, usage by kind, and the updatedAt that rename_tag and delete_tag need. One call returns all of them (at most 500). Read-only · CLI: obss tags list Result fields Each tag has id, name, usage (total and byKind), createdAt, and updatedAt. inspect_catalog and list_experiments filter by tag with tags and untagged.

update_tags

Add or remove tags on records. Adds and removes tags on up to 100 records at once. Adding a name that the project doesn’t have yet creates the tag. Changes data · CLI: obss tags add, obss tags remove Result fields records (each record’s id, kind, and all its tags after the change) and createdTags (the tags this call created). You can send the same call again safely. A change that would give a record more than 20 tags changes nothing and returns tag_limit_exceeded. Tags never change a record’s definition, versions, or updatedAt.

rename_tag

Rename a tag. Renames a tag. Every record keeps it, because its tag_ ID doesn’t change. Changes data · CLI: obss tags rename Result fields The renamed tag, with its id, name, usage, createdAt, and updatedAt. A tag that changed since you read it returns tag_changed with the current updatedAt. A name that another tag in the project has, in any case, returns tag_name_taken with that tag’s ID.

delete_tag

Delete a tag from every record. Deletes a tag and removes it from every record that has it. Tell the person how many records have it, from list_tags, before you call it. Destructive · Irreversible · CLI: obss tags delete Result fields tag (the deleted tag’s id and name) and removedFrom (how many records had it). A tag the project doesn’t have returns tag_not_found.

Data

Query datasets with fields, filters, and aggregates, for questions the other reads don’t answer, such as results by model or mode.

inspect_data_model

List the data you can query. Lists every dataset, its fields and types, each field’s filters and aggregates, the default order, and the limits. Read-only · CLI: obss data describe Result fields datasets, each with name, title, description, grain, fields, identityFields, defaultSelect, and defaultOrder, and limits. Each field has name, type, nullable, description, filterOperators, and aggregates.

query_data

Query Obss data. Runs one bounded query over a dataset. It never accepts SQL or returns internal IDs, secrets, or credential-like values. Read-only · CLI: obss data query Result fields columns, rows, query (the query as Obss ran it), page (returned, hasMore, nextCursor, truncated, and reasons), and limits. The datasets and their fields are listed under obss data, including evals.mode, the model fields of evals and insights, and the per-turn runTurns series.

Agents, models, and setup

Manage a project’s agents, custom agents, and models, your Obss AI features, and the setup steps that need the dashboard.
Custom agents are in private beta. Email support@obss.app to enable them for your organization.

inspect_project_ai

Get built-in and custom agents, models, and Obss AI settings. Returns agents, provider and model readiness, your personal connections, model refresh state, and each Obss AI feature’s model and whether it can run. It never returns secrets or starts a refresh. Read-only · CLI: obss project ai inspect Result fields The roles, providers, agents, and models views return agent, provider, model, and role counts, provider readiness, model refresh counts, and blockers. blockers is empty when you can run models; otherwise it says why not, with the dashboard setupUrl where a person fixes it. A custom agent’s custom block has status (needs_values, testing, needs_attention, not_tested, or ready), description, revision, connectionId, url, definition, env, envNote, testBlockers, handshake (start-up details such as failure.message, failure.detail, and authMethods), fix (the failed step and what to change), mcpServersRefused, and latestRun. See Custom agent fields. In the providers view, each provider’s modelRefresh has status, listedAt, and the counts of the last refresh. listedAt is set when the refresh has read the provider’s list, so the models view has those models while status is still running. See obss project ai inspect. A cursor returns stale_cursor when the configuration changed between pages, usually from a model refresh.

manage_project_ai

Create custom agents and change agents, models, and Obss AI settings. Creates or changes custom agents, turns agents and models on or off, refreshes model lists, states context windows, and sets each Obss AI feature’s model and instructions. No operation carries a secret. Changes data · CLI: obss project ai apply A custom agent’s source is one of:
  • kind: "registry" with registryId and an optional version: an agent from the ACP Registry.
  • kind: "manual" with install and optional args. install is an npm package (kind: "npm", package), a Python package (kind: "python", package), or a download (kind: "download", url, sha256, command).
  • kind: "agent_json" with agentJson: an agent.json definition.
env lists up to 64 environment variable names. Values are entered only in the dashboard, and the result returns the agent’s page and the dashboard link for each value. skillsFolder is the folder where the agent reads skills, up to 256 characters. A stale revision changes nothing and returns configuration_changed, with details.currentRevision when the target has one. An invalid operation returns validation_error. Read inspect_project_ai again before you retry. When a public catalog lists a context window for a model, inspect_project_ai reports it as suggestedContextWindow, which applies only after you state it. Connecting, signing in again, switching accounts, and entering credentials stay on the provider’s dashboard link from inspect_project_ai.

delete_project_ai

Remove custom agents or provider connections. Removes a custom agent, deletes a custom provider connection, resets a built-in organization connection, or disconnects your personal provider account. Destructive · CLI: obss project ai delete A removed custom agent no longer shows in new experiments or the Agents list, and Obss clears its environment variable values. Past results keep its name. Obss refuses the removal while a run of the agent is in progress, either a Test agent run or an experiment. Removing it again returns the same result. Pass the revisions from inspect_project_ai. A refused removal changes nothing and returns one of these codes:

test_custom_agent

Test a custom agent. Checks that a custom agent installs, starts, and answers, with one run of the Agent check task: Agent default, the Raw Agent treatment (no tools), no project files, and Evals and Insights off. It uses 1 run. Changes data · You approve each call · CLI: obss project ai test-agent Poll the returned launchId with get_launch_status. Each finished run with these settings updates the agent’s status in inspect_project_ai. A Test agent run is a check, not an experiment. It sends no notice and doesn’t show on Home or in the dashboard’s experiments list, but the CLI and the plugin still list it. It refuses while custom.testBlockers names a blocker, with the first one’s code: Get secure setup links. Returns the dashboard links for OAuth connections, API keys, secret headers, and other secret values. Read-only · CLI: obss setup links Result fields issues lists each machine, MCP server, CLI, and SDK in the project that still needs a required value or your account connection, with the setupUrl where a person adds it. An MCP server whose saved values aren’t bound to its current address has a field of scope address, and its setupUrl is the server’s Tools tab, where a person selects Refresh. The list is empty when nothing is missing.

inspect_mcp_account

Check your MCP server connection. Checks whether your account can use an MCP server in a treatment, and returns the latest problem, a next step, and the dashboard setup link. A failed reconnect never replaces a working connection. Read-only · CLI: obss mcp accounts inspect Result fields ready, recovery (with message, action, setupUrl, retryAt, and apiKeyAlternative), setupUrl, redirectUri, and oauthAppConfigured. Their meanings are under obss mcp accounts inspect. When the server’s saved values aren’t bound to its current address, ready is false and setupUrl is the version’s Tools tab, where a person selects Refresh.

detect_mcp_auth_mode

Detect how an MCP server authenticates. Probes the server’s current definition and reports how it authenticates. It never returns the server’s endpoint or credentials, and the OAuth app’s Client ID and Client secret are entered only in the dashboard. See MCP server sign-in. Read-only · CLI: obss mcp accounts detect-auth Result fields

list_mcp_server_tools

List an MCP server’s tools. Reads the tools the server listed at Obss’s last check, with the check’s status, message, time, tool count, and Tools page (setupUrl). Titles, descriptions, and hints are the server’s own text. Read-only · CLI: obss mcp tools list Continue with after_position while truncated is true. Result fields The check’s status, message, and time, the tool count, setupUrl, truncated, nextAfterPosition, usageRunCount, lastChange, headerValuesBound, and each tool with canTurnOff, onRunCount, runCount, callCount, offInTreatmentCount, and offInTreatmentIds. Their meanings are under obss mcp tools list. When headerValuesBound is false, only a person’s Refresh at setupUrl binds the saved values: refresh_mcp_server_tools fails with mcp_tool_check_needs_web and launches stop with mcp_header_values_unbound. Give the person setupUrl instead of calling it (Saved header values and the server address).

refresh_mcp_server_tools

Refresh an MCP server’s tools. Checks the server’s tools now and returns the list as list_mcp_server_tools does. A running check is joined, not restarted. A server that needs your account connection or a header value returns that status and its setupUrl. Changes data · CLI: obss mcp tools refresh Result fields The fields of list_mcp_server_tools, plus refresh, which is running until the check finishes. If the wait ends first, read the list again with list_mcp_server_tools. It never sends saved header values to an address they aren’t bound to, and it never binds them. When headerValuesBound is false, it fails with mcp_tool_check_needs_web. Give the person details.setupUrl (the version’s Tools tab) to check the address and select Refresh, and don’t call the tool again until they have.

Quota and notifications

Read your run balance and manage your notifications. Buying runs stays in the dashboard.

inspect_quota

Get the run balance. Returns the run balance, and whether a launch of requested_runs runs can start now. It offers no purchases or billing actions. Read-only · CLI: obss quota inspect Result fields remainingRuns, reservedRuns, usedRuns, totalRuns, remainingBreakdown, state, and readiness, as in obss quota inspect. The plugin returns no billingUrl.

list_quota_ledger

List run balance history. Returns a page of run balance transactions. It offers no purchases or billing actions. Read-only · CLI: obss quota ledger Result fields Each entry has a qtx_ transactionId, type, amount, effectiveAt, expiresAt, source, and the experimentId and link of the experiment that caused it, if any. hasMore and nextCursor page the list.

list_notifications

List notifications. Returns a page of your notifications, optionally for one project. Read-only · CLI: obss notifications list Result fields Each notification has notificationId, insertedAt, status, title, and a safe link. A schedule’s paused, skipping, and short-of-runs notices, and the problem notices of the experiments it launched, also carry its scheduleId, which you can read with get_schedule.

acknowledge_notifications

Mark notifications as read. Marks one notification, or up to 500 unread ones, optionally in one project. Repeating a call gives the same result. Changes data · CLI: obss notifications acknowledge Result fields acknowledgedCount and, with all, hasMore. Repeat the call while hasMore is true.

inspect_notification_preferences

Get notification preferences. Returns your account-wide setting for each category and channel, and whether each can be changed, is required, or doesn’t apply. Read-only · CLI: obss notifications preferences This tool takes no parameters.

manage_notification_preferences

Change notification preferences. Changes only the categories and channels you send. The rest stay as they are. Changes data · CLI: obss notifications preferences update For example, {"experiment-updates": {"email": false, "in-app": true}} keeps experiment updates in-app only.

Ask Obss

Ask Obss, the in-app assistant, uses the tools on this page.

disconnect_ask_oqo

Disconnect Ask Obss. Disconnects Ask Obss from your account in this organization until you connect it again on the Ask Obss page. Your chats stay. Destructive · CLI: obss ask disconnect This tool takes no parameters other than expected_organization_id. Result fields removed is false when Ask Obss wasn’t connected, and then nothing changes.

Next steps