# Tools - Obss

Build treatments
Tools
Add skills, MCP servers, CLIs, and SDKs by hand or from GitHub, give them the values they need, and mark official versions.
Tools are the skills, MCP servers, CLIs, and SDKs that your product gives agents. Add them here, then attach them to a
treatment
to measure how much each one helps. They’re separate from the
Obss Plugin
and
Obss CLI
, which you use to operate Obss itself.
Manage → Tools
has a page for each type:
MCP servers
,
Skills
,
CLIs
, and
SDKs
.
Tool types
Tool
What it is
What to measure
Skill
Instructions and helper files the agent should follow for your product
Whether the agent loads it for the right tasks, and pass-rate lift
MCP server
Tools your product gives agents over MCP
Clear, distinct tool names and descriptions, and whether the agent picks the right tool first
CLI
A command-line program agents install and call
Help text, error messages, and subcommands the agent can find
SDK
A library agents install and import
Method names, examples, and errors that point to the right fix
Every tool is versioned, and each experiment keeps the tool versions it launched with. A tool’s
Runs
tab lists every run whose treatment gave the agent the tool, in any version and every experiment, whether or not the agent called it. Its
Experiments
and
Frictions
tabs list those runs’ experiments and frictions.
Skills
A skill is a folder with a
SKILL.md
file, plus any other files the agent needs. Open
Skills
, select
Add skills
, and drop a folder, a
.zip
, or a
.skill
file, or choose
From GitHub
. One upload can hold several skills.
A skill takes the
name
in its
SKILL.md
, which is the name the agent sees, whatever folders it sits in:
skills/cloud/onboarding/SKILL.md
with
name: cloud-onboarding
becomes
cloud-onboarding
. When a repository holds the same skill at two paths with the same files, Obss imports it once. Two different skills with the same
name
can’t be imported together. Obss names each clash, so you can rename one of the skills or choose a path that holds only one.
To save a new version, open the skill and select
Import new version
. A skill’s page also has
Overview
and
Files
tabs.
MCP servers
An MCP server gives agents your product’s tools over MCP. Add it, then check which tools it offers and how agents use them.
Add an MCP server
Open
MCP servers
, select
New MCP server
, and choose a
Transport
:
Transport
When to use it
HTTP
The standard remote connection. Most servers use it
SSE
A remote server whose docs say server-sent events
Local
A command on the agent’s machine. Set its
Arguments
and
Environment variables
For a remote server, enter its
URL
and choose how it signs in:
Sign in with provider
uses OAuth: each member connects their own account, and only that member can use that connection. Some providers need an OAuth app you create with them (
MCP server sign-in
).
API key or token
sends a key or token in
Headers
, shared by everyone in the project.
Add
Additional headers
when the server’s docs ask for more, such as an account or team ID. You can also edit the whole server as JSON with
View as JSON
.
Keys in a URL or an argument
When a server takes its key in the URL (
https://mcp.example.com/mcp?apiKey=…
), or a local server takes a value such as a database address as an argument, write
${NAME}
there instead of the value. Then add
NAME
with its value under
Headers
for a remote server, or under
Environment variables
for a local one.
${NAME}
works only in the URL’s path or query, not in the host, the port, or after
#
.
Obss fills in the value when a run starts and when it checks the server’s tools. The agent never sees a remote server’s value; a local server gets it in the run’s environment.
In a local server’s shell script, such as
sh -c '…'
, write
"$NAME"
or
"${NAME}"
in double quotes, because single quotes stop the shell from reading the variable.
Dashboard
CLI
Plugin (MCP)
In the server’s
URL
, write
https://mcp.example.com/mcp?apiKey=${EXA_API_KEY}
. Under
Headers
, add
EXA_API_KEY
and enter the key. Select
Save
.
Declare the name in
headerNames
and write it in
url
, then enter the value from the link in the result.
{
"action"
:
"create"
,
"kind"
:
"mcp"
,
"mutationAttemptId"
:
"create-exa-mcp-01"
,
"name"
:
"Exa"
,
"definition"
: {
"transport"
:
"http"
,
"url"
:
"https://mcp.example.com/mcp?apiKey=${EXA_API_KEY}"
,
"headerNames"
: [
"EXA_API_KEY"
]
}
}
Save it as
exa.json
and run
obss resources apply --operation-file exa.json
(
obss resources apply
). For a local server, declare the name in
envNames
and write
${NAME}
in
args
.
Ask your AI assistant. It creates the server with
Create or update a resource
(
author_resource
) and gives you the link to enter the key.
Add the Exa MCP server at https://mcp.example.com/mcp. It takes the API key in the apiKey query parameter.
See an MCP server’s tools
An MCP server’s
Tools
tab lists the tools the server offered at Obss’s last check: each tool’s name, description, parameters, and the server’s own hints, such as
Read-only
or
Destructive
. In the
MCP servers
list, the
Tools
column shows the count and opens the tab.
Obss checks a remote server’s tools when you save a new version, connect your account to it, open the
Tools
tab more than a day after the last check, or ask for a check:
Dashboard
CLI
Plugin (MCP)
Open the MCP server, select the
Tools
tab, and select
Refresh
. Or, in the
MCP servers
list, open the server’s
⋯
menu and select
Refresh tools
.
obss
mcp
tools
refresh
mcp_2e8b6f13
--wait-ms
20000
--output
json
See
obss mcp tools refresh
.
Ask your AI assistant to check the server again. It calls
Refresh an MCP server’s tools
(
refresh_mcp_server_tools
).
Check the tools of our Docs MCP server again.
The tab shows each added, removed, or changed tool and when it changed. A failed check says why and keeps the last list, with a link to setup when the server needs your account connection or its header values.
Obss can’t check a local server’s tools outside a run. A server can show different tools to different accounts, so the list is the one Obss saw with the account of the person who checked last. You can also
turn off some of a server’s tools
in one treatment.
See how runs use each tool
The
Tools
tab also shows how agents used each tool. The count at the top is the number of runs whose treatment included this version.
Label
What it means
N calls in R runs
Calls agents made to the tool, and in how many runs, or
Not called
3 calls in 2 runs of the 4 it was on
Some treatments turned the tool off, so only runs that had it on count (or
Not called in the 4 runs it was on
). No count shows when every run had it off
Off in N treatments
Treatments that turn the tool off, on any server version. Opens
Treatments
filtered by
MCP tool turned off
New
or
Updated
The server’s last change added or changed the tool
Counts appear once a run of this version is processed. They include failed calls, because Obss doesn’t record whether a call succeeded, so the tab shows no error rate. They also include calls where the agent changed a dot in the tool name to
_
or
-
. A long list adds search,
Show only never called
, and
Show only off in a treatment
.
Dashboard
CLI
Plugin (MCP)
Open the MCP server and select the
Tools
tab. To see another version, choose it in the version list first.
obss
mcp
tools
list
mcp_2e8b6f13
--output
json
usageRunCount
is the number of runs whose treatment included the version. Each tool has
onRunCount
(the runs that had it on),
runCount
,
callCount
,
offInTreatmentCount
, and up to 20
offInTreatmentIds
. See
obss mcp tools list
.
Ask your AI assistant which tools agents use. It calls
List an MCP server’s tools
(
list_mcp_server_tools
), which returns the same fields as the CLI.
Which tools of our Docs MCP server did agents call, and which treatments turn tools off?
Test without a tool
To find out how much one MCP tool helps, run a treatment with the tool and the
same treatment without it
.
Test without this tool
makes the second treatment for you. Then
launch an experiment
with both.
Dashboard
CLI
Plugin (MCP)
Open the MCP server and select the
Tools
tab.
Select
Details
on the tool.
Select
Test without this tool
, then a treatment that includes this version of the server.
Obss copies that treatment with the tool turned off, names it “
treatment
without
tool
”, and opens it. If a treatment with the same tools and turned-off tools already exists, Obss opens it instead. A treatment that already turns the tool off shows
Already off
and opens when you select it.
When no treatment includes this server version, the button reads
New treatment with this server
. The button is unavailable for a tool that can’t be turned off.
Read the treatment’s tools (its
facts
) with
obss catalog list --kind treatment --projection full
. Then create the copy with
obss resources apply
, using the same
skillVersionIds
,
mcpServerVersionIds
,
cliVersionIds
, and
sdkVersionIds
. Copy its
disabledMcpTools
and add the tool to the server version’s entry, because the field replaces every choice.
Turn off MCP tools
has the operation file.
If a treatment with the same tools already exists, the result is
matched_existing
with that treatment.
Ask your AI assistant to make the copy. It reads the treatment with
Search the project catalog
(
inspect_catalog
,
projection
full
), then creates the copy with
Create or update a resource
(
author_resource
).
Make a copy of the "Docs MCP" treatment without the ask_question tool of our Docs MCP server.
Saved header values and the server address
Saved header values, such as the API key in
Headers
, and the
Client secret
of your own OAuth app are bound to server addresses that a person chose. A launch, a run, a tool check, and
Connect
send them only to a bound address (
Connect
sends the secret to the sign-in server that address names). So a key can’t reach an address no person chose, such as one an AI assistant saved through the CLI or the plugin.
A person binds values in the dashboard in one of three ways:
Save the server with values.
Obss binds them to the server’s address after the save, and a renamed header keeps its value.
Save the server at a new URL.
Obss binds all saved values to the new address. The old one stays bound, so you can compare two versions of one server at two addresses in one experiment.
Select Refresh on a version’s Tools tab.
Under
Refresh
, the tab shows
Binds the saved values to …
with the full address. Obss binds the values to it, then checks the tools.
When you enter a value again, it’s bound only to the server’s current address. If a key leaks, enter the new key, then select
Refresh
on each other version that must keep it.
When an AI assistant saves a version at another address through the CLI or the plugin, nothing is bound:
Where
What you see
Tools
tab
Saved values are not bound to this address
, and Obss doesn’t check that version on its own
MCP servers
list
Check address
, and the project home lists the server with setup steps
Server page, for a server that signs in with your account
Open Tools tab
in place of
Connect account
Launch
Stops with
mcp_header_values_unbound
, naming the server, version, and full address, with
Open Tools tab
obss mcp tools refresh
,
refresh_mcp_server_tools
Fail with
mcp_tool_check_needs_web
and a link to the version’s
Tools
tab. They never bind values
If you expect the address, open the
Tools
tab and select
Refresh
. If you don’t, save a new version of the server with the correct address instead.
CLIs and SDKs
Open
CLIs
or
SDKs
, select
New CLI
or
New SDK
, and set these fields:
Install
: a package manager and package, or
Custom
for a shell install command. It runs while Obss builds the machine, before variables exist, so use a post-install command for anything that needs one.
Post-install commands
(CLIs only): commands that run at the start of every run,
after the machine’s setup commands
and before the agent starts. They can read the tool’s variables as
${NAME}
, for example to sign the CLI in.
Variables
: values the post-install commands and the agent need, such as an API key. In the tool’s treatments, they
replace a machine variable with the same name
.
A CLI also has
Commands
: every executable name its install puts on
PATH
. A shell call that runs one of them counts as use of the CLI in
Run details
. One package can install several names; for example,
@brightdata/cli@0.4.1
installs
bdata
and
brightdata
.
Install
Commands
An exact npm or crates.io version, such as
@brightdata/cli@0.4.1
, or a
go install
Read from the package when you save. Leave
Commands
empty to save all of them. A name the package doesn’t install stops the save, and Obss shows the names it does install
pip, gem, apt, custom, or a version range such as
@brightdata/cli@^0.4
Type the names yourself, separated by spaces or commas
The CLI and the plugin use the same
commands
field and rules, and their result names every executable that counts as use of the CLI.
Import from GitHub
Import
Skills
and
Files
from a public GitHub repository instead of uploading them.
1
Open the GitHub import
In the add dialog, choose
From GitHub
.
2
Choose the repository
In
GitHub source
, type a repository name,
owner/name
, or
owner/
to list one owner’s public repositories, and pick one from the list.
3
Choose a branch, tag, or commit
In
Branch, tag, or commit
, pick a ref, or leave it empty for the default branch (first in the list). In a repository with many refs, type the start of a branch or tag name to find it.
4
Choose a path
In
File or folder path
, pick a path, or leave it empty to import the whole repository. For skills, the list shows each skill folder and each folder that holds skills, with its skill count, and the dialog says how many skills your selection imports.
5
Import
For skills, turn on
Mark as official
to make each imported skill its
official version
. Select
Import
, and Obss saves a snapshot at the resolved commit.
You can also paste a GitHub tree or blob URL into
GitHub source
and pick the repository, and Obss fills the ref and path from the URL. A ref that contains
/
, such as
feature/docs
, resolves the same way it does on GitHub. Every field also accepts typed text, such as a commit SHA or a path in a large repository.
To get later changes, open the skill or folder and select
Check for updates
. A skill gets a new version when its content has changed, a folder takes the newer snapshot, and finished runs keep the content they used.
From the CLI or the plugin,
obss github browse
and
Browse public GitHub repositories
(
browse_github_source
) find the same repository, ref, and path without saving anything. Pass the choice to
obss resources apply
or
author_resource
as
source.repository
,
source.ref
, and
source.path
.
Add the values a tool needs
Some tools need a value, such as an API key or your own account sign-in, before a run can use them. You add these values only in the dashboard, and the CLI and the plugin return a link to the page.
A tool with an empty required value shows
Needs values
in its list, and its box on
Tools
counts how many need values.
An MCP server that uses
Sign in with provider
shows
Connect account
until you connect your own account.
A notice at the top of
Home
names each of these, with
Add values
or
Connect account
to finish it. An empty optional value never blocks a run.
Official versions
An official version is the one version of a tool that you declare part of your product’s interface. Turn on
Mark as official
on a tool’s page, or when you import skills.
The read-only treatments
All official skills
,
All official MCPs
,
All official CLIs
, and
All official SDKs
collect them, so you can measure your whole interface at once. Each one appears when the project has at least one official version of that type.
Attach tools to a treatment
After you create a tool, Obss offers
Create treatment
to make a treatment with it. You can also include it, at a version you choose, in any treatment under
Manage → Treatments (Toolsets)
(
create a treatment
). When you save a new tool version,
Update linked treatments?
lets you
move those treatments to it
.
Approvals during runs
During a run, every tool call is approved automatically, so agents can call every tool in the treatment, including MCP tools marked destructive (
destructiveHint
) or as needing a person’s approval (
_meta["anthropic/requiresUserInteraction"]
). Give the task data that the agent can safely change or delete.
Approval marks outside Obss
Outside Obss, Claude Code in headless mode (
claude -p
) refuses a tool marked
anthropic/requiresUserInteraction
, because no person can approve it. Keep the mark on tools whose actions can’t be undone, because it protects people who use your server interactively.
Delete a tool
You can’t delete a tool version that a treatment still uses, and the delete preview says why, for example “Remove this skill from treatments before deleting it.” Remove a skill from those treatments first, or delete them in the same deletion. An MCP server, CLI, or SDK version that an experiment used can’t be deleted, even after you remove it from every treatment.
A deletion can also delete the experiments, runs, Evals, and Insights that depend on what you delete. Deleting a skill version or a task version deletes every run that used it, and you can’t undo it. Read the delete preview, which lists everything it removes and anything that blocks it, before you confirm.
If a
schedule
launches an experiment that can’t launch again after the deletion, the deletion pauses that schedule, and the delete dialog names it first.
From the CLI and the plugin
The CLI and the plugin name the tool types
skill
,
mcp
,
cli
, and
sdk
.
Task
CLI
Plugin (MCP)
List tools
obss catalog list --kind skill,mcp,cli,sdk
inspect_catalog
Create or change an MCP server, CLI, or SDK, or a skill from GitHub
obss resources apply
author_resource
Upload skills from a folder or ZIP
obss uploads create --kind skill
create_upload
,
complete_upload
Set an official version, or check GitHub for updates
obss resources lifecycle
manage_resource
Check how an MCP server signs in
obss mcp accounts detect-auth
detect_mcp_auth_mode
List an MCP server’s tools
obss mcp tools list
list_mcp_server_tools
Check an MCP server’s tools again
obss mcp tools refresh
refresh_mcp_server_tools
Get the link to add values
obss setup links
get_setup_links
List the experiments, runs, and frictions that used a tool
--tool-id
(or
--tool-version-id
for one version) on
obss experiments list
,
runs list
,
frictions list
tool_id
(or
tool_version_id
) on
list_experiments
,
list_runs
,
list_frictions
Preview and delete
obss resources delete --dry-run
preview_delete_resources
,
delete_resources
Next steps
Treatments (Toolsets)
Group your tools into treatments and keep a baseline.
Launch an experiment
Compare treatments on the same tasks.
Measure a tool's impact
Follow the whole loop for one tool.
Machines and files
MCP server sign-in
Powered by
This documentation is built and hosted on Mintlify, a developer documentation platform