> ## Documentation Index
> Fetch the complete documentation index at: https://docs.surfacearea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# gateway bench

> Push a world as a content-hashed version, address any version with slug@ref, branch and tag it, file proposals instead of pushes, and dispatch hosted runs from the terminal.

`gateway bench` treats a [world](/worlds) as a versioned bundle. Every push is content-hashed and browsable in the app, every run records the version it ran against, and any version stays addressable afterwards.

Building a benchmark? A benchmark is a world with tasks; [gateway benchmarks](/cli/benchmarks) runs the worlds commands in benchmark order (`init`, `check`, `push`, `run`, `results`) and uses `bench` underneath for versions.

<Info>
  The product says **world**; the command group is still called `bench` and its
  arguments still say `container`. The words on screen moved to the six in the
  [Glossary](/glossary) while identifiers stayed put, so existing pipelines keep
  working. Read `container` as `world`.
</Info>

## Push a version

`bench push` uploads the directory as a new version on a branch. The bundle is content-hashed, so pushing an unchanged tree lands on the existing version instead of creating a duplicate.

```bash theme={null}
gateway bench push ./acme-world -m "add the refund route"
gateway bench push ./acme-world --branch refunds -m "wip"
```

| Flag | What it does |
| - | - |
| `--slug` | Container slug; defaults to the directory name. |
| `--branch`, `-b` | Branch to push to. Defaults to the directory's tracked branch, else `main`. |
| `--message`, `-m` | Change reason, shown in History. |
| `--author` | Attribution shown in History; defaults to `GATEWAY_AUTHOR`. |
| `--name`, `--description`, `--tag`, `--visibility` | Container metadata (`PRIVATE` or `PUBLIC`); `--tag` repeats. |
| `--auto-bump`, `--rc`, `--post` | Rewrite `[project].version` in `pyproject.toml` before hashing, so the bump lands inside the bundle. |
| `--image` | A pre-built OCI image reference, recorded on the version for image and agentic worlds. |
| `--force` | Skip the fast-forward check and move the branch even if it diverged. |

A world is pushed as the tree it is: `schema/world.json`, `connector.toml`, tasks and verifiers. The platform compiles and hosts that tree, so nothing is repackaged.

<Info>
  Pushing onto a branch whose tip moved since you pulled is refused with a
  `409`. Pull or `checkout` the new tip and push again. Use `--force` only when
  you mean to discard what moved.
</Info>

## Address a version with slug\@ref

Most read commands take `slug[@ref]`. A `ref` resolves in four ways.

| Ref form | Example | Resolves to |
| - | - | - |
| Branch name | `acme-world@main` | The branch's current tip. |
| Tag | `acme-world@v1.2.0` | The exact version the tag points at, permanently. |
| Semantic version | `acme-world@0.8.0` | The version published under that `pyproject` version. |
| Content hash prefix | `acme-world@939cd2df` | That exact bundle. |

A bare `slug` means the head. `bench pull` also accepts `@latest`, and `bench checkout` reads a bare slug as `main`.

```bash theme={null}
gateway bench resolve acme-world@main     # pinned version id + task count
gateway bench tasks acme-world@v1.2.0     # the task list of that version
```

A branch moves; a tag does not. Pin a release gate or a published demo to a tag, and let CI track a branch.

## Get a version onto your disk

| Command | What it does |
| - | - |
| `bench pull <slug>` | Reconstruct a version's source directory. `--target`/`-t` (or `--dir`) chooses where; the default is `./<slug>`. |
| `bench checkout <slug@ref>` | Pull *and* record the branch and base version in the directory, so later pushes fast-forward correctly. |
| `bench status [path]` | Where the directory stands: tracked branch, base version, whether it has local edits, whether the remote tip moved. |

```bash theme={null}
gateway bench checkout acme-world@main -t ./acme-world
gateway bench status ./acme-world
```

Use `checkout` when you intend to push back, `pull` when you only want to read a version's files.

<Info>
  Pulling is for authoring. To run an agent against a world, open a hosted
  session by slug instead of unwrapping a copy on your own disk; see
  [gateway worlds](/cli/worlds#open-live-sessions).
</Info>

## Read the history

| Command | What it does |
| - | - |
| `bench log <slug>` | A branch's version lineage, tip first: hash, semver, author, reason. `--branch`/`-b` (default `main`), `--limit`/`-n` (default 20). |
| `bench refs <slug>` | The container's branches and tags. |
| `bench diff <base> <head>` | Per-file diff between two version ids: added, removed, modified, unchanged. |

```bash theme={null}
gateway bench log acme-world -b main -n 10
gateway bench diff <baseVersionId> <headVersionId>
```

## Branch, tag and fork

| Command | What it does |
| - | - |
| `bench branch [name]` | List branches and tags with the directory's current branch marked `*`, or create a branch with `name`. `--slug`, `--path`, `--from` or `--version-id` for the start version. |
| `bench tag <name>` | Create an immutable tag pointing at a version. `--slug`, `--path`, `--version` or `--version-id` (default: the `main` tip). |
| `bench fork <slug>` | Fork a container into a new one, sharing the bundle and recording `forkedFromVersionId`. `--into` names the target slug, defaulting to `<slug>-fork`. |

```bash theme={null}
gateway bench branch refunds --from <versionId>
gateway bench tag v1.2.0 --version <versionId>
gateway bench fork acme-world --into acme-world-eu
```

Fork when a world diverges permanently — a second tenant, a different vendor plan. Branch when the change is meant to come back.

## Propose a change instead of pushing one

`bench propose` files the directory's local edits as a change proposal against a branch, which someone then reviews and applies. Use it where a push would be a pull request: an agent editing a world it does not own, or a change that needs a human before it reaches `main`.

```bash theme={null}
gateway bench propose ./acme-world --title "add refunds" -m "the Sept incident needs a write path"
gateway bench proposals acme-world --status open
```

`propose` takes `--title`, `--message`/`-m` (aliased `--reason`), and `--branch`/`-b` for the branch it merges into (default `main`). `proposals <slug>` lists them and filters with `--status open|applied|dismissed`.

## Dispatch hosted runs

A **run config** is a saved recipe — models, tasks, parallelism — that the platform executes on its own runners. Dispatching one from the CLI sends the same request as the **Run** button.

| Command | What it does |
| - | - |
| `bench run-configs [action]` | `list` (default), `create` or `delete` a run config. |
| `bench dispatch <run_config_id>` | Start a hosted run. `--container-id` when the project hosts several; `--version-id` pins a `READY` version instead of the head. |
| `bench run <run_id>` | One run's status, score and per-task rewards. `--wait` polls to terminal, bounded by `--timeout-min` (default 45). |
| `bench validate [path]` | Push this checkout, run the named run config against it, and gate — the pull-request entry point. |
| `bench secrets <action>` | `set`, `list` or `unset` the environment variables every hosted run receives. Write-only. |

```bash theme={null}
gateway bench run-configs create --name nightly --models openai/gpt-5-mini --parallelism 4
gateway bench dispatch <runConfigId>
gateway bench run <runId> --wait
```

`dispatch` returns as soon as the platform accepts the runs. `bench run <runId> --wait` polls one to terminal.

`run-configs create` takes `--name`, `--models` (comma-separated), `--parallelism` (1–16), `--tasks` (bundled task names) or `--world-tasks` (stored task refs as `<id>` or `<id>@<version>`, for api worlds and not alongside `--tasks`), `--agent` for a harbor agent, and `--auto-run-on-push` to dispatch on every pushed `READY` version. `delete` takes `--id`.

`bench validate` combines the three CI steps: `--run-config <name>` names the config to dispatch, `--min-score` fails any run below the bar, `--summary-file` writes a markdown summary for a pull-request comment, and `--slug`, `--branch` and `--timeout-min` control the rest. For a world whose agent runs locally, use [`gateway worlds ci`](/cli/worlds#test-a-world-and-gate-a-pull-request) instead.

```bash theme={null}
gateway bench secrets set --env acme-world --key VENDOR_TOKEN   # value from stdin
gateway bench secrets list --env acme-world
```

Secrets are named in `UPPER_SNAKE_CASE` and scoped to one world with `--env`. Omit `--value` on `set` and the value is read from standard input, which keeps it out of shell history.

## Read the tests a world ships

`bench tests <slug>` lists the runs of a world's own test suite, newest first — those triggered on push, from the CLI and from the API alike.

```bash theme={null}
gateway bench tests acme-world --limit 5
gateway bench tests acme-world --run <runId>
```

`--run <id>` prints one run's full report instead of the list. Authoring and running the suite is [`gateway worlds test`](/cli/worlds#test-a-world-and-gate-a-pull-request).

## Commands that need the Python package

`bench eval` (local verifiers and harbor engines), `bench install` (pip-installs a version's wheel), `bench harvest-harbor` (harvests a local harbor jobs directory) and `bench push --build-wheel` execute Python locally and belong to the Python package — see the [Python SDK](/sdk) overview. For a hosted run, use `bench dispatch`.

## Where to go next

* [gateway worlds](/cli/worlds) — authoring, data, sessions and tests.
* [gateway benchmarks](/cli/benchmarks) — a world with tasks, run k times per model.
* [The gateway CLI](/cli) — credentials, the local runtime, output and exit codes.
* [Worlds hub](/sdk/benchmark-hub) — the same push and version surface from Python.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.