> ## Documentation Index
> Fetch the complete documentation index at: https://docs.surfacearea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Keep real data out of a shared world

> Declare a [redact] policy so a world built from real vendor data can be shared. The four modes, the rules that decide which one a field falls under, the salt, and how to redact rows on the way in.

A world built from a real vendor holds real records. A `[redact]` policy in `connector.toml`
names the fields that must not travel, and decides what the world stores in their place. The
policy is one block, and every surface that writes rows reads the same one.

Redaction is optional. A world with no `[redact]` block stores every field as given, and every write path (`data import`, batches, `session seed`, a task's seed) defaults to `--redact off` even when a policy exists; you opt in per write with `apply` or `refuse`. Each mode costs the
world something — a search route, a join, a format — so pick per field and record the reason.

## The four modes

```toml filename="connector.toml" theme={null}
[connector]
slug = "spycloud"
base_url = "https://api.spycloud.io"

[redact]
hash     = ["email", "full_name", "password"]
drop     = ["ssn", "results.*.raw_record"]
preserve = ["account_number", "routing_number", "phone"]
round    = { latitude = 2, longitude = 2 }
```

| Mode | What the store holds instead | What it costs |
| - | - | - |
| `hash` | `"sha256:<64 hex>"`, salted with the connector's slug | Prefix, substring and fuzzy search on the field stop working |
| `drop` | Nothing. The key is removed from the row | The field answers no question at all |
| `preserve` | A format-preserving digest: same length, digits stay digits, letters stay letters of the same case | Nothing is readable, but the value still looks like an identifier |
| `round` | The number rounded to the decimals you keep | Precision, and only precision |

`hash` is the safe default. The digest is deterministic, so an exact-match lookup still
resolves and a join between two entities still lines up.

`preserve` exists because `hash` breaks format-constrained fields. A 15-digit account number
becomes a 71-character `sha256:` string, and the field's own length and format bounds then
refuse the row. A shape-preserving digest stays inside the contract. Use it for identifiers —
account and routing numbers, IBANs, phone numbers — and leave free text on `hash`, since a
non-ASCII character is passed through unchanged.

`round` keeps the numeric type and the row's shape and drops only the precision. Coordinates
are the usual case: `round = { latitude = 2 }` turns `37.774929` into `37.77`.

`drop` is the only mode that answers nothing. A lookup on a dropped field misses rather than
errors, because the field was never written.

<Info>
  Both `hash` and `preserve` walk into lists and redact element by element, so an array stays
  an array and the world's contract still accepts the row. Objects are left alone — name their
  fields instead.
</Info>

## Which fields a mode applies to

**A bare name matches at any depth.** `hash = ["email"]` hashes every `email` key anywhere in
the row, however deeply nested.

**A dotted path matches that place only**, with `*` standing for a list item. `drop = ["results.*.raw_record"]`
drops `raw_record` from each item of the top-level `results` list and leaves a `raw_record`
elsewhere in the row untouched. Use a path when a bare name would catch a key you need — a
count map whose keys happen to include `email`, for instance.

**There is no entity scope.** `hash = ["contacts.phone"]` is not "the `phone` of `contacts`"; it is a
`phone` key under a `contacts` object inside a row, which no row has. A bare `phone` applies to every
entity's `phone`. `gateway worlds schema check` refuses a dotted path that starts with an entity name,
and — in a world without `connector.toml`, where the policy only ever sees rows — one whose first
segment is no entity's top-level field. A bare name is never refused: it may be a vendor key the model
drops before the contract sees the row, or sit inside an open nested object. `gateway worlds describe`
shows which declared fields a policy actually lands on; a name that tags nothing there redacts nothing.

**`round` takes bare field names only**, mapped to the decimals kept, `0` through `10`.

**A field falls under exactly one mode.** Naming the same field in two lists is refused when
the connector is parsed, with `A field is hashed, dropped, rounded or preserved, never two of
those`. When a name and a path could both match a value, the modes rank `drop`, then `hash`,
then `preserve`, then `round`.

## The salt is the connector's slug

A hashed value is `sha256` of the string `"<connector slug>:<value>"`, written as
`sha256:<64 hex>`. The slug is `[connector] slug` from `connector.toml`, **not** the world's
platform slug. The two often differ: the `spycloud-world` world hashes under `spycloud`.

The plaintext is hashed exactly as given: no lowercasing, no trimming, no other salt. Applying
the same rule anywhere reproduces the digest the world stored.

<Info>
  Computing a digest by hand with the platform slug produces a value that never matches a
  stored row. Read the salt from the world itself: `gateway worlds describe <slug>` prints it,
  and `storedForms` in [the TypeScript SDK](/sdk-ts/describe) reads it from `world.redact.slug`.
</Info>

Each world salts with its own slug, so the same email is a different digest in two worlds and
nothing can be joined on disk. A [link between worlds](/worlds/links) recomputes both forms
from the plaintext you supply.

## Where the policy runs on its own

Two paths redact without being asked:

* **`gateway worlds connector capture`** redacts each response body before it is written to
  `captures/`, so a capture on disk is already redacted.
* **`gateway worlds data extract`** redacts the rows it shapes, unless you pass `--no-redact`.
  Its stderr summary counts what it did: `{"read": N, "rows": M, "unknownFields": {...}, "redacted": {"dropped": n, "hashed": n}}`.

Every other write path stores rows exactly as given, by default. `data import`, a data batch
and a session seed all assume the caller may already hold the rows redacted — an export of
another world, or a hashed extract — because hashing a digest twice would break every join.

<Info>
  Captures hold real records even after redaction, because a field the policy does not name is
  stored in the clear. Keep `captures/` out of anything you share.
</Info>

## Redact on the way in

Every write path takes a `redact` mode, and every one of them defaults to `off`.

| Mode | What happens to a row |
| - | - |
| `off` (default) | The row lands as given, byte for byte |
| `apply` | The row goes through the world's `[redact]` policy before its contract sees it — hash, drop, preserve and round, exactly as a capture would have stored it |
| `refuse` | A row carrying plaintext in a redacted field is refused, naming the entity, index, field and policy. Nothing is rewritten |

```bash theme={null}
# Rows from a real run, redacted here rather than by hand.
gateway worlds data import vendor-world rows.json --redact apply

# Check an already-redacted export before importing it.
gateway worlds data check vendor-world rows.json --redact refuse

# One live session only.
gateway worlds session seed <sessionId> rows.json --redact apply
```

A task can carry the mode so every session the task opens seeds the same way:

```json theme={null}
{
  "seed": {
    "mode": "append",
    "redact": "apply",
    "rows": { "people": [{ "id": "P1", "email": "jo@acme.test" }] }
  }
}
```

The same field reaches the other surfaces under the same name:

| Surface | Where `redact` goes |
| - | - |
| REST | `redact` in the body of `POST /api/public/worlds/{slug}/data/import` and `POST /api/public/worlds/{slug}/data/batches` |
| MCP | `redact` on `import_world_data` and on `seed_world_session` |
| TypeScript SDK | `session.seed(rows, { redact })`, and `redact` on a task's `seed` block |

### What `refuse` checks

`refuse` verifies what is verifiable from the stored value alone:

* a **hashed** field must hold a `sha256:<64 hex>` digest, or a list of them;
* a **dropped** field must be absent;
* a **rounded** number must carry no more decimals than the policy keeps.

A **preserved** value is indistinguishable from a real identifier by design, so `refuse` lets
it through. Use `apply` for a world with `preserve` fields.

<Info>
  `apply` and `refuse` both need the world's policy, which lives in `connector.toml`. A world
  without one is refused up front rather than silently storing plaintext:
  `redact=apply needs the world's [redact] policy, and this world has no connector.toml`.
</Info>

A refusal names the row and what was wrong with it:

```
people row carries plaintext in 'email' under [redact] hash: expected a sha256:<hex> digest;
import with redact=apply to redact it here, or store the redacted form
```

## Read a redacted world back

Three surfaces report how a field is stored and what a plaintext value becomes:

* **`gateway worlds describe <slug>`** stamps every field with its policy — `[hash]`,
  `[preserve]`, `[round:n]`, `[drop]` — alongside entities, keys and row counts.
* **`gateway worlds data query <world> <entity> --where email=jo@acme.test --resolve`** turns
  each value you pass into the form the store holds and runs the query on that. See
  [Put data in a world](/worlds/data#ask-for-the-rows-you-mean).
* **`describeWorld` and `storedForms`** do the same programmatically. See
  [Describe a world](/sdk-ts/describe).

Plan queries from `describe` before you write them. A field marked `hash` or `preserve`
answers a plaintext lookup through resolution; a field in the clear answers it as given; a
field marked `drop` answers nothing.

## Choosing, in practice

For each field the policy touches, run the exact lookup (it must still resolve) and the search
route that filters on it (it will not), then record which trade you took.

Two options when a search route breaks: keep the field in plaintext because it is not
sensitive, or drop the route from the world and write it into the limits. See
[What makes a world good](/worlds/good-world#redact-in-a-way-that-keeps-the-shape).

## Where to go next

* [Put data in a world](/worlds/data) for the rows contract and the import path.
* [The files a world is made of](/worlds/contract) for every other `connector.toml` key.
* [Mock any vendor API](/worlds/custom-connections) for captures, handlers and conformance.
* [Describe a world and look things up by plaintext](/sdk-ts/describe) for the programmatic form.
* [Link worlds together](/worlds/links) for resolution across worlds with different salts.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.