# Overview (/docs)
Omnigraph is a lakehouse native graph engine built for context assembly. It
acts as the operational state and coordination layer for agents, with
git-style versioning and branching, multimodal retrieval, and object storage
native deployment on the [Lance](https://github.com/lance-format/lance) format.
## Quick install [#quick-install]
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
```
Or with Homebrew:
```bash
brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph
```
New here? Follow the [install guide](/docs/getting-started/install), or clone a
[starter cookbook](https://github.com/ModernRelay/omnigraph-cookbooks) to get a
working graph with seed data in one command.
## Explore the docs [#explore-the-docs]
# CLI guide (/docs/cli)
The `omnigraph` CLI can work directly with a graph store, through a running
server, or with a cluster definition. Start with the form that matches the
resource you have:
```bash
# One graph, opened directly
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' --store ./graph.omni
# One graph on a multi-graph server
omnigraph query sources_for_claim --params '{"claim":"lower-latency"}' \
--server prod --graph knowledge
# A reusable scope from ~/.omnigraph/config.yaml
omnigraph query sources_for_claim --params '{"claim":"lower-latency"}' \
--profile prod-knowledge
```
Run `omnigraph --help` for the flags supported by your installed
version. The [CLI reference](/docs/cli/reference) summarizes addressing, commands,
configuration, and output formats.
For a managed cluster, select it with `use`. The CLI acquires identity credentials
when needed. Run `graphs list` to discover graph names, then `query` or `mutate`
from that folder with an explicit `--graph`. Applied Cedar policy determines your
permissions. See [managed data access](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-data.md) for expiry, offline
operation and credential clearing.
Organization administrators can [invite, enroll and remove
teammates](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-team.md) with `organization --managed`, using the same cached
login. Cluster admission and control grants remain separate from Cedar graph
permissions.
## Create, load, and query a graph [#create-load-and-query-a-graph]
```bash
omnigraph init --schema schema.pg ./graph.omni
omnigraph load --data evidence.jsonl --mode overwrite ./graph.omni
omnigraph query sources_for_claim \
--query queries.gq \
--params '{"claim":"lower-latency"}' \
--format table \
--store ./graph.omni
```
`load` always requires a mode:
* `overwrite` replaces each node or edge type represented in the batch. Types
absent from the batch are unchanged.
* `append` inserts new entities and rejects duplicate IDs.
* `merge` inserts new IDs and updates existing IDs.
Use `mutate` for `.gq` insert, update, and delete queries:
```bash
omnigraph mutate add_source \
--query mutations.gq \
--params '{"slug":"incident-review","title":"Incident review"}' \
--store ./graph.omni
```
For a stored server query, omit `--query`; the positional name selects the
query from the server's registry:
```bash
omnigraph query sources_for_claim --server prod --graph knowledge \
--params '{"claim":"lower-latency"}'
```
## Work with branches [#work-with-branches]
```bash
omnigraph branch create review/new-data --from main --store ./graph.omni
omnigraph load --data batch.jsonl --mode merge \
--branch review/new-data ./graph.omni
omnigraph query inspect --query review.gq \
--branch review/new-data --store ./graph.omni
omnigraph branch merge review/new-data --into main --store ./graph.omni
```
Each branch operation is also a GQ statement, run through the same verbs that
run any `.gq` source: the control writes through `mutate`, the listing through
`query`. A statement names its branches itself, so it takes no `--branch`:
```bash
omnigraph mutate -e 'branch create "review/new-data" from main' --store ./graph.omni
omnigraph query -e 'branch list' --format table --store ./graph.omni
omnigraph mutate -e 'branch merge "review/new-data" into main' --store ./graph.omni
omnigraph mutate -e 'branch delete "review/new-data"' --store ./graph.omni
```
See [Branches and commits](/docs/branching) for isolation, history, and
merge behavior.
## Session settings [#session-settings]
`--set NAME=VALUE`, repeatable, gives a [session
setting](/docs/queries-search/queries#session-settings) a value for one invocation; the
value is spelled as after `=` in a `set` line (`--set merge_lineage=off`). A `set`
line in the source applies after `--set`, so the file wins.
* `query --set`: applies to the read, `show` included.
* `mutate --set`: applies to the mutation or branch statement.
* `branch merge --set`: applies to the merge (`--set merge_lineage=verify`).
* `commit changes --set`: sent as `set=NAME=VALUE` on the commit's change
read. The values are validated and select nothing in this release; a direct
run (`--store`) accepts them with no effect.
* `changes poll --set`: sent as `set=NAME=VALUE` on the change feed read, with
the same validation and the same absence of effect.
* `load --set`: a direct run (`--store`) applies them to the
staged write (`--set stage_write_concurrency=16`); a served run refuses any
`--set` before sending, since the served load routes carry no
`settings` field.
```bash
omnigraph query reports --query queries.gq --set merge_lineage=off --store ./graph.omni
omnigraph branch merge review/new-data --into main --set merge_lineage=verify --store ./graph.omni
```
A direct run (`--store`) is the process, so it accepts a `process` setting
from `--set`, the file and the environment. A served run (`--server`) sends
`--set` values in the request's `settings` field, and a `--set` or `set` line
naming a `process` setting is refused before anything is sent: `setting
`` is a process setting; it is read from the server's environment, not
from a request`.
A stored query takes no `--set`: `omnigraph query ` without `-e` or
`--query` runs under the process defaults.
## Read and write Blob values [#read-and-write-blob-values]
Read a managed Blob cell to a file, inspect its metadata, or replace it:
```bash
omnigraph blob get node Document doc-42 body \
--store ./graph.omni --out body.bin
omnigraph blob stat node Document doc-42 body \
--store ./graph.omni --json
omnigraph blob put node Document doc-42 body \
--store ./graph.omni --file body.bin
```
`blob get` writes bytes to stdout when `--out` is omitted. Add `--offset` and
`--length` for a range. The CLI reports external references but does not follow
them. `blob put` reads stdin when `--file` is omitted, and `blob clear` sets a
nullable cell to null; both accept `--if-match`. See [Blob values](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md)
for the complete contract.
## Inspect and maintain a graph [#inspect-and-maintain-a-graph]
```bash
omnigraph snapshot ./graph.omni --json
omnigraph commit list ./graph.omni --json
omnigraph schema show ./graph.omni
omnigraph optimize ./graph.omni
omnigraph repair ./graph.omni # preview only
omnigraph cleanup --keep 10 --older-than 7d ./graph.omni
omnigraph cleanup --keep 10 --older-than 7d --confirm ./graph.omni
```
Maintenance commands open storage directly; they do not run through
`--server`. For a cluster-managed graph, address it with
`--cluster --graph `. Read the
[maintenance guide](/docs/operations/maintenance) before repair or cleanup.
## Use a server [#use-a-server]
Declare the server once, store its token, then address graphs by ID:
```yaml
# ~/.omnigraph/config.yaml
servers:
prod:
url: https://graph.example.com
```
```bash
printf '%s' "$OMNIGRAPH_TOKEN" | omnigraph login prod
omnigraph graphs list --server prod
omnigraph query sources_for_claim --params '{"claim":"lower-latency"}' \
--server prod --graph knowledge
```
The server resolves the actor from its token; clients cannot override it with `--as`.
`login` writes the token to `~/.omnigraph/credentials` with mode `0600`.
`OMNIGRAPH_TOKEN_`, the server name uppercased with `-` written as `_`,
supplies it instead and outranks the stored one: `OMNIGRAPH_TOKEN_PROD` here.
Two server names that spell the same variable, such as `a-b` and `a_b`, are
refused when the config loads.
The CLI sends the `prod` token only to URLs under its `url`: the same scheme,
host and port, and a path that starts with the `url`'s path once `.` and `..`
segments are resolved. A `unix://` server's token goes only to that socket.
Any other request carries no credential, including a literal
`--server https://…` URL that matches no declared server. If such a request
gets HTTP 401, the CLI names the server to declare or the `omnigraph login`
command to run; `--json` output carries this as `missing_credential` beside the
server's own error. A `--graph` value must be one path segment of ASCII letters,
digits, `-` or `_`; any other value is refused before a request is sent.
Each invocation captures its operator home and ambient profile, then reads
configuration once when needed. Editing it does not change the destination,
actor defaults or presentation settings; unknown keys warn once. Ordinary graph
clients retain the token selected during setup across registry discovery and
execution, even if the credentials file changes. Managed session renewal follows
its existing authentication flow.
Named-server `login` without `--token` reads the first line of stdin. Credential
writes by `login` and `logout` use the invocation's captured operator home.
## Output and errors [#output-and-errors]
For `query` and `alias`, output selection follows `--json`, then `--format`,
then the alias's `format`, then operator `defaults.output`, then `table`.
The selected JSON or JSONL format also applies to structured execution errors:
JSON writes an error object to stdout; JSONL writes one error line. Explicit
`--format table` selects human output and diagnostics even when a default
requests JSON.
Before valid operator configuration and, for an alias, its selected binding have
been captured, error formatting uses only explicit command-line flags.
`mutate` still requires explicit `--json` for structured errors. For reads,
unstructured transport failures and malformed error bodies remain diagnostics
on stderr.
Output delivery failures return nonzero without undoing completed work. A failed
login/logout report may follow a completed credential change; skill export or
embedding generation may have already written its files. The CLI does not repeat
those operations to recover output. `skill show` keeps its existing exception:
a reader closing the pipe early is accepted. Help and version also retain their
normal exit status when their output pipe closes.
A final flush failure does not append another JSON document after a returned
result. A human data-write refusal delivered on stderr keeps its documented exit
code even when stdout is unavailable. If a managed command has accepted a
deployment or lifecycle operation but cannot report it on stderr, it attempts
one recovery-only JSON record on stdout with the accepted identity and any
retained idempotency key. This still exits nonzero and does not resubmit work.
See [output formats](/docs/cli/reference#query-inputs-and-output)
and [diagnostics](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/diagnostics.md).
## Use a server through a Unix socket [#use-a-server-through-a-unix-socket]
A server may also be reached through a Unix-domain socket, for example a local
gateway in front of `omnigraph-server`. A sandboxed agent that is denied network
access may still be allowed to connect to one socket path. Address the socket
with a `unix://` URL followed by its absolute path:
```bash
omnigraph query sources_for_claim --params '{"claim":"lower-latency"}' \
--server unix:///run/omnigraph/server.sock --graph knowledge
```
```yaml
# ~/.omnigraph/config.yaml
servers:
local:
url: unix:///run/omnigraph/server.sock
```
The whole path names the socket, so a socket URL has no host, path prefix,
query or fragment. A path may contain spaces, written as is or as `%20`. The
CLI sends the same HTTP/1.1 requests over the socket as over TCP: the
`HEAD /healthz` discovery, the contract header, the bearer token, and no
redirects or automatic retries. Proxy settings do not apply. `unix://`
addresses work on Unix platforms only.
A `--server` value or server `url:` must use `http://`, `https://` or
`unix://`. Any other scheme fails as an invalid server address; use `--store`
for graph storage.
## Manage a cluster [#manage-a-cluster]
Self-hosted `cluster` commands read a directory containing `cluster.yaml`, or address
a running server explicitly. They ignore managed folder context:
```bash
omnigraph cluster validate --config ./company-brain
omnigraph cluster plan --config ./company-brain --as act-alice
omnigraph cluster apply --config ./company-brain --as act-alice
```
They manage graph definitions, schemas, stored queries, policies, embedding
providers and external Blob rules. Plan is always an observation and never takes
the writer lock. Removing a graph declaration deletes its managed storage and
history. Apply refuses unmanaged existing roots, missing graphs that remain
declared, and schema drift; it has no lifecycle or repair override. See
[deployment boundaries](/docs/clusters#deployment-boundaries).
For a managed cluster, log in to its Intent API and select the cluster for
your config directory:
```bash
omnigraph login --api https://control.example
omnigraph use CLUSTER_ID --api https://control.example --config ./company-brain
omnigraph cluster plan --managed --config ./company-brain --json > plan.json
omnigraph cluster apply --managed --config ./company-brain --plan "$(jq -r .data.preview_id plan.json)" --json
omnigraph cluster status --managed --config ./company-brain --json
omnigraph cluster history --managed --config ./company-brain --json
omnigraph logout --api https://control.example
```
Commit and push external configuration before planning. The API plans its
bound head, or the full pushed commit ID selected with `--rev`. Apply uses that
exact preview and your current permissions. It waits for the native result,
active runtime and archived evidence. `--no-wait` returns the reserved delivery
ID; follow it with `cluster status --managed DEPLOYMENT_ID --wait`. Cancellation
is available only before that delivery's first dispatch attempt.
The explicit `--managed` flag uses the folder's `.omnigraph/context` to select
the API and cluster. Without it, cluster commands ignore that context.
Missing or malformed context and unavailable APIs refuse without
falling back to local deployment. See [Managed cluster commands](/docs/cli/reference#managed-cluster-commands)
for credential storage, automation, bounded waits, and exit codes.
## Validate source before running it [#validate-source-before-running-it]
```bash
omnigraph lint --schema schema.pg --query queries.gq
omnigraph queries validate --cluster ./company-brain --graph knowledge
```
`lint` checks one `.gq` source. `queries validate` checks the applied stored
query registry for a cluster graph.
## Give an agent the matching skill [#give-an-agent-the-matching-skill]
The binary carries the `omnigraph` agent skill written for its own commands and
HTTP contract. Export it where your agent loads skills:
```bash
omnigraph skill export ~/.claude/skills/omnigraph --force --json
omnigraph skill show references/queries.md
```
`skill export` creates the directory and refuses to replace existing files
unless `--force`. It records the files it wrote in `.omnigraph-skill.json`; a
later `--force` export removes a recorded file this skill no longer has only
while its bytes are unchanged, and touches no other file. `--json` reports the
`files` written, `skill_version`, `cli_version` and `http_contract`.
`skill show` prints `SKILL.md`, or the named skill file, to stdout. Neither
command reads configuration or contacts a server.
# CLI reference (/docs/cli/reference)
This page maps the CLI; the installed binary is the exact reference:
```bash
omnigraph --help
omnigraph --help
omnigraph --help
```
## Addressing a graph [#addressing-a-graph]
Most graph commands accept one of these scopes:
| Scope | Use |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| positional URI | Direct access for commands whose positional slot is not used by another value |
| `--store ` | Direct access to one `file://`, `s3://`, or `az://` graph; the exact token `og_catalog` (also as `--graph og_catalog` on a server) is the [engine catalogue](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/catalogue.md), serving `query` and `schema show` with no store opened |
| `--server --graph ` | A graph served by a multi-graph server at an `http://`, `https://` or [`unix://`](/docs/cli#use-a-server-through-a-unix-socket) URL |
| `--cluster --graph ` | Direct maintenance of a cluster-managed graph |
| `--profile ` | A named scope from operator config |
A bare local path is accepted where a graph URI is expected. `--server` and
`--store` are mutually exclusive. A store already identifies one graph, so it
cannot be combined with `--graph`.
Common global flags:
| Flag | Meaning |
| -------------- | ------------------------------------------------------------------- |
| `--as ` | Actor for direct writes and cluster operations |
| `--yes` | Non-interactive consent for destructive writes to non-local storage |
| `--quiet` | Suppress the resolved write target printed to stderr |
A served write refuses `--as` ("`--as` is not allowed on a served write"): the
server resolves the actor from the bearer token. Drop it, or use `--store `.
## Commands [#commands]
| Command | Purpose | Scope |
| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
| `init` | Create an empty graph from a `.pg` schema | direct |
| `query` | Run a read query, or a branch read (`branch list`, `diff`, `merge-base`, `merge … dry run`), `show`, or `explain` statement | direct or served |
| `mutate` | Run an insert/update/delete query (`as commit` leaves it unpublished; `as unattached branch [on ""]` records a snapshot that never publishes), or a `branch create`, `branch delete`, `branch merge`, `commit publish`, `commit drop`, or [`load csv`](/docs/queries-search/mutations#csv-loading) statement | direct or served |
| `load` | Load graph JSONL in `overwrite`, `append`, or `merge` mode | direct or served |
| `blob get/stat/put/clear` | Read, inspect, replace or clear one Blob cell | direct or served |
| `branch create/list/delete/merge/diff/merge-base` | Manage graph branches; `merge --as-commit` leaves the merge unpublished, `merge --dry-run` reports the outcome or conflicts and writes nothing, `diff` reviews what a branch changed since its merge base, `merge-base` prints that base | direct or served |
| `snapshot` | Show a branch snapshot: `internal_schema_version`, `graph_manifest_version`, and the datasets of one captured graph version | direct or served |
| `commit list/show/changes/publish/drop` | Inspect history (`list --limit`/`--page-token` for one page of it, `list --unpublished` for a branch's unpublished commits, `list --unattached` for its live unattached snapshots; the two flags exclude each other and paging) or one commit's entity changes; publish an unpublished commit onto its recorded target branch (an unattached snapshot refuses with `unattached_cannot_publish`), or drop it (an unattached snapshot takes every descendant with it); `publish` and `drop` take `--as` on a direct graph, the reads refuse it | direct or served |
| `diff` | Net entity changes between two points: branches or commit ids (`--from`, `--to`) | direct or served |
| `changes poll/baseline` | Consume a branch change feed or establish a new baseline | direct or served |
| `export` | Stream a branch as JSONL | direct or served |
| `schema show` | Read the accepted schema | direct or served |
| `schema apply` | Apply a schema to a standalone graph | direct |
| `schema plan` | Preview a schema migration | direct |
| `schema upgrade-system-columns` | Respell a graph's system columns in place; needs a graph that normal open accepts and keeps its storage format | direct |
| `lint` | Validate `.gq` source | local schema or direct graph |
| `upgrade` | Convert a v8, v9 or v13 graph's storage to the format this binary serves, offline; `--check` writes nothing | direct standalone |
| `optimize` | Compact data and reconcile declared indexes | direct |
| `rebuild-full-text-indexes` | Replace full-text indexes on one branch | direct |
| `repair` | Report each table's Lance history against its registration (`no_drift` or `foreign_drift`) | direct |
| `cleanup` | Delete table versions that no retained graph commit pins, under an explicit retention policy ([Maintenance](/docs/operations/maintenance#cleanup)) | direct |
| `graphs list` | List graph metadata or minimal identity discovery | served |
| `queries list/validate` | Inspect or validate a cluster query registry | cluster |
| `cluster validate/plan/apply/...` | Operate self-hosted declarative state; [deployment/recovery flags](/docs/clusters) | config, server, explicit root |
| `cluster --managed` | Operate a managed service cluster | managed API and folder context |
| `policy validate/test/explain` | Validate or evaluate applied policy | cluster |
| `embed` | Generate, clean, or refresh seed embeddings | local tooling |
| `login`, `logout` | Manage a named server credential or a managed API session | local or managed API |
| `use` | Select a managed cluster for a config directory | managed API |
| `profile list/show` | Inspect operator profiles | local |
| `alias` | Invoke a personal stored-query alias | served |
| `skill export/show` | Export the agent skill built into this binary, or print one of its files ([matching skill](/docs/cli#give-an-agent-the-matching-skill)) | local |
| `version` | Print the package version, HTTP contract and storage formats; `--json` prints `version`, `http_contract` and `internal_schema` (`writes`, `serves`) ([Verify](/docs/getting-started/install#verify)) | local |
`rebuild-full-text-indexes` accepts `--branch` (default `main`), `--json`, and
`--as` for actor attribution. Direct maintenance does not load server policy;
see the [rebuild procedure](/docs/operations/maintenance#rebuild-full-text-indexes).
## Query inputs and output [#query-inputs-and-output]
For ad-hoc source, pass `--query ` or `-e/--query-string `; with
multiple declarations the positional name selects one. A stored server query is
its registry name alone. Parameters come inline, `--params '{"name":"Ada"}'`,
or from a file, `--params-file params.json`. `--set NAME=VALUE`, repeatable,
gives a [session setting](/docs/cli#session-settings) a value for the run of
`query`, `mutate`, `branch merge`, `commit changes`, `changes poll`, and `load`.
The source may instead be one branch statement, `mutate -e
'branch create b0'` or `query -e 'branch list'` (writes through `mutate`, the
listing through `query`; no `--branch`, `--snapshot`, `--if-commit`, name, or
params; see [Work with branches](/docs/cli#work-with-branches)), or one
`explain` statement, `query -e 'explain query q() { … }'`, answering the plan
as rows under the query's own target and params; see [Explain](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/explain.md).
Read output supports `table`, `json`, `jsonl`, `csv`, and `kv`. `--json` is the
stable machine-readable form for commands that do not use `--format`. Result
cells use the [JSON result spelling](/docs/queries-search/queries#json-result-spelling);
`table`, `csv`, and `kv` print strings unquoted. `--format json` prints the
envelope pretty and the `rows` array compact; a refusal follows [Diagnostics](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/diagnostics.md).
### Machine-readable read and write positions [#machine-readable-read-and-write-positions]
`query --json` returns `graph_commit_id` when its read snapshot has a graph
head. The id and rows share one pinned snapshot; use that id for a later
conditional mutation.
Successful `mutate --json` and `load --json` responses include `commit`, the exact commit published by
that attempt. It contains `graph_commit_id`, optional `graph_branch`,
`graph_manifest_version`, optional parent and merged-parent ids, optional
`actor_id`, and `created_at` in Unix microseconds. A successful mutation
that changes no entities returns `"commit": null`.
`--json` and read `--format json` preserve structured errors on stdout. Data-write failures
report [whole-command outcomes and exits](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/troubleshooting.md#failed-data-write-commands).
Verified HTTP conditional mismatches exit 4; embedded mismatches exit 1 because writable open can complete earlier work.
### Conditional mutations [#conditional-mutations]
```bash
omnigraph query find_person --query queries.gq --store graph.omni --json
omnigraph mutate update_person --query queries.gq --store graph.omni \
--if-commit --json
```
`--if-commit` runs the mutation only while the target branch is still at that
commit. Any intervening commit on the branch invalidates the condition, even
when it changed unrelated data. A mismatch has no effect and exits with code
4 against a server, 1 on an embedded `--store` run; JSON output includes `precondition_failure` with `expected` and optional
`actual` commit ids. Re-read and decide again instead of retrying blindly.
## Storage upgrade [#storage-upgrade]
```bash
omnigraph upgrade ./graph.omni --check --json
omnigraph upgrade ./graph.omni --json
omnigraph schema upgrade-system-columns ./graph.omni --check --json
```
`--store` is an alternative to the positional storage URI. `omnigraph upgrade`
converts a standalone v8, v9 (release 0.11.x) or v13 graph to v16, offline and
in place, keeping branches and commit history, and stamps a v14 or v15 graph
to v16 with one metadata-only commit per live branch; stop every process using
the graph and retain a verified whole-root backup first. `--check` writes
nothing; `--to-format` accepts 16 only. `check_passed`, `already_current` (a
v16 graph) and `completed` exit 0; `check_failed` (`unsupported_source` for any
other format below 14, `newer_than_binary` above 16, `unsupported_target`)
and `recovery_required` (a pending attempt, finished as `recovery.action` says,
or leftover recovery files) exit 1. The report names the formats, the route,
the `work` counts, findings and the recovery action. Server and cluster
addressing are refused; see [storage upgrade](/docs/operations/upgrade#storage-upgrade).
`schema upgrade-system-columns` is a separate operation on a served graph: [system-column upgrade](/docs/operations/upgrade#system-column-upgrade-legacy-spellings).
## Load modes [#load-modes]
`load --mode` is required:
| Mode | Existing entities | Typical use |
| ----------- | ------------------------------------------------------------------------------- | ------------------------------------------- |
| `overwrite` | Each node or edge type represented in the batch is replaced; other types remain | Initial load or import of a complete export |
| `append` | Kept; duplicate IDs fail | Strict batch insertion |
| `merge` | Updated by ID | Idempotent synchronization |
`--branch ` selects an existing branch. Add `--from ` to create a
missing branch from an explicit base. Overwrite is destructive and may require
`--yes` for non-local storage.
In a selected managed folder, implicit `load --graph ` uses the separate
cached data credential. It requires `change`, plus `branch_create` when
`--from` is present. Managed loads bound input to 32 MiB, responses to 8 MiB,
and one request to 300 seconds; uncertain writes are never automatically
retried. See [managed bulk loading](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-data.md#bulk-loading) for limits,
permissions, ordinary addressing, and reconciliation.
Diffs, branch review and change feeds: [Changes, Diffs and Change Feeds](/docs/branching/changes).
## Blob commands [#blob-commands]
```text
omnigraph blob get|stat|put|clear [scope] [options]
```
`get` accepts `--branch` or `--snapshot`, `--offset`, `--length`, and
`--out `. `stat` accepts `--branch` or `--snapshot` and `--json`. `put`
reads `--file ` or stdin; `put` and `clear` accept `--branch`,
`--if-match `, `--json` and, on `--store`, `--as`. See [Blob values](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md).
## Operator configuration [#operator-configuration]
The default path is `~/.omnigraph/config.yaml`. Set `OMNIGRAPH_HOME` to use a different directory.
```yaml
operator:
actor: act-alice
defaults:
output: table
server: prod
default_graph: knowledge
servers:
prod:
url: https://graph.example.com
clusters:
company:
root: s3://company-data/omnigraph
profiles:
prod-knowledge: {server: prod, default_graph: knowledge}
company-admin: {cluster: company, default_graph: knowledge}
local-dev: {store: file:///tmp/dev.omni}
aliases:
experts:
server: prod
graph: knowledge
query: find_experts
args: [topic]
params:
limit: 20
format: table
```
Each profile binds exactly one of `server`, `cluster`, or `store`. Select it with
`--profile` or `OMNIGRAPH_PROFILE`. Explicit flags override values filled by a profile.
Bearer tokens never belong in `config.yaml`. Bind one to a declared server with
`omnigraph login ` or `OMNIGRAPH_TOKEN_`; the CLI sends it only to
URLs under that server's `url` (see [server credentials](/docs/cli#use-a-server)).
## Managed cluster commands [#managed-cluster-commands]
Add `--managed` to select the managed service explicitly. Without it, `cluster`
uses self-hosted configuration or an explicit server/root and ignores managed
folder context. The flag can appear before or after the subcommand.
| Session command | Behavior |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `omnigraph login --api ORIGIN [--organization ORG_ID]` | Reuse cached access or print a WorkOS device verification URL/code; credentials stay in the OS keychain, JSON contains identity/expiry only |
| `omnigraph logout --api ORIGIN [--organization ORG_ID]` | Clear only the selected organization's control credentials and request provider revocation; `provider_revocation_confirmed` reports its outcome; accepted work continues |
| `omnigraph use CLUSTER_ID --api ORIGIN [--organization ORG_ID] [--config DIR] [--json]` | Verify cluster access and atomically bind the selected directory |
The secret-free `DIR/.omnigraph/context` is:
```yaml
version: 2
cluster: CLUSTER_ID
api: https://control.example
organization_id: org_customer
```
Without a folder or `--organization`, exactly one cached organization is required.
Fresh login selects its organization using only its new session; failure or
uncertainty preserves cached sessions and requires fresh login. OS-keychain
consent is outside the 35-second selection and verification network budget.
Fully explicit `login`/`logout` select independently of the folder; ordinary
commands refuse conflicts. To replace an old or invalid regular context, sign in
with `--api` and `--organization`, then run fully explicit `use`. Context lookup
uses only `--config` (default `.`); links, special files and files over 16 KiB refuse.
See the [authentication contract](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-09-09-identity-credentials-and-applied-policy.md#provider-native-access-and-standard-clients)
for origin validation, separate organization sessions, identity pins and renewal bounds.
| Managed command | Behavior |
| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `cluster list --managed --api ORIGIN [--organization ORG]` | List clusters visible to the selected organization |
| `cluster members list --managed` | List [explicitly enrolled people](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-lifecycle.md#list-cluster-members); requires the bound cluster's `status` permission |
| `cluster pull --managed --out NEW_DIR [--rev REVISION]` | Download the bound cluster's exact configuration and context into a new directory |
| `cluster plan --managed [--rev REVISION]` | Prepare a preview of the full pushed commit ID, or resolve the bound head when omitted |
| `cluster apply --managed --plan PREVIEW_ID` | Deliver that exact preview under current permissions |
| `cluster status --managed [DEPLOYMENT_ID] [--wait]` | Read labeled cluster observations, or an exact delivery; `--wait` requires its ID |
| `cluster history --managed [--limit N] [--since RFC3339]` | Read newest deliveries first, default and maximum 100; filter creation time inclusively before the limit |
| `cluster cancel --managed DEPLOYMENT_ID` | Cancel your queued delivery before its first dispatch attempt |
See [managed lifecycle](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-lifecycle.md) for cluster operations and [team administration](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-team.md) for `organization --managed` (`org`), invitation commands and versioned member changes.
All accept `--config DIR` and `--json`. Plan and apply also accept
`--idempotency-key KEY`, `--no-wait`, and `--timeout SECONDS`. An omitted key is
generated and printed before submission. Retain it and the exact request after
uncertainty: replaying the same key/body retrieves the same reservation;
changed input refuses. The CLI never repeats a submission automatically.
Once an ID is known, observe it. Cancellation is idempotent, but refuses after
possible dispatch and cannot stop native effects.
Plan and apply read pushed source, not uncommitted local edits; use
`cluster push --managed` to upload. Previews hold no change lease. Source edits
grant no deployment permission: applied policy authorizes native effects.
A stale generation or changed policy refuses without substituting a preview.
Plan returns its synchronous preview. Replaying its key returns metadata,
without redisclosing the saved native plan. Apply and exact-ID status waiting
poll every two seconds for up to 300 seconds. `--timeout` accepts 1–3600
seconds. Reaching the deadline stops only the local wait; continue with
`cluster status --managed DEPLOYMENT_ID --wait`. Apply `--no-wait` prints the
durable delivery reservation and both product/native IDs, then exits 0;
it does not establish native acceptance or activation. Every HTTP request has a
10-second deadline and an 8 MiB response limit; configuration download allows
120 seconds and 64 MiB encoded (8 MiB of file contents). Redirects are refused.
`--json` prints one API envelope to stdout with its native fields and separate
source, delivery, archive and runtime observations intact. Progress and idempotency keys use
stderr; refusals use a JSON problem object with a `type` field.
| Managed deployment result | Exit code |
| -------------------------------------------------------------------------------------------- | --------- |
| Prepared preview, valid observation, confirmed cancellation, or explicit no-wait reservation | 0 |
| Waited apply/status: fresh converged native result, active runtime and archived evidence | 0 |
| Native failure/nonconverged result, transport or protocol error | 1 |
| Explicit refusal | 2 |
| Local wait deadline with delivery, observation, activation or archive unresolved | 5 |
Snapshot reads exit 0 without implying deployment success. Historical results
cannot establish current activation. Unresolved preview replay exits 5 with its
ID. There is no plan abandonment or inferred retry safety/exit 75 from HTTP.
Cloud lifecycle commands retain their [separate outcomes](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-lifecycle.md#recover-an-uncertain-response).
Applied policy authorizes execution. `--as`, `--server`, `--profile`, `--graph`,
`--store`, and global `--cluster` do not select managed cluster operations.
For unattended execution, supply an accepted service bearer and its API origin:
```bash
export OMNIGRAPH_CONTROL_API=https://control.example
# Supply OMNIGRAPH_CONTROL_TOKEN through your CI secret mechanism.
omnigraph cluster apply --managed --plan PREVIEW_ID --idempotency-key DEPLOYMENT_KEY --json
```
The canonical `OMNIGRAPH_CONTROL_API` must match the selected context. A
missing or mismatched pair refuses before any request. These credentials are
separate from [server credentials](/docs/cli#use-a-server) and operator profiles.
`--managed` requires context except `create`, `list` and explicit-origin operation
lookup; `--direct` is rejected. Managed API failures never trigger local deployment.
Service-only `create`, `list`, `pull`, `push`, `delete`, `undo-delete`, `token`, `operation`, `history` and `cancel` require `--managed`. Local `validate`,
`observe`, `force-unlock`, and `upgrade-ledger` reject it. Managed `plan` takes
`--rev` and managed `apply` requires `--plan`; self-hosted deployment flags cannot
be combined with them.
Use `cluster operation --managed OPERATION_ID [--wait]` for service lifecycle observation;
self-hosted `cluster status --deployment-id ID` addresses a native receipt.
## Managed data access [#managed-data-access]
After login and cluster selection, use `graphs list` to discover graphs, then
`query`, `mutate`, `load`, or commit reads with `--graph` from the managed folder.
Missing or expired identity credentials are acquired before the operation;
applied Cedar policy decides permissions. See [managed data access](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-data.md)
for offline behavior, identity binding, discovery and credential clearing.
## Confirmation rules [#confirmation-rules]
`cleanup` changes nothing until `--confirm` is present. Destructive operations
against non-local storage also require interactive confirmation or `--yes`; in
non-interactive and JSON modes they fail closed. The same non-local consent
rule applies to overwrite loads and branch deletion, verb or statement.
# Changes, Diffs and Change Feeds (/docs/branching/changes)
OmniGraph can describe what one commit changed, compare any two points in
history, review what a branch changed, or deliver an ordered feed of changes
on a branch. Every surface reports logical nodes and edges, not storage
details, in one change shape.
HTTP calls follow the [HTTP contract](/docs/operations/server#http-contract),
including the required request header and response check before consuming a
JSON or streamed body.
## Inspect one commit [#inspect-one-commit]
```bash
omnigraph commit changes --store graph.omni --json
```
The equivalent HTTP route is
`GET /graphs/{graph_id}/commits/{commit_id}/changes`. The commit is compared
with its first parent. Each response includes:
* `cause`: the commit id, parent, optional merged parent, authored branch,
optional actor, and authorship time in Unix microseconds.
* `changes`: inserts with `after`, updates with `before` and `after`,
and deletes with `before`. Each image contains `properties`; an edge image
also contains `endpoints: {from, to}`. A Blob property holds its export
value (`base64:…` or an external URI), or, for a stored reference to a byte
range of an object, `{"uri", "offset", "length"}` (see
[Blobs](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md)).
Filter with repeatable `--kind node|edge`, `--type `, and
`--op insert|update|delete` options. A page holds at most 8,192 changes; the
HTTP routes default to 1,000. Results are deterministic by entity kind, type
identity, logical id, and operation.
Large results are paginated. The CLI normally follows every
`next_page_token` and asks for 8,192 changes per page, because every page
restarts its comparison and fewer, larger pages cost less; passing
`--page-token` fetches exactly one page, of 1,000 changes unless `--limit`
sets the size. A page token continues that one commit result and is not a
change-feed cursor.
A parentless commit returns `409` with reason `parentless_commit`. A diff
that crosses a schema boundary returns `409` with reason
`schema_boundary`; take a new baseline instead of treating either response
as an empty change.
## Compare two points [#compare-two-points]
```bash
omnigraph diff --from --to main --store graph.omni --json
```
Each side is a branch name, meaning its head when the diff starts, or a commit
id: a published commit, an unpublished proposal, or an unattached snapshot. A
string that names both a branch and a commit is refused; name the branch's
head by its commit id instead. The equivalent HTTP route is
`GET /graphs/{graph_id}/diff?from=&to=`, and the GQ statement
`branch diff to ` answers the same comparison at `POST /query`,
a commit id quoted (`branch diff "" to main`).
The result is the net change from the `from` state to the `to` state, in the
`commit changes` change shape and order, with `from_commit_id` and
`to_commit_id` naming the exact commits the sides resolved to. There is no
`cause`: a range of commits has no single author. Intermediate history is not
replayed, so an update that was later reverted produces nothing, and a delete
followed by an equal reinsert produces nothing. A diff from a commit's first
parent to the commit equals that commit's `commit changes`, without the cause.
Comparing two branch heads also shows what the target changed since the
branches diverged, reversed. To review a branch, use the branch form below.
The kind, type and operation filters, `--limit` and `--page-token` work as for
`commit changes`. A page token keeps the commits the first page resolved, so
later pages answer for them even after the branches move. Each continuation is
checked against the request's sides, on the server and in the embedded CLI
alike. A side spelled as a commit id reaches only that commit. A branch side
reaches its head now, any of its earlier heads within its last 1,024 commits,
and any commit the caller may read as a side of its own. A token naming any
other commit is refused with `400`; start the diff again without it. A diff
across a
schema change returns `409` with reason `schema_boundary`, and a side whose
history was reclaimed returns `410` with `change_feed_gap`.
Cost grows with the size of each changed type, not with the number of
changes: apart from a single commit, every changed type is compared in full.
Types unchanged between the two points are skipped.
## Review a branch [#review-a-branch]
```bash
omnigraph branch diff feature --into main --store graph.omni --json
omnigraph branch merge-base feature --into main --store graph.omni --json
```
`branch diff` reports what `feature` changed since its merge base with `main`
(`main` when `--into` is omitted): the net change from the merge base to the
head of `feature`. Changes `main` made after the merge base do not appear. The
response also names the base as `base_commit_id`. `branch merge-base` prints
that commit alone, the one a merge of `feature` into `main` would classify
against. While `feature` has never merged `main` in, the base is on its
first-parent history, and `changes poll --branch feature --start
after:` replays its commits since then; a merge commit in that
range still carries the changes it brought in. A base off `feature`'s
first-parent history is refused as a start: for example, after `main` is
merged into a `feature` that had diverged from it, the base is the `main`
commit that merge brought in. `branch diff` gives the net change from any
base.
The HTTP routes are `GET /graphs/{graph_id}/branches/diff?source=&target=` and
`GET /graphs/{graph_id}/branches/merge-base?source=&target=`; the GQ statements
are `branch diff feature into main` and `branch merge-base feature into main`.
A statement answers rows through `POST /query` or `omnigraph query -e`. A
`branch diff` row has `kind`, `type` (the type's name), `id`, `op`, `before`
and `after`; the compared commits and the token for the next page travel in
the answer's `page` object, and `after page ""` (then `limit `)
continues it. A `branch merge-base` row names `source`, `target`, both heads
and `merge_base_commit_id`. Statements take no kind, type or operation filter
yet; use the routes or CLI commands for those. To see whether
the merge would succeed, run it as a [dry run](/docs/branching/merge#preview-a-merge).
## Follow a branch [#follow-a-branch]
```bash
# Start at the current head. Existing history is not replayed.
omnigraph changes poll --start now --store graph.omni --json
# Replay the branch's first-parent history.
omnigraph changes poll --start beginning --store graph.omni --json
# Continue after a known commit, or resume from a saved cursor.
omnigraph changes poll --start after: --store graph.omni --json
omnigraph changes poll --cursor --store graph.omni --json
```
`now` is the default when neither `--start` nor `--cursor` is supplied.
Use `--branch` to follow a branch other than `main`. The same kind, type,
operation filters, and `--limit` supported by `commit changes` are available
here.
Each poll captures a fixed branch head and returns complete commits in
first-parent order. A commit with no logical entity changes may still appear
as an empty block and advance the feed.
The durable `cursor` appears only on the terminal page, after every returned
commit block is complete. A page ending partway through a commit has only a
`next_page_token`. On the terminal page, `caught_up` says whether the poll
reached its captured head. The CLI consumes feed page tokens itself and prints
the cursor only after reaching that terminal page.
Delivery is at least once: retrying a cursor may replay the next complete
commit. Apply each block idempotently by `graph_commit_id`, then persist the
terminal cursor atomically with the applied blocks.
Cursors are opaque and caller-owned; the server stores no consumer position.
A cursor is bound to its graph, branch lifetime, and filter scope. Reusing it
with a different scope, or using a page token as a cursor, returns `400`.
The HTTP route is `GET /graphs/{graph_id}/changes`. Its `cursor`, `start`,
and `page_token` parameters are mutually exclusive.
## Recover from a retention gap [#recover-from-a-retention-gap]
A cursor does not prevent `cleanup` from reclaiming old history. When a feed,
commit diff or diff can no longer be read, the server returns `410` with
`change_feed_gap`, including `first_unreadable_commit_id` and, when
available, the rejected `cursor`. Retrying the same cursor cannot close the
gap; install a new baseline:
```bash
omnigraph changes baseline --out snapshot.jsonl --store graph.omni --json
```
If a poll has already accumulated complete readable blocks before it reaches a
gap, it first returns those blocks with `caught_up: false`; polling the
returned cursor then produces the deterministic `410`.
The HTTP equivalent is `POST /graphs/{graph_id}/changes/baseline`. Send `{}`
for the default `main` scope or include branch and filter fields. The
`application/x-ndjson` response streams an exact entity snapshot followed by
one terminal record:
```json
{"baseline":{"snapshot_commit_id":"...","resume_cursor":"..."}}
```
Snapshot records use the export line format, and a Blob property holds the
same value shapes as a change image. That includes the
`{"uri", "offset", "length"}` object for a stored reference to a byte range of
an object, which export refuses; a snapshot holding one does not reload with
`load` (see [Blobs](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md)).
An interrupted stream has no terminal record and therefore no usable cursor.
Install the complete snapshot durably before saving `resume_cursor`; the
resumed feed begins with commits after the captured snapshot.
Baseline kind and type filters select the snapshot contents. An operation
filter applies only to the resumed feed. Baselines require the `export`
policy action; commit changes, diffs and feed polling require `read`. A diff
needs `read` on the branch each side resolves to; a side the actor cannot read
answers `404` exactly as a side that names nothing, whatever the other side
names. A commit whose branch was deleted answers the same `404`.
`branch diff` and `branch merge-base` need `read` on both branches, every
page included, and answer `403` otherwise. On a later page, a commit the
token names that its side no longer resolves to needs `read` on the branch
that wrote it, unless it was an earlier head of the side's branch, such as a
fork's first head that `main` wrote. An unknown or unreadable one answers the
same `400`.
The CLI's durable `--out` installation is available on POSIX platforms. The
file contains only snapshot entity records, not the terminal handshake. The CLI
syncs and atomically replaces it before printing
`{"snapshot_commit_id":"...","resume_cursor":"..."}` to JSON stdout. Concurrent
captures to the same path stay serialized until the cursor output is flushed.
A cursor-output failure returns an error; the installed snapshot remains in place.
# Branches, Commits, and History (/docs/branching)
A branch is an isolated, durable graph history. Use branches to prepare and
review a multi-step change without exposing intermediate results on `main`.
## Branch workflow [#branch-workflow]
```bash
# Create an isolated branch from main.
omnigraph branch create review/add-benchmark --from main --store graph.omni
# Write and inspect it.
omnigraph load --data benchmark.jsonl --mode append \
--branch review/add-benchmark graph.omni
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' \
--branch review/add-benchmark --store graph.omni
# Publish its result to main, then remove the source branch.
omnigraph branch merge review/add-benchmark --into main --delete-branch \
--store graph.omni
```
List and delete branches with:
```bash
omnigraph branch list --store graph.omni
omnigraph branch delete review/abandoned --store graph.omni
```
Creating a branch defaults to `main` when `--from` is omitted. A load can create
a missing target branch by combining `--branch ` with `--from `.
Each branch operation is also a GQ statement, so a client that already sends
`.gq` source needs no second transport. The three control writes go to
`POST /mutate`, and the listing and the branch reads to `POST /query`, each
with no request target, name, or parameters:
```text
branch create "review/add-benchmark" from main
branch merge "review/add-benchmark" into main
branch delete "review/abandoned"
branch list
branch diff "review/add-benchmark" into main
branch merge-base "review/add-benchmark" into main
branch merge "review/add-benchmark" into main dry run
```
`from` defaults to `main` and `into` defaults to `main`. A name outside the
identifier alphabet (a lowercase letter or `_`, then letters, digits, or `_`),
such as one containing `/`, `-`, or `.`, is quoted, as above; `main`, `b0`,
and `_x` are bare. A control write answers a
`ChangeOutput` whose `outcome.kind` is `created`, `deleted`, or `merged`
(`outcome.merge` holds the merge result), with zero affected counts; a merge
that publishes a commit reports it in `commit`. `branch list` answers one row
per branch in column `name`, sorted by name. Each statement runs under the same
policy check as its HTTP route (`POST /branches`, `DELETE /branches/{branch}`,
`POST /branches/merge`, `GET /branches`), and `branch list` is a read, so it is
refused on `POST /mutate` as a control write is refused on `POST /query`. The
branch reads `branch diff`, `branch merge-base` and `branch merge … dry run`
answer rows as `branch list` does and take their routes' checks; see
[reviewing a branch](/docs/branching/changes#review-a-branch) and
[previewing a merge](/docs/branching/merge#preview-a-merge). A bare `dry run` after the
names is the clause, so a branch named `dry` is quoted.
Branches are cheap until written: unchanged data remains shared with the source.
A branch remains after a normal merge, so prefer `--delete-branch` or delete it
when review is complete. Live branches retain the history they depend on and can
prevent `omnigraph cleanup` from reclaiming old data.
## Branch names [#branch-names]
Branch names may contain `/`, but live names must not be path prefixes of one
another. For example, `review` and `review/alice` cannot coexist. `main` is
reserved. A parent branch can be deleted while descendants remain. A path segment
may not end in `.` followed by 26 upper-case letters and digits: that shape is
reserved for OmniGraph's internal per-life branch identity. Ordinary dotted
names such as `release.1.2` are fine.
A branch name is used exactly as written. A name that is empty, has leading or
trailing whitespace, or contains a control character is refused wherever a
branch is named: the branch a read or write targets, `from`, merge sources and
targets, branch create and delete, Blob writes, change feeds and branch
statements. The server answers `400 Bad Request` before authorization, the
embedded engine returns a bad-request error before any effect, and the CLI
reports either error. Names are never trimmed, so `main ` does not name
`main`.
Deleting a branch and creating another with the same name yields a fresh
branch lifetime. Readers that captured the deleted branch fail with a typed
error rather than seeing the replacement. Deletion retires the logical name
and leaves native manifest history and table storage for explicit `omnigraph cleanup`; it starts no background table-reclaim work. Cleanup preserves data still needed by
other branches and their underlying histories.
Branch-control operations are safe across handles in one writer process. Do not
run branch create/delete control concurrently from separate writer processes
against the same graph.
## Atomicity model [#atomicity-model]
OmniGraph does not provide connection-scoped `BEGIN` and `ROLLBACK`.
| Scope | Guarantee |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| One mutation query | All statements publish as one commit or none become visible. |
| One load request | The complete batch publishes as one commit or none becomes visible. |
| Several commands on a branch | Each command that publishes a change is a durable branch commit. Earlier commands are not rolled back if a later one fails. |
| Branch merge | The resulting source state becomes visible on the target in one atomic commit. |
Deleting an abandoned branch discards that workspace from normal access, but it
is an explicit lifecycle action rather than transaction rollback.
## Commits [#commits]
Every write that publishes a change records a graph commit. A successful
mutation that matches no entities publishes no commit. A commit includes its
id, parent, branch, actor when known, and timestamp. Merge commits have two
parents.
```bash
omnigraph commit list graph.omni --branch main
omnigraph commit show --uri graph.omni
```
`commit list` is newest first. Omitting `--branch` shows history reachable from
`main`; selecting a branch includes the history inherited at its fork plus its
own commits.
A long history can be read a page at a time:
```bash
omnigraph commit list graph.omni --branch main --limit 50 --json
omnigraph commit list graph.omni --branch main --page-token --json
```
`--limit` (1 to 1,000) returns one page with `head_commit_id`, the head the
listing resolved, and `next_page_token` while older commits remain. Pass the
token back with the same `--branch` for the next page, of 100 commits unless
`--limit` sets the size. Later pages answer for that head even after the
branch moves, so a page never repeats or skips a commit. A page reads only the
history it returns; without `--limit` or `--page-token` the whole history is
read and listed at once. Served under a policy, a caller whose `read` does
not cover every branch continues only from a commit of the branch's history,
and is refused with `400` otherwise: a commit the branch still holds, or one a
branch the caller may read published, costs nothing more, and any other is
looked for down the branch's history, which costs what the whole listing
costs. Served, the same listing is
`GET /commits?branch=&limit=` continued with `page_token=`.
Paging applies to a branch's history, not to `--unpublished` or
`--unattached`.
### Unpublished commits [#unpublished-commits]
A mutation or a merge written `as commit` records its result as an unpublished
commit: the branch head is unchanged, the result is readable at the commit's
id (`--snapshot `), and the commit lists with `published: false`.
```graphql
query propose_bob() as commit {
insert Person { name: "bob", age: 41 }
}
branch merge feature into main as commit
commit publish ""
commit drop ""
```
`commit publish` installs the commit on the branch it recorded as its target.
The published commit keeps the actor and timestamp of the write that created
it; the publish records no publisher. A published commit is answered as it
stands by a repeat publish and is never dropped. An unpublished commit expires
once an ordinary write moves its target branch's head past the commit's
parent, or when that branch is deleted or recreated, and can then only be
dropped: a publish is refused (a moved head fails its precondition), reads at
its id (`--snapshot`, `commit show`, `commit changes`) answer `not found`, and
it leaves the `--unpublished` listing.
```bash
omnigraph branch merge feature --into main --as-commit --uri graph.omni
omnigraph commit list graph.omni --branch main --unpublished
omnigraph commit publish --uri graph.omni
omnigraph commit drop --uri graph.omni
```
Served, the same control writes are `POST /commits/{commit_id}/publish`
(the published commit), `DELETE /commits/{commit_id}` (204), and
`GET /commits?unpublished=true&branch=`; `POST /branches/merge` takes
`as_commit`, which excludes `delete_branch`. Publishing and dropping run under
the `commit_publish` and `commit_drop` policy actions on the recorded target
branch ([Policy](/docs/operations/policy)).
### Unattached branches [#unattached-branches]
A mutation written `as unattached branch` records its result as an unattached
snapshot: the branch head is unchanged, the result is readable at the
snapshot's id (`--snapshot `), and the snapshot never publishes. A second
write `on ""` (or `on $base` with a `String` parameter) builds on that
snapshot and returns a new id; the caller keeps the latest id as its tip. The
snapshot's branch is the one the first write ran on (`main` when the request
names none); a write `on` a snapshot runs on that recorded branch, and a
request that names another branch is refused. Two writes on one base each get
their own snapshot and see nothing of each other's.
```graphql
query add_person($name: String) as unattached branch {
insert Person { name: $name }
}
query add_address($id: String, $base: String) as unattached branch on $base {
insert Address { owner: $id, city: "Delft" }
}
```
A write that changes nothing returns `commit: null` and creates no snapshot;
the caller keeps its previous tip. The receipt's `commit` carries
`kind: "unattached"`, `published: false` and `root_commit_id`, the first
snapshot of the chain (its own id on that first snapshot); `parent_commit_id`
is the base snapshot, or the branch head the first write ran on. A base must
be a live unattached snapshot: a published commit or an unpublished commit is
refused.
```bash
omnigraph commit list graph.omni --branch main --unattached
omnigraph query people --query queries.gq --snapshot --store graph.omni
omnigraph commit drop --uri graph.omni
```
`commit list --unattached` shows the branch's live snapshots, newest first,
flat; `parent_commit_id` tells the chains apart. `commit drop` of a snapshot
also drops every snapshot built on it; dropping the first one discards the
whole chain, and a snapshot built on another chain is untouched. `commit
publish` of an unattached snapshot is refused with HTTP 409 and the error
code `unattached_cannot_publish`; nothing is written. Every snapshot of a
chain expires when an ordinary write moves the branch head past the head the
first write captured, or when the branch is deleted or recreated: expired
snapshots answer `not found` to reads and writes and leave the listing. Dropping or expiring
a snapshot promises no storage reclamation; `cleanup` reports what it retains.
Served, the listing is `GET /commits?unattached=true&branch=`
(`unattached` and `unpublished` together are refused) and the drop is the same
`DELETE /commits/{commit_id}` (204). Writing needs `change` on the recorded
branch; listing and reading a snapshot need `read` on it
([Policy](/docs/operations/policy)).
## Historical reads [#historical-reads]
Every read targets either a live branch or an immutable graph commit. Take a
`graph_commit_id` from `commit list --json` and pass it as the snapshot:
```bash
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' \
--snapshot --store graph.omni
```
A query stays on one snapshot for its entire lifetime. Wildcard traversal is
refused on an explicit historical target because historical schema membership
is not reconstructed. Explicit edge alternatives retain named-edge historical
read rules.
A historical query is written against the current schema. A type renamed since
the commit still reads under its current name, but a property reads only where
the commit stores that same property under its current name. A property added
since the commit, renamed since, or dropped and re-added with the same name is
refused at that commit, naming the property, rather than answering with null or
with the dropped property's values. A whole-node `return { $p }` reads every
current property, so it is refused at commits made before a property was added.
Table data written before v0.10 records no property identity, so a commit that
reads such data refuses its properties too, unless that data is still the
table's current version.
Historical reads can eventually fail after destructive cleanup removes the
versions that commit needs. Branch deletion can likewise end access to branch-only history.
## Changes and feeds [#changes-and-feeds]
Inspect the logical changes made by one commit, compare two points, or review
what a branch changed since it left `main`:
```bash
omnigraph commit changes --store graph.omni --json
omnigraph diff --from --to main --store graph.omni --json
omnigraph branch diff feature --store graph.omni --json
```
For continuous consumption, `omnigraph changes poll` follows complete commits
on one branch and returns an opaque resume cursor. If cleanup has reclaimed
required history, create a coherent snapshot and new cursor with
`omnigraph changes baseline`.
See [Changes, Diffs and Change Feeds](/docs/branching/changes) for filters, pagination,
cursor checkpointing, and `410` retention-gap recovery.
See [Merging Branches](/docs/branching/merge) for merge outcomes and conflict handling.
# Merging Branches (/docs/branching/merge)
A merge integrates one branch into another and publishes the target change
atomically across the whole graph.
```bash
omnigraph branch merge review/2026-04-25 --into main \
--store s3://bucket/graph.omni
```
The source is positional. `--into` defaults to `main`.
The same merge as a GQ statement, sent to `POST /mutate` with no request
target, name, or parameters:
```text
branch merge "review/2026-04-25" into main
```
`into` defaults to `main`; a name with `/`, `-`, or `.` is quoted, a bare
identifier such as `main` is not. The answer is a `ChangeOutput` with
`outcome.kind = "merged"` and `outcome.merge` one of `already_up_to_date`,
`fast_forward`, or `merged`; after a `fast_forward` or `merged` result,
`commit` identifies that merge's own publication, even if another writer has
since advanced the target. The statement has no `--delete-branch`
composition: follow it with `branch delete `. A merge statement takes
no commit precondition -- `POST /mutate/if-graph-commit` and `--if-commit`
refuse one beside it -- and no front offers a conditional merge today.
A merge preserves changes already integrated into a branch when you later
merge that branch back. For example, after merging a new edge from `main`
into `review`, editing only a node on `review` and merging it into `main`
preserves that edge. A merge whose source and target tables reached the same
version count on different branches fast-forwards; earlier releases refused
it with a `table version … already exists` error.
## Outcomes [#outcomes]
* **Already up to date**: the target already contains the source changes.
* **Fast-forward**: the target still has the merge base's schema and table
versions, so it takes the source's state. The check reads state, not
history: a target merged again from a branch it fast-forwarded before, with
no write in between, fast-forwards again. One exception: when a
fast-forward into `main` carried a table whose changes cancel out on the
branch, or a table on a fork left by a branch created before storage format
v11, `main` ends on its own version of that table, so a later merge from
that branch can report **Merged**. The merged data is the same.
* **Merged**: both branches changed, so OmniGraph performs a three-way,
entity-level merge and creates a commit with two parents.
`branch merge --json`, the HTTP merge route and the GQ merge statement all
return `commit` with the exact commit id, manifest version, parents, branch,
actor and creation timestamp (Unix microseconds). Fast-forward returns the new
target commit; its merged parent identifies the source head. An already-up-to-date
merge publishes nothing and returns `commit: null`.
If a remote response is lost or incomplete, the merge may have committed. The
CLI does not resubmit it automatically. Inspect the intended target changes and
relevant history before deciding what to do next; a later head alone cannot
identify the missing merge receipt.
When only the source changed a table since the merge base, a merge into a
named branch can adopt that exact table snapshot without copying its rows.
This also works when the target already has its own table history. Tables
changed on both sides still use the normal three-way merge and constraint
checks.
The target can continue writing after adoption. Other live branches keep their
current table snapshots through those target writes and cleanup;
creating a replacement branch is unnecessary. Cleanup retains the underlying
histories they still need. Avoiding a table copy does not eliminate the reads
needed to validate a merge.
The source branch is unchanged by the merge. Use `--delete-branch` for the
normal review-branch lifecycle:
```bash
omnigraph branch merge review/2026-04-25 --into main --delete-branch \
--store s3://bucket/graph.omni
```
Deletion happens after a successful merge and has its own authorization check.
If deletion is denied or the source still has descendants, the merge remains
successful and durable. The CLI exits 0 and warns in human output; HTTP and CLI
JSON preserve `commit`, report `branch_deleted: false` and include the structured
`branch_delete_error_details` (`error`, plus typed `code` and details where available).
Successful deletion reports `branch_deleted: true`; without `--delete-branch`,
both deletion fields are absent. v0.12 removes the old `branch_delete_error`
string. Inspect and retry deletion separately; do not replay a successful merge.
Deleting a branch is irreversible and may make branch-only commits unavailable.
It also releases old history for a later cleanup.
## Preview a merge [#preview-a-merge]
```bash
omnigraph branch merge feature --into main --dry-run --store graph.omni --json
```
A dry run runs the merge through classification and constraint validation,
then stops. It reports the outcome the merge would have now, or fails with the
same structured conflict list, and writes nothing: `commit` is `null` and
`dry_run` is `true`. It is a read of both branches, so it needs `read` on each
and no `branch_merge` grant. It refuses `--as-commit` and `--delete-branch`.
Over HTTP, send `dry_run: true` to `POST /graphs/{graph_id}/branches/merge`.
The GQ statement `branch merge feature into main dry run` is the same read at
`POST /query`: it answers one row with `source`, `target` and `outcome`, and a
conflict is the 409 a merge answers.
A dry run answers whether the merge would succeed. To see what it would change,
use [`branch diff`](/docs/branching/changes#review-a-branch). To review the exact result
before it lands, merge `--as-commit` and inspect the proposal with
`commit changes`; the proposal expires once the target branch moves.
## Conflicts [#conflicts]
When both sides changed the same logical data incompatibly, the merge returns a
structured conflict list and publishes nothing.
| Kind | Meaning |
| ---------------------------- | ------------------------------------------------------------------- |
| `divergent_insert` | Both branches inserted the same id. |
| `divergent_update` | Both branches updated the same entity differently. |
| `delete_vs_update` | One branch deleted an entity the other updated. |
| `orphan_edge` | The result would contain an edge whose endpoint was deleted. |
| `unique_violation` | The result would violate uniqueness. |
| `cardinality_violation` | The result would violate edge cardinality. |
| `value_constraint_violation` | The result would violate an enum, range, or other value constraint. |
A merge classifies both sides against their merge base, the newest commit
both branches descend from, counting a merged branch as an ancestor: a
commit both branches descend from that no other such commit descends from.
When two such commits are each not an ancestor of the other, the base is the
one nearest the two heads (the fewest commits between it and the heads,
summed), then the one with the higher generation (its distance from the
first commit along the longest path), then the one with the smaller commit
id. Two branches that
each merged the same third branch therefore share that branch's commit as
their base, and an entity only one of them changed after that import merges
cleanly. The record of such a commit lives in the branch
it was merged from, and the merge can read it from a live branch or retained
retired history. Cleanup preserves the exact selected base and the retired
histories needed to resolve it. Recreating the logical branch name cannot
substitute the replacement branch's same-numbered snapshot.
Each conflict identifies the affected type and, when applicable, entity id. The
HTTP server returns conflicts with status `409`, the same answer for
`POST /branches/merge` and for a `branch merge` statement on `POST /mutate`.
Reconcile the data on one or both branches, then run the merge again.
### Edges inserted on both sides [#edges-inserted-on-both-sides]
Whether the same edge added on both branches converges depends on the edge
type's declared identity:
* No declaration: edge ids are generated, so each side's insert is its own
row and the merge keeps both. This is the documented multiset default;
parallel edges are legitimate data.
* `@unique(@src, @dst)`: each branch's write succeeds on its own, and the
merge reports `unique_violation`. Releases before edge keys provide this
guard as `@unique(src, dst)`; from 0.11 it is spelled
`@unique(@src, @dst)`.
* `@key(@src, @dst)`: both sides derive the same id, so identical inserts
converge to one row with no conflict. If the sides disagree on a non-key
property, the merge reports `divergent_insert` on the edge type with the
derived id as the entity id (for `@key(@src, @dst)` a JSON array such as
`["Alice","Bob"]`; the elements are the endpoint node ids, so they are
generated ids when the endpoint type declares no key). Reconcile it like any divergent insert: align the
property on one branch (insert the same key again with the agreed
values; the insert upserts), then merge again.
## Merge classification mode [#merge-classification-mode]
The `merge_lineage` [session setting](/docs/queries-search/queries#session-settings)
(`request` scope; `set merge_lineage = off;` before the statement, `--set
merge_lineage=off`, or the request's `settings` field; process default
`OMNIGRAPH_MERGE_LINEAGE`) selects how a branch merge finds what changed. `on`
(the release default) discovers candidates from Lance version metadata —
fragment lists and deletion files — and compares candidate rows. Known deleted
row positions use bounded direct reads. Candidate filtering may still scan data
when an applicable index is absent; a fail-closed
precondition gate falls back to the full three-way scan whenever any
assumption cannot be proven (Blob-bearing schema, differing schemas or storage
paths across the pins, version pins not matching the manifest entries, missing
stable identifiers or the exact-id primary-key contract, or a candidate set
past its byte budget). The table version a side pins need not descend from
the one the merge base pins, so merging a branch back after an earlier merge
still qualifies. `off` forces the full three-way scan everywhere — the
operational fallback if merge results are ever in question.
`verify` runs both, compares their decisions entity by entity, publishes the
scan's result, and fails the merge loudly on any divergence (the debug-build
default, used for validation; it costs both paths). A merge that succeeds
produces the same result in every mode; only cost differs. An unrecognized
value is refused where it arrives: in the environment it refuses startup, in
a request or a file it refuses that request.
Later writes create fresh table storage when needed and leave unused former
storage for explicit cleanup. They do not reclaim a previous table history as
part of preparing the write or merge.
## After a large merge [#after-a-large-merge]
Indexes do not define merge correctness. Newly merged entities remain queryable even
when index coverage has not caught up, but some searches may scan them. Run
`omnigraph optimize` after a large merge to restore efficient layout and index
coverage.
A merge that writes rows is also bounded by the Blob payload it copies; see
[Blob limits](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md#limits).
See [Branches, Commits, and History](/docs/branching) for the complete branch workflow.
# Cluster configuration reference (/docs/clusters/config)
Cluster commands read a directory containing `cluster.yaml`:
```bash
omnigraph cluster validate --config ./company-brain
```
`--config` defaults to the current directory. Unknown fields and duplicate YAML
keys are errors so that misspelled intent is never ignored.
## Complete shape [#complete-shape]
```yaml
version: 1
metadata:
name: company-brain
# Omit for a cluster stored in this directory.
storage: s3://company-data/omnigraph/company-brain
state:
backend: cluster
lock: true
providers:
embedding:
default:
kind: openai-compatible
base_url: https://api.example.com/v1
model: text-embedding-3-large
api_key: ${EMBEDDING_API_KEY}
graphs:
knowledge:
schema: knowledge.pg
queries: queries/
embedding_provider: default
external_blobs:
allow:
- base: s3://company-assets/knowledge/
scope: server_safe
policies:
graph-access:
file: graph.policy.yaml
applies_to: [knowledge]
server-access:
file: server.policy.yaml
applies_to: [cluster]
```
## Top-level fields [#top-level-fields]
| Field | Required | Meaning |
| --------------------- | -------: | -------------------------------------------------------------------------------------------------- |
| `version` | yes | Configuration schema; currently `1` |
| `metadata.name` | no | Display name |
| `storage` | no | Cluster root; direct default is the config directory, remote apply defaults to the selected server |
| `state.backend` | no | Omit or set to `cluster` |
| `state.lock` | no | Exclusive writer admission; omit or set to `true`; `false` refuses execution |
| `providers.embedding` | no | Named embedding provider profiles |
| `graphs` | no | Graph declarations keyed by graph ID |
| `policies` | no | Policy bundles keyed by bundle name |
Credentials are process configuration and must not appear in
`cluster.yaml`.
## Graphs [#graphs]
Each graph requires a schema file:
```yaml
graphs:
knowledge:
schema: knowledge.pg
```
Optional fields:
| Field | Meaning |
| -------------------- | ---------------------------------------------------------- |
| `queries` | Stored-query files, directories, or explicit name mappings |
| `embedding_provider` | Name under `providers.embedding` |
| `external_blobs` | Allow-list for new external Blob references |
Query declarations support three forms:
```yaml
# Every declaration in top-level *.gq files in a directory
queries: queries/
# Every declaration in these files or directories
queries: [people.gq, reports/]
# Explicit registry names
queries:
find_experts:
file: knowledge.gq
```
Unreadable files, parse errors, duplicate query names, and queries that do not
type-check against the graph's desired schema fail validation.
## Embedding providers [#embedding-providers]
Provider `kind` may be `openai-compatible`, `openai`, `gemini`, or `mock`.
Real providers require `api_key: ${ENVIRONMENT_VARIABLE}`; inline secrets are
rejected. The serving process resolves the environment variable at boot and
during live deployment preflight for affected graphs. `cluster validate`,
`plan`, and direct apply do not resolve it. Vector dimensions remain part of
the graph schema.
See [Embeddings](/docs/queries-search/embeddings) for provider behavior.
## External Blob references [#external-blob-references]
New external references are denied unless their normalized URI falls under an
allowed base:
```yaml
external_blobs:
allow:
- base: s3://company-assets/knowledge/
scope: server_safe
```
`server_safe` permits the base for a served graph. `embedded_only` is for an
embedded host and may permit a local `file://` directory; it is not installed
by the HTTP server or direct-store CLI. Bases must be absolute, non-overlapping,
and free of credentials, query strings, fragments, and path traversal.
A base must also name storage outside the cluster's storage root: the config
directory when `storage` is omitted, or the `storage` URI. That root holds every
graph and the applied state, and ingress reads with the process's own storage
credentials, so a base over it would let any writer copy another graph's data
or the cluster ledger into a readable Blob value. `cluster validate`, `plan`,
and `apply` refuse such a base with `external_blob_base_overlaps_storage_root`,
whatever its scope. Put external objects under a sibling prefix instead, for
example `s3://company-assets/cluster-external/` beside
`storage: s3://company-assets/cluster`. A server that finds an overlapping
`server_safe` base in the applied state quarantines that graph and serves the
others; if no applied graph is left to serve, startup fails with
`cluster_no_healthy_graphs`. An embedded handle refuses a policy whose base
overlaps its own graph root.
A base is compared only with a storage root of its own kind: an `s3://` base
with an `s3://` root, a `file://` base with a local root. When the root is
spelled with a path component a base URI cannot express (an empty component
such as `s3://bucket/a//cluster`, or a percent sign in a local path), a
same-kind base is refused with `external_blob_storage_root_uncomparable`,
because disjointness cannot be proven. Moving the base does not clear that
code; the storage root spelling does.
The allow-list controls which external objects an authorized writer may cause
the process to inspect: a new external Blob reference, and the source of a
`load csv` statement, which the engine reads in full once
([CSV loading](/docs/queries-search/mutations#csv-loading)). Cedar policy separately
decides who may write. See [Blob values](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md).
## Policies [#policies]
```yaml
policies:
graph-access:
file: graph.policy.yaml
applies_to: [knowledge, catalog]
registry-access:
file: server.policy.yaml
applies_to: [cluster]
```
A bundle targets either graph IDs or the `cluster` server scope, never both.
Only one bundle may bind a given graph or the cluster scope. See
[Authorization](/docs/operations/policy).
## Storage [#storage]
For direct apply, omitted `storage` puts applied state and graph data under the
config directory. `cluster apply --server` instead uses the selected server’s
canonical root when `storage` is omitted; an explicit absolute path or `file://`,
`s3://` or `az://` URI must match it. Remote apply rejects relative storage paths
and reads only the source bundle from the caller’s filesystem.
Use the standard storage credential environment for the chosen backend. Azure
is a qualification preview and requires the admission wrapper for every writer;
see [Deployment](/docs/getting-started/deployment#azure-blob-preview).
## Declared paths [#declared-paths]
Every `schema`, `queries`, and policy `file` path is resolved against the
directory that holds `cluster.yaml`. A relative path must stay inside that
directory: a `..` segment is refused with `config_path_escape`, and a path
that reaches its file through a symbolic link, on the way to it or as a
query file discovered inside a declared directory, is refused with
`config_path_symlink`. Each diagnostic names the setting that declared the
path. The bundle is one directory of files read exactly as declared, so what
gets applied never depends on something outside it.
An absolute path is accepted as given and is not checked for either shape.
Prefer relative paths; they are what keep a bundle portable and hermetic.
## Command behavior [#command-behavior]
| Command | Changes graph or cluster state? | Use |
| ---------------- | ------------------------------: | -------------------------------------------------------------------------------- |
| `validate` | no | Parse and type-check the declaration |
| `plan` | no | Preview desired differences; unsupported deployment effects still refuse |
| `apply` | yes | Converge graph inventory, schemas, queries, policies and provider/Blob settings |
| `status` | no | Read recorded deployment and lock status; server status also observes activation |
| `observe` | no | Report current observations without changing authority |
| `force-unlock` | yes | Remove one proven-stale lock by exact ID |
| `upgrade-ledger` | state only | Convert a stopped legacy ledger without resetting graph data |
All execution uses ledger v2 and requires `state.lock: true`. With `--server`,
apply submits to the existing writer and activates without restart. Direct
apply owns exclusive admission and requires an explicit handoff before serving.
Retained graph roots and storage formats stay fixed. Policy/provider/Blob bindings
can change; removing a graph declaration deletes its managed storage and history. See
[deployment boundaries](/docs/clusters#deployment-boundaries).
Apply does not load rows or start servers. Root-addressed status, reconciliation
and conversion do not use `cluster.yaml`; see [the deployment workflow](/docs/clusters).
## Limits [#limits]
Configuration loading refuses limits before schema effects. Shared files with
identical bytes count once toward the aggregate source limit.
| Resource | Limit |
| --------------------------------------------------------------------- | ----------------------------------: |
| Each `cluster.yaml`, schema, query or policy source | 1 MiB |
| Distinct source bytes in one captured bundle | 8 MiB |
| Declared resources (each graph and schema count separately) | 4096 |
| Query discovery paths plus directory entries, including ignored files | 4096 |
| Encoded immutable deployment bundle | 16 MiB |
| Encoded cluster ledger | 16 MiB |
| Lock metadata read | 64 KiB |
| Outstanding deployments | 1 |
| Retained deployment results | 32 records, 4 MiB total, 1 MiB each |
| Deployment actor / resource address / canonical root | 256 / 512 / 4096 UTF-8 bytes |
Deployment admission checks the actual encoded prepared intents and reserves
ledger/result space to record completion before accepting schema effects.
JSON escaping and prepared state can make a deployment exceed its encoded limit
even when raw source bytes fit. Result history may evict older completed records
to admit a new deployment; outstanding authority is retained. These bounds do
not cap graph data, native manifest-history scans or total process memory.
See [Operating a cluster](/docs/clusters) for the end-to-end workflow.
# Operating a cluster (/docs/clusters)
An OmniGraph cluster is a declarative bundle of graphs, schemas, stored queries
and authorization policies. Apply converges graph inventory, schemas, queries,
policies and provider/Blob settings. Submit to the running server to activate
changes without a restart.
Use a cluster for a multi-graph server or shared operational configuration. For
one local graph, the [quickstart](/docs/getting-started/quickstart) is simpler.
## Create a bundle [#create-a-bundle]
```text
company-brain/
├── cluster.yaml
├── knowledge.pg
├── queries/
│ └── people.gq
├── cluster.policy.yaml
└── graph.policy.yaml
```
```yaml
version: 1
metadata:
name: company-brain
graphs:
knowledge:
schema: knowledge.pg
queries: queries/
policies:
cluster-access:
file: cluster.policy.yaml
applies_to: [cluster]
graph-access:
file: graph.policy.yaml
applies_to: [knowledge]
```
Paths are relative to `cluster.yaml`. Its configuration version is independent
of the deployment ledger version. The [configuration reference](/docs/clusters/config)
covers storage roots, embedding providers, external Blob policy and limits.
For server-owned deployment, the applied policy must grant the operator
`config_manage` at cluster scope. Schema changes and graph deletions additionally
require `read` and `schema_apply` on the affected graphs. Direct deletion also
enforces an installed graph policy and requires an authorized `--as` actor.
Deployment status reveals management
metadata under `config_manage`; it does not require data access on unrelated
graphs. New graphs need suitable declared policies too.
The server derives the actor from its bearer token; `--as` is for direct access.
## Bootstrap a cluster [#bootstrap-a-cluster]
```bash
omnigraph cluster validate --config ./company-brain
omnigraph cluster plan --config ./company-brain
omnigraph cluster apply --config ./company-brain --as act-alice --json
```
Fresh apply creates the deployment ledger and declared graphs. It captures all
source bytes before execution, prints the original `Deployment-ID`, and records
exact outcomes. It does not load rows; use `load` or `mutate` for data changes.
Direct apply retains its admission lock after completion. Establish that the
owner and its accepted I/O have settled, then follow
[ownership transfer](/docs/getting-started/deployment#writer-topology) using the exact printed
lock ID before starting the server:
```bash
omnigraph --cluster file:///srv/company-brain cluster force-unlock ''
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-alice":"secret"}' \
omnigraph-server --cluster file:///srv/company-brain --bind 0.0.0.0:8080
```
Use the actual root printed by apply. A directory boot resolves its storage root
through `cluster.yaml`; a root URI boots directly from applied resources. Editing
local files alone never changes serving behavior. See
[HTTP server](/docs/operations/server) for authentication and routes.
### First boot from a bootstrap receipt [#first-boot-from-a-bootstrap-receipt]
Administrators using the `omnigraph-cluster` library can prepare a fresh S3
cluster with `bootstrap_serving`, containing only its initial cluster management
policy. After that initializer has stopped, supply its exact receipt at first boot:
```bash
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-alice":"secret"}' \
omnigraph-server --cluster s3://company-data/company-brain \
--bootstrap-handoff /run/omnigraph/bootstrap-receipt.json
```
The receipt must be a regular file containing strict JSON, at most 16 KiB.
It must match the selected root and its completed empty bootstrap. The server
claims ownership once before listening; replay, mismatched state and uncertain
claims refuse. Any failure after the claim retains ownership, including invalid
trust or server settings. Investigate that exact attempt; do not retry by
removing the lock or omitting the flag.
The receipt supplies no user permissions. Configure static tokens or identity
trust normally; the applied policy controls who can create the first graph
through `cluster apply --server`. Protect the receipt and boot configuration as
administrator inputs. This path supports only first boot of a fresh S3 cluster;
it is not a restart, recovery or writer-fencing mechanism. Local and Azure roots
are unsupported, and other writers must remain excluded.
## Deploy without restarting [#deploy-without-restarting]
Edit and validate the bundle, then submit it to the running owner:
```bash
omnigraph cluster validate --config ./company-brain
OMNIGRAPH_TOKEN_PRODUCTION='secret' omnigraph cluster apply \
--server production --config ./company-brain --json
```
`production` names a server declared in the operator config, and the CLI sends
its token only to that server's URL:
```yaml
# ~/.omnigraph/config.yaml
servers:
production:
url: https://graph.example.com
```
`omnigraph login production` stores the token instead of the variable. A
literal `--server https://…` URL that matches no declared server gets no token;
see [server credentials](/docs/cli#use-a-server).
With `--server`, omitted `storage` binds the bundle to the selected server’s
canonical root. An explicit absolute `storage` must match that root; relative
storage paths refuse. The CLI reads only local source files and needs no local
mount or storage credentials for the server’s root.
Preview the same local configuration through the server before applying:
```bash
omnigraph cluster plan --server production --config . --json
omnigraph cluster apply --server production --config . --timeout 1800 --json
```
The served plan lists resource changes and schema migrations, including the exact
managed root and history that a graph removal deletes. It writes nothing and keeps
serving admission open. Its ledger CAS and input digest identify an observation;
apply rechecks current authority and physical execution eligibility.
The CLI prints the deployment ID before submission. The server keeps its PID,
listener and writer ownership. It closes admission on affected graphs,
finishes their admitted requests, publishes schema changes, and activates
matching schemas, queries and runtime permissions together. Unaffected graphs
keep serving.
Graph additions and removals use the same deployment; see
[deletion semantics](#deployment-boundaries) before removing a declaration.
The response separates the durable deployment result from `active`, which means
that result's affected bindings are installed in this process. An unrelated
blocked graph does not invalidate that activation. Default apply waits for
convergence and activation by polling the original ID. `--no-wait` instead returns
after durable acceptance; drain and preparation may precede that acknowledgment.
`--timeout SECONDS` bounds caller waiting, including acceptance (default 300,
maximum 3600). Expiry exits 5 with the original ID and last observation; it does
not cancel execution or prove failure. After reporting expiry, the CLI does not
wait for an unfinished background DNS lookup before exiting. A failed recovery
hint also preserves the server's original refusal, including an outstanding
deployment ID. Resume observation with:
```bash
omnigraph cluster status --server production --deployment-id ID --wait --timeout 1800 --json
```
The exact response contains `deployment`, `active` and `in_progress`. Default
apply and status `--wait` return at activation, which can precede release of
deployment admission. Repeat exact-ID status reads until `in_progress: false`
before submitting another deployment; this confirms the observed deployment has
released its admission. General cluster status retains its aggregate `status` object. An authenticated submitter
can read its own durable receipt even after its deployment removes its management
permission; general status and later deployments still require current permission.
Lost submission responses are followed only by original-ID reads, never automatic
resubmission. While waiting, observation retries transient HTTP 429/503 responses
and interrupted response bodies within the same budget; malformed receipts fail immediately.
Expired receipts and unknown outcomes require investigation.
Each graph publishes atomically; deployment
across multiple graphs is not one transaction. Query-only changes create no graph
commit and also work with multiple branches. Schema changes remain main-only
and require a single live branch.
Policies, provider definitions and graph bindings, and external-Blob rules can
change on existing graphs. Current permissions authorize the deployment; proposed
permissions cannot authorize themselves. Provider changes do not re-embed stored
vectors. Roots, format and credential/trust configuration stay fixed.
A refusal before effects restores unchanged serving views, including after a
drain timeout. That timeout bounds draining admitted requests; once drained,
the server owns preparation, completion and activation through the existing
shutdown boundary. There is one deployment protocol and no legacy execution fallback.
Upgrade the CLI, server and cluster tools together. Finish outstanding deployments
with the build that accepted them before upgrading: captured inputs are exact
and are not translated into a different request. Use the stopped-ledger upgrade
command to remove obsolete completed-result runtime fields; graph data and exact
achieved receipts are preserved. The graph storage format is unchanged.
## Direct deployments and conversion [#direct-deployments-and-conversion]
Without `--server`, apply executes under its own exclusive admission and requires
the serving owner to have stopped and handed off the lock. Start the server after
settlement to activate the applied revision. Direct apply never takes over a live
server or writes around its lock.
Use explicit stopped-ledger conversion for a v1 ledger or a v2 ledger whose
completed receipts still contain obsolete runtime activation fields:
```bash
omnigraph --cluster file:///srv/company-brain \
cluster upgrade-ledger --writers-stopped --json
```
Stop serving, writers and maintenance and establish prior graph/control I/O
quiescence first. Conversion preserves rows, graph identities, branches, history
and applied resources. It does not reset graphs, replay old work or convert graph
storage formats. There is no automatic migration or v1 execution fallback.
For v2, conversion removes only those obsolete runtime fields, preserving the
ledger identity, next deployment sequence and exact achieved receipts. Outstanding
deployments and unsupported receipt shapes refuse; finish accepted work with
its originating build before upgrading.
Normal reads report `ledger_upgrade_required` with the stopped-upgrade command
for a recognized prior receipt; malformed or unknown state remains an error.
A completed read-only preflight refusal releases a newly acquired direct lock.
Accepted work, cancellation and uncertain effects retain it for reconciliation.
`--as` labels a storage-owning operator; installed graph policies still govern
schema effects. It is not remote authentication.
## Inspect and recover a deployment [#inspect-and-recover-a-deployment]
A lost connection does not mean failure. Observe the original ID:
```bash
OMNIGRAPH_TOKEN_PRODUCTION='secret' omnigraph cluster status \
--server production --deployment-id '' --json
```
An output failure also returns nonzero without proving that deployment failed.
Served apply must write and flush its deployment ID before submitting. Direct
apply finishes its one deployment attempt even if its identity diagnostic cannot
be written, then returns the output error with the original ID, root and lock
information. Observe that identity before taking further action; failed output
does not cause automatic resubmission.
Status is read-only. `active` is false for an older result after a newer revision
activates. Only the current runtime can prove activation; a durable completion
receipt alone cannot. Repeating apply with the same ID and identical captured input
returns its recorded outcome; it never executes again. Different input refuses.
If the server stopped with an outstanding deployment, inspect the storage root:
```bash
omnigraph --cluster file:///srv/company-brain \
cluster status --deployment-id '' --json
```
Establish prior-owner and accepted-I/O quiescence, exclude concurrent admissions
and unlocks, then reconcile that exact ID:
```bash
omnigraph --cluster file:///srv/company-brain cluster force-unlock ''
omnigraph --cluster file:///srv/company-brain \
cluster apply --deployment-id '' --writers-stopped --json
```
Reconciliation uses captured input and exact publication evidence; it never
replays an uncertain schema or graph-creation invocation. Unstarted schema or
graph-creation work can be recorded as not attempted. A partial graph birth can
be abandoned only when its exact unpublished, empty artifacts belong to that
attempt. A foreign or committed graph is never reset. Unknown outcomes stay
outstanding and block new writes. A later creation may reuse an empty local
directory tree left by that cleanup; files, symlinks and cloud markers still refuse.
A settled partial result permits a corrective successor from achieved state;
it does not roll back graphs that committed. Recovery retains a new admission
lock, so perform ownership transfer before starting another owner. Never
allocate a new ID to retry an unknown outcome. Result eviction reports acceptance
and outcome as unknown; it never authorizes replay. See [limits](/docs/clusters/config#limits).
## Deployment boundaries [#deployment-boundaries]
`cluster apply` creates graphs, deletes removed graphs, and updates schemas,
stored queries, policies, providers and Blob rules. Removing a graph declaration
and applying is destructive: it deletes that graph's managed storage, including
all branches, retained history and managed Blob bytes. Review the deletion in
`cluster plan` and keep any required backup before applying. There is no separate
delete flag or unregister mode. External Blob source objects are not deleted.
On object stores, deletion removes the active namespace; provider version history
and retention policies may keep older objects. This is not secure erasure.
Live apply closes the affected graph to new requests and waits for its admitted
work and response bodies before deletion. Other graphs continue serving. If
deletion is interrupted, reconcile the original deployment ID; the recorded
operation resumes removal of the exact root. Editing the desired configuration
does not cancel accepted deletion, and absence during recovery does not create a
replacement graph.
Importing an existing root, recreating a missing managed graph, accepting schema
drift and repairing catalog payloads are not apply modes. Restore missing or
damaged authoritative data from a verified backup before deploying changes to
that graph.
Every present affected root is checked against its achieved schema identity,
even when source text changes. Removing an already-absent graph still records a
completed deletion; it never recreates the root. An out-of-band replacement or schema change refuses with
`applied_schema_drift`; matching text cannot authorize a different graph identity.
Plan always observes without taking the cluster writer lock. It reports the
existing owner and runs the same effect-free preparation used by apply, with
the intended actor. Its migration steps come from that preparation, not a second
schema-file read. Apply prepares again under writer admission because a preview
reserves nothing. Unavailable affected graphs, schema drift and migration
restrictions are errors, except that deletion accepts an already-absent root. An unavailable unrelated graph does not block an
independent change. Live apply also validates provider secrets and serving
settings on the server.
## Operational boundaries [#operational-boundaries]
* One mutation-capable process owns a cluster. Online deployment runs inside it;
direct maintenance requires an ownership handoff.
* Object-storage clusters may boot directly from `s3://bucket/prefix` or
`az://container/prefix`; source files are not needed for serving or recovery.
* Azure remains a qualification preview. Its running server retains the
mandatory [admission wrapper](/docs/getting-started/deployment#azure-blob-preview) during HTTP
deployment; submission does not acquire a second storage-writer lease.
* A schema drop removes data from the branch head without reclaiming storage.
Retained historical commits remain readable until cleanup removes them.
# Concepts (/docs/concepts)
OmniGraph is a typed property-graph database. A graph contains **nodes**,
**edges**, and a schema that defines their properties and constraints. The same
graph can be queried with structured graph patterns, full-text search, and
vector search.
## Graph model [#graph-model]
* A **node type** describes an entity such as `Person` or `Document`.
* An **edge type** connects two node types, such as `WorksAt: Person -> Company`.
* A **property** has a declared type and can be nullable.
* Constraints such as keys, uniqueness, ranges, and edge cardinality are checked
on every write path.
Schemas use the [`.pg` language](/docs/schema). Reads and mutations use the
[`.gq` language](/docs/queries-search/queries).
## Consistency [#consistency]
Every query reads one consistent snapshot of the whole graph. A concurrent
write cannot become visible halfway through a query.
One mutation query or load publishes one graph commit. If it fails, none of its
changes become visible. Separate mutation queries are separate commits.
[Branches](/docs/branching) provide isolated, durable workspaces for
multi-step changes. Each change on a branch is still its own commit; merging the
branch makes the combined result visible on the target in one atomic step.
## History [#history]
Commits form a graph-wide history. You can:
* inspect commits and their actors;
* read a branch head or an earlier commit;
* compare two points in history;
* merge branches with entity-level conflict reporting.
History remains available until destructive cleanup removes the storage
versions it needs. Live branches can retain older versions, so delete branches
you no longer need.
## Storage and indexes [#storage-and-indexes]
A graph is stored at one local, S3, or Azure Blob root. That root is the graph's
consistency and history boundary; cross-graph transactions are not supported.
See [storage](/docs/concepts/storage) for supported URI forms and backend configuration.
Indexes improve performance but do not define logical correctness. A declared
index may be missing or cover only part of recent data; queries still return
matching entities outside its coverage. Run `omnigraph optimize` to compact
data and refresh index coverage.
## Access paths [#access-paths]
The embedded engine, CLI, and HTTP server share the same graph semantics. The
server adds bearer authentication and Cedar authorization; direct storage
access does not pass through server authentication.
* [Quickstart](/docs/getting-started/quickstart)
* [CLI guide](/docs/cli)
* [HTTP server](/docs/operations/server)
* [Clusters](/docs/clusters)
# Storage (/docs/concepts/storage)
One OmniGraph graph lives at one storage root. The root contains the graph's
schema, data, branches, and commit history. Treat it as a single unit: do not
edit files beneath it with storage-provider tools or write to its underlying
datasets directly.
## Supported graph URIs [#supported-graph-uris]
| URI | Use |
| ----------------------------------------------------------------- | ---------------------------------------- |
| `/absolute/path/graph.omni` or `file:///absolute/path/graph.omni` | Local filesystem |
| `s3://bucket/prefix/graph.omni` | Amazon S3 or an S3-compatible service |
| `az://container/prefix/graph.omni` | Azure Blob Storage qualification preview |
An `http://` or `https://` URL addresses an OmniGraph server; it is not a graph
storage URI. CLI commands use `--server` for server URLs and `--store` for direct
storage access.
Azure support is a qualification preview: local Azurite validation and a live
managed-identity smoke deployment are complete, but the adversarial live-Azure
matrix is still pending. Every
mutation-capable Azure process must use the admission wrapper described in
[deployment](/docs/getting-started/deployment#azure-blob-preview).
## Consistency and history [#consistency-and-history]
Although a graph contains separate data for each node and edge type, OmniGraph
presents one graph-wide snapshot. A successful write makes all affected types
visible together. Readers never see half of a graph commit.
Renaming a type with `@rename_from` preserves its identity and history. Dropping
and later recreating a declaration starts a new lifetime, even when the public
name is reused.
`omnigraph optimize` rewrites physical layout without deleting history.
`omnigraph cleanup` is different: it permanently removes old storage versions,
which can make earlier commits unreadable. Review the retention policy and
backups before confirming it, and quiesce long-lived historical or Blob readers
first. An unconfirmed cleanup echoes the policy but does not inspect candidates.
## Local filesystem requirement [#local-filesystem-requirement]
Local read-write graphs require filesystem hard-link support. Read-only access
can work without it, but initialization and writes fail loudly when the
filesystem cannot provide the required create-if-absent behavior. Use a native
local filesystem rather than a mount that emulates files incompletely.
## S3 configuration [#s3-configuration]
OmniGraph uses the standard AWS credential chain. Common settings include:
* `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optional
`AWS_SESSION_TOKEN`;
* `AWS_REGION` or `AWS_DEFAULT_REGION`;
* `AWS_ENDPOINT_URL_S3` for an S3-compatible endpoint;
* `AWS_S3_FORCE_PATH_STYLE=true` when the endpoint requires path-style URLs;
* `AWS_ALLOW_HTTP=true` only for a trusted local development endpoint.
## Azure configuration [#azure-configuration]
Azure roots use the strict form `az:///`. Credentials never
belong in the URI.
* `AZURE_STORAGE_ACCOUNT_NAME` selects the account.
* `AZURE_STORAGE_CLIENT_ID` optionally selects a user-assigned managed identity.
* `AZURE_STORAGE_ENDPOINT` selects a custom Blob endpoint.
* `AZURE_STORAGE_USE_EMULATOR=true` enables Azurite; use
`AZURITE_BLOB_STORAGE_URL` to override its endpoint.
Azure Container Apps supplies its managed-identity endpoint variables at
runtime. Restart the process after changing storage-account, endpoint, or
identity selection.
For deployment, server boot, and cloud-specific safety requirements, see
[Deployment](/docs/getting-started/deployment).
# Deployment (/docs/getting-started/deployment)
OmniGraph can keep a cluster on a local filesystem, S3-compatible object
storage, or Azure Blob Storage. The server always boots from one cluster root
and exposes its ready applied graphs under `/graphs/{id}/…`; use
`--require-all-graphs` when any blocked graph must fail startup.
Servers use the v0.13 HTTP contract and open graph storage formats v14 to v16. The
cluster ledger has a separate version: explicitly converting it to v2 preserves
graph data and history. Start with [Operating a cluster](/docs/clusters) to
create and apply a deployment bundle.
## Binary [#binary]
```bash
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-service":"secret"}' \
omnigraph-server \
--cluster /srv/omnigraph/company-brain \
--bind 0.0.0.0:8080
```
For object storage, pass the applied storage root:
```bash
AWS_REGION=us-east-1 \
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-service":"secret"}' \
omnigraph-server \
--cluster s3://company-data/omnigraph/company-brain \
--bind 0.0.0.0:8080
```
Use `GET /healthz` for process health and `GET /readyz` for readiness:
`/readyz` reports `loading` (startup is still pending), `serving`,
`degraded` (some graphs unavailable), `blocked` (none ready), or `draining`.
It includes the applied `config_digest` it booted from
(`booted_serving_digest`), the ledger revision, and `served_graph_count`,
`ready_graph_count`, `loading_graph_count` and `blocked_graph_count`.
`served_graph_count` counts the whole registry. Graphs closed for a transition also
contribute to `blocked_graph_count`. Readiness stays 503 while any graph is
loading. Once startup finishes, it returns 200 when at least one graph is ready
or the applied inventory is empty, and 503 when all graphs are unavailable or
shutdown has begun. The listener opens after configuration and
authorization validation, before graphs open. Healthy graphs become available
as they finish loading, while readiness holds aggregate traffic until loading
finishes. A nonempty cluster with no healthy graph exits with a
startup error after its opening attempts finish. With `--require-all-graphs`,
all graphs remain unavailable until every graph opens successfully; any failure
stops the server. Wait for readiness before sending data requests: the printed
listening address and `/healthz` confirm only that the listener is live.
Readiness is unauthenticated and names no graph. Authorized `GET /graphs`
returns one `graphs` list, including blocked entries, with each graph's `state`
(`loading`, `ready`, `blocked`, `transitioning` or `stopping`), `read_available`, `write_available` and
`action`. Availability describes the runtime, not the caller's permissions.
Blocked entries include a sanitized `failure`: `invalid_configuration`,
`invalid_policy`, `invalid_external_blob_policy`, `open_failed` or
`invalid_stored_queries`. Details remain in server logs.
`apply_correction_or_restart` means deploy corrected queries, policies or provider
configuration, or restore damaged storage from a verified backup and restart.
It never authorizes adopting a different graph identity. `wait_for_restart` means
shutdown. `wait_for_startup` means the single startup admission is incomplete;
it promises no retry time or additional open.
`wait_for_transition` means that graph's admissions are closed while its owner
finishes a transition. Wait for that owner, or restart if the transition was
abandoned or expired; there is no promised retry time. Other ready graphs can
continue serving and transitioning. Server-owned deployments activate schema and
stored-query changes through this transition; a proved pre-effect refusal restores
the unchanged views. See [live deployment](/docs/clusters#deploy-without-restarting).
Ready entries use `none`. There is no separate `quarantined` list or automatic startup retry.
`GET /cluster/deployments/{id}` reports `active` only when this process has installed
the exact deployment and all its graph bindings are ready. Loading, blocked graphs,
partial convergence and shutdown keep it false; a restart must verify its bindings
again. Successful apply alone is not activation, which never rewrites the durable
achieved result or receipt.
Known loading, blocked or transitioning graphs return 503 (`graph_unavailable`) to callers authorized to
read `main` or list the management inventory; other callers cannot use this response to discover
them. Unknown graphs return 404. A 503 does not authorize replaying a write.
`omnigraph graphs list --server ` displays graph ID, state and URI;
`--json` preserves the full inventory fields. Minimal discovery remains IDs/names only.
Admitted writes and reads with fully accepted request bodies retain capacity
until execution finishes, even after client disconnect. Writes include optional
source deletion after a merge. A lost response leaves the write's outcome
uncertain; follow [failure outcome](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/troubleshooting.md#failed-data-write-commands)
before resubmitting. MCP cancellation and its 30-second response deadline stop
waiting; the admitted read continues and retains its tool slot until completion.
SIGTERM closes operation admission and stops accepting connections. The server
waits for entered graph opens, admitted writes, executing reads, read bodies and
stream producers for at most `--shutdown-grace-seconds` (else
`OMNIGRAPH_SHUTDOWN_GRACE_SECONDS`, else 25), then logs unfinished work and exits 2.
A thread enforces the deadline even if the runtime stalls. Signal handling starts
after configuration loading, before graph opening. Disconnected work remains in
the wait; a graph finishing startup during shutdown cannot become available.
A panic or uncertain startup/write completion closes admission for every graph
and starts the same shutdown without renewing its deadline. Once HTTP connections
and other known logical owners finish, it exits 2 immediately; the watchdog is
the upper bound. Unresolved reservations stay charged until exit. This contains
the process; it does not prove native I/O settled or the engine can be reused.
A failure proven to leave the graph unchanged can keep admission open when its
physical storage requests have also settled. Error status or category alone
proves nothing; uncertain publication and unclassified failures still fail closed.
A clean native shutdown exits 0. Set the orchestrator's own
termination grace longer than this value; a cutoff is crash-equivalent for the
work it interrupts, and the next open recovers it as after any crash.
On local storage and S3, the server also closes its storage scope, waits for
accepted graph and control writes, and releases its own cluster admission.
The next server can start normally with the same `--cluster`. An initial
`--bootstrap-handoff` receipt is only for the first process; omit it on later
starts. The data, history and applied configuration remain in place.
Uncertain or interrupted work retains admission and exits nonzero. S3 release
uses a conditional marker, so a lost release response cannot delete a later
owner's lock. A nonclean exit is still a reason to inspect the original outcome,
not to replay writes. Azure keeps its existing shutdown behavior: even a clean
exit retains admission and requires the independent Azure recovery procedure.
The server's storage clients disable hidden mutation retries; if
`OBJECT_STORE_CLIENT_MAX_RETRIES` is set, it must be `0`. An ambiguous storage
failure stops new writes rather than allowing a later success to hide it.
Other direct writers still follow the [ownership-transfer procedure](#writer-topology).
## Admission limits [#admission-limits]
Admission refuses excess operations and input immediately. Set these process-wide
environment variables before startup; sizes are integer bytes. Invalid numbers
warn and use the default; zero capacity refuses the corresponding lane.
| Variable | Default | Resource |
| ------------------------------------- | ------------------: | ---------------------------------------------------------------------------- |
| `OMNIGRAPH_GLOBAL_INFLIGHT_MAX` | 64 | Admitted write operations across actors |
| `OMNIGRAPH_GLOBAL_BYTES_MAX` | 268435456 (256 MiB) | Estimated retained write-input bytes across actors |
| `OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX` | 16 | Admitted writes per actor |
| `OMNIGRAPH_PER_ACTOR_BYTES_MAX` | 4294967296 (4 GiB) | Estimated retained write-input bytes per actor |
| `OMNIGRAPH_ACTIVE_ACTORS_MAX` | 1024 | Actors with admitted writes; idle records are removed |
| `OMNIGRAPH_INGRESS_INFLIGHT_MAX` | 64 | Write requests retained by collection, operations or responses |
| `OMNIGRAPH_INGRESS_BYTES_MAX` | 268435456 (256 MiB) | Reserved/retained write-body bytes |
| `OMNIGRAPH_READ_INGRESS_INFLIGHT_MAX` | 64 | Read/MCP requests with bodies retained by collection, execution or responses |
| `OMNIGRAPH_READ_INGRESS_BYTES_MAX` | 67108864 (64 MiB) | Reserved/retained read/MCP body bytes |
| `OMNIGRAPH_BODY_TIMEOUT_SECONDS` | 30 | Time allowed to collect one complete request body |
The default ingress allowances total 320 MiB across the independent lanes;
configure both byte limits when setting an instance's input budget.
Body timeouts above 86400 seconds also warn and use the 30-second default.
Ingress reserves the route's maximum before reading, then reduces the reservation
to the actual body size. Body collection limits remain 1 MiB for ordinary JSON
requests, 32 MiB for bulk load and 64 KiB for MCP. Cluster deployment POSTs
allow 16 MiB plus 1 KiB for the request envelope; the captured bundle itself is
limited to 16 MiB. MCP uses the read-body lane after authentication. Only registered
bulk routes receive the larger limit; a stored query named `load` or `ingest` keeps the
ordinary JSON limit. Stored-query admission uses the registry's typed read/write
kind. Disconnect does not release capacity held by executing work; the body timeout
ends at complete collection and cannot cancel admitted execution. Input limits do
not bound process RSS or all engine memory and I/O. Size instances for graph,
workload and concurrency as well as request bytes.
The server allows 128 read-response observers and, independently, 64 write-response
observers. Executing reads, pending results, bodies, yielded bytes and their server
producers retain the slot until their final owner releases it. Bodyless reads
consume only a read observer; slow reads cannot exhaust write-body or write-response
capacity. Status routes bypass
ordinary admission so they remain callable during saturation.
Export and change-baseline responses reserve 256 KiB each from a fixed 2 MiB
process allowance, permitting eight simultaneous reservations. Each response
allows three outstanding 64 KiB chunks plus one pending producer chunk; at most
two chunks wait in its queue. Yielded chunks, including retained clones and
slices, keep their reservation until released. A new response waits at most
250 ms for process capacity before HTTP 413. Once admitted, a producer waits
for chunk capacity or disconnect without a per-chunk deadline; ordinary queue
backpressure has no additional deadline. An interrupted baseline supplies no
usable terminal cursor. Consumers must release each received server buffer to
continue; an in-process consumer retaining every buffer can exhaust its lane.
An HTTP client may still collect its received response: those client buffers are
separate from the server's transport buffers. These limits cover transport
payload buffers, not engine row/Arrow encoding, native work or total process RSS.
The engine separately limits retained keyed batches, keyed parse estimates
and removed-ID collections across each operation's tables; see
[mutation limits](/docs/queries-search/mutations#limits-and-conflicts). These fixed limits
return HTTP 413 with structured `resource_limit` details before the operation's
data fragments or publication, and leave the server available for smaller work.
They do not undo an implicitly created load branch or earlier command effects,
and do not establish a process memory ceiling or automatic retry safety.
## Container [#container]
The container entrypoint reads `OMNIGRAPH_CLUSTER` and binds to port 8080:
```bash
docker run --rm -p 8080:8080 \
-e OMNIGRAPH_CLUSTER=s3://company-data/omnigraph/company-brain \
-e AWS_REGION=us-east-1 \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e OMNIGRAPH_SERVER_BEARER_TOKENS_JSON \
ghcr.io/modernrelay/omnigraph-server:
```
For a local cluster, mount the complete cluster directory and point
`OMNIGRAPH_CLUSTER` at the mount:
```bash
docker run --rm -p 8080:8080 \
-v /srv/company-brain:/var/lib/omnigraph/cluster \
-e OMNIGRAPH_CLUSTER=/var/lib/omnigraph/cluster \
-e OMNIGRAPH_SERVER_BEARER_TOKEN \
ghcr.io/modernrelay/omnigraph-server:
```
Terminate TLS at a load balancer or trusted reverse proxy. Keep bearer tokens
and storage credentials in the platform's secret store.
## Unix-socket access [#unix-socket-access]
`omnigraph-server` listens on TCP only. Some local clients can open a Unix-domain
socket but no TCP connection, such as an agent whose sandbox allows one socket
path. To serve them, run a gateway that relays the socket to the server's
loopback address unchanged, for example:
```bash
socat UNIX-LISTEN:/run/omnigraph/server.sock,fork TCP:127.0.0.1:8080
```
Clients address it with `--server unix:///run/omnigraph/server.sock`. The
gateway must pass requests and responses through unchanged, including the
`Omnigraph-Http-Api` header. Restrict who may connect with the permissions of
the socket's directory; bearer authentication and policy apply as over TCP.
## S3-compatible storage [#s3-compatible-storage]
For AWS S3, configure the standard AWS credential chain and region. For a
compatible service, these variables may also be needed:
```bash
export AWS_ENDPOINT_URL_S3=https://objects.example.com
export AWS_S3_FORCE_PATH_STYLE=true
```
Set `AWS_ALLOW_HTTP=true` only for a trusted local development endpoint. Do not
put credentials in `cluster.yaml` or graph URIs.
The same storage root must be visible to servers and out-of-band cluster or
maintenance jobs. Submit schema/query changes and graph additions to the running
server with `cluster apply --server`; the root's applied revision remains the
deployment artifact. Direct apply requires ownership transfer before serving.
## Azure Blob preview [#azure-blob-preview]
Native `az:///` roots are implemented and tested with
Azurite. A live managed-identity smoke deployment is complete, but Azure
remains a qualification preview until the adversarial live-Azure matrix is
complete.
Every mutation-capable Azure process must be admitted under the cluster root:
```bash
AZURE_STORAGE_ACCOUNT_NAME=companygraph \
AZURE_STORAGE_CLIENT_ID= \
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-service":"secret"}' \
omnigraph-azure-admission run \
--mode server \
--root az://omnigraph/company-brain \
-- \
omnigraph-server \
--cluster az://omnigraph/company-brain \
--bind 0.0.0.0:8080
```
Use the same wrapper with `--mode job` and the same canonical `--root` for
bootstrap/apply jobs, `cluster upgrade-ledger`, root-addressed reconciliation,
`cluster force-unlock`, direct graph writers, and maintenance. Root-addressed
deployment status is read-only and does not need the lease. A v2 cluster's persisted
lock is additional admission; it does not replace the Azure lease or its
qualification boundary. The container entrypoint wraps an `az://` cluster automatically.
Replica-count settings are not a correctness fence: the admission lease is.
The checked-in [Azure reference deployment](https://github.com/ModernRelay/omnigraph/blob/main/deploy/azure/README.md)
contains the supported Container Apps topology, validation command, and
stuck-lease runbook. A stranded Azure lease and a retained v2 cluster lock are
separate: complete the lease recovery procedure before starting a wrapped
cluster-unlock or reconciliation job. Do not break a lease until the owner has
been identified and stopped.
## Writer topology [#writer-topology]
Run one mutation-capable writer process per cluster. This includes servers,
direct CLI writes, deployments, branch controls, and maintenance. On a v2
cluster, supported servers and direct CLI writers acquire the same exclusive
cluster admission before writable graph work and recheck the applied inventory.
CLI queries, exports, schema inspection/planning, lint and stored-query
validation check the applied identity without acquiring or changing that lock;
they refuse outstanding deployments or schema drift. Native maintenance,
including `repair` preview, uses writer admission. An outstanding deployment
admits only reconciliation of its exact original ID.
Keep older binaries, raw storage tools and embedded writers outside this
cooperating boundary stopped; the lock cannot fence their native storage I/O.
Admission remains held after an ordinary direct write command succeeds or a
deployment completes. A clean server shutdown follows the bounded release
described above; ending work without that proof retains admission. A completed read-only
preflight refusal during new deployment, conversion or admission construction
releases its admission. Recovery of accepted work, cancellation and uncertain
effects retain it. A finished response, zero active HTTP
requests, lock age or a stopped PID alone does not prove that previously
accepted storage writes have settled. Before transferring ownership, stop the
prior owner and establish that its graph work and control-store I/O are terminal.
Exclude new admissions and other unlock attempts until the exact-ID unlock
finishes. Then start the next owner. Use the root-addressed
[status and recovery commands](/docs/clusters#inspect-and-recover-a-deployment)
to obtain the lock ID; unlocking never resolves an uncertain deployment.
Use the same current toolchain for restart. Older binaries refuse the released
S3 lock record; do not delete that record to bypass the version check.
A legacy v1 ledger refuses ordinary serving and writes. Stop its previous
owners and explicitly [convert the ledger](/docs/clusters#direct-deployments-and-conversion)
before using the current admission protocol.
Read replicas and zero-downtime overlapping writer replicas are not currently a
supported topology. Prefer stop-then-start replacement for a mutation-capable
server.
## Authentication and policy [#authentication-and-policy]
Configure tokens from environment or a mounted secret file:
* `OMNIGRAPH_SERVER_BEARER_TOKEN`
* `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON`
* `OMNIGRAPH_SERVER_BEARER_TOKENS_FILE`
* `OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET` in the AWS-enabled build
Policy bundles come from the applied cluster. See
[HTTP server](/docs/operations/server) and
[Authorization and actors](/docs/operations/policy).
## Upgrade and backup [#upgrade-and-backup]
Back up the whole cluster root, not selected physical files. Before a release:
1. read the release notes;
2. quiesce writers;
3. verify backups or exports;
4. upgrade the fleet together;
5. restart and run representative reads and writes.
Graph-format upgrades and cluster-ledger conversion are separate operations.
Follow the [storage upgrade guide](/docs/operations/upgrade) for qualified format
routes; `cluster upgrade-ledger` never resets or rewrites graph history.
# Install (/docs/getting-started/install)
## macOS and Linux [#macos-and-linux]
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
```
The installer downloads a release archive, verifies its SHA-256 checksum, and
installs the binaries in `~/.local/bin`.
Prebuilt archives support Linux x86\_64 and arm64, macOS arm64, and Windows
x86\_64. The Linux archives need glibc 2.39 or newer, as on Ubuntu 24.04 and
Debian 13. Build from source on other platforms, including Intel macOS.
Homebrew is also supported:
```bash
brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph
```
## Windows [#windows]
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -Command "iwr -UseBasicParsing https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.ps1 | iex"
```
Binaries are installed in `%USERPROFILE%\.local\bin`.
## Release channels [#release-channels]
The default is the latest stable release; if none exists, the installer falls
back to `edge`.
Edge builds are prereleases built once a day from the tip of `main`, after its
whole CI run passes, when `main` has moved since the last one. Each is
published as its own release, `edge--` (for example
`edge-20261016-9c2b367`), whose files never change; the rolling `edge` release
always holds the newest one. The 30 newest edge builds are kept, and so is the
newest build of every storage format they no longer cover, since that build can
still export a graph written at that format. To install the newest:
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh \
| RELEASE_CHANNEL=edge bash
```
```powershell
iwr -UseBasicParsing https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.ps1 -OutFile install.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File .\install.ps1 -ReleaseChannel edge
```
To deploy an edge build, pin it by its tag, so every machine installs the same
binaries and you can return to them: `VERSION=edge-20261016-9c2b367` on
macOS/Linux or `-Version edge-20261016-9c2b367` on Windows.
Each edge build is also a [container image](/docs/getting-started/deployment#container) with the
same tag, `modernrelay/omnigraph-server:edge-20261016-9c2b367` on Docker Hub
and `ghcr.io/modernrelay/omnigraph-server:edge-20261016-9c2b367`, holding that
build's Linux x86\_64 binaries on Debian trixie. The `edge` image tag follows the
rolling `edge` release; `latest` stays on the newest stable release. Pin a
deployment to the image's digest, which the run's summary lists, and check it
with `gh attestation verify oci://@ --repo ModernRelay/omnigraph`.
Read the build's release description before pointing it at a graph that a
stable release must still read. It lists the storage format the build writes
and the HTTP contract it speaks next to the latest stable release's. A branch
the build writes to is left at the build's storage format, and a stable binary
refuses a format newer than its own, so test on a copy of the graph or back it
up first ([upgrades](/docs/operations/upgrade)). A client and a server must speak
the same HTTP contract.
Install a specific stable tag with `VERSION=v0.12.0` on macOS/Linux or
`-Version v0.12.0` on Windows. Set `INSTALL_DIR` or `-InstallDir` to choose a
different destination.
Documentation on `main` may describe behavior newer than the latest stable
binary. Check `omnigraph version` and the [release notes](https://github.com/ModernRelay/omnigraph/blob/main/docs/releases/).
## Build from source [#build-from-source]
Install the Rust stable toolchain and the Protocol Buffers compiler (`protoc`),
then:
```bash
RUSTFLAGS= cargo build --release --locked \
-p omnigraph-cli \
-p omnigraph-server \
-p omnigraph-azure-admission
```
The empty `RUSTFLAGS` replaces the workspace `.cargo/config.toml` build flags,
which exist for the development test suites; release binaries build without
them.
Or use the source installer:
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install-source.sh | bash
```
## Installed programs [#installed-programs]
Release archives contain:
* `omnigraph` — CLI;
* `omnigraph-server` — HTTP server;
* `omnigraph-azure-admission` — Azure writer-admission utility.
The admission utility's `inspect`, `break`, and help commands are available on
Windows. Its child-supervision `run` mode is supported only by the Unix
deployment images used for the Azure reference topology.
## Verify [#verify]
```bash
omnigraph version
omnigraph-server --help
omnigraph-azure-admission --help
```
`omnigraph version` reports what the binary itself speaks, without opening a
graph or contacting a server. For example, the edge build of commit `9c2b367`
prints:
```text
omnigraph 0.14.0-edge+9c2b367
internal-schema 16 (serves v14 to v16)
http-contract 0.13
commit 9c2b3679e8f66062252e7ed6270a9478b4b77413
```
`omnigraph version --json` prints the same facts as one object:
```json
{
"version": "0.14.0-edge+9c2b367",
"channel": "edge",
"commit": "9c2b3679e8f66062252e7ed6270a9478b4b77413",
"http_contract": "0.13",
"internal_schema": {
"writes": 16,
"serves": [14, 16]
}
}
```
An edge build reports a SemVer pre-release of the package version, such as
`0.14.0-edge+9c2b367`. Edge builds are published only while the package version
is above the latest stable release, so an edge version sorts after every
published stable release and before the release it precedes. `channel` is `stable`
or `edge` for a GitHub release archive and null for any other build. `commit`
is the commit a packaged build was compiled from; a plain source build reports
it as null and prints no `commit` line. The server reports the same commit as
`source_version` on `GET /healthz`.
`http_contract` is the `Omnigraph-Http-Api` value the CLI sends. A server
accepts only an exact match and returns its own value on every response,
including `HEAD /healthz` ([HTTP contract](/docs/operations/server#http-contract)).
When several CLIs are installed, for example one in `~/.local/bin` and an
older one from Homebrew, choose by this field, not by the package version.
`internal_schema.writes` is the storage-format stamp this binary's commits
write, and `internal_schema.serves` is the inclusive range, lowest first, that
normal open accepts; `snapshot --json` reports a graph's own stamp as
`internal_schema_version`. `omnigraph --version` (`-v`) prints only the first
line.
Release archives from v0.14.0 on, and every `edge--` build, carry
a build provenance attestation. Check that an archive was built by this
repository's release workflow, and from which commit:
```bash
gh attestation verify omnigraph-linux-x86_64.tar.gz --repo ModernRelay/omnigraph
```
The [container image](/docs/getting-started/deployment#container) carries the same stamp from
v0.14.0, and its GHCR digest carries the attestation:
```bash
docker run --rm --entrypoint /usr/local/bin/omnigraph \
ghcr.io/modernrelay/omnigraph-server: version
gh attestation verify oci://ghcr.io/modernrelay/omnigraph-server: \
--repo ModernRelay/omnigraph
```
Continue with the [quickstart](/docs/getting-started/quickstart).
# Quickstart (/docs/getting-started/quickstart)
This walks the core loop end to end: define a schema, initialize a graph, load
data, query it, and use a branch. It uses a local file-backed graph; swap the
path for an `s3://…` URI to run the same flow against object storage.
[Install](/docs/getting-started/install) the `omnigraph` CLI first.
The example builds a small evidence graph in which sources support a claim.
## 1. Write a schema [#1-write-a-schema]
A schema (`.pg`) declares your node and edge types. Save this as `schema.pg`:
```graphql
node Source {
slug: String @key
title: String
}
node Claim {
slug: String @key
statement: String
}
edge Supports: Source -> Claim
```
See the [schema language](/docs/schema) for types, constraints, and edges.
## 2. Initialize the graph [#2-initialize-the-graph]
```bash
omnigraph init --schema schema.pg graph.omni
```
`init` creates an empty graph at the given URI with your schema applied.
## 3. Load data [#3-load-data]
Data is newline-delimited JSON, one node or edge per line. Save this as
`evidence.jsonl`:
```jsonl
{"type":"Claim","data":{"slug":"lower-latency","statement":"The migration reduced request latency."}}
{"type":"Source","data":{"slug":"load-test","title":"Load test report"}}
{"type":"Source","data":{"slug":"production-metrics","title":"Production metrics"}}
{"edge":"Supports","from":"load-test","to":"lower-latency"}
{"edge":"Supports","from":"production-metrics","to":"lower-latency"}
```
The keyed `slug` values become node IDs, so the edges can refer to
`load-test`, `production-metrics`, and `lower-latency`. One load can contain
several node and edge types and publishes them as one graph commit. `--mode` is
required (`overwrite | append | merge`):
```bash
omnigraph load --data evidence.jsonl --mode overwrite graph.omni
```
For finer-grained writes and the node/edge JSONL shapes, see
[mutations and loading](/docs/queries-search/mutations).
## 4. Query [#4-query]
Write a query (`.gq`) — save as `queries.gq`:
```graphql
query sources_for_claim($claim: String) {
match {
$source: Source
$claim_node: Claim { slug: $claim }
$source supports $claim_node
}
return { $source.title as source }
order { source asc }
}
```
Run it:
```bash
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' --format table --store graph.omni
```
This returns `Load test report` and `Production metrics`.
The query name is positional; `--query` points at the `.gq` source and
`--store` addresses the graph's storage directly.
The [query language](/docs/queries-search/queries) covers `match`/`return`/`order`, and
[search](/docs/queries-search/search) covers vector and full-text search.
## 5. Work on a branch [#5-work-on-a-branch]
Branches isolate changes until you merge them — Git-style, across the whole
graph. Save an additional source and evidence link as `benchmark.jsonl`:
```jsonl
{"type":"Source","data":{"slug":"independent-benchmark","title":"Independent benchmark"}}
{"edge":"Supports","from":"independent-benchmark","to":"lower-latency"}
```
`branch` commands reserve their positional argument for the branch name, so
they address the graph with `--store` (the same flag `query` uses above):
```bash
omnigraph branch create review/add-benchmark --store graph.omni
omnigraph load --data benchmark.jsonl --mode append \
--branch review/add-benchmark graph.omni
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' \
--branch review/add-benchmark --store graph.omni
omnigraph branch merge review/add-benchmark --into main --store graph.omni
```
See [branches & commits](/docs/branching) and [merging](/docs/branching/merge).
## Next steps [#next-steps]
* [CLI reference](/docs/cli/reference) — every command and flag.
* [Schema language](/docs/schema) and [query language](/docs/queries-search/queries).
* [Operating a cluster](/docs/clusters) and [running the server](/docs/operations/server)
for multi-graph, multi-user deployments.
# Embeddings (/docs/queries-search/embeddings)
OmniGraph can turn query text into a vector for
`nearest($document.embedding, $text)`. Use the same provider and model that
produced the stored document vectors; vectors from different models are not
comparable.
Generated vectors are finite, nonzero, and L2-normalized before use. The target
`Vector(N)` property determines their required dimension.
## Providers [#providers]
| Provider | Configuration |
| ------------------- | ---------------------------------------------------------------------------------- |
| `openai-compatible` | Default. Uses an OpenAI-compatible `/embeddings` endpoint; defaults to OpenRouter. |
| `openai` | Uses OpenAI directly. |
| `gemini` | Uses Google's embedding API. |
| `mock` | Deterministic local vectors for tests and development. |
For direct or embedded use, configure the provider with environment variables:
| Variable | Meaning |
| -------------------------------------------------------- | -------------------------------------------------- |
| `OMNIGRAPH_EMBED_PROVIDER` | `openai-compatible`, `openai`, `gemini`, or `mock` |
| `OMNIGRAPH_EMBED_BASE_URL` | Override the provider endpoint |
| `OMNIGRAPH_EMBED_MODEL` | Override the model id |
| `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY` | Provider credential |
| `OMNIGRAPH_EMBED_DEADLINE_MS` | Total call deadline; default 60,000 ms |
| `OMNIGRAPH_EMBED_TIMEOUT_MS` | Per-request timeout; default 30,000 ms |
| `OMNIGRAPH_EMBED_RETRY_ATTEMPTS` | Maximum attempts; default 4 |
| `OMNIGRAPH_EMBED_RETRY_BACKOFF_MS` | Initial retry backoff; default 200 ms |
| `OMNIGRAPH_EMBEDDINGS_MOCK` | Force the mock provider |
The default OpenRouter model is `openai/text-embedding-3-large`. The direct
OpenAI default is `text-embedding-3-large`; Gemini defaults to
`gemini-embedding-2`.
## Cluster configuration [#cluster-configuration]
Cluster-served graphs select a named provider in `cluster.yaml`:
```yaml
providers:
embedding:
default:
kind: openai-compatible
base_url: https://openrouter.ai/api/v1
model: openai/text-embedding-3-large
api_key: ${OPENROUTER_API_KEY}
graphs:
knowledge:
schema: knowledge.pg
embedding_provider: default
```
Inline API keys are rejected. `${ENV_VAR}` references are resolved when the
server starts, not when the cluster configuration is planned or applied.
## Schema annotation [#schema-annotation]
Associate a vector with its source text:
```graphql
node Document {
slug: String @key
body: String
embedding: Vector(1536) @embed("body", model="openai/text-embedding-3-large") @index
}
```
The vector is the embedding of the record's `body` value exactly as stored,
and a text `nearest($d.embedding, $q)` embeds `$q` exactly as given. Neither
side adds a type name, a field label or any other template, and neither trims
whitespace, so a record whose `body` equals the query text embeds the same
input. The one difference a provider can see is the retrieval role: Gemini
receives `RETRIEVAL_DOCUMENT` for stored text and `RETRIEVAL_QUERY` for a
query, and the other providers receive identical requests. Vectors you compute
yourself must embed the source value the same way to be comparable with text
queries.
When `model` is recorded, a text `nearest` query is rejected unless the active
provider resolves to exactly that model id. Changing the recorded source or
model is not an in-place schema migration; rebuild or re-embed the data instead.
`@embed` does not populate the property during a load, even with an embedding
provider configured. Supplied vectors are preserved, omitted nullable vectors
remain null, and missing required vectors refuse the load. Supply vectors in
input data or prepare seed files with the offline command.
Successful HTTP and CLI JSON load results include `embedding_generation`:
`"unsupported"` means the load touched a node type with an `@embed` property;
`null` means it did not. This describes the load capability, including when
every vector was supplied. Human CLI output prints the corresponding guidance.
## Offline file pipeline [#offline-file-pipeline]
`omnigraph embed` reads and writes JSONL files; it does not mutate a graph.
```bash
omnigraph embed --input raw.jsonl --output embedded.jsonl --spec embeddings.json
```
The spec names, for each node type, the vector property to fill and the one
property whose value it embeds:
```json
{
"dimension": 1536,
"types": {
"Document": { "target": "embedding", "fields": ["body"] }
}
}
```
`fields` must name exactly one property, the source the target's `@embed`
declares; a spec that lists none or several is refused. The command does not
read the schema, so keep `target`, `fields`, and `dimension` equal to the
`@embed` declaration and its `Vector(N)`. The source's string value is embedded
unchanged, as described under [Schema annotation](#schema-annotation). A
processed record whose source is absent, null, or blank gets no vector: whatever
its target held, including a null or an empty list, is removed and counted as
cleaned. A source value that is not a string is refused.
By default it fills missing vectors. Use `--reembed-all` to replace selected
vectors or `--clean` to remove them. `--type` and `--select` restrict the
records processed. A seed manifest can be supplied with `--seed` instead of
separate input, output, and spec paths; its `embeddings` entry takes the same
shape as the spec.
# Mutations and Loads (/docs/queries-search/mutations)
Mutation statements live inside a named `.gq` query. Run them with
`omnigraph mutate`.
```graphql
query hire($person_id: String, $company_id: String, $role: String) {
insert WorksAt {
from: $person_id,
to: $company_id,
role: $role
}
}
```
Edge endpoints use the reserved assignments `from` and `to`; their values are
the logical ids of existing endpoint nodes.
## Statements [#statements]
```graphql
insert Person { email: $email, display_name: $name }
update Person set { display_name: $name } where email = $email
delete Person where email = $email
```
A `where` on `update` or `delete` takes the same expressions as a read filter
(see [Boolean expressions and nulls](/docs/queries-search/queries#boolean-expressions-and-nulls)):
`and`, `or`, `not`, parentheses, the six comparisons, `starts_with`,
`contains`, `in`, `is null` and `is not null`. Its operands are the target type's
properties, named bare, the system fields `@id`, `@src` and `@dst`, literals,
parameters, and `now()`; `from` and `to` stay accepted for `@src` and `@dst`.
It keeps only the rows whose expression is true. On four `Knows` edges, `ab1`
and `ab2` from `a` to `b`, `ac`, and `db`:
```graphql
query exact() { delete Knows where @src = "a" and @dst = "b" }
```
removes `ab1` and `ab2` and nothing else: `affected: nodes=0 edges=2`. A
binding variable in a `where` is refused at compile time (`T14`); an
aggregate, search, or ranking call is `T44`, for example ``T44: `count` cannot
appear in a mutation where; a where compares the row's own properties,
parameters and now()``. A compound `where` runs under either `engine` setting.
Assignment values are constants: literals, parameters, `now()`, and
comparisons or Boolean operators over them, such as `adult: true or $flag`.
A constant's value is fixed per invocation: parameters and `now()` are bound
once before any retry, so a retried mutation computes the same value. It follows the read rules
for null: with `$flag` null, `true or $flag` is `true` and `false and $flag`
is `false`, and `$age > 30` with `$age` null is null. A null result assigned
to a nullable property writes null; on a `Blob` property it clears the cell.
`null` is a reserved word, so clear a Blob through a nullable parameter such
as `$content: Blob?`. Assigned to a non-nullable property a
null result is refused with a typed error that names the property, never
written as a default. A
property or system field in a value is refused, for example ``T45: `age`
cannot appear in an assignment value; assignments and binding matches are
constants per invocation``.
A mutation query may contain several inserts and updates, or several deletes,
but it cannot mix inserts or updates with deletes. Split that workflow into two
queries, or run the queries on a branch and merge when the combined result is
ready.
## Atomicity [#atomicity]
An effectful mutation query publishes one graph commit. All of its statements
become visible together, or none do. Separate mutation commands are separate
commits, even when they run consecutively.
With `--json`, a successful effectful mutation returns `commit` with the
exact `graph_commit_id` and commit metadata published by that attempt. A
successful mutation that matches no entities publishes no commit and returns
`"commit": null`. Load JSON responses use the same exact receipt.
For a multi-command workflow, use a branch as isolated staging. Earlier changes
remain committed on that branch; merging makes the resulting branch state
visible on the target in one atomic step. See
[Branches, Commits, and History](/docs/branching).
## Insert and update identity [#insert-and-update-identity]
* Inserting a node or edge with `@key` is an upsert by its derived id.
Inserting the same key again updates the existing row.
* Inserting a node without a key is a strict insert with a generated or supplied
id.
* Inserting an edge without a key is a strict insert with a generated or
supplied id; the same pair inserted twice is two edges.
* Key properties cannot be changed by an update. Edge types do not support
update at all: change a keyed edge by inserting its key again with the new
values (an upsert), and an unkeyed edge by delete and re-insert.
All declared value, uniqueness, endpoint, and cardinality constraints are
checked before publication.
## Bulk loading [#bulk-loading]
`omnigraph load` accepts newline-delimited JSON. One file can contain nodes and
edges of several types:
```jsonl
{"type":"Person","data":{"email":"ada@example.com","display_name":"Ada"}}
{"type":"Company","data":{"slug":"acme","name":"Acme"}}
{"edge":"WorksAt","from":"ada@example.com","to":"acme","data":{"role":"Engineer"}}
```
Here, `Person.email` and `Company.slug` are single-property String keys, so their
derived ids are exactly `ada@example.com` and `acme`. The edge uses those ids in
`from` and `to`.
A `Date` value is an integer day count since 1970-01-01 or a `YYYY-MM-DD`
string; a `DateTime` value is an integer millisecond count since the Unix epoch
or an ISO 8601 string. Any other JSON type (a float such as `19723.0`, a
boolean, an object) fails the load with `invalid Date value` or `invalid
DateTime value` naming the property. A `Date` string carries no time of day:
`"2024-01-01T02:00:00+05:00"` fails the load with `invalid Date literal`, as it
does in a mutation param, a `date(...)` literal, or a read filter; an instant
belongs in a `DateTime` property. A `DateTime` holds milliseconds: a string with
a non-zero digit past the third fractional digit, such as
`"2024-01-01T00:00:00.123456Z"`, fails the load with `invalid DateTime literal`,
as it does in a mutation param, a `datetime(...)` literal, or a read filter.
Trailing zeros, as in `.123000`, are accepted.
Choose the mode explicitly:
| Mode | Existing id | Use |
| ----------- | ------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| `append` | Fails with `key_conflict` | Add entities without replacing anything. |
| `merge` | Updates the existing entity | Upsert a batch. |
| `overwrite` | Replaces every node or edge type represented in the batch; types absent from the batch remain unchanged | Rebuild from a complete export or seed. |
For a keyed edge the existing-id column applies to its derived id: `append`
reports `key_conflict` on an already-committed pair, `merge` upserts it, and
a supplied top-level `id` must equal the derivation exactly.
```bash
omnigraph load --data batch.jsonl --mode merge graph.omni
```
One load request is one graph commit. Use `--branch --from ` to
create a missing review branch and load onto it in the same workflow.
Loads preserve supplied embeddings and do not generate them. Results report
`embedding_generation: "unsupported"` when a loaded node type declares
`@embed`, or `null` otherwise; see [Embeddings](/docs/queries-search/embeddings).
### CSV loading [#csv-loading]
The `load csv` statement reads one CSV object into one declared node or edge
type, with every cell typed by the `.pg` declaration. It is a GQ statement, so
it runs wherever a `branch create` statement runs: `omnigraph mutate -e`, a
`.gq` file, or `POST /mutate` with the request `branch` as its target (`main`
when omitted). It is the one CSV surface: `omnigraph load` reads JSONL only.
```graphql
load csv "s3://imports/person.csv" into node:Person mode append delimiter "|" header
load csv "s3://imports/knows.csv" into edge:Knows mode append delimiter "|" header columns (from, to, creationDate)
```
* `mode` is required: `overwrite`, `append`, or `merge`, with the meanings of
the table above.
* `delimiter ""` is one printable ASCII character; `,` when unspelled.
* `header` makes the first record name the columns: each name is the property
it feeds, and an edge file's first two columns are its endpoints whatever
the header calls them (`Person.id|Person.id` in an LDBC file).
* `columns (a, b, …)` names the property each column feeds, by position. It
is required without `header` and overrides a header's names. `from` and
`to` name an edge file's endpoint columns and are refused on a node file.
* An empty field is null; a null for a non-nullable property, a column the
type does not declare, a count of `columns` that differs from the source, a
malformed number, a repeated name, and an edge file without `from` or `to`
are refused before anything is staged.
Numbers are typed by the reader. `Bool` takes exactly `true` or `false`.
`Date` and `DateTime` cells are read as text and pass the checks above
before conversion: `2024-01-01T02:00:00+05:00` in a `Date` column and
`2024-01-01T00:00:00.123456Z` in a `DateTime` column fail the load exactly
as they fail a JSONL load, and a numeric offset without a colon, as LDBC
writes it (`2010-02-14T15:32:10.447+0000`), stores the instant. List, Vector
and Blob properties have no CSV encoding: a column mapped to one is refused
with the kind named; an unmapped one stores null when nullable.
Float values must be finite and representable by their declared `F32` or
`F64` type; NaN, infinity and overflow refuse the load. Each decoded batch
is held against the statement's `query_memory_bytes` allowance before it
is retained, and each staged-write input against `write_max_bytes`.
Node ids derive from the `@key` as in a JSONL load; an edge's `from` and `to`
are the canonical ids of its endpoints, so a single-property integer key is
written as its decimal value. The statement never forks a branch.
The source is a URI the engine reads under the graph's external Blob policy
([External Blob references](/docs/clusters/config#external-blob-references)):
`s3://` objects beneath a configured
base, and for an embedded run `file://` paths beneath an `embedded_only`
base. A graph with no policy denies every source; a URI outside every base,
an unsupported scheme, and an object over the
[`load_source_max_bytes`](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/settings.md) setting (default 32 MiB)
are refused before any byte is read, and the read is bounded by a 120-second
deadline. A refused source names `csv_source_bytes`; raising the setting in
the statement's `set` prefix (or the request's `settings` field) admits a
larger object. A direct `omnigraph mutate`
run installs an embedded-only base over a `file://` source's directory for
that statement, so the file is admitted by the same rule; a served run
refuses a `file://` source, since the server reads only what it may reach.
`explain` before a `load csv` statement prints the load's plan, the same
document a query's explain prints: the source, the one validation with each
check and its access path, and the write.
An overwrite runs that plan; append and merge run the keyed adapters.
It reads the source's metadata and no payload byte.
See [Explaining a load statement](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/explain.md#explaining-a-load-statement).
#### CLI and HTTP requests [#cli-and-http-requests]
```bash
omnigraph mutate --store graph.omni -e 'load csv "file:///data/person_0_0.csv" into node:Person mode append delimiter "|" header'
omnigraph mutate --server dev -e 'load csv "s3://imports/knows.csv" into edge:Knows mode append delimiter "|" header'
```
The CLI uses `--branch`; HTTP uses the request `branch`, with `main` as the
omitted default. The statement accepts no `name`, `params` or commit
precondition (`--if-commit` or `Omnigraph-If-Graph-Commit`). It requires
`change` on the branch. The response is a mutation's `ChangeOutput`: the
branch, `load csv` as `query_name`, loaded node and edge counts, and the commit.
```bash
curl -sS http://localhost:8080/graphs/knowledge/mutate \
-H 'Omnigraph-Http-Api: 0.13' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-d '{"query":"load csv \"s3://imports/person.csv\" into node:Person mode append delimiter \"|\" header","branch":"main"}'
```
The HTTP body carries the statement; the server reads its source under the
graph's `external_blobs.allow` policy, with server-safe bases only. Before
fetching, workload admission reserves the request's `load_source_max_bytes`
(default 32 MiB) plus the statement bytes against both global and per-actor
budgets, so a raised setting must still fit the server's own limits. This reservation lasts until the write
settles, including when the HTTP caller disconnects.
## Limits and conflicts [#limits-and-conflicts]
Insert/update mutations and incremental keyed loads are bounded to 8,192
entities per touched type. The [`write_max_bytes`](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/settings.md)
setting (default 32 MiB) bounds retained row data per type and across all
touched types. Blob payloads have a separate allowance of the same size,
including inline bytes, copied external bytes and carried update values.
Every version of a key written in one mutation counts, so an insert and an
update of the same row charge its Blob twice.
Keyed loads also bound their parser estimate independently. A strict JSONL
load retains its projected input check, `strict_input_arrow_bytes`.
A CSV load has no whole-statement input check.
It applies `write_max_bytes` to each staged-write input instead.
An overwrite stages the reader's 8,192-row batches, and a batch over the
allowance refuses as `staged_write_input_bytes`.
An append or merge keeps the keyed adapter's ceilings
(`retained keyed batch bytes per operation`, `keyed entities for
`,
`keyed entity bytes for
`).
The [`query_memory_bytes`](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/settings.md) setting bounds one
`load csv` statement's tracked memory as a whole: the retained source, every
decoded batch, validation state and the staging working set.
A statement the allowance cannot hold refuses with the `query_memory_bytes`
resource error before any fragment write.
A CSV load needs roughly the source plus twice its typed input under that
allowance.
The source object is bounded by `load_source_max_bytes` before it is read
(refused as `csv_source_bytes`).
The loader reads the object once per statement, under the size, etag and
version its metadata probe captured, and every reprepare attempt reuses those
bytes.
An object replaced between the probe and the read refuses as an
external-source error, and nothing is staged.
Raising `write_max_bytes` lifts `strict_input_arrow_bytes` and
`staged_write_input_bytes`; raising `query_memory_bytes` admits a larger
statement; raising `load_source_max_bytes` lifts `csv_source_bytes`. See
[Blob limits](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md#limits).
Deletes, including cascades, and overwrite loads collecting replaced IDs have
a separate `write_max_bytes` allowance per operation for those IDs, summed over all
touched types. Each removed ID is charged its UTF-8 length plus 24 bytes, so
the default allowance holds 671,088 IDs of 26 bytes, the length of a generated ID. A
delete and the edges it cascades to draw on the same allowance. An overwrite of
entities loaded without a `@key` and without an explicit `id` removes every
committed ID of that type, because those IDs are generated again on each load.
Oversized work returns a resource-limit error before its data is staged or
published. These checks do not bound total engine memory. The setting can
change each allowance from its 32 MiB default; a CSV overwrite stages its
input as a vector of reader batches, each bounded by the allowance, under
the statement's `query_memory_bytes`. Split larger inserts, updates, keyed
loads and deletes into explicit commits. An overwrite replaces each
represented type as one image and cannot be split: it keeps its bulk-input
behavior and remains subject to its removed-ID check.
Independent existing constructive datasets stage concurrently. The
`stage_write_concurrency` [session setting](/docs/queries-search/queries#session-settings)
controls that width for both Load and insert/update mutations (default 8,
range `1..=64`); it is `process` scope, so the server takes it from
`OMNIGRAPH_LOAD_CONCURRENCY` and a direct CLI run from that variable or from
`--set` on `load`, `ingest` or `mutate` (a JSONL input carries no `set` line),
and an invalid or `0` value refuses startup instead of running the default.
Delete staging remains serial. This affects preparation only—one
request still publishes exactly one graph commit.
A stale strict update, delete, or overwrite can return `read_set_conflict`.
Refresh the branch and retry deliberately. A `key_conflict` means an append or
strict insert found an existing id; it never silently becomes an upsert.
If a write returns `recovery_required`, do not immediately resubmit it. Reopen
the graph read-write or restart the server, then retry from a fresh branch head.
## Blobs [#blobs]
Blob assignments accept managed `base64:` data and, when allowed by graph
policy, external URI references. Ownership differs by load mode, and Blob bytes
count toward write limits. See the canonical [Blob guide](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md) before
loading them, and its [limits](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md#limits) for the bounds that apply.
## Conditional mutations [#conditional-mutations]
Use `--if-commit` when a mutation should apply only to the graph state that
the caller read:
```bash
# The JSON response includes the commit pinned for these rows.
omnigraph query find_person --query queries.gq --store graph.omni --json
omnigraph mutate update_person --query queries.gq --store graph.omni \
--params '{"name":"Ada"}' \
--if-commit --json
```
The condition compares the effective head of the target branch. Any
intervening commit invalidates it, including a commit that changed an unrelated
entity or type. On mismatch, nothing is written: the CLI exits with code 4
against a server and with code 1 on an embedded graph such as this `--store`
run, and JSON output contains `precondition_failure` with `expected` and
optional `actual` commit ids. Re-read the branch and decide again; do not blindly retry
the old mutation. The precondition is still checked when the mutation would
match no entities.
Use the `graph_commit_id` returned with the original query rows. Fetching a
head id afterward creates a race between the read and the precondition. For the
HTTP header and dedicated conditional routes, see the
[HTTP server guide](/docs/operations/server#conditional-mutations).
# Query Language (`.gq`) (/docs/queries-search/queries)
A `.gq` file contains named, typed queries. Read queries match graph patterns
and return columns; mutation queries use the same declaration form and are
covered in [Mutations](/docs/queries-search/mutations). A file may instead hold exactly
one statement, never beside a query declaration: a branch statement (`branch
create`, `delete`, `merge`, `list`, `diff`, or `merge-base`; see [Branches,
Commits, and History](/docs/branching)), a `show` statement (see
[session settings](#session-settings)), or an `explain` statement, which
answers the plan a read query would run under instead of its rows (see
[Explain](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/explain.md)). Any of them may open with
[session settings](#session-settings) lines.
```graphql
query engineers($title: String) @description("People with a title") {
match {
$p: Person { title: $title }
$p worksAt $c
}
return { $p.name, $c.name as company }
order { company asc, $p.name asc }
limit 50
}
```
Run an ad-hoc query with:
```bash
omnigraph query engineers --query queries.gq \
--params '{"title":"Engineer"}' --store graph.omni
```
## Declarations and parameters [#declarations-and-parameters]
```text
query ($required: String, $optional: I32?) { ... }
```
Parameter types use the schema scalar types. Every non-nullable declared
parameter must be supplied; omission fails before query execution. A trailing
`?` accepts `null` or an omitted value. `@description("...")` and
`@instruction("...")` attach metadata for clients that expose stored queries
as tools.
## Match patterns [#match-patterns]
A match binds nodes and follows typed relationships. For edge alternatives,
wildcard, bounded hops, direction and edge identity, see
[Traversal and match patterns](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/traversal.md). Omitted bounds always mean
exactly `{1,1}`; multiple hops require an explicit finite range.
### Boolean expressions and nulls [#boolean-expressions-and-nulls]
Filters combine with `and`, `or`, `not` and parentheses. `or` binds loosest,
then `and`, then `not`, then comparison, so `not $p.email is null` reads as
`not ($p.email is null)`. Comparisons do not chain: `$a < $b < $c` is a parse
error. `not {` stays the pattern negation. Operands of `and`, `or` and `not`
must be `Bool` (`T41`), as in
`($d.stage = "open" or $d.stage = "paused") and not $d.amount < $cut`.
A comparison (`contains` and `starts_with` included) with a null operand is
null, and `not null` is null. A filter or mutation `where` keeps only rows
whose expression is true, so `not ($p.age > 30)` skips rows whose `age` is
null.
Numeric comparisons choose their types before execution. A literal may use a
property's type when its value is exactly representable in that type; otherwise
both operands use a common numeric type. Parameters use their declared type,
regardless of the supplied value. An `F32` property storing `0.1` therefore
differs from the `F64` literal `0.1`, while `0.5` compares exactly. These rules
also apply to list membership and mutation predicates, whether a filter runs
in a scan or in memory. Mixed signed and `U64` values compare without losing
integer precision.
| `x` | `y` | `x and y` | `x or y` |
| ------- | ------- | --------- | -------- |
| `true` | null | null | `true` |
| `false` | null | `false` | null |
| null | null | null | null |
| `true` | `false` | `false` | `true` |
`x is null` and `x is not null` are always `Bool` and are the only null tests:
GQ has no `null` literal, so `$p.x = null` is a parse error.
`not ($p.age > 30) or $p.age is null` selects every row the comparison did
not. A parameter bound to `null` makes every comparison on it null. `is null`
refuses a `Blob`.
`and`, `or`, `not`, `is`, `null` and `in` are reserved words: none is a bare operand
or a return alias. `$p.and` after a dot and `and: 1` in an assignment or
binding match stay legal; `nothing` and `android` are ordinary identifiers.
A traversal names its edge bare or as a string, the one spelling for an edge
a reserved word names: `$i "in" $b` follows the edge `In`; `$i in $b` tests membership.
A mutation `where` takes the same expressions over the target type's
properties, `@id`, `@src`, `@dst`, literals, parameters and `now()`, never a
binding variable, aggregate or search call:
`delete Knows where @src = "a" and @dst = "b"`. Assignment values and binding
matches take constants evaluated once per invocation, such as
`adult: true or $flag`; a property or system field there is `T45`. See
[Mutations](/docs/queries-search/mutations).
### Correlated blocks [#correlated-blocks]
`not { ... }`, `exists { ... }`, `count { ... } op value` and
`sum(expr) { ... } op value` (also `min`, `max`, `avg`) match a pattern once per
outer row. Each block must read a variable bound outside it. The row is kept
when the aggregate comparison is true: `not` means `count = 0`, and `exists`
means `count > 0`. The right side is a literal, `now()` or typed parameter;
numeric bounds follow the [comparison rules](#boolean-expressions-and-nulls)
for the aggregate's result type. Its argument is a scalar expression in the
block's scope: numeric for `sum` and `avg`; numeric, `String`, `Bool`, `Date`
or `DateTime` for `min` and `max`, as in `return`. With no matches, `sum`,
`min`, `max` and `avg` satisfy no comparison; `count` is `0`.
Integer block `sum` accumulates exactly in 128 bits, refuses overflow of that
range, and rounds the total once to `F64` before comparison. For example,
`9007199254740993` and `-9007199254740992` sum to `1`, while a single value
`9007199254740993` rounds to `9007199254740992` and does not satisfy
`sum(...) { ... } > 9007199254740992`.
As in `return`, `avg` converts each non-null input to `F64` before summing
and dividing by the count. For those same two values, the average is `0`,
because the first value rounds before accumulation. `min` and `max` retain
the column's declared type, including the full `U64` range; comparing a `U64`
extremum with an `I64` parameter preserves both integer ranges. Row count and
column `count` use `I64` and refuse a count beyond its range; column `count`
skips null values.
The block narrows the rows before `order` and `limit`, so a paged listing
filtered by a relationship count is exact. A binding the block's traversal
connects to the outer row is reached through it, or the block runs once from a
node it binds by key if that is cheaper (`strategy` in [explain](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/explain.md));
a binding correlated only through a filter, or read by a text search, is
scanned. A single-hop, filter-free `count { ... }` over a directed, unbound
edge is answered from the graph index's degree. A bare aggregate in `match`,
`count($d) > 2` without a block, is refused: it names no row to group by.
```
query prolific($least: I64) {
match {
$a: Author
count {
$p: Post { published: true }
$a authored $p
} >= $least
}
return { $a.name }
order { $a.name }
limit 20
}
```
### Strings and lists [#strings-and-lists]
* `$x.tags contains "rust"` tests membership when `tags` is a list.
* `$x.number in $numbers` tests membership in a list parameter or literal
(`$x.number in ["A-1", "A-2"]`): `$numbers contains $x.number` with the
list, of the value's type (`T7`), on the right. On a matched binding's
property it filters that binding's scan. An empty list has no member: `in`
keeps no row, `not (... in [])` keeps the rows whose value is not null.
* `$x.title contains "graph"` tests exact, case-sensitive substring containment
when `title` is a String.
* `$x.title starts_with "Omni"` tests an exact, case-sensitive prefix.
`NULL` never matches these predicates. `%` and `_` are ordinary characters,
not wildcards. The predicates remain correct without an index; do not assume a
free-text String index accelerates exact prefix or substring filters.
Use [full-text search](/docs/queries-search/search) for tokenization, fuzzy matching, and
relevance ranking.
### System fields [#system-fields]
A node's identity and an edge's endpoints are system fields, read through
`@`-prefixed meta-fields that no user property can shadow: `$p.@id` is the
identity of any binding, `$e.@src` and `$e.@dst` the endpoints of a bound
edge. They work in filters, projections, and orderings (`$p.@id = $who`,
`return { $p.@id }`, `order { $p.@id asc }`) and answer on every graph,
whatever the stored column is called. In a mutation predicate, which carries
no binding, the bare form serves: `delete Person where @id = $who`,
`delete Knows where @src = $who`. A bare `id` (`$p.id`, `where id = ...`)
always names a user property called `id`; where none is declared it is the
unknown-property error, which names the meta-field. Edge inserts keep
addressing endpoints as `from` and `to`. `$e.@type` returns the bound edge's
canonical schema type name as a non-null `String`, including on ordinary named
traversals; it is synthesized query metadata and cannot be written. `@src` and
`@dst` preserve stored orientation, including when the query traverses in
reverse. In type position the same prefix names an engine-owned, read-only
type: `$o: @Operator` binds the rows of the [engine catalogue](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/catalogue.md).
## Return, order, and limit [#return-order-and-limit]
```graphql
return { $person.name, count($company) as companies }
order { companies desc, $person.name asc }
limit 20
```
Return expressions include variables, properties, literals, `now()`, and the
aggregates `count`, `sum`, `avg`, `min`, and `max`. Each gives one column named by
its alias or expression (`$p.name` gives `p.name`); duplicates are refused (`T25`).
Integer `sum` accumulates exactly in 128 bits and rounds once to `F64`; `avg`
uses a floating-point accumulator. `min` and `max` accept numeric, `String`,
`Bool`, `Date`, or `DateTime` columns and keep their type (false before true,
dates chronologically). With no matches, an aggregate-only return gives one row
(`count` 0, the rest null); a return that also projects a group value gives no rows.
A bare node returns one object with `@id` and every property except `Blob` and
`Vector`: `$p` produces column `p`; project a property for one field. `count($p)`
counts rows and `count(distinct $p)` distinct nodes; `count(distinct $p.city)`
counts distinct non-null values, so a node reached by several paths counts once.
Other aggregates require a property (`T8`) and refuse `distinct` (`T57`).
A comparison or Boolean expression requires an alias (`T43`), such as
`$p.age > 30 as adult`; its result is `Bool`, or `Bool?` with a nullable operand,
and `is null` is always `Bool`. In an aggregated return its operands are
aggregates, literals, `now()` and parameters, evaluated per group after
aggregation: `count($p) > 0 as any` is `Bool?` because `count` is `I64?`. A group
value beside an aggregate is refused (`T58`), any other Boolean item too (`T9`).
`return distinct { $p.name }` keeps one row per complete projected tuple.
Repeated traversal matches and equal values on different entities collapse;
nulls compare equal. Every projection is a key, including comparisons, and
bare nodes deduplicate by identity. Deduplication follows `match` and precedes
ordering and limiting, so the limit counts distinct rows. Aggregates with
`distinct` are refused (`T56`). Except for a leading search function, DISTINCT
order keys must be projected values or aliases, including property keys
(`T42`). Remaining projected columns break ties ascending; no identity tiebreak is added.
For ordinary returns, explicit order is total: entity ids break ties only when
they can change the visible order, and no id is read when every projection is a
key. Ascending puts nulls first, descending last. `nearest(...)` requires a limit.
On ungrouped, non-DISTINCT rows, property and system-field order keys need not be projected;
other keys, except the leading search function, must be return aliases or
expressions written in return (`T42`), like `count($d)` after `return { count($d) as deals }`.
For ordinary returns, `nearest(...)` orders by ascending distance and `bm25`
and `rrf` by descending score, including through traversals, ranking the binding named whichever binding `match` declares first ([details](/docs/queries-search/search#ranking-a-traversal-reached-binding); `rrf` arms on two connected bindings: `T59`).
The search function must lead; subsequent keys precede the identity tiebreak. Projected
scores must repeat that leading function (`T33`); the [Search guide](/docs/queries-search/search)
defines their names and types. Rank expressions under aggregates (`T32`) and
search predicates in return (`T35`) are refused.
For aggregate or DISTINCT returns, the leading search function selects the
input population; it does not order the output. Subsequent keys order output
rows, followed by every group or DISTINCT key ascending. Without subsequent
keys, output is unordered. Aggregates cannot project scores (`T9`). DISTINCT
uses the selected nearest population without retry overfetch, so duplicates
can leave fewer rows than the limit. BM25 reads every matching entity before
the final limit.
## Blobs [#blobs]
Blob properties are not ordinary read-query values. They cannot be projected,
filtered, ordered, or passed to an aggregate. Read one logical Blob cell with
the CLI or HTTP Blob endpoint described in [Blobs](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md). Blob parameters
remain valid for mutation assignment.
## Branches and historical reads [#branches-and-historical-reads]
Reads default to `main`. Select another branch or an immutable commit with
`--branch` or `--snapshot`:
```bash
omnigraph query engineers --query queries.gq --branch review \
--params '{"title":"Engineer"}' --store graph.omni
```
When the snapshot has an effective graph head, `omnigraph query --json`
includes its `graph_commit_id`, pinned with the returned rows. Use that
same-snapshot id with a later mutation's `--if-commit` option when implementing
read-modify-write. On a newly created, unmodified branch, it is the head
inherited from the source branch and is valid for the branch's first
conditional mutation.
See [Branches, Commits, and History](/docs/branching).
## JSON result spelling [#json-result-spelling]
JSON `rows` follow Arrow's JSON conventions: OmniGraph writes them with the
`arrow-json` writer from the result batches and keeps no per-type spelling of
its own. The spellings a consumer sees:
* A null cell's key is omitted from its row, and from a struct cell; a null
element inside a list value stays `null`.
* `Date` is `"2024-01-01"`; `DateTime` is `"2024-01-01T12:34:56.789"` in UTC
with no `Z`, and no fractional part when it is zero.
* Integers of every width are bare numbers; JavaScript's `JSON.parse` rounds
values beyond 2^53.
* `F32` prints at 32-bit width (`0.99`) and `F64` at 64-bit width; integral
floats carry `.0`; magnitudes from 1e10 up or below 1e-5 take exponent form
(`1.0e20`, `1.0e-7`); a non-finite computed value is `null`.
* `Vector(N)` and list properties are JSON arrays.
On input, a JSON number for an `F64` parses to the nearest `F64` value, the one
a GQ literal with the same digits names, so an `F64` value read from `rows`
loads back unchanged. A `Date` string is a calendar day, `"2024-01-01"`; a
string that carries a time of day, such as `"2024-01-01T02:00:00+05:00"`, is
refused as a load value, a param, or a `date(...)` literal, and an instant
belongs in a `DateTime` property. A `DateTime` holds milliseconds: a string
with a non-zero digit past the third fractional digit, such as
`"2024-01-01T00:00:00.123456Z"`, is refused as a load value, a param, or a
`datetime(...)` literal; trailing zeros, as in `.123000`, are accepted.
A `Date` or `DateTime` count outside the range the writer can format is refused
on load. A read that meets one fails with status 500; the error names the
column, the result row, and the count, and an `update` of that row repairs it.
## Session settings [#session-settings]
Use `set`, `reset`, and `show` to configure query execution. See
[Session settings](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/settings.md) for syntax, scopes, defaults, and CLI/HTTP usage.
## Linting [#linting]
Validate queries without running them; a refusal reports a stable code, its
position or stage, the expectation and one fix ([Diagnostics](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/diagnostics.md)):
```bash
omnigraph lint --query queries.gq --schema schema.pg --json
```
`Q000` identifies a file the parser refused, at `line , column `; a
[branch statement](/docs/branching) where declarations were expected
and a [settings line](#session-settings) naming an unknown setting or a value
outside its row report the same way. `L201` warns when a nullable property is
never set by any update query in the inspected set. Type errors report the
affected query and their `T…` code. The command exits nonzero when the overall
status is an error.
For every query that compiles successfully, JSON output includes an
`operation` descriptor:
* `result` lists projected fields in return order. Each field has `name`,
`kind`, and `nullable`; list fields also have `item_kind`, and vectors
have `vector_dim`.
* `reads` conservatively lists every node or edge type the query may inspect.
* `writes` lists every node or edge type the query may change and is empty
for a read query.
Read and write entries are sorted, deduplicated objects with `kind`
(`node` or `edge`) and the case-sensitive `type_name`. Result kinds use
the spellings `string`, `bool`, `int`, `bigint`, `float`, `date`,
`datetime`, `blob`, `vector`, `list`, and `object`. A parse or type
error has no `operation` descriptor.
A mutation target appears in both `reads` and `writes`. An edge insert also
reads its endpoint node types, and a node delete includes incident edge types
that its cascade may remove.
# Search (/docs/queries-search/search)
OmniGraph combines vector, full-text, and graph patterns in one `.gq` query.
Search expressions can filter or order a matched node set; a `limit` is required
for nearest-neighbor ordering.
## Functions [#functions]
| Function | Meaning |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `nearest($d.embedding, $q)` | Rank vectors by L2 distance. `$q` may be a vector or text that the configured embedding provider converts to a vector. |
| `search($d.body, $q)` | Full-text token search. |
| `fuzzy($d.body, $q [, max_edits])` | Full-text search with edit-distance tolerance. |
| `match_text($d.body, $q)` | Match a full-text query in a `match` block. |
| `bm25($d.body, $q)` | BM25 relevance score. |
| `rrf(rank_a, rank_b [, k])` | Fuse two rankings with Reciprocal Rank Fusion. The default `k` is 60. |
Search arguments use their declared types. `fuzzy` defaults `max_edits` to 2
when omitted; a supplied value must be an integer from 0 to 4,294,967,295.
`rrf` defaults `k` to 60 when omitted; a supplied value must be an integer
from 1 to 4,294,967,295. A supplied null is an error, including a nullable
parameter that was omitted from the parameter map. Query text and vectors
must also be non-null; vector elements must convert to finite F32 values.
Filters in the `match` block are applied before ranking, so `limit 10` means the
top ten matches that satisfy the graph and property filters.
A `bm25()` ordering reads every matching entity before applying the final
limit. Equal scores are then ordered by entity identity, including when the
smallest identity lies beyond the first search candidates.
Full-text rankings inside `rrf()` are
never bounded this way: each full-text arm scans every matching entity, and
fusion ranks the entities that satisfy the graph and property filters, so
bounding an arm could silently drop an entity's contribution and shift fused
results. When a traversal constrains the ranked variable and the graph shows
few entities could satisfy it, the full-text arms instead rank only those
entities (an unbounded, index-served prefilter — results are identical, the
scan is just smaller); broad traversals keep the full scan. A `nearest()`
ranking inside `rrf()` is inherently top-k, as vector
search always is: an entity outside its window adds no vector contribution
to its fused score, so a traversal that drops the window's top matches can
shift fused ranks.
## Vector search [#vector-search]
```graphql
query similar($q: Vector(4)) {
match { $d: Document }
return { $d.slug, $d.title }
order { nearest($d.embedding, $q) }
limit 10
}
```
Raw vectors are ranked with L2 distance. Vectors produced by OmniGraph's
embedding client are normalized, so L2 and cosine similarity produce the same
ordering for those generated vectors. See [Embeddings](/docs/queries-search/embeddings) for text
queries and provider configuration.
IVF vector searches keep Lance's adaptive one-partition minimum and cap the
partitions a `nearest` scan reads at 20 per index delta by default. The cap
prevents Lance's centroid-distance heuristic from expanding a small search
across the entire index; larger values can improve recall at the cost of
latency and object-store I/O.
A capped scan that returns fewer candidates than requested with partitions
left unread is rerun with the cap raised four-fold, then without a cap, until
the candidates are found or the index is exhausted. A scan that ended short
for any other reason (every matching row found, the whole type read) is not
rerun. When a traversal or a filter above the scan drops candidates and
leaves `limit` unfilled, the query asks the scan for four times, then
sixteen times, the requested candidates; a query whose survivors are rarer
than one in sixteen of the nearest candidates then runs one exact pass over
the whole type (every row with an embedding ranked, no cap), taken sooner
once a rung would cover the whole type anyway, so `limit` is
filled whenever that many survivors exist; a query whose survivors are permanently fewer than
`limit` pays that whole-type pass on every execution. A `nearest` ordering constrained by a traversal is first restricted
to the entities that can satisfy the traversal's first hop when few entities
can (the same gate `rrf()` uses), and ranks only those. A `nearest()` arm
inside `rrf()` is a top-k window: the arm's scan widens its own cap, but a
traversal that drops the arm's rows shortens the fused answer. As with every
IVF search, a full candidate count does not make the ANN ranking exact; the
cap remains a recall/latency tradeoff.
| Setting or variable | Meaning |
| ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ann_nprobes` ([session setting](/docs/queries-search/queries#session-settings), `request` scope; process default `OMNIGRAPH_ANN_NPROBES`) | Partition cap per index delta of a `nearest` scan; default 20, `0` removes the cap, an invalid value refuses startup |
| `OMNIGRAPH_RRF_GATE_RATIO` | Fraction of the ranked type below which a traversal-constrained `nearest` or `rrf()` prefilters its scan; default 0.10, `0` turns the gate off, an invalid value is the default |
| `OMNIGRAPH_RRF_GATE_MAX_IDS` | Largest eligible set the gate pushes into the scan; default 100000, `0` turns the gate off, an invalid value is the default |
| `rrf_plan` ([session setting](/docs/queries-search/queries#session-settings), `process` scope; process default `OMNIGRAPH_RRF_PLAN`) | `auto` (default), `force_prefilter`, or `force_postfilter`, for diagnosis. On a traversal-constrained `nearest`, `force_postfilter` can leave `limit` unfilled and `force_prefilter` ranks the eligible entities regardless of the size threshold |
## Full-text search [#full-text-search]
Use full-text functions for token search, fuzzy terms, and relevance. Use the
query language's exact `contains` and `starts_with` predicates for literal,
case-sensitive substring and prefix matching.
A full-text function needs a full-text index on its property: a one-column
`@index` or `@key` on a free-text String property of a node. The index's
analyzer folds case and stems, so `"deep"` matches `"Deep Learning"`. A call
on any other String property is refused at type checking (`T27`); for an
enum, compare its values with `=` instead. The check reads only the schema.
`search`, `match_text`, and `fuzzy` are positive predicates in `match`: use
a standalone call or compare the call to literal `true` with `=`. Other
comparisons, including `= false`, `!= true`, a Boolean parameter, and a
call on the right side, are refused at type checking (`T38`).
```graphql
query relevant($q: String) {
match { $d: Document }
return { $d.slug, bm25($d.body, $q) as score }
order { bm25($d.body, $q) desc }
limit 10
}
```
`score` is the relevance the ordering used, one computation observed twice:
the projected `bm25(...)` must repeat the leading `order` key (`T33`), and
`nearest(...)` projects its distance the same way; that distance is the
squared L2 distance Lance ranks by. Without an alias the column is
`d._score` or `d._distance`. `rrf(...)` projects the fused score under the
same rule, an `F64` named `d._rrf` after its first arm's binding without an
alias. A `nearest(...)` or `bm25(...)` that appears only as an `rrf` arm is
refused (`T33`), and so are the predicates `search(...)`, `fuzzy(...)` and
`match_text(...)` (`T35`). A relevance ranking orders most relevant first:
`asc` on a `bm25(...)` or `rrf(...)` order key is refused (`T54`), as any
modifier on `nearest(...)` is at parse time.
Exact String predicates remain correct without an index. A free-text index does
not accelerate equality, `starts_with`, or literal substring `contains`.
## Hybrid ranking [#hybrid-ranking]
Reciprocal Rank Fusion combines rankings without assuming their raw scores use
the same scale:
```graphql
query hybrid($vector: Vector(4), $text: String) {
match { $d: Document }
return { $d.slug, $d.title }
order { rrf(nearest($d.embedding, $vector), bm25($d.body, $text)) }
limit 10
}
```
Ranking order is a contract, not a side effect: search-ordered results are
sorted on the search score itself, including through multi-hop traversals, with
secondary keys and the entity-id tie-break applied after the score. A fused
order sorts every row by the fused score, then the keys written after
`rrf(...)`, then every binding's id, and `limit` counts rows after that order;
an entity that fans out to several rows contributes each of them. The full
ordering contract lives on the [queries page](/docs/queries-search/queries).
## Ranking a traversal-reached binding [#ranking-a-traversal-reached-binding]
A search order may rank any binding of the pattern, whichever binding the
`match` block declares first, and whether or not the ranked binding is
declared at all:
```graphql
query passages($matter: String, $q: String) {
match {
$r: SourceRevision { matter_number: $matter }
$r hasPassage $p
}
return { $r.path, $p.locator }
order { bm25($p.text, $q) }
limit 5
}
```
The ranking starts the traversal: the query scans every `Passage`, ranks it
and keeps the passages the traversal reaches from the selected revision, in
score order. Ranking a destination therefore costs a scan of the ranked type,
however few entities the other end reaches. A `nearest()` ranking ranks the
reached entities, never a global top list that the traversal later cuts.
An `rrf()` whose two arms rank two different bindings one traversal connects,
directly or over several hops, is refused at type checking (`T59`): a ranking
starts the traversal at the binding it ranks, and one traversal cannot start
at both. Rank one binding in both arms, or bindings no traversal joins.
## Indexes [#indexes]
`@index` and `@key` declare index intent. For a single-property node declaration,
OmniGraph currently creates:
| Property | Index use |
| ---------------------------------------- | --------------------------------------------- |
| Enum, number, Boolean, Date, or DateTime | Equality, range, membership, and null filters |
| Free-text String | Full-text functions |
| Vector | `nearest` |
Node ids and edge ids/endpoints are indexed automatically. Lists and Blobs do
not receive property indexes. Composite declarations and edge-property
declarations do not currently create property indexes.
Indexes are derived performance data. A new declaration may still be pending,
and newly written entities may fall outside existing coverage. Queries remain
correct by scanning missing or uncovered data: full-text functions match the
scanned rows with the index's analyzer, and vector search falls back to an
exact scan when needed. Every write records a declared full-text index's
analyzer before its postings are built. A table version written by an older
release may hold rows and no part of the index at all; a full-text call on
it is refused before any scan with HTTP `409` and a
`full_text_index_required` detail naming `Type.property`, never answered
with a case-sensitive scan. Build the index to lift it: `omnigraph optimize`
on `main`, or `omnigraph rebuild-full-text-indexes --branch ` on
another branch. Historical snapshots of that version stay refused. Run:
```bash
omnigraph optimize graph.omni
```
after a large load or merge, and on a regular maintenance cadence, to refresh
coverage and compact data. An empty vector property remains pending until it has a
non-null vector to index.
# Maintenance (/docs/operations/maintenance)
OmniGraph provides four direct-storage maintenance commands:
* `optimize` compacts data and reconciles declared indexes.
* `rebuild-full-text-indexes` replaces full-text indexes on one branch.
* `repair` diagnoses foreign storage drift without adopting it.
* `cleanup` permanently removes unretained table versions and unused table
forks left by branches created before storage format v11.
They do not run through the HTTP server. Address a standalone graph directly,
or select a graph from a cluster root:
```bash
omnigraph optimize ./graph.omni
omnigraph optimize --cluster s3://company/omnigraph --graph knowledge
```
Follow the command-specific concurrency requirements below. Azure writers
must also run through `omnigraph-azure-admission`; Azure support remains a
qualification preview pending the adversarial live-Azure matrix.
## Optimize [#optimize]
```bash
omnigraph optimize ./graph.omni
omnigraph optimize ./graph.omni --json
```
Optimize rewrites small fragments into fewer larger fragments, rewrites every
fragment that still stores a dropped property's values, rebuilds each scalar or
vector index whose coverage lags behind appended rows, and builds missing
declared indexes that are ready to build. It does not delete old versions or
collect unused table forks. Use `cleanup` for storage reclamation.
Dropping a property removes it from the schema without rewriting its table
(see [schema changes](/docs/schema#schema-changes)), so its values stay
in the table's data files. Every optimize rewrites each fragment that still
holds them, whatever its size, copying only the remaining properties; the
dropped values are not read. A `cleanup` then deletes the old files, Blob
values included, once nothing it retains references them: no commit its
retention keeps, no live branch created from a commit before that optimize,
and no tag naming such a commit. Merging a branch does not release its files;
deleting it does. A table holding no dropped values and nothing else to
compact is left alone.
Each table's work is staged as detached Lance versions of the table's current
pin and published in one graph commit, like any other write. A run
that fails before that commit leaves the graph unchanged and no recovery
state; after it the published pins are the tables' versions, with nothing
left to finish. Optimize runs beside
live writers: a write that lands on a table while its compaction is staged
fails the run with a read-set conflict, and the next run re-plans from the
new state. An update or delete prepared before an optimize published reports
the same conflict and is retried by its caller. Lance compacts neighbouring
fragments only when the same indexes cover them, so a table whose index
coverage was uneven before a run may coalesce fully only on the next run,
after the rebuilt coverage is in place.
Compacting a table with Blob properties reads each managed Blob value into
memory to rewrite it. Optimize sizes the read batches of each compaction task
(a group of neighbouring fragments compacted together) from the task's largest
row, summing that row's managed Blob values, so one batch materializes at most
32 MiB of managed Blob payload whatever the fragment sizes. External Blob
references are carried without reading the referenced object and count
nothing. A task holding a row over 16 MiB compacts one row per batch, and a
single row over 32 MiB is materialized whole. This bounds the payload of a
batch, not the process heap: Lance's writer copies each inline payload into
the arrays it prepares while it still holds the batch, so memory use exceeds
the payload. Every task of a Blob table gets this batch size explicitly, 1 to
8192 rows, so it takes precedence over `LANCE_DEFAULT_BATCH_SIZE` there,
including when the variable is smaller. When a fragment being compacted holds
a Blob descriptor the engine's decoder rejects, sizing can refuse the
compaction with a Blob integrity error before that table is rewritten, and
nothing is committed; a table with nothing to compact is not scanned.
Optimize works on up to
`OMNIGRAPH_MAINTENANCE_CONCURRENCY` tables at once (default 8), so budget for
that many Blob tables compacting together.
Optimize also persists the traversal-adjacency artifact
(`__graph_index/csr-current.bin`), which cold traversal builds load instead of
scanning every edge dataset. The artifact is derived and regenerable: optimize
is its only writer, every load is verified and falls back to the in-memory
build, and a failed write is a warning, not an optimize error. The artifact is
rewritten by optimize runs that advance an edge dataset; on a store with no
edge work it is left as-is, and a stale copy costs only the artifact's speedup,
never correctness.
Existing full-text indexes are preserved rather than incrementally merged.
Unindexed rows remain searchable, but a growing uncovered tail can cost more to
scan. Use `rebuild-full-text-indexes` to refresh that coverage or migrate an old
analyzer generation. Optimize reports uncovered full-text coverage under
`pending_indexes` with this remedy. Deferred coverage alone creates no graph
commit or maintenance work.
A vector index whose property has no usable vectors remains pending rather than
failing the run. Run optimize again after loading or generating vectors.
## Rebuild full-text indexes [#rebuild-full-text-indexes]
```bash
omnigraph rebuild-full-text-indexes ./graph.omni --branch main
omnigraph rebuild-full-text-indexes --cluster s3://company/omnigraph \
--graph knowledge --branch review --json
```
The command fully replaces text-search indexes from current entities using the
engine's default English analyzer, including already indexed entities. It builds
declared node full-text indexes and replaces existing physical full-text indexes
on nodes or edges; it does not create edge-property indexes from declarations.
External custom tokenizer settings are not preserved. Completed rebuilds warn
on stderr, or in the JSON `warnings` array, that those settings were replaced.
All rebuilt datasets become visible in one graph commit. JSON reports `branch`,
`graph_commit_id`, and `rebuilt_indexes` with each `type_key` and `property`.
Success means the selected branch's planned rebuild was published, or an explicit
no-op with an empty index list, null commit, and empty `warnings` array. The default
branch is `main`.
`--as` supplies actor attribution for this command. Direct CLI access, including
`--cluster`, does not load the server's Cedar policy; storage permissions are its
trust boundary. An embedded host that installs a policy checker also enforces
the Change permission before rebuilding.
Other branches and historical snapshots are not rewritten. Rebuild every live
branch that needs full-text search; restoring an older snapshot may require
rebuilding again. The operation does not regenerate embeddings or alter entity
values. Stop overlapping writers and preserve a backup before an upgrade; see
[full-text upgrades](/docs/operations/upgrade#full-text-index-upgrade).
Unknown legacy or external index kinds require a controlled migration, not a
guessed replacement; see [unsupported inventory](/docs/operations/upgrade#unsupported-index-inventory).
## Repair [#repair]
Repair reports, per node or edge type, how the backing dataset's Lance linear
history stands against the graph's registration. Preview:
```bash
omnigraph repair ./graph.omni --json
```
Every OmniGraph table write is a detached Lance commit that a graph commit
pins, so a table's linear history ends where it was created, and the
registration records that point as its last linear version
(`omnigraph.last_linear_version`; `1` for a table created under storage
format v11). A graph table is classified `no_drift` when its linear HEAD
equals that version and `foreign_drift` when linear commits sit above it. A
foreign commit came from something other than OmniGraph writing the table
directory; no read or write of the graph resolves the linear HEAD, so
foreign drift changes no query result and blocks no writer. For a
`foreign_drift` table `repair` prints the last linear version, the HEAD and
the number of foreign versions in `operations`, takes no action (`no_op`)
and exits 0. It never adopts the foreign commit. `cleanup` leaves foreign versions and their files in
place and lists them per table under `foreign_versions`. A HEAD below the
recorded last linear version is reported as an internal manifest error.
Each `repair --json` row carries `type_key`, `published_dataset_version`,
`lance_head_version`, `classification`, `action`, `operations` and `error`.
The removed `--confirm` and `--force` options are rejected. To discard foreign commits,
export the graph and load it into a new one; see
[Troubleshooting](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/troubleshooting.md#foreign-drift).
## Cleanup [#cleanup]
Cleanup is a tracing collector over the graph's table storage. Every table
write is a detached Lance commit that a graph commit pins, so what a table
keeps is decided by which graph commits the run retains. Per live branch
the run computes the graph commits its policy keeps, takes the table
versions those commits pin as roots, marks every file the root manifests
reference (base and overlay data files, deletion files, index directories,
transaction files), and deletes unretained table versions and the files only
they reference. Selected merge bases and native tagged snapshots are retained
in addition to the versions selected by the policy.
Historical table lifetimes participate even after a type is dropped or its
name is re-added under a new identity. Blob sidecars follow their parent data
file's reachability.
A table version no graph commit ever named is unpublished staging: a write
in flight, or one that failed before its publication. Unless a native tag
protects it, it is deleted when the branch incarnation and graph head its
commit recorded can no longer be published against: its valid incarnation is absent from a complete inventory taken
after the staged manifests were listed, or the captured head moved past its
publication without naming it. Missing or malformed ownership is kept.
Cleanup validates its complete branch and tag capture after all table
inventories; an overlapping change refuses the plan before deletion. Cleanup also removes table forks
left by branches created before storage format v11 once nothing references
them and the graph branch incarnation in their name is gone. Branch deletion
and later writes leave reclamation to this command. Without `--confirm`, the
CLI only echoes the requested retention policy and exits before opening the
graph; it does not enumerate candidates:
```bash
omnigraph cleanup --keep 10 --older-than 7d ./graph.omni
```
Run the reviewed policy with `--confirm`:
```bash
omnigraph cleanup --keep 10 --older-than 7d --confirm ./graph.omni
```
At least one retention option is required:
| Option | Meaning |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--keep N` | Retain the newest `N` graph commits of every live branch, and every table version they pin |
| `--older-than DURATION` | Also retain every graph commit newer than the cutoff, on every live branch; and collect a pre-v11 table fork only once every data and branch-reference object of the fork is older than the cutoff |
A graph commit survives when either option retains it. The current HEAD of
every live branch is always retained, and a named branch also keeps its
oldest commit, the copy of the parent's commit it was created from, while
the branch lives. Cleanup also retains the selected merge bases between
current branch heads and accepted merge inputs. Native tags protect the exact
snapshots they name. A tagged graph snapshot retains all its table versions; a
tagged table snapshot retains that version, including a detached version or a
version on main.
`--keep` counts graph commits on that branch, never Lance versions of a
dataset: `--keep 10` keeps the last ten graph commits of each live branch
readable, with every table version one of them pins, whatever their age. A pre-v11 fork that no
registration references is collected once the graph branch incarnation in
its name is gone: immediately with `--keep` alone, and with `--older-than`
only after every object of the fork is older than the cutoff. Native tags,
readable graph snapshots and required native ancestors also protect forks. Choose a policy that matches
your rollback and audit needs.
For `s3://` and `az://` targets, destructive execution also requires an
interactive confirmation or `--yes`. Non-interactive and JSON runs refuse
without `--yes`.
Before cleanup:
1. stop long-lived Blob readers and readers of snapshots the policy removes;
2. verify important branches and snapshots;
3. make or verify a backup/export;
4. review the retention policy against the snapshots and branches you still
need;
5. review the exact retention command and confirmation target.
Cleanup fails closed per table: a trace that does not finish (a pinned
version missing from the table's version listing, or a read that failed)
deletes nothing for that table and reports why in its result row; fix the
cause and rerun cleanup to converge. Cleanup deletes a table version's unused
files before the version itself. If a run stops between the two steps, the
next run finds that version again, counts the files already gone as removed,
and deletes the version. The run derives each branch's history
and pins from one captured version, then validates that the live inventory,
versions, incarnations and tags still match after all table inventories.
A changed observation refuses the whole plan before deletion; rerun with the
same policy. This includes a branch deleted after cleanup listed it: cleanup
returns a retryable conflict, and a new run captures the current branches.
Retirement archives exact branch identity and ancestry inside
its native tree, then removes the active ref. Creation does not scan retired
histories. Cleanup removes unneeded retired trees and their archives, while
preserving trees needed by live descendants, merge bases or tags.
Ordinary graph writes and branch merges may overlap cleanup. A merge protects
its accepted source, target and base snapshots with durable native tags;
source advancement does not revoke those inputs. A cancelled merge or one
whose publication returns an in-doubt error keeps its tags until the target
witness proves it can no longer publish. An unchanged or unresolvable target
keeps them. Graph-branch creation/deletion and cleanup retain the existing
single-writer-process control boundary.
Legacy native snapshots can borrow another location's files. Cleanup retains
those physical origins, including overlay and index files. An incomplete origin
trace prevents physical-file deletion across the run. Unregistered native trees
can retain extra origin files until a later cleanup after those trees vanish.
Ordinary writes retain unpublished staging, because the branch head its commit recorded is still the
branch's head. A write that publishes during the run either landed before
the snapshot, in which case its version is a root, or after it, in which
case its staging is kept for the same reason. Staging left by a write that
failed before publication is kept until that branch publishes again or is
deleted; on a branch that never publishes again it stays, and its table's
result row counts it under `unpublished_manifests`. Age alone never decides
anything: an old staging is not proof that its writer stopped, and an old
pinned version is never removed while a retained graph commit names it.
Files no listed manifest references follow Lance's seven-day unverified-file
rule, including temporary manifests. Each orphan Blob sidecar must satisfy
that age rule itself, even when its parent data file is older.
Each table's result row (under `--json`) carries `bytes_removed`,
`manifests_removed` (table versions deleted: published ones no retained
graph commit names, and dead staging), `unpublished_manifests` and
`unpublished_bytes` (the staging the run found, dead ones included),
`foreign_versions` (Lance versions above the table's last linear version,
which the run never deletes) and `error`, set only when the table's trace
did not finish, in which case its counts are zero. `old_versions_removed`
stays for one release beside `manifests_removed` and carries the same
number; nothing defers a table, so the row has no `deferred` field.
Exit 0 means every table was visited and that nothing a retained graph
commit pins, and no file such a commit references, was removed. It does not mean every table was collected: read the result rows
for tables with an `error`.
## Suggested cadence [#suggested-cadence]
* Run `optimize` after large loads or on a regular cadence for write-heavy
graphs.
* Run `repair` when you want to know whether something outside OmniGraph
committed to a table's Lance history.
* Run `cleanup` from an explicit retention policy after backups and rollback
requirements have been reviewed.
Moving a graph to another storage format is not maintenance. See
[Upgrading](/docs/operations/upgrade).
# Authorization and actors (/docs/operations/policy)
OmniGraph uses Cedar policy bundles to authorize graph, server, and cluster
configuration actions.
Policies are declared in `cluster.yaml` and installed during cluster bootstrap
or graph creation. Servers load them at startup or when activating a new graph.
## Actions [#actions]
Graph-scoped actions:
| Action | Covers |
| ---------------- | -------------------------------------------------------------------- |
| `read` | Queries, snapshots, branches, and commits |
| `export` | Snapshot export |
| `change` | Mutations, loads, and source metadata probes for `explain load csv` |
| `schema_apply` | Schema changes |
| `branch_create` | Branch creation |
| `branch_delete` | Branch deletion |
| `branch_merge` | Branch merge |
| `commit_publish` | Publication of an unpublished commit onto its recorded target branch |
| `commit_drop` | Drop of an unpublished commit |
| `invoke_query` | Entry to a stored query |
| `admin` | Reserved graph administration |
`graph_list` is server-scoped and controls the metadata catalog at
`GET /graphs`. It does not control the minimal identity-authenticated
`GET /graphs/discovery` route: every valid cluster identity can discover all
applied graph IDs and display names, without gaining permission to read their
schema or contents. See [signed data credentials](/docs/operations/server#signed-data-credentials).
`config_manage` is cluster-scoped. It authorizes configuration changes through
the identity-authorized cluster API, including stored queries, graph creation,
and policy membership/binding changes on existing graphs. Grants and revocations
are authorized by the currently applied policy, never the proposed policy.
Changing an existing
graph's schema requires that graph's `schema_apply` permission;
reading its remote schema or migration preview requires `read` on `main`.
The reserved graph `admin` action does not grant cluster management.
A stored mutation requires both `invoke_query` and `change`. A stored read
requires `invoke_query` and `read`.
## Bind a policy [#bind-a-policy]
```yaml
# cluster.yaml
policies:
graph-access:
file: graph.policy.yaml
applies_to: [knowledge]
server-access:
file: server.policy.yaml
applies_to: [cluster]
```
A bundle may target graph IDs or the `cluster` scope, but not both. Only one
bundle may bind a given scope. The target `og_catalog` binds the
[engine catalogue](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/catalogue.md), which every server serves and no
`graphs:` entry declares: its `read` rule decides catalogue reads with the
resource `og_catalog` and no branch, and the bundle is applied and activated
like a graph's.
Example graph policy:
```yaml
version: 1
groups:
readers: [act-alice, act-bob]
rules:
- id: readers-can-read
allow:
actors: { group: readers }
actions: [read]
branch_scope: any
- id: readers-can-invoke
allow:
actors: { group: readers }
actions: [invoke_query]
```
A cluster-bound bundle can contain separate configuration and inventory rules:
```yaml
version: 1
groups:
owners: [principal:alice]
rules:
- id: owners-manage-config
allow:
actors: { group: owners }
actions: [config_manage]
- id: owners-list-graphs
allow:
actors: { group: owners }
actions: [graph_list]
```
`config_manage` uses the Cedar `Cluster::"root"` resource; `graph_list` uses
`Server::"root"`. Put them in separate rules and do not give either a branch
scope. Bind this file with `applies_to: [cluster]` as above.
For an identity-authorized apply, the **current applied policy** authorizes all
planned effects before any graph or configuration effect. A proposed policy
cannot give its author permission to install itself. A saved plan does not
transfer its author's permissions: another caller must pass the current policy
checks. Explicit first initialization can install a declared initial management
policy under an exact bootstrap capability; missing policy on an existing
cluster is an error, not permission to bootstrap again. Existing storage-holder
cluster APIs keep their explicit trust boundary.
Graph rules may use `branch_scope` for a source branch or
`target_branch_scope` for a destination branch. Values are `any`, `protected`,
or `unprotected`; a rule may not set both. A branch is protected when the
bundle's `protected_branches` list names it exactly:
```yaml
protected_branches: [main, release]
```
Branch names are compared as written. A `protected_branches` entry that is
empty, has leading or trailing whitespace, or contains a control character
fails policy validation, and a request naming such a branch is refused as a
bad request before any rule is evaluated (see
[branch names](/docs/branching#branch-names)). Server actions and graph-wide
`invoke_query` rules do not take branch scopes. `commit_publish` and
`commit_drop` take `target_branch_scope`: the scope is the branch the
unpublished commit recorded as its target. Writing an unpublished commit
needs no new action: a mutation `as commit` is a `change` on its branch and a
merge `as commit` is a `branch_merge`, so an actor with ordinary write rights
can omit `as commit`; this is a voluntary review workflow, not an approval
boundary (see [Unpublished graph commits](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-10-10-unpublished-graph-commits.md)).
A mutation `as unattached branch` is likewise a `change` on the branch it
records as its target: the first write's branch, and for a write `on ""`
the target the base snapshot recorded, whatever branch the request names.
`commit list --unattached` and a query `--snapshot ` of an unattached
snapshot need `read` on that recorded target; `commit show ` keeps its
scope-free `read`; `commit drop` takes `commit_drop` on the recorded target as
for an unpublished commit (see [Unattached graph branches](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-10-11-unattached-graph-branches.md)).
## Validate and test [#validate-and-test]
Policy commands read the applied bundle from a cluster:
```bash
omnigraph policy validate --cluster ./company-brain --graph knowledge
omnigraph policy test --tests policy.tests.yaml \
--cluster ./company-brain --graph knowledge
omnigraph policy explain \
--cluster ./company-brain --graph knowledge \
--actor act-alice --action read --branch main
```
Edit the policy file or its `applies_to` bindings, then run
`cluster apply --server URL --config DIR`. The server finishes admitted requests
before activating matching HTTP and engine permissions. Later requests use the
new rules. Cluster management policy changes take effect in the same deployment;
a newly granted administrator cannot authorize that grant.
## Actor identity [#actor-identity]
For HTTP requests, the server maps the bearer token to an actor. Headers,
query parameters, and request bodies cannot override that identity.
Signed credentials use `principal:`; groups and
permissions come from applied policy. An identity credential contains no
graph/action grants. Version-1 restricted credentials are not accepted.
For direct CLI writes, actor resolution is:
1. `--as `;
2. `operator.actor` in `~/.omnigraph/config.yaml`;
3. no actor.
When a policy is installed, a missing actor is denied. Served writes reject
`--as` because only the server may resolve their actor.
Successful graph commits record the actor for the whole atomic change. Inspect
the audit trail with:
```bash
omnigraph commit list --store ./graph.omni --json
omnigraph commit show --store ./graph.omni --json
```
## Server startup modes [#server-startup-modes]
For static token authentication:
| Tokens | Policy | Startup and authorization |
| ---------- | ---------- | --------------------------------------------------------------- |
| none | none | Requires explicit `--unauthenticated`; otherwise startup fails |
| configured | none | Only the `read` action is allowed; all other actions are denied |
| configured | configured | The policy decides each action |
| none | configured | Startup fails because no request could establish an actor |
`GET /graphs` is denied unless a `cluster`-scoped policy grants `graph_list`,
including when graph policies exist.
Signed-token trust also enables authenticated startup. Signed credentials
require an explicit policy permit for protected operations even when no
static tokens are configured. Only the minimal identity discovery route is
independent of policy membership; it exposes existence, not graph access.
Policy is enforced for graph writes inside the engine as well as at the HTTP
boundary. This keeps direct and embedded writers subject to the same action
checks when a policy engine is installed. Per-entity and per-property
authorization is not currently supported; authorization is graph/branch scoped.
# HTTP server (/docs/operations/server)
`omnigraph-server` serves every healthy graph in one applied cluster under
`/graphs/{graph_id}/…`. It has no single-graph boot mode. Directory boot reads
the current `cluster.yaml` to resolve storage and validate the source location;
graph, query, and policy resources come from applied state. URI boot is
config-free.
The [OpenAPI document](https://github.com/ModernRelay/omnigraph/blob/main/openapi.json) defines the graph API. A running
server returns it from `GET /openapi.json`, which is not self-listed.
## Start a server [#start-a-server]
From a local cluster bundle:
```bash
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-alice":"secret"}' \
omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080
```
For object storage, replace `./company-brain` with the cluster root, for example
`s3://company-data/omnigraph/company-brain`.
The default bind address is `127.0.0.1:8080`. `--require-all-graphs` makes any
graph startup failure fatal. Otherwise failed graphs remain in the authorized
inventory as blocked while healthy graphs serve; see [readiness](/docs/getting-started/deployment).
An applied empty cluster creates no default graph and serves an empty inventory.
`/readyz` reports its applied digest, ledger revision/CAS and zero graph counts.
Missing or unapplied state, or a nonempty cluster whose graphs all fail, refuses
startup. Authentication, policy and data-token root checks still apply.
Use `cluster apply --server URL --config DIR` for schema, query, policy,
provider and Blob-rule changes without restart. Graph additions and removals
use the same deployment; removal deletes the graph's managed storage and history. Direct apply requires stopped serving and a subsequent
start; see [cluster deployments](/docs/clusters).
An unapplied resource edit does not activate it, although changing or breaking
the directory's config can change where boot looks for applied state.
For first boot of a fresh, empty S3 cluster initialized by the library, use
`--bootstrap-handoff FILE` with its exact receipt. This is a one-time ownership
claim, not a restart or recovery path; see
[first boot from a bootstrap receipt](/docs/clusters#first-boot-from-a-bootstrap-receipt).
### Logs [#logs]
The server logs to standard output at `info` unless `RUST_LOG` sets other
levels. `omnigraph_server::init_tracing()` limits three dependency targets, and
their `::` children, to warnings and errors even with `RUST_LOG=trace`: `rmcp`,
whose verbose SDK logs contain query arguments and results, and Lance's
`lance::dataset_events` and `lance::file_audit`, which at `info` name every
table's full storage URI, every file Lance creates or deletes, and each delete
predicate with its entity IDs. Other targets keep their configured levels.
Embedders using their own subscriber must enforce the same filter.
A failed Blob delivery, change-route request or streamed response is logged by
its error class, never by its message; see
[HTTP errors](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/troubleshooting.md#http-errors).
## HTTP contract [#http-contract]
Upgrade the v0.13 CLI, server and HTTP integrations together. Every protected
request requires exactly one `Omnigraph-Http-Api: 0.13`: graph, registry and
cluster-deployment calls, including `/graphs/discovery`, JSON, streams and Blob GET/HEAD. Missing,
duplicate, combined or unsupported values return `400 api_contract_mismatch`
after authentication but before graph lookup, body decoding or execution. This
HTTP identifier is separate from package/storage versions and grants no permission.
The previous `0.12` value is refused; update integrations to the current routes,
request/response shapes and credentials before changing their header.
Ordinary responses, including errors and streams, carry the same header. Check
it before decoding or emitting the body. A missing/incompatible response header
means unknown effects, never permission to replay. Proxies must preserve it both ways.
Server URLs name the service root, including its proxy prefix. Select a graph
with `--graph`, `default_graph` or `alias.graph`. Migrate graph-qualified server
URLs to that root plus graph selection; the CLI never guesses a root from a path. A `unix:///absolute/path.sock` URL names a Unix-domain socket instead: its whole path is the socket, so it has no proxy prefix, and requests carry `Host: localhost`; see [Unix-socket access](/docs/getting-started/deployment#unix-socket-access).
Public `/healthz`, `/readyz` and `/openapi.json` need no header and identify the
contract in their responses. Before every graph, registry or deployment request, the CLI sends
`HEAD /healthz` to that base with a five-second bound, no bearer, body consumption
or cache. Failure prevents data dispatch. Each request adds one discovery round
trip; its existing deadline stays separate. Neither request follows redirects
or retries automatically. Older clients/servers without this contract are unsupported.
MCP/OAuth metadata neither require nor return this header; managed control-plane
APIs keep their separate protocol. The accepted [HTTP admission decision](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-09-30-v012-http-admission.md)
defines exact request/response and refusal rules.
## Authentication [#authentication]
Choose one static token source:
| Environment variable | Value |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIGRAPH_SERVER_BEARER_TOKEN` | One bearer token; actor is `default` |
| `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON` | Actor-to-token object, e.g. `{"act-alice":"secret-a"}` |
| `OMNIGRAPH_SERVER_BEARER_TOKENS_FILE` | Path to that JSON object |
| `OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET` | Secrets Manager mapping; requires an AWS-enabled build: the container images, the release archives from v0.14.0 and every edge build, or a source build with `cargo build -p omnigraph-server --features aws` |
Send `Authorization: Bearer secret-a`. The token selects the actor for
authorization and commit attribution; clients cannot claim another actor.
See [Authorization and actors](/docs/operations/policy).
A server with neither static tokens, signed-token trust, nor policy refuses to start unless you explicitly
pass `--unauthenticated` (or set `OMNIGRAPH_UNAUTHENTICATED=1`). Use that only
on a trusted development network. Static tokens without a policy allow only the
`read` action. Stored-query invocation, export, graph listing, writes, and other
actions remain denied.
### Signed data credentials [#signed-data-credentials]
To accept short-lived credentials from an issuer, mount its public trust file
and select it explicitly:
```bash
omnigraph-server --cluster s3://company-data/company-brain \
--data-token-trust /run/omnigraph/data-token-trust.json
```
The file binds public signing keys to the exact storage root, issuer, account,
cluster id, and cluster incarnation. Invalid trust or a root mismatch refuses
startup before graphs open. Trust alone requires bearer authentication. The
server verifies tokens locally; it does not contact an identity or control
service. The provisioning operator owns supplying the correct identity binding.
The [trust and credential format](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/0053-offline-data-token-verification.md#public-trust-and-root-binding)
defines the machine-written file.
Signed credentials use the actor `principal:`. A caller
cannot change its actor through request headers or JSON. The server accepts
version-2 identity credentials, which bind the principal to the cluster and
contain no permissions. Applied Cedar policy decides graph operations;
missing policy or an unknown policy actor denies protected access.
Every valid identity credential can call `GET /graphs/discovery` for applied
graph IDs and names (currently identical), including blocked graphs. It
requires no policy membership and returns no locations, availability, schema,
queries or data. Static credentials cannot use it. `GET /graphs`
requires `graph_list` permission. Discovery grants no graph access or reachability.
See [managed data access](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/managed-data.md) for issuance and CLI discovery.
Tokens live for 60–86,400 seconds from issuance. The server permits an issuance
clock up to 30 seconds ahead, so at most 86,430 seconds can remain on admission.
Expiry has no grace period. Logout or a permission change at the issuer does
not revoke an issued token; already accepted operations can finish after
expiry. Stored-query calls need `invoke_query` plus `read` or `change` for the
body. Apply policy changes through `cluster apply --server`; editing a source
file alone does not change permissions. Deployment uses
[current-policy authorization](/docs/operations/policy#actions); the identity credential supplies no bypass.
Static credentials can coexist for operator recovery. An exact configured
static credential keeps its existing authority, including credentials with
dots; an invalid signed credential never falls back to static or anonymous
access. Restart to change public trust. Install new and old keys together
before issuing with a new key, and retain the old key for at least 86,430
seconds after its final issuance before removing it with another restart.
### OIDC resource identities and MCP [#oidc-resource-identities-and-mcp]
Enable OIDC with a public admission file alongside signed or static credentials:
```bash
omnigraph-server --cluster s3://company-data/company-brain \
--oidc-identity-trust /run/omnigraph/provider-access.json
```
The file binds an exact HTTPS issuer, resource audience, organization, account,
cluster incarnation and canonical root. Public RSA keys verify RS256 tokens;
explicit subject mappings select stable `principal:` actors. Tokens carry
no graph permissions: applied Cedar governs graph and schema operations, while
every admitted identity can discover graph IDs and names.
Human access tokens must name exactly one configured resource audience, the
configured organization and an admitted subject, and expire within 300 seconds
of issuance. OAuth-client ID tokens, delegated or impersonated credentials and
unqualified machine identities refuse. Every refresh must request the exact
resource again and check the returned audience.
Supply at most four RSA keys, 1,000 subject mappings and 256 KiB per public file.
The [resource identity contract](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-09-09-identity-credentials-and-applied-policy.md#provider-native-access-and-standard-clients)
defines the versioned format. Authority is held by whoever can publish this
file; it contains no provider secret or graph policy. Protect its filesystem
permissions and publish complete updates atomically.
Boot validates identity and root before opening graphs. Local refresh runs every
five seconds: the binding stays fixed, revisions increase, and equal revisions
require identical bytes. Every request checks the original snapshot deadline,
at most 300 seconds after capture; invalid updates cannot extend it. Requests
never fetch keys or call a control service, so publisher outages eventually
prevent new OIDC access; accepted operations can finish. Admission refresh does
not restart writers. Graph configuration retains its normal activation.
With this profile configured, the server also serves public
protected-resource metadata and an OAuth challenge on its [`/mcp`](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/mcp.md)
endpoint, so standard OAuth clients such as Claude can sign in. The [MCP guide](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/mcp.md) describes the
endpoint, its tools, settings and limits.
## Route families [#route-families]
| Route family | Purpose |
| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /healthz` | Process health |
| `GET /readyz` | Replica readiness and booted applied revision |
| `GET /openapi.json` | Runtime copy of the OpenAPI document |
| `GET /graphs` | Graph metadata catalog; requires `graph_list` policy |
| `GET /graphs/discovery` | Graph IDs and display names only; requires an identity credential |
| `POST /cluster/deployments` | Submit an exact-ID deployment: graph creation/deletion or schema, query, policy, provider and Blob-binding changes |
| `GET /cluster/deployments`, `GET /cluster/deployments/{id}` | Authorized deployment status and current-process activation observation |
| `GET /.well-known/oauth-protected-resource`, also at the resource's path suffix | Public OIDC resource metadata; only when OIDC trust is configured |
| `/mcp` | The [MCP endpoint](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/mcp.md); when static tokens, signed data tokens or OIDC trust are configured |
| `/graphs/{id}/query`, `/mutate` | Run inline GQ source |
| `/graphs/{id}/mutate/if-graph-commit` | Run an inline conditional mutation |
| `/graphs/{id}/queries` | List and invoke stored queries, including conditional mutations |
| `/graphs/{id}/load`, `/load/ndjson` | Bounded batch loading |
| `/graphs/{id}/blob` | GET/HEAD one Blob cell; PUT/DELETE to replace or clear it |
| `/graphs/{id}/branches` | Branch management and merge, a merge dry run (`dry_run: true`), a branch's diff against its merge base (`GET /branches/diff`) and that merge base (`GET /branches/merge-base`) |
| `/graphs/{id}/snapshot`, `/commits` | Snapshot (`internal_schema_version`, `graph_manifest_version`, and the datasets of one captured graph version), history (paged with `limit` and `page_token`), per-commit changes, and the unpublished commits of a branch (`GET /commits?unpublished=true`, `POST /commits/{commit_id}/publish`, `DELETE /commits/{commit_id}`) |
| `/graphs/{id}/changes`, `/diff` | Poll a branch feed, establish a baseline, or diff two points (branches or commit ids) |
| `/graphs/{id}/schema` | Show the accepted schema |
| `/graphs/{id}/export` | Stream a branch snapshot as JSONL |
`/query` also serves `branch list`, the branch reads (`branch diff`, `branch merge-base`, `branch merge … dry run`), `show`, and [explain](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/explain.md).
`/mutate` serves `branch create`, `branch delete`, `branch merge`, `commit publish`,
and `commit drop`. See [Branching](/docs/branching).
Each of `/query`, `/mutate`, `/mutate/if-graph-commit` and `/branches/merge`
takes an optional `settings` field, and the two GET change routes a `set=`
parameter; see [Session settings](/docs/queries-search/queries#session-settings).
Change a managed graph's schema through `cluster apply --server URL --config DIR`; see
[cluster deployments](/docs/clusters#deploy-without-restarting).
## Run an inline query [#run-an-inline-query]
```bash
curl -sS http://localhost:8080/graphs/knowledge/query \
-H 'Omnigraph-Http-Api: 0.13' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-d '{
"query":"query find($name: String) { match { $p: Person { name: $name } } return { $p.name } }",
"name":"find",
"params":{"name":"Ada"}
}'
```
Use `branch` or `snapshot` to select a read view; they are mutually exclusive.
When the read snapshot has an effective graph head, the canonical `/query`
response includes its `graph_commit_id`, pinned with the returned rows. Inline
writes go to `/mutate` and may select a target `branch`.
## Invoke a stored query [#invoke-a-stored-query]
```bash
curl -sS http://localhost:8080/graphs/knowledge/queries/find_person \
-H 'Omnigraph-Http-Api: 0.13' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-d '{"params":{"name":"Ada"}}'
```
Authorization denials for stored-query invocation appear as `404`, preventing
callers from probing registry names. A stored query then receives the normal
`read` or `change` authorization check for its body.
## Conditional mutations [#conditional-mutations]
Use a dedicated route and the commit returned by the read whose result you are
acting on:
```bash
curl -sS http://localhost:8080/graphs/knowledge/mutate/if-graph-commit \
-H 'Omnigraph-Http-Api: 0.13' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-H 'Omnigraph-If-Graph-Commit: ' \
-d '{"query":"query rename($name: String) { update Person set { name: $name } where email = \"ada@example.com\" }","name":"rename","params":{"name":"Ada"}}'
```
Stored mutations use `POST /graphs/{id}/queries/{name}/if-graph-commit` with
that header. Ordinary `/mutate` and `/queries/{name}`
reject it rather than ignore the condition. Never fall back to an unconditional
route after a conditional request fails.
The header must contain one raw commit id; wildcard, quoted, weak-ETag, and
comma-list forms are invalid. The condition covers the whole target branch.
If its effective head differs, the server returns `412` with
`precondition_failure { expected, actual? }` and writes nothing. Re-read and
decide again.
Successful mutation and load responses contain `commit`, the exact commit
receipt for that attempt. A successful mutation with no matching entities
returns `"commit": null`.
## Load NDJSON [#load-ndjson]
`POST /graphs/{id}/load/ndjson` accepts logical node and edge records with
`Content-Type: application/x-ndjson`. The request is one bounded atomic graph
batch; it is not a durable stream or an unbounded ingestion session. Split a
larger feed into batches and wait for each response before acknowledging it
upstream.
HTTP loading defaults to `mode=merge` and branch `main`. A missing target branch
is an error unless the request supplies `from`. This differs from the CLI,
where `--mode` is always required.
See [Mutations and loading](/docs/queries-search/mutations) for the record shape and load
modes, including [CSV statements on `/mutate`](/docs/queries-search/mutations#csv-loading).
## Deliver Blob values [#deliver-blob-values]
`GET`/`HEAD /graphs/{id}/blob` select a cell by `entity`, `type`, `id` and `property`,
support managed ranges/ETag conditions, and report external references without fetching
them. `PUT` and `DELETE` replace or clear the cell; see [Blob values](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md),
[Replacing one Blob value](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md#replacing-one-blob-value) and [Blob limits](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md#limits).
## Changes and baselines [#changes-and-baselines]
`GET /graphs/{id}/commits/{commit_id}/changes` compares a commit with its first
parent, `GET /graphs/{id}/diff?from=&to=` any two points, and `/branches/diff`
a branch with its merge base. `GET /graphs/{id}/changes` polls branch commits
and returns a durable cursor; unreadable history returns `410 change_feed_gap`.
`POST /graphs/{id}/changes/baseline` streams entities, a snapshot commit and a
resume cursor ([Changes, Diffs and Change Feeds](/docs/branching/changes)).
## Errors and retries [#errors-and-retries]
Application errors preserve structured details. Admission limits use `429` with
`Retry-After`; size limits use `413`, and blocked graphs or closed admission use
`503`. Admitted writes continue after disconnect. Only the
CLI's qualified whole-command admission refusal permits exit 75 and caller retry;
generic 409/503 and lost responses do not. See [failure outcomes](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/operations/troubleshooting.md#failed-data-write-commands).
For TLS, secrets, storage credentials, container examples and Azure admission,
see [Deployment](/docs/getting-started/deployment).
# Upgrading OmniGraph (/docs/operations/upgrade)
Normal open accepts storage formats v14 to v16 (a v14 or v15 graph is stamped v16 at its first
publish, or by the [storage upgrade](#storage-upgrade)) and never migrates a graph. A standalone
graph at v8 or v9 (release 0.11.x) or at v13 is converted in place by that offline upgrade. A graph at any other
format, and every cluster-managed graph below v14, is refused and takes the
[export/import rebuild](#rebuild) with the binary that wrote the graph.
Check the [release notes](https://github.com/ModernRelay/omnigraph/blob/main/docs/releases/) for storage and index compatibility.
## Storage upgrade [#storage-upgrade]
`omnigraph upgrade` converts a standalone v8, v9 or v13 graph to v16 in place, and stamps a v14 or v15 graph to v16 in place by the route `stamp-v14-to-v16`: one metadata-only commit per live branch still below v16, main first, and nothing else (no pending marker, nothing under `__history/legacy/`, no table row rewritten); stop every reader and writer and take the backup as for any upgrade, and a run that stops part-way is finished by running it again. That upgrade is required before `as commit`, `commit publish` or `commit drop`, which are refused on a branch still at v14, and before `as unattached branch`, which is refused on a branch still at v14 or v15.
Branches, commit ids, commit history and table data are kept; no table row is
rewritten. Only main builds from 2026-10-01 to 2026-10-04 wrote v13.
The upgrade is offline, and it cannot verify that itself:
1. Stop every server, embedded reader or writer and maintenance process that
has the graph open, and keep them stopped until the upgrade reports
`completed`. An old process opened earlier can still write to a branch; the
upgrade then stops and the root is restored from the backup.
2. Take and verify a backup of the whole graph root. There is no reverse
conversion: rollback is restoring that backup with the old binary.
3. Check, then convert, then verify with the new binary:
```bash
omnigraph upgrade ./graph.omni --check --json
omnigraph upgrade ./graph.omni --json
omnigraph commit list ./graph.omni --json
```
`--check` writes nothing. A run without `--check` first marks the graph as
pending, which every normal open refuses, writes the pre-upgrade commits as
immutable objects under `__history/legacy/`, converts each live branch,
validates what it wrote and only then removes the marker. A run that stops
after the marker is resumed by running the same command again with the same
executable. Never delete the marker or anything under `__history/`.
| `outcome` | Meaning | Exit |
| ------------------- | ------------------------------------------------------------------------------- | ---- |
| `check_passed` | v8, v9, v13, v14 or v15 graph, every check passed, nothing written | 0 |
| `already_current` | v16 graph, no live branch below v16, nothing to do | 0 |
| `completed` | converted, validated and served as v16, or every live branch stamped v16 | 0 |
| `check_failed` | a finding below; nothing was written | 1 |
| `recovery_required` | the graph is pending or holds leftover recovery files; follow `recovery.action` | 1 |
The report's `work` object counts what the run read and writes: `live_refs`,
`retired_refs` (deleted branches whose commits are kept), `orphan_writers`,
`legacy_commits`, `bookkeeping_versions`, `absent_parents`, `data_files`,
`id_shards`, `writer_shards`, `schema_contents`, `census_reads`,
`census_cells`, `legacy_bytes` and `table_opens` (the tables of a v8 or v9
graph opened to compare their columns with the root schema objects). A
resumed run that finds the legacy objects complete reads no history again and
reports `retired_refs`, `orphan_writers`, `bookkeeping_versions`,
`schema_contents`, `census_reads`, `census_cells`, `legacy_bytes` and
`table_opens` as zero.
| Finding | Names | What to do |
| ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `unsupported_target` | the target | `--to-format` accepts 16 only |
| `newer_than_binary` | the format | upgrade the binary |
| `unsupported_source` | the format, layout, over-budget branch, or the unreadable, over-bound or table-mismatched root schema objects of a v8 or v9 graph | [rebuild](#rebuild) with the build that wrote the graph. For a retired branch, instead of rebuilding run `cleanup --keep --confirm` with that build, then `upgrade --check` again; rebuild only when the retired branch stays. For root schema objects, restore the whole root from one backup, objects and tables together, then check again; the objects alone, from an older backup, no longer match the tables and are refused. An unreadable contract refuses the 0.11.x open too, so export with 0.11.x is no way out of it |
| `source_recovery_required` | leftover recovery files, or a `.staging` root schema object of a v8 or v9 graph, each with or without a live `__schema_apply_lock__` branch | stop every writer, then open the graph read-write with 0.11.x: `omnigraph snapshot ` is such an open (`upgrade --check` is not). The open removes the recovery files and finishes or rolls back a schema apply that changed the table set; a property-only `.staging` it refuses and names the manual choice between the live and the staging schema. The lock stays live after the open; the upgrade handles it (next row). Then check again |
| `schema_apply_lock_retired` | a live `__schema_apply_lock__` branch on a root with no `.staging` object and no recovery files: the lock of a v8 or v9 schema apply killed before it released it | nothing: the upgrade handles it, and the converted graph does not hold it as a live branch; `--check` reports it, the run handles it |
| `history_objects_present` | object keys | restore the whole root, `__history/` included, from the backup of the earlier attempt |
| `legacy_lineage_incomplete`, `legacy_lineage_corrupt` | up to 20 commit ids or branches | rebuild |
| `legacy_record_over_bound`, `legacy_census_over_bound`, `legacy_directory_over_bound` | the measured value and its limit | rebuild |
| `legacy_uncommitted_change`, `legacy_head_record_mismatch` | the branch, version and field | report it; rebuild meanwhile |
| `preflight_failed` | the error | fix the cause and check again; nothing was written. Above the live-branch limit: rebuild |
| `pending_upgrade`, `upgrade_interrupted` | the attempt | rerun without `--check` with the same executable |
| `unknown_upgrade_ownership`, `fence_publication_attempted` | the marker state | keep the root, run `--check`, finish with the executable that started the attempt |
| `legacy_plan_changed`, `legacy_objects_differ` | digests | finish with the executable that marked the graph; restore the backup if an object under `__history/legacy/`, or the schema archive under `__history/schemas/` that a fenced v8 or v9 rerun reads instead of the root schema objects, was changed |
Limits, checked before anything is written: 1,024 live branches including
main and 1,024 retired branches; 1,000,000 catalog rows or 64 MiB per branch
head, live or retired, and 64 MiB per root schema object of a v8 or v9 graph;
256 KiB of commit fields and 16 MiB per commit record; a history-read budget
that refuses a single v13 branch line above about 75,000 commits; and a
history-memory budget of 1 GiB that admits about 3.8 million table entries,
counted for each branch as its commits plus its head, times the tables of its
head (10,000 commits over 383 tables, or 1,000 commits over 3,800). That count
multiplies every commit of a branch by the tables of its head, so a graph
whose tables were added late is over-counted. The budget counts only the
fixed-size part of each table entry: table names and paths are not counted
and planning holds a second copy, so the process needs several times 1 GiB of
memory. A graph over a limit is rebuilt. A retired branch over a limit cannot
shrink; 0.11.x `cleanup --keep --confirm` removes it when no live branch,
merge base or tag needs it (without `--confirm` it is a dry run), then run
`upgrade --check` again.
After the upgrade, commit ids from before it resolve as before, and commit
listing, the change feed, merges and `cleanup` read across the upgrade. A
snapshot addressed by a catalog version below the upgrade serves the commit
at that version, or the nearest commit below it. The upgrade lands the graph
at v16, so a v14 or v15 development build older than this upgrade refuses the
upgraded graph before any open: upgrade every binary first.
A graph written by 0.11.x (v8 or v9) has limits of its own. Its checks read every catalog version since `init`, so a long history is slow to check.
| Subject | After the upgrade |
| ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Root schema objects | `_schema.pg`, `_schema.ir.json` and `__schema_state.json` at the graph root are read as the graph's schema and must be present and consistent. The upgrade leaves them in place. v14 to v16 never read them, and they go stale at the next schema apply. `_graph_commit_recoveries.lance/` also stays, as an unreferenced dataset |
| Unfinished schema apply | A `.staging` copy of a root schema object, or a leftover recovery file, is refused as `source_recovery_required`, with or without a live `__schema_apply_lock__` branch: open the graph read-write with 0.11.x (`omnigraph snapshot `), which removes the recovery file and finishes or rolls back an apply that changed the table set. An apply that changed properties only leaves a `.staging` that 0.11.x refuses as well; its message names the manual choice between the live and the staging schema. 0.11.x releases the lock only inside the process that took it, so the lock stays live after that open. A live lock on an otherwise clean root is handled by the upgrade: the converted graph does not hold it as a live branch, and the next schema apply is not refused as already in progress |
| Schema of old commits | 0.11.x kept only the latest schema, so every commit from before the upgrade is recorded under the one schema live at the upgrade. The tables and rows of an old commit are exact. A read at an old commit id is planned against the live schema, as in 0.11.x: a type added later is absent there, and a type dropped later cannot be queried |
| Table versions above a commit | An interrupted 0.11.x write can leave them. The upgrade opens each table only at the version main's head pins and does not report them. Reads and writes ignore them, `repair` reports `foreign_drift` and `cleanup` lists them as foreign versions |
| `cleanup` after a 0.11.x `cleanup` or `schema apply --allow-data-loss` | Both removed table versions that old catalog versions still name. `cleanup` deletes no catalog version of a live branch, so a catalog version can outlive the table versions it names, after these 0.11.x runs and after any earlier `cleanup` whose policy was narrower. A `cleanup` whose policy retains such a catalog version reports `is absent from the listing` for its tables, deletes nothing from them and still exits 0. `--older-than D` passes once those versions are older than `D`, and `--keep N` once they fall outside the newest `N`. A branch base, a merge base and a tag are retained under every policy: while one of them is such a version, every policy refuses its tables, and deleting that branch or tag releases it. `omnigraph cleanup --keep 1 --confirm` retains only those and each branch head, so it passes unless one of them is such a version. A read at a commit id whose table version 0.11.x removed fails as it did before |
Server and cluster selectors, cluster profiles and cluster-layout roots are
refused: a cluster-managed graph is exported with the build that wrote it and
[rebuilt](#rebuild). Paths, file URIs and symlinks resolve before that check.
## v0.9 to v0.10 [#v09-to-v010]
Released v0.9.0 uses Lance 9; v0.10.0 uses Lance 11. Both use graph storage
format v6, so existing entities, branches, and retained history do not need an
export/import migration. Development builds using Lance 10 follow the same
full-text upgrade procedure below.
The CLI/API vocabulary changes are not rolling-compatible. Update the CLI,
server, and client integrations together, with application traffic stopped.
Even without full-text indexes, do not run an old and new fleet against the
same graph. Keep the old executables and a verified whole-root backup until the
upgrade is proven. See the [v0.10 compatibility notes](https://github.com/ModernRelay/omnigraph/blob/main/docs/releases/v0.10.0.md#compatibility).
## Full-text index upgrade [#full-text-index-upgrade]
The v0.9-to-v0.10 Lance 11 transition keeps graph storage format v6, entities,
branches, and history. It changes the English stemmer used by full-text search.
Old indexes cannot safely be searched by the new analyzer, so OmniGraph explicitly refuses
full-text queries until the selected indexes have been rebuilt.
1. Stop application readers and writers. Using the old CLI, inventory the live
branches and record which need full-text search, including `main`:
```bash
old-omnigraph branch list --store ./graph.omni --json
```
2. Preserve and verify a backup of the **entire graph root**: metadata, branch
references, history, data, and indexes. A single branch's JSONL export is not
a rollback backup. For a cluster-managed graph, the root is
`/graphs/.omni`; retain the deployment bundle and
configuration too. Keep any externally referenced storage available.
3. Stop every old server, embedded writer, and maintenance process. Upgrade the
CLI and entire serving fleet together, leaving application traffic stopped.
Do not mix Lance 9/10 and Lance 11 readers or writers. Keep the server stopped
while the new direct-storage maintenance CLI rebuilds indexes.
4. Use the new operator CLI to rebuild each branch on the inventory checklist:
```bash
omnigraph rebuild-full-text-indexes ./graph.omni --branch main --as operator --json
omnigraph rebuild-full-text-indexes ./graph.omni --branch review --as operator --json
```
`--as` is optional actor attribution, not server-policy authorization. Inspect
`branch`, `graph_commit_id`, `rebuilt_indexes`, and `warnings`. A successful
non-empty rebuild publishes all planned indexes on that branch in one graph
commit. An empty index list with a null commit means no work was needed; it
does not mean other branches or historical snapshots were migrated.
Rebuilding replaces custom tokenizer settings with the default English
analyzer and reports that warning. See [maintenance](/docs/operations/maintenance#rebuild-full-text-indexes).
5. Verify representative searches and entity counts on **every rebuilt branch**.
Include words affected by stemming, such as `organism` and `university`.
Start only the new fleet, keeping application traffic stopped, and verify
CLI/API integrations against the new response fields before resuming traffic.
6. Keep the backup for rollback. Old binaries can silently miss matches in newly
rebuilt indexes; roll back by restoring the whole pre-upgrade backup with
the old fleet, not by pointing an old binary at the upgraded store.
Ordinary reads, traversal, and vector search remain available without the
full-text rebuild. Historical snapshots are unchanged and may refuse full-text
search; rebuilding a live branch does not upgrade its earlier snapshots.
Branch creation from a pinned historical snapshot is not supported.
The operation is explicit and can be expensive on large graphs; `optimize` is
not a replacement for this migration.
### Unsupported index inventory [#unsupported-index-inventory]
Automatic full-text rebuilding supports engine-created v6 indexes from Lance
9/10 with recorded index-kind metadata. If legacy or externally created
inventory does not identify its physical index kind, the command refuses before
publishing any index changes. It cannot safely guess whether an unknown index
is full-text, scalar, or vector; repeating the command does not fix this.
Use compatible old tooling for a controlled export/import into a new graph root,
or restore a trusted backup with supported inventory. Export/import preserves
entities but not shared branch ancestry or history, as described below. Keep the
source intact until verification and cutover; do not drop unknown indexes or
edit their metadata to bypass the refusal.
### Cluster state and rollback [#cluster-state-and-rollback]
Keep the pre-upgrade cluster deployment bundle, configuration, and state backup
alongside the graph-root backups. A rollback restores that consistent set with
the old executables; it does not point an old server at upgraded index files.
If v0.10 applied an `external_blobs` policy and the cluster state itself is not
being restored, v0.9 cannot read that new optional state field. While every
writer is stopped, remove `external_blobs` from each graph's configuration,
review `cluster plan`, and use the v0.10 `cluster apply` to converge the removal
before starting v0.9. Editing only the YAML is insufficient; do not hand-edit
the state ledger. This state-shape step does **not** replace restoring the
pre-upgrade graph backup after full-text rebuilding.
A downgrade also removes v0.10's default-deny external-URI enforcement: v0.9
writers admitted arbitrary supported sources, including `file://`. Do not roll
back to v0.9 if that ingress boundary is required.
## What an export/import rebuild preserves [#what-an-exportimport-rebuild-preserves]
This table describes rebuilding a graph at a new URI for a graph-format change,
not the full-text index rebuild above. An index rebuild retains the existing
graph's entities, branches, and history.
| Preserved | Starts fresh |
| -------------------------------------- | --------------------------- |
| Nodes and edges in the exported branch | Commit history |
| IDs and property values | Branch topology |
| Stored vectors | Old snapshots |
| Managed Blob values | Physical indexes and layout |
Export each branch you need separately. Each import becomes an independent
main branch in a new graph; shared ancestry is not reconstructed.
## Choose the export binary [#choose-the-export-binary]
The refusal message names the release line that wrote the graph. The known
mapping is:
| Storage generation | Export with |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| v1 | 0.3.1 or earlier |
| v2 | latest 0.6.x |
| v3 | latest 0.7.x |
| v4 | latest 0.8.x |
| v5 | the exact unreleased development build that wrote it |
| v6 | latest 0.10.x (the refusal names 0.9.x or 0.10.x) |
| v7 | the exact unreleased development build that wrote it |
| v8, v9 | latest 0.11.x (v8: legacy system column spellings). A standalone graph takes the [storage upgrade](#storage-upgrade) instead |
| v10 to v13 | the main development build that wrote it; the refusal names its build dates. A standalone v13 graph takes the [storage upgrade](#storage-upgrade) instead |
| v14 to v16 | current binary; entity export/import is not required within this storage generation |
If the graph's generation is newer than the binary, upgrade the binary instead.
## System-column upgrade (legacy spellings) [#system-column-upgrade-legacy-spellings]
`omnigraph schema upgrade-system-columns ` respells a served graph's
legacy system columns in place (`id`/`src`/`dst` to `__id`/`__src`/`__dst`).
It needs a graph that normal open accepts. The respelling
changes no storage layout and publishes the replacement contract with the
renamed table references in one new graph commit. Columns are renamed by field id: no table
rows are rewritten, indexes survive, and data and existing history/commit IDs are preserved;
`--check` runs the preflight and writes nothing; `--json` prints the report.
The preflight refuses a graph with any non-main branch (merge what you need,
then delete the branches: a merge alone leaves the source live) and a
property whose name starts with `_` (rename it with `@rename_from` first),
naming every offender under `system_columns_preflight`. Edge constraints
such as `@unique(src, dst)` become `@unique(@src, @dst)`. Stop every server
serving the graph and retain a verified backup first. Concurrent graph movement
can refuse the attempt; stop writers and rerun from a fresh capture.
There is no reverse operation. A run interrupted before it publishes changes nothing: rerun
it. Once publication succeeds, fresh read-only and read-write opens use the
accepted schema; a rerun reports `already_current`. A graph that
already spells `__id`/`__src`/`__dst` reports `already_current`;
cluster-managed graphs are refused.
## Rebuild [#rebuild]
Keep separate executables and target URIs so the source remains recoverable.
When an export already carries top-level `id`, the rewrite preserves it and any user property named `id`. For predecessor schemas, change endpoint constraint references from `src`/`dst` to `@src`/`@dst` before the new `init`. See [ingestion](https://github.com/ModernRelay/omnigraph/blob/main/docs/dev/ingestion.md#strict-graph-batch) for the envelope contract.
```bash
# Old binary
old-omnigraph schema show s3://bucket/graph.omni > schema.pg
old-omnigraph export s3://bucket/graph.omni > graph.jsonl
# Relocate predecessor export identities
jq -c 'if has("id") then . else .id = .data.id | del(.data.id) end' \
graph.jsonl > graph-current.jsonl
# New binary (schema.pg endpoint constraints use @src and @dst)
omnigraph init --schema schema.pg s3://bucket/graph-new.omni
omnigraph load --mode overwrite --data graph-current.jsonl \
s3://bucket/graph-new.omni
# Verify with the new binary
omnigraph snapshot s3://bucket/graph-new.omni --json
omnigraph schema show s3://bucket/graph-new.omni
```
For another branch:
```bash
old-omnigraph export --branch review s3://bucket/graph.omni \
> review.jsonl
jq -c 'if has("id") then . else .id = .data.id | del(.data.id) end' \
review.jsonl > review-current.jsonl
omnigraph init --schema schema.pg s3://bucket/graph-review-new.omni
omnigraph load --mode overwrite --data review-current.jsonl \
s3://bucket/graph-review-new.omni
```
## Verify before cutover [#verify-before-cutover]
At minimum:
* compare entity counts by type;
* run representative queries and mutations in a staging copy;
* sample IDs, vectors, null values, and Blob values;
* rebuild or reconcile declared indexes with `optimize`;
* verify server policy, stored queries, and external Blob access in the target
cluster;
* keep the old graph read-only until the new fleet is serving successfully.
Embeddings are copied as stored vectors; they are not regenerated. If the model
changed, re-embed after the import.
External Blob references require the target graph's allow-list to admit the
same sources. A direct-store CLI has no cluster allow-list and therefore cannot
admit new external references. Rebuild such data through a configured cluster
server or an embedded host that installs the policy. See
[Blob values](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md).
## Cluster cutover [#cluster-cutover]
Cluster graph roots are derived as `/graphs/.omni`; a
graph declaration cannot point at an arbitrary replacement root. Choose one of
these cutovers:
* **New graph ID in the same cluster.** Add (for example)
`knowledge_next` with the desired schema, query, provider, and policy
bindings. Validate, plan, and submit `cluster apply --server` so the running
owner creates and activates its derived root. Load through that server, verify
the new ID, back up what you need, and move clients. Removing the old declaration
and applying deletes its managed storage and all retained history.
* **Same graph ID in a parallel cluster root.** Copy the source bundle, set a
new `storage` root, and keep the original cluster untouched. Validate, plan,
and apply the new bundle; load the export into its derived graph root; then
point the server fleet at the new cluster root and restart together. This
preserves the public graph ID while changing the whole deployment artifact.
In either case, quiesce writers before export and cutover. External Blob
references must be loaded through a policy-aware server or embedded host after
the target allow-list is applied.
Do not run a mixed fleet of binaries that disagree on the storage format. Do
not edit internal metadata, overwrite the source root with `init --force`, or
copy files or backing datasets between graph roots.
# Schema Language (`.pg`) (/docs/schema)
A schema declares the node and edge types a graph accepts. OmniGraph validates
the same schema and constraints for mutation queries, loads, and branch merges.
Actor attribution on a commit does not create a graph node. Applications may
declare their own `Actor`, `OmniActor`, or other provenance types; these are
ordinary customer-owned types with no automatic rows or permission grants.
```graphql
node Person {
email: String @key
display_name: String
age: I32?
@range(age, 0..150)
}
node Company {
slug: String @key
name: String
}
edge WorksAt: Person -> Company @card(0..) {
role: String?
@unique(@src, @dst)
}
```
Comments use `// ...` or `/* ... */`.
## Declarations [#declarations]
```text
interface { }
node [implements , ...] { }
edge : -> [@card(...)] { }
```
Interfaces provide reusable properties. An edge's endpoint node types must
already be declared. Query traversals spell an edge with a lowercase first
letter (`worksAt` for `WorksAt`); lookup is otherwise case-insensitive.
## Property types [#property-types]
| Type | Values |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `String` | UTF-8 text |
| `Bool` | `true` or `false` |
| `I32`, `I64` | Signed integers |
| `U32`, `U64` | Unsigned integers |
| `F32`, `F64` | Floating-point numbers |
| `Date` | Calendar date |
| `DateTime` | Timestamp with millisecond precision |
| `Vector(N)` | `N` 32-bit floating-point values |
| `Blob` | Managed bytes or an external reference; see [Blobs](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/blobs.md) |
| `enum(a, b, ...)` | One of the declared strings |
| `[T]` | A list of scalar `T` values |
| `T?` | A nullable value |
Property names starting with `_` are reserved for system columns and are
rejected when a schema is admitted. The reserved namespace covers Lance's
virtual system columns (`_rowid`, `_rowaddr`, `_rowoffset`,
`_row_created_at_version`, `_row_last_updated_at_version`) and OmniGraph's
own implicit stored columns, spelled `__id` on nodes and edges and
`__src`/`__dst` on edges for newly created graphs.
Supported existing graphs without the `system-columns` feature keep the earlier spellings `id`, `src`,
and `dst` for their implicit columns. They reserve `id` on nodes and edges,
and `src`/`dst` on edges. On current graphs all three names are ordinary
property names. Query meta-fields use `@id`, `@src`, and `@dst`; projected
node objects use `@id`, and JSONL exports use a top-level `id`. The schema
endpoint (`GET /schema`) reports the physical spellings in `system_columns`.
`_distance` and `_score` are also reserved for new declarations: search-ordered
queries rank results by those columns. A graph whose schema already declared
either name before this reservation keeps opening; only new schemas are
refused.
On edge types the names `from` and `to` are also reserved: they are the
insert parameters that name the endpoints.
## Constraints [#constraints]
Constraints can be written in the type body. `@key`, `@unique`, and `@index`
also have a single-property shorthand.
| Constraint | Applies to | Meaning |
| --------------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@key(p, ...)` | node or edge | The property tuple identifies the entity; its id is derived from it. Key properties must be non-null scalar values. An edge key must include both `@src` and `@dst`, and a key may be declared only when the type is created. |
| `@unique(p, ...)` | node or edge | No two entities may share the property tuple. Edge constraints may include `@src` and `@dst`. |
| `@index(p, ...)` | node or edge | Declares index intent. Indexes affect performance, not correctness. |
| `@range(p, min..max)` | node | Restricts a numeric property, or each element of a numeric list, to an inclusive range; either bound may be omitted. Values compare with the bounds exactly at any magnitude, and a decimal bound on an integer property admits the integers on its side: `@range(n, -2.5..2.5)` admits -2 through 2. |
| `@check(p, "regex")` | node | Requires a String property to match the expression. |
| `@card(min..max)` | edge | Restricts the number of edges; omit `max` for an unbounded range, as in `@card(1..)`. The default is unbounded from zero. |
`@range(p, ..10)` supplies only a maximum, and `@range(p, 10..)` supplies
only a minimum; both endpoints are inclusive. Integer bounds must fit `I64`
(`-9223372036854775808` through `9223372036854775807`), even for a `U64`
property. Decimal bounds use their `F64` value.
Already accepted graph schemas keep their stored bounds. Older versions
interpreted an upper-only bound as a minimum and converted oversized integer
bounds to `F64`, possibly rounding their value. Opening or upgrading a graph
does not rewrite those accepted meanings. Reapplying that source, or using an
exported schema to create another graph, uses the corrected admission rules:
to preserve the accepted constraint, spell the historical minimum as `min..`
and a stored floating-point bound as a decimal with the recorded value.
Complete outstanding deployments containing affected schema source before
upgrading the binary; deployment source validation uses the admission rules.
`Blob`, list, and vector properties cannot be keys. Blob properties also cannot
be unique or indexed.
Current automatic property indexes are created for single-property node declarations:
orderable scalars and enums receive a scalar index, free-text Strings receive a
full-text index, and vectors receive a vector index. Composite declarations and
edge-property declarations are accepted as schema intent but do not currently
create a property index. Edge endpoints are indexed independently for traversal.
See [Search](/docs/queries-search/search#indexes).
## IDs [#ids]
Every node and edge has a String id. On the wire it rides at the top level of
the load or export envelope, under the fixed key `id`, beside `type` (nodes) or
`edge`; `data` holds user properties. Legacy-vintage graphs also accept
`data.id` as identity when the top-level `id` is absent; providing both is
refused. In queries the id is the meta-field
`$p.@id` and an edge's endpoints are `$e.@src` and `$e.@dst`
([system fields](/docs/queries-search/queries#system-fields)); a bare `id` in a query
is always a user property of that name.
* A node with `@key` derives its id from the complete typed key tuple. Renaming a
key property with `@rename_from` does not change existing ids.
* A node without a key receives a generated id unless input supplies one.
* An edge with `@key` derives its id the same way, in the catalog's key
order: `@src`, then `@dst`, then any scalar members. A composite id encodes
as a JSON array of the member values, for example `["Alice","Bob"]`.
* An edge without a key uses generated or supplied ids. Edges store their
endpoints as the graph's endpoint columns, `__src` and `__dst` on graphs
created by this binary.
For hand-authored load data, omit a keyed node's or keyed edge's top-level `id`
and let OmniGraph derive it; a supplied id on a keyed edge must equal the
derived id exactly. Export includes ids so a graph can be rebuilt without losing edge
references.
## Annotations [#annotations]
* `@rename_from("OldName")` on a node, edge, or property declares a rename
during schema migration. Use the current name everywhere after applying it.
* `@description("...")` adds human-readable metadata to types and properties.
* `@instruction("...")` adds usage guidance to node and edge types.
* `@embed("source_property", model="model-id")` associates a vector with its
source String property. The model is optional; see
[Embeddings](/docs/queries-search/embeddings).
Property-level `@key`, `@unique`, and `@index` are shorthands for their
single-property constraints. Unknown annotations are retained as metadata but
have no built-in behavior.
## Schema changes [#schema-changes]
Always preview a direct schema change before applying it:
```bash
omnigraph schema plan --schema next.pg graph.omni
omnigraph schema apply --schema next.pg graph.omni
```
Supported changes include adding types, adding nullable properties, renaming
nodes, edges, or properties with `@rename_from`, adding index declarations,
widening an enum with new values, updating descriptions or instructions, and
dropping node, edge, or property declarations. Dropping a property also drops
every constraint that names only dropped properties: its own `@index`,
`@unique`, `@range` or `@check`, and a composite constraint whose members are
all dropped.
None of these changes rewrites existing rows. Adding a nullable property,
renaming a property, or dropping a property changes only the table's metadata:
existing rows read the new property as null, no stored value or Blob is read,
and the indexes of the remaining properties keep covering the rows they
covered. A dropped property's index is removed by the same commit, and index
maintenance does not build it again. The time and memory
the apply takes therefore do not grow with the number of rows or the size of
their values. They still grow with the table's metadata: its columns, its
indexes and its fragments, of which a table written by many small commits has
more until `omnigraph optimize` compacts them.
Applying changed schema text, including comments or formatting, records the new
source in one graph commit even when no table changes are needed. Applying the
same accepted source again leaves the graph unchanged.
Changes such as adding a required property to existing entities, changing a property
type (except enum widening), changing edge endpoints or cardinality, changing a
node's implemented interfaces, adding a constraint other than `@index`,
removing a constraint from a property that stays, and adding, removing or
changing an existing type's `@key` are rejected. A drop is rejected when it
would remove a constraint that also names a field that stays, such as
`@unique(a, b)` when only `a` is dropped. The plan reports the exact
unsupported step before anything changes.
A drop removes the declaration from the current schema and reclaims no storage
at apply. Older commits still read the dropped data until `omnigraph cleanup`
stops retaining them. Re-adding a property under a dropped name starts a new
property, so a historical query never reads the dropped values through it; see
[historical reads](/docs/branching#historical-reads). A dropped type's data is reclaimed by that cleanup. A
dropped property's values also stay in the table's current data files, beside
the properties that remain, until the next `omnigraph optimize`: every
optimize rewrites each fragment of a table that still stores a dropped
property's values, copying only the remaining properties. Optimize rewrites
main only, and cleanup deletes the old files only once nothing it retains
references them: a live branch keeps the commit it was created from and its
head, a tag keeps the snapshot it names, and the retention policy keeps older
commits. Merging a branch releases none of its files; only deleting the branch
does. To erase dropped data, run optimize after the drop, delete every branch
created from a commit before that optimize and every tag naming such a commit,
then run cleanup with a retention that excludes the commits before the
optimize, for example `--keep 1`; after that, the dropped data cannot be
recovered. Review the plan carefully, and treat that cleanup as the step that
cannot be undone. See [optimize](/docs/operations/maintenance#optimize) and
[cleanup](/docs/operations/maintenance#cleanup).
Cluster-managed graphs change schema through `omnigraph cluster apply`. Direct
schema apply and the server schema-apply endpoint refuse cluster-managed graphs.
## Diagnostic codes [#diagnostic-codes]
Migration rejections may include a stable `OG-...` code. Match automation on the
code rather than the message text.
| Code | Meaning |
| ----------- | --------------------------------------------------------------- |
| `OG-DS-102` | Drop a node type that contains entities. |
| `OG-DS-103` | Drop an edge type that contains entities. |
| `OG-DS-104` | Drop a populated property. |
| `OG-MF-103` | Add a required property to a populated type. |
| `OG-MF-106` | Change a property's type, including enum narrowing or renaming. |
`omnigraph schema plan` includes these codes in human and JSON output.