Manifests Reference — For Suite Authors

This page is for people who maintain the conformance suite. SDK authors should usually start with SDK Integration Guide.

The conformance suite is built around machine-readable JSON documents with schemas in the nnrp-conformance/schemas/ directory. Human-authored files should stay readable and parameterized; generated byte-level artifacts are outputs, not hand-maintained source files.

Document Types

Document	Schema	Owner	Purpose
Protocol manifest	protocol-manifest.schema.json	Suite	Root entry point for one protocol line.
Case manifest	case-manifest.schema.json	Suite	Declares cases, layers, status, and required capability tokens.
Capability manifest	capability-manifest.schema.json	SDK	Declares implementation name, target protocol line, and claimed support tokens.
Semantic vector recipes	semantic-vector-recipes.schema.json	Suite	Human-readable source for canonical vectors.
Generated vector manifest	vector-manifest.schema.json	Suite tooling	Generated bytes consumed by SDK tests.
Conformance report	report.schema.json	Runner	Case selection report for one capability manifest.
Adapter execution plan	adapter-execution-plan.schema.json	Runner/action	Dynamic behavior cases passed to an SDK adapter command.
Adapter case results	adapter-case-results.schema.json	SDK adapter	Machine-readable pass/fail/skip results for the adapter plan.
Benchmark execution plan	benchmark-execution-plan.schema.json	Runner/action	Benchmark scenarios passed to an SDK benchmark command.
Benchmark results	benchmark-results.schema.json	SDK benchmark	Measured latency/throughput/allocation results and evidence paths.
API profile suite manifest	api-profile-suite.schema.json	Suite	Root entry point for one application-level API compatibility profile.
API profile recipe	api-profile-recipe.schema.json	Suite	Human-readable request/expectation source for API-level compatibility cases.
API profile capability manifest	api-profile-capabilities.schema.json	Adapter	Declares adapter, profile level, supported operations, and optional API extensions.
API profile execution plan	api-profile-execution-plan.schema.json	Runner/action	Profile recipes selected for an adapter capability manifest.
API profile results	api-profile-results.schema.json	Adapter	Machine-readable pass/fail/skip results for API profile recipes.
Wire conformance suite manifest	wire-conformance-suite.schema.json	Suite	Root entry point for endpoint-level wire scenarios.
Wire conformance scenario	wire-conformance-scenario.schema.json	Suite	Human-readable source for frame-level client/server/proxy scenarios.
Wire conformance target	wire-conformance-target.schema.json	Implementation	Declares endpoint modes, transports, scenario capabilities, and limits.
Wire conformance plan	wire-conformance-plan.schema.json	Runner/action	Concrete wire scenarios selected for one target.
Wire conformance results	wire-conformance-results.schema.json	Runner/action	Machine-readable wire scenario outcomes and evidence references.

Protocol Manifest

Example: protocol/nnrp-1-preview3/manifest.json

The protocol manifest is the root entry point for one versioned baseline. It binds the protocol line, suite version, lifecycle status, case manifests, vector recipe manifests, generated vector manifests, and report schema.

{
  "$schema": "../../schemas/protocol-manifest.schema.json",
  "protocol_version": "nnrp-1-preview3",
  "suite_version": "0.1.0",
  "status": "draft",
  "case_manifests": [
    "cases/mandatory-core.json",
    "cases/l0-wire-vectors.json",
    "cases/l0-wire-errors.json",
    "cases/l1-control-plane.json",
    "cases/l1-session-container.json",
    "cases/l1-data-plane.json",
    "cases/l1-schema-cache.json",
    "cases/l2-binding-driver.json"
  ],
  "vector_recipe_manifests": [
    "vectors/semantic-vectors.json"
  ],
  "vector_manifests": [],
  "report_schema": "../../schemas/report.schema.json"
}

Version fields must match across protocol manifests, case manifests, capability manifests, adapter plans, and result documents.

Case Manifest

Example: protocol/nnrp-1-preview3/cases/mandatory-core.json

Case manifests group test cases by layer or feature area. A case with required_capabilities: [] is always selected for every implementation; capability-specific cases must list the tokens required to select them.

{
  "$schema": "../../../schemas/case-manifest.schema.json",
  "protocol_version": "nnrp-1-preview3",
  "manifest_name": "mandatory-core",
  "cases": [
    {
      "id": "l0.header.roundtrip.basic",
      "layer": "L0",
      "status": "mandatory",
      "feature": "header.roundtrip.basic",
      "required_capabilities": [],
      "description": "Round-trip the current common header without changing frozen fields."
    },
    {
      "id": "l1.handshake.basic",
      "layer": "L1",
      "status": "mandatory",
      "feature": "handshake.basic",
      "required_capabilities": ["handshake.basic"],
      "description": "Complete the minimum client/server handshake and capability negotiation."
    }
  ]
}

Semantic Vector Recipes

Example: protocol/nnrp-1-preview3/vectors/semantic-vectors.json

Recipe files are the human-authored source of truth for canonical vectors. They describe intent with readable fields such as message type, flags, lengths, session id, frame id, and trace id. Do not hand-maintain large hex strings in source recipes.

{
  "$schema": "../../../schemas/semantic-vector-recipes.schema.json",
  "protocol_version": "nnrp-1-preview3",
  "vectors": [
    {
      "recipe_type": "header",
      "name": "current.header.ping_can_drop",
      "description": "Common header for a PING frame that may be dropped.",
      "version_major": 1,
      "wire_format": 0,
      "message_type": "ping",
      "flags": ["can_drop"],
      "meta_len": 0,
      "body_len": 0,
      "session_id": 7,
      "frame_id": 11,
      "view_id": 0,
      "route_id": 0,
      "trace_id": 19
    }
  ]
}

Generated vector manifests contain the encoded bytes and are produced by:

nnrp-conformance-runner generate-vectors \
  --recipe protocol/nnrp-1-preview3/vectors/semantic-vectors.json \
  --output /tmp/canonical-vectors.json

Use verify-vectors to confirm that generated byte output remains deterministic.

Capability Manifest

Capability manifests are implementation-authored input. They contain only identity, protocol version, and support tokens.

{
  "$schema": "../../schemas/capability-manifest.schema.json",
  "implementation_name": "nnrp-py",
  "protocol_version": "nnrp-1-preview3",
  "supports": [
    "handshake.basic",
    "session.open_close",
    "frame_submit.tensor.inline",
    "result_push.basic"
  ]
}

The low-code generator on this site reads the versioned case manifests and emits this shape. It does not decide which tokens an implementation may claim.

API Profile Suite Manifest

Example: profiles/openai-compatible/1/manifest.json

API profile suite manifests freeze application-level compatibility profiles separately from the wire protocol baseline. The OpenAI-compatible profile binds one profile name, one schema version, one compatibility level, the protocol baselines it can run on, and the declarative recipe manifests that define the test surface.

{
  "$schema": "../../../schemas/api-profile-suite.schema.json",
  "profile": "openai-compatible",
  "schema_version": "openai-compatible/1",
  "level": 1,
  "protocol_baselines": ["nnrp-1-preview3"],
  "recipe_manifests": [
    "recipes/chat-streaming-text.json",
    "recipes/chat-non-streaming.json"
  ]
}

This document is intentionally profile-owned. It is not an SDK capability manifest and it is not a protocol case manifest.

API Profile Recipe

API profile recipes are readable request/expectation files. They avoid hard-coded byte fixtures and describe API behavior with operation names, request bodies, expected events, and terminal outcomes.

{
  "$schema": "../../../../schemas/api-profile-recipe.schema.json",
  "id": "openai-compatible.chat.streaming-text",
  "profile": "openai-compatible",
  "schema_version": "openai-compatible/1",
  "operation": "chat.completions.create",
  "required_capabilities": [
    "api.level1",
    "api.chat.completions.create",
    "api.streaming"
  ],
  "status": "mandatory",
  "request": {
    "body": {
      "model": "${MODEL_ID}",
      "messages": [{ "role": "user", "content": "Say hello." }],
      "stream": true
    }
  },
  "expect": {
    "events": [{ "type": "response.output_text.delta", "min_count": 1 }],
    "terminal": "success"
  }
}

API Profile Capability Manifest

Adapters provide an API profile capability manifest to declare which profile level and operation features they support. The low-code generator on this site can emit this shape from the profile suite manifest and recipe catalog.

{
  "$schema": "../../schemas/api-profile-capabilities.schema.json",
  "adapter": "vllm-nnrp-adapter",
  "profile": "openai-compatible",
  "schema_version": "openai-compatible/1",
  "compatibility_levels": [1],
  "operations": [
    {
      "name": "chat.completions.create",
      "streaming": true,
      "non_streaming": true,
      "tool_calls": true,
      "cancellation": true
    }
  ],
  "extensions": []
}

API Profile Execution Plan

The runner combines a profile suite manifest with an adapter capability manifest and emits a recipe execution plan. The plan includes a coverage_matrix so CI can show why each recipe was selected, skipped, or left unclaimed.

nnrp-conformance-runner api-profile-plan \
  --protocol protocol/nnrp-1-preview3/manifest.json \
  --profile profiles/openai-compatible/1/manifest.json \
  --capabilities conformance/openai-compatible-1.api-capabilities.json \
  --output artifacts/api-profile-plan.json

Wire-Level Conformance Manifests

Wire-level conformance is endpoint-oriented. It is used when the runner must act directly as a client, server, or proxy instead of calling an SDK-owned adapter command.

The suite manifest is the root of one wire baseline:

{
  "$schema": "../../schemas/wire-conformance-suite.schema.json",
  "protocol_version": "nnrp-1-preview4",
  "suite_version": "0.1.0",
  "status": "frozen",
  "scenario_manifests": ["scenarios/runtime-control.json"],
  "modes": ["client", "server", "proxy"],
  "transports": ["tcp", "quic", "ipc", "websocket"]
}

The target manifest is maintained by the implementation repository. It says which runner modes and transport endpoints the suite may use:

{
  "$schema": "../../schemas/wire-conformance-target.schema.json",
  "target_name": "nnrp-rs-preview4",
  "protocol_version": "nnrp-1-preview4",
  "suite_version": "0.1.0",
  "wire_conformance": {
    "modes": ["client", "server"],
    "transports": [
      { "name": "tcp", "endpoint": "127.0.0.1:19091", "tls": false }
    ],
    "capabilities": ["control.cancel_abort", "control.trace_context"],
    "limits": {
      "max_frame_bytes": 16777216,
      "max_in_flight": 256
    }
  }
}

Runner flow:

nnrp-conformance-runner wire-plan \
  --suite wire-conformance/nnrp-1-preview4/manifest.json \
  --target conformance/nnrp-1-preview4.wire-target.json \
  --output artifacts/wire-plan.json \
  --results-path artifacts/wire-results.json \
  --evidence-dir artifacts/wire-evidence

nnrp-conformance-runner wire-run \
  --plan artifacts/wire-plan.json \
  --target conformance/nnrp-1-preview4.wire-target.json \
  --output artifacts/wire-results.json

nnrp-conformance-runner validate-wire-results \
  --plan artifacts/wire-plan.json \
  --results artifacts/wire-results.json

Wire results keep scenario ids identical to the plan and attach evidence paths for frame captures, terminal states, and transport-level diagnostics.

Adapter Documents

The suite action creates an adapter execution plan and invokes the SDK-provided adapter command. The adapter command writes adapter case results. The important invariant is that result id values must exactly match plan case ids.

Python SDK adapter command:

python -m nnrp.tools.adapter_conformance

Benchmark Documents

Benchmark execution plans describe scenarios with category, feature, required_capabilities, and workload. SDK benchmark commands write benchmark results with an environment block and per-scenario metrics such as p50_us, p95_us, throughput_ops_per_sec, cpu_percent, peak_memory_bytes, and gc_alloc_bytes.

Python SDK benchmark command:

python -m nnrp.tools.benchmark \
  --plan benchmark-plan.json \
  --output artifacts/benchmark-results.json

What The Runner Does Not Freeze

The public contract is the JSON boundary described above. Internal Rust type layouts, SDK adapter object trees, and runner in-memory state are intentionally outside the frozen surface.