NNRP Conformance Suite

The NNRP Conformance Suite is a shared, language-neutral protocol interoperability baseline that is independent of any single SDK. It gives implementations in different languages and from different organizations a common, versioned standard to prove that they speak the same protocol.

What it is

As the number of NNRP implementations — official and third-party — continues to grow, a natural question arises: how do you know these implementations are actually speaking the same protocol, rather than each one just passing its own tests?

The conformance suite answers exactly that question. It consists of:

1. Versioned baselines: each protocol line (for example nnrp-1-preview3) has its own baseline directory containing a protocol manifest, case manifests, semantic vector recipes, and generated artifacts.
2. Case manifests: machine-readable JSON files that describe each test case's protocol layer (L0–L4), its status (mandatory / optional / experimental), and the capability declarations required to run it.
3. Capability manifests: each implementation repository provides a JSON declaration listing the protocol capabilities it has completed and is willing to publicly claim support for.
4. API profile recipes: application-level profiles, such as OpenAI-compatible NNRP APIs, use readable recipe manifests and adapter capability declarations instead of hard-coded request traces.
5. Wire-level targets: endpoint-level tests let the runner act directly as client, server, or proxy and validate protocol frames without going through an SDK adapter.
6. Runner: a Rust CLI that loads a baseline and capability manifests, then produces execution plans, adapter result validation, API profile plans, benchmark plans, and reports.

Why it exists

Without a shared conformance baseline, three failure modes naturally emerge in any multi-implementation phase:

1. Every CI is green, but nobody owns interoperability. Each repository only runs its own unit tests. Everything looks passing, but nothing proves that two implementations can complete a handshake, negotiate capabilities, and run a full frame-submit and result-push cycle together.
2. Protocol interpretation slowly drifts. Without external constraints, each implementation develops its own understanding of protocol edge cases. By the time real interoperability is needed, the gap is hard to close.
3. "Tests pass" loses its meaning. If tests only validate internal self-consistency rather than alignment with a protocol standard, green CI is a false sense of security.

The conformance suite removes all three failure modes by providing an external, versioned, machine-executable protocol standard that each implementation can validate itself against independently.

What you can do with it

During development: run only completed capabilities

You do not need to wait until every capability is done before adopting conformance. The capability manifest lets you declare what you currently support, and the runner only executes cases for those capabilities. Cases for unclaimed capabilities are marked not_claimed, not as failures.

This means you can wire in conformance at any point during active development and get a meaningful report rather than a wall of noise from features you have not started yet.

In CI: explicitly bind a protocol version

CI does not guess which protocol version your implementation targets. You pass an explicit baseline (for example protocol/nnrp-1-preview3/manifest.json) and the runner validates three things:

1. Protocol manifest version = case manifest version = capability manifest version.
2. Only capabilities the implementation claims are promoted into the mandatory or optional execution set.
3. A version mismatch causes an immediate failure — ambiguous "close enough to this version" states are never allowed to enter the merge flow.

For SDK integration: use suite-owned adapter and benchmark contracts

SDK repositories do not hand-maintain byte fixtures. The suite owns readable semantic recipes, generated canonical vectors, adapter execution plans, adapter result schemas, benchmark execution plans, and benchmark result schemas. SDK repositories provide only capability claims, an adapter command, and optional benchmark runner evidence.

For third-party implementations: a verifiable interoperability claim

If you are building a third-party NNRP implementation, the conformance suite gives you a concrete way to say "this implementation passes the nnrp-1-preview3 mandatory core and supports the following capabilities" rather than just "I implemented NNRP".

Where to go next

Quick Start

Minimum bring-up path. Get a first execution-plan report in five minutes.

Capability Manifest Generator

Generate protocol capability manifests, OpenAI API profile adapter declarations, and wire-level target manifests without hand-writing JSON.

Capability Catalog

Browse versioned capability tokens, combination rules, and the conformance obligations attached to each claim.

Manifests Reference

For suite authors. Complete JSON field reference for protocol manifests, case manifests, vector recipes, adapter plans, benchmark plans, and reports.

SDK Integration Guide

For SDK developers. How to create a capability manifest, implement the adapter command, run benchmarks, and wire CI end-to-end.

CI and Version Selection

How conformance version binding works and what errors to avoid in CI.