What does relayorb check compare?

The result or error of each recorded request, after dropping the JSON-RPC id and any keys you pass with --ignore-key. Differences are printed as path: old -> new.

Does it test tools/list too?

Yes. Every recorded request except initialize and ping is re-sent, so tool schema changes show up as failures.

How to test an MCP server: record real sessions, check them in CI

Updated 2026-10-05

Unit tests cover your functions, but the thing that breaks agents is the protocol surface: tool schemas, result shapes, error behavior. The most realistic test input is a session a real agent actually produced. Record one, then check every future build against it.

1. Record a real session

Point your agent at the server through RelayOrb and do the task you care about.

Record

relayorb record --name my-server -- node dist/server.js

2. Save it as a fixture

Export the session to a JSON file and commit it. Review it first: recordings contain everything the tools returned.

Export

relayorb export my-server -o tests/fixtures/session.json

3. Check the server in CI

relayorb check starts the server, replays the handshake, re-sends every recorded call, and diffs each answer. It exits 0 when everything matches, 1 when an answer changed, and 2 if the server couldn't start. Use --ignore-key for fields that change on every run, like timestamps.

GitHub Actions step

- name: MCP regression check
  run: |
    curl -fsSL https://relayorb.com/install.sh | sh
    ~/.local/bin/relayorb check tests/fixtures/session.json --ignore-key timestamp -- node dist/server.js

4. Test agents without the real server

The reverse also works. relayorb replay pretends to be the server and answers from the recording, so agent tests run offline, without API keys, and get the same answers every time.

Replay

relayorb replay tests/fixtures/session.json

Install RelayOrb

Free and open source (Apache-2.0). A single binary for macOS, Linux, and Windows. No account, no cloud, no telemetry.

macOS / Linux

curl -fsSL https://relayorb.com/install.sh | sh

Windows and other options: GitHub Releases.

FAQ

What does relayorb check compare?: The result or error of each recorded request, after dropping the JSON-RPC id and any keys you pass with --ignore-key. Differences are printed as path: old -> new.
Does it test tools/list too?: Yes. Every recorded request except initialize and ping is re-sent, so tool schema changes show up as failures.