CMLabs · Psyclone AIOS

# 7. Recordings & Formats

Record part or all of a live system’s dataflow with the Replay Catalog, play it back offline for analysis and repeatable testing, and understand what lands on disk.

## Why record dataflow?

A running Psyclone system is a stream of typed messages. Being able to capture that stream and replay it later — at the original pace, in a different system, without the original sensors or hardware attached — is one of Psyclone’s most useful operational tools:

- Repeatable testing. Record a live session once; replay it into a development system as many times as you like. Downstream modules cannot tell the difference between replayed and live messages.
- Offline analysis. Capture a production incident and inspect it later in PsyProbe, message by message.
- Demos without hardware. The CoCoMaps robots shipped `*_static_*.xml` spec variants that replay a recorded demo instead of driving real robots — same modules, recorded input.
- Regression baselines. Feed a recorded input set into a changed module and compare outputs.

Note. Recording is done by catalogs declared in the PsySpec — primarily the Replay Catalog. The per-component `<recording>`/`<playback>` child tags are a separate mechanism that is currently Stub parsed-only (parameters stored, not acted on) — see below.

## The Replay Catalog

The Replay Catalog is a built-in catalog type (`type="ReplayCatalog"`, backed by the `PsySystem` library). It has two modes, selected simply by what you declare inside it:

Mode | You declare | Behaviour
--- | --- | ---
Record | `<trigger>` entries | Every message matching a trigger is appended to the recording under the catalog’s `root` directory, subject to the size/count caps.
Playback | `<post>` entries | Recorded messages are re-posted into the running system at their originally recorded intervals (or a fixed override interval).

### Recording

Declare a catalog with a storage `root`, caps, and one trigger per message type you want captured:

    <catalog name="Replay1" type="ReplayCatalog" root="./replay1"
             maxsize="10240000" maxcount="2000">
      <trigger name="Video"  type="robot.sensor.video" />
      <trigger name="Bumper" type="robot.sensor.bumper" />
    </catalog>

This keeps the last 2000 messages and at most ~10 MB on disk — whichever cap is hit first wins. Because subscription is by ordinary triggers, you can record a narrow slice (two sensor streams, as here) or, with wildcard message types, a broad swathe of the system.

A real production example — the CoCoMaps demo recorder captured ~30 message types with a 1 GB budget:

    <catalog name="DemoRecording" type="ReplayCatalog"
             root="%RecordingDir%/Demo2MasterRecording" maxsize="1000000000">
      <trigger name="RobotStatus"    type="Self.Position.Data" />
      <trigger name="HumanDetection" type="Human.Self.Detected" />
      <trigger name="IncomingSpeech" type="input.speech.detected" />
      <!-- ...one trigger per message type of interest (~30 in total)... -->
    </catalog>

### Playback

To replay, declare a catalog over the same root with `<post>` entries instead of triggers. The names must match the trigger names used when recording; the message types posted may differ if you want to remap them into the target system:

    <catalog name="Replay1" type="ReplayCatalog" root="./replay1">
      <post name="Video"  type="robot.sensor.video" />
      <post name="Bumper" type="robot.sensor.bumper" />
    </catalog>

Messages are posted at the intervals at which they were recorded, preserving the original temporal shape of the data. Two extra attributes control playback:

Attribute | Meaning
--- | ---
`interval="1000"` | Override the recorded pacing: post one message every N ms regardless of original timing.
`rotate="yes"` | Loop: when the recording is exhausted, start again from the beginning. Useful for soak tests and standing demos.

Tip. Record in the live system, then copy the `root` directory to your development machine and point a playback catalog at it there. Recording in one system and replaying in another is the designed workflow. The bundled `Examples/messagerecord.xml` and `Examples/messageplayback.xml` specs are a minimal working pair to start from.

## Configuration parameters

Recording catalogs accept the standard component storage attributes (they share the generic catalog attribute set):

Attribute | Unit / values | Default | Meaning
--- | --- | --- | ---
`root` | path | — | Directory (record) / source directory (playback) for the recording files.
`maxsize` | bytes | 0 (unlimited) | Maximum total bytes kept on disk; oldest data is dropped first.
`maxcount` | messages | 0 (unlimited) | Maximum number of messages kept.
`interval` | ms | 0 (recorded pace) | Playback only: fixed posting interval override.
`rotate` | `yes`/`no` | `no` | Playback only: loop from the beginning when exhausted.

Warning. With both `maxsize` and `maxcount` at their defaults (0 = unlimited), a recorder subscribed to high-volume streams will grow without bound. Always set at least one cap in production, and put `root` on a volume with headroom — video-frame streams fill disks fast.

## What’s on disk

A recording is a self-contained directory tree under `root`. Messages are stored in Psyclone’s native binary DataMessage serialisation — the same container format used on the wire between nodes — together with the per-trigger index the playback catalog needs to reproduce ordering and timing:

- Message payloads are stored verbatim: every user entry (strings, integers, floats, times, binary blobs, nested messages) survives the round trip bit-for-bit.
- Timestamps are the original creation times in microseconds, which is how playback reconstructs the recorded intervals.
- Trigger names are stored with each message; this is why playback matches on name, not type — the name is the join key between the recording and your `<post>` declarations.
- The size/count caps are enforced by rotating out the oldest data, so a capped recording always holds the most recent window.

Note. The format is an internal binary format, not a documented interchange format: read it back with a ReplayCatalog (or inspect messages live in PsyProbe during playback) rather than parsing the files directly. If you need data in an external tool, replay into a small export module, or use a RequestStore/MessageDataCatalog to serve JSON/XML over HTTP.

## Per-component <recording> / <playback> tags

The PsySpec grammar also accepts `<recording root="..." filesize="..." overwrite="...">` and `<playback root="..." interval="...">` as children of an individual component, intended for transparent per-module record/playback. These are currently Stub: the parameters are parsed and stored (as `componentrecording*` / `componentplayback*`), but the engine does not yet act on them. Use a Replay Catalog instead — it covers the same use cases explicitly and works today.

## Related stores: Data, File and request catalogs

Recording is one member of a family of built-in storage catalogs. Choosing the right one keeps your spec simple:

Catalog | Stores | Typical use
--- | --- | ---
ReplayCatalog | Message stream + timing | Record/replay of live dataflow (this chapter).
DataCatalog | Named entries in one datastore file, flushed on an interval (e.g. `<catalog name="MyData" type="DataCatalog" interval="2000" root="./mydata.dat" />`) | Central key/value persistence shared by modules.
FileCatalog | Plain files under a root (`ReadOnly` parameter blocks writes) | Serving/collecting files through query/reply.
RequestStore / MessageDataCatalog | Last-N messages (or extracted fields/blobs) per named store, with a mimetype | Exposing live values, images and status documents to PsyProbe and HTTP clients (`GET /api/query?from=RequestStore&query=VideoFrame`).

Whiteboards (see the User Guide) complete the picture for in-memory retrieval with a query language. For the full catalog concept and query API, see User Guide chapter 8.

## Operational checklist

- Set `maxsize`/`maxcount` caps on every recorder; check disk headroom under `root`.
- Keep trigger names stable between record and playback specs — they are the matching key.
- Version your recording directories (CoCoMaps used `%RecordingDir%` spec variables per demo) so a spec + recording pair stays reproducible.
- For standing demos and soak tests, use `rotate="yes"`; for latency-sensitive replay analysis, keep the recorded pacing (no `interval` override).
- Watch the recorder in PsyProbe’s Activity tab while recording — it shows exactly which messages are being captured.

PreviousDebugging & Traces NextDistributed Admin