9. Advanced Applications: the CoCoMaps Case Study

The system

CoCoMaps (Collaborative Cognitive Maps, EU Echord++) put two TurtleBot robots and humans in a shared space: speech dialogue with turn-taking, face and human detection, ROS navigation, and joint task execution (including robots operating a simulated wall control panel). Each robot ran its own complete Psyclone system — partly onboard, partly on a server over WiFi, on top of ROS — and the systems cooperated by sharing observations, roles and tasks through one master Collaborative Cognitive Map. It is the best available worked example of “advanced Psyclone”, and everything in this chapter is drawn from its real specs and source.

The heart of the design is a set of four custom catalogs. A catalog in Psyclone is just a crank function with state and query handling — an exported int8 Fn(PsyAPI*), declared with <catalog type="...">, whose <setup> XML arrives as the componentsetup parameter string. All four follow the same skeleton: a waitForNewMessage loop that answers CTRL_QUERY messages from modules and HTTP calls from PsyProbe from the same code path.

System shape: one federation, many Psyclone systems

Each robot is an independent Psyclone system; the master robot (or server) runs the real CCM, and every other system runs a same-named proxy. Drag the diagram to explore the real dataflow:

Blue = modules/pipelines, orange = catalogs, charcoal = system boundaries. The CCM proxy on each slave forwards queries to the master over a remote <query host=... port=...>; modules never know the CCM is remote.

• Perception → CCM: ROS robot status and vision modules post Self.Position.Data, Human.Self.Detected, input.speech.detected…; CCMCollector catalogs map these into typed observations and push them to the CCM.
• Roles drive modes: a RoleNegotiator per robot takes/leaves CCM roles (Searcher/Communicator…); small gating modules translate Self.Role.* into dialog.on/off, navigation.auto.on/off, cmd.input.audio.on/off — switching the whole system’s operating mode via messages.
• Tasks close the loop: TaskNegotiator/TaskExecutor modules negotiate CCM tasks; executing one posts Robot.Request.Navigate.NamedPoint (resolved against CCM named points) and panel commands: robot ⇆ world model ⇆ actuation.
• Dialogue stack: microphone → Nuance speech reco/TTS, pitch tracking, and a turn-taking model run as a Python crank (<crank name="YTTM" language="python2" script="%YTTMDir%/yttm.py"/>), with the GUI in its own external space.
• ROS is just messages: the ROSInterface library wraps navigation/odometry/camera topics as cranks (including a simulator variant), so ROS is another message source/sink inside the psySpec.
• Observability & replay: a MessagesOfInterest whiteboard subscribes wildcards (dialog.*, Role.*, Task.*…) and a ReplayCatalog records ~30 message types for the repeatable-demo *_static_* spec variants (see chapter 7).
• PsyProbe as the app UI: <psyprobe location="%PsyDir%/html"> with an <alias name="robot" .../> serving a custom robot web UI, plus manual-test <post> entries so operators can inject commands from the browser.

CCMCatalog: the Collaborative Cognitive Map

The CCM is the shared world model: maps (floor plans with origin/offset/scale/bitmap), identities (known robots and humans, with portrait and face images), objects (live entities with type, identity link, first/last-seen, presence, media streams), timestamped observations (typed time series per object — e.g. Location points, String utterances), vantage points and named points (navigation targets), plus distributed roles and tasks with full assignment history. One master CCM serves the whole federation.

Behaviour

Startup reads three parameters — SystemID (this system’s ID in the federation), MaxKeep (max observations kept per typed store, default 1000) and StorageDir (disk directory, default CCMData) — then parses its <setup> (maps, identities, vantage/named points, roles). The main loop waits on api->waitForNewMessage(100) and answers CTRL_QUERY messages whose Query string entry selects the operation:

Replies go back with api->queryReply(inMsg->getReference(), status, replyMsg). Roles and tasks are EXCLUSIVE or SHARED, task states run Created → Assigned → Accepted → Status → Success/Failed/Timedout/Cancelled, and every transition is kept in a history log.

PsySpec declaration (real, trimmed — master)

<catalog name="CCMMaster" type="CCMCatalog">
  <parameter name="SystemID" type="Integer" value="%SystemID%" />
  <parameter name="StorageDir" type="String" value="%DataDir%/CCMData" />
  <trigger name="ObjectInfo" type="Object.Information" />
  <post name="CommandNavigateTo" type="Robot.Request.Navigate" />
  <post name="RoleAssigned" type="Role.Assigned" />
  <post name="TaskCreated"  type="Task.Created" />
  <post name="TaskTimeout"  type="Task.Timeout" />
  <!-- ...more Robot.Command.* / Report.* / Role.* / Task.* posts... -->
  <setup>
    <map name="main" view="10" invertx="yes" />
    <identity id="1" type="robot" name="RobotOne" nickname="Turtle1"
              image="%DataDir%/turtlebot.bmp" />
    <identity id="3" type="human" name="Thor List" nickname="Thor"
              image="%DataDir%/thor.bmp" face="%DataDir%/thor_face720.bmp" />
    <vantagepoints>
      <point id="1" label="%VantagePoint1Name%" data="%VantagePoint1Data%"
             overlap="%VantagePoint1Overlap%" />
    </vantagepoints>
    <namedpoints>
      <point label="ControlPanel1" data="%PanelPoint1Data%" />
    </namedpoints>
    <roles>
      <role name="Searcher" type="Shared" />
      <role name="Communicator" type="Exclusive" />
    </roles>
  </setup>
</catalog>

The proxy: transparent federation

Slave systems run CCMProxyCatalog under the same name as the master CCM (“must be named the same as the master CCM in every other system”). It forwards queries to the real master through a remote query route and re-emits the master’s events locally through the same post list — so slave modules are written exactly as if the CCM were local:

<catalog name="CCMMaster" type="CCMProxyCatalog">
  <parameter name="SystemID" type="Integer" value="%SystemID%" />
  <query name="Master" source="CCMMaster" host="%MasterAddress%" port="%MasterPort%" />
  <trigger name="ObjectInfo" type="Object.Information" />
  <!-- same post list as the master, so events re-emit locally -->
</catalog>

Feeding it: CCMCollector

CCMCollector is a bridge catalog that turns ordinary pub/sub messages into CCM observations. Its <setup> holds <obs> entries with <mapping> children; each trigger extracts fields from the message and sends one Query=AddObservations bundle to the CCM via api->queryCatalog(&resultMsg, "Master", outMsg):

<catalog name="PositionCollector1" type="CCMCollector">
  <query name="Master" source="CCMMaster" />
  <trigger name="RobotStatus"    type="Self.Position.Data" />
  <trigger name="HumanDetection" type="Human.Self.Detected" />
  <trigger name="IncomingSpeech" type="input.speech.detected" />
  <setup>
    <obs name="Location" obstype="Location" trigger="HumanDetection">
      <mapping entry="x" key="PosX" />
      <mapping entry="y" key="PosY" />
    </obs>
    <obs name="Utterance" obstype="String" trigger="IncomingSpeech">
      <mapping entry="val" key="Utterance" />
    </obs>
  </setup>
</catalog>

PsyProbe element

The catalog registers its own view with api->addPsyProbeCustomView("Cognitive Map", "elements/ccmcatalog.html") — a jCanvas 2-D map that draws the floor-plan bitmap, plots objects at their latest Location observations, overlays identity portraits (fetched via /api/query?...&query=identity&datatype=image&id=N), shows role/task badges per identity, and lets an operator click to post commands. It is the canonical example of a catalog serving both DataMessage queries (to modules) and JSON/image HTTP queries (to PsyProbe) from one query loop.

Companion cranks in the same source complete the pipeline: RoleNegotiator (default/primary/secondary role climbing with UpgradeRole/DowngradeRole triggers), TaskNegotiator (reacts to PerformTask/CancelTask and CCM task events), TaskExecutor (reacts to TaskAssigned/TaskTimeout/TaskCancelled and drives robot actions), and CCMTester (a periodic self-test).

TaskManagerCatalog: hierarchical task scripting

A declarative hierarchical task machine. Tasks (with arbitrarily nested subtasks) are defined in a JSON file; each task lists message triggers and message posts per event type (fail, success, complete, stop, timeout, start, next, previous, restart, startnow), plus timeout and start/between delays. The catalog turns the whole tree into pub/sub behaviour: incoming messages advance tasks, timers fire timeouts, and transitions post new messages.

Its single parameter is TaskFile (required). Two implementation details make it a masterclass in advanced catalog technique:

• Runtime subscription registration. After parsing the JSON it collects all trigger/post types and builds a <subscription> XML fragment at runtime, registering it with api->addSubscription(...) — subscriptions generated from a data file, not the psySpec. Posts that would self-trigger are dropped with a warning.
• Timer queue + trigger matching. The loop computes its wait from the next pending timer, matches incoming types via api->typeToText(inMsg->getType()), advances the tree, and emits api->postOutputMessage(post) for each resulting post.

<psySpec>
  <catalog name="TaskManager" type="TaskManagerCatalog">
    <parameter name="TaskFile" type="String" value="tasks.json" />
  </catalog>
</psySpec>

And the task file it consumes (real, trimmed — the robot searches for, then identifies, a participant):

[{ "name": "Search Participant",
   "triggers": { "start":   ["task.participants.none"] },
   "posts":    { "success": ["task.conversation.start"] },
   "delays":   { "start": 1000, "between": 1000 },
   "subtasks": [
     { "name": "Search for Participant", "timeout": 120000,
       "triggers": { "success": ["task.participants.tracked"],
                     "fail":    ["task.participants.none"] },
       "posts":    { "start":   ["task.participants.search"],
                     "timeout": ["task.participants.none"] } },
     { "name": "Identify Participant", "timeout": 30000, "...": "..." } ] }]

For PsyProbe it registers a TaskView tab (elements/hierarchicaldiagram.html) and publishes the live task tree as private data (api->setPrivateData("JSON", ..., "application/json")). The viewer is built on vis.js Network — the same offline bundle this guide uses — fetching the JSON via /api/getcomponentdata and rendering an interactive node graph with live status.

TimeSeriesCatalog: typed time-series & live charting

Records values extracted from messages into named, colored, capped datasets (integer, float, string, time) and serves them to queriers and to a live scrolling PsyProbe chart. CoCoMaps used it to visualise turn-taking dynamics — who has the floor, speech on/off, pitch — in real time. It has no parameters at all: everything is in the <setup>, with two tag kinds:

• <dataset name datatype color shoulders shading maxkeep maxfrequency /> — declares a series (default maxkeep 1000; maxfrequency f becomes a minimum ms-apart). shoulders is "0" (drop to zero around each sample) or "last" (repeat previous value) and controls the step shape; shading is fill opacity.
• <store dataset trigger key timekey durationkey value datatype /> — binds a message trigger to a dataset: extract key from the message (single values, arrays or maps — spread across the duration), or store a fixed value (e.g. 50 when a state turns on).

<catalog name="TimeSeriesCatalog" type="TimeSeriesCatalog">
  <trigger name="SpeechOff"      type="input.speech.off"/>
  <trigger name="IHaveTurn"      type="dialog.on.i-have-turn"/>
  <trigger name="OtherHasTurn"   type="dialog.on.other-has-turn"/>
  <trigger name="IStartSpeaking" type="output.audio.started"/>
  <trigger name="IStopSpeaking"  type="output.audio.ended"/>
  <setup>
    <dataset name="OtherPitch" datatype="integer" color="orange" shading="0"  shoulders="0" />
    <store dataset="OtherPitch" trigger="SpeechOff" key="pitchvalue" datatype="float" />

    <dataset name="ISpeak" datatype="integer" color="purple" shading="50" shoulders="last" />
    <store dataset="ISpeak" trigger="IStartSpeaking" value="50" datatype="integer" />
    <store dataset="ISpeak" trigger="IStopSpeaking"  value="0"  datatype="integer" />

    <dataset name="IHaveTurn" datatype="integer" color="blue" shading="80" shoulders="last" />
    <store dataset="IHaveTurn" trigger="IHaveTurn"    value="20" datatype="integer" />
    <store dataset="IHaveTurn" trigger="OtherHasTurn" value="0"  datatype="integer" />
  </setup>
</catalog>

Queries: Query=<datasetname> (with sinceid/sincetime/maxcount/maxage/minage) returns the dataset as JSON; List=all returns metadata for every dataset. The PsyProbe view (elements/timeseriescatalog.html) is a live strip chart built on SmoothieChart: it discovers datasets via list=all, then polls query=<dataset>&sinceid=<lastid> per series; clicking pauses/resumes the stream. (Smoothie for streaming, vis.js for graphs, Chart.js for static charts — three complementary options for custom views.)

ControlPanel: an interactive screen/menu catalog

ControlPanel simulates an operator console: a hierarchy of screens with selectable options, optional PIN-protected screens, and flashing selection feedback. Robots (via dialogue and tasks) or humans (via the PsyProbe view) navigate it — CoCoMaps demos had a robot walk to a wall panel and shut down “Generator 1”. It is a catalog rather than a module because it holds queryable state (the current screen) rather than transforming message streams.

Parameters: inifile (path to a JSON screen definition — despite the name; required) and print (debug). Operations return a PanelResponse: SUCCESS, UNKNOWN, BUSY, NOACCESS, ENTERPIN, NONE, FAILED. Queries distinguish web calls (an HTTP_OPERATION entry present) from module queries: HTML (rendered screen for PsyProbe), OptionSelect, BackNavigation, Reset, EnterPIN, AddPIN, ReadScreen, GetUpdatedView.

The bridge pattern: ControlPanelController

A companion crank bridges pub/sub to the panel: for every incoming trigger it copies the message, sets Query=<triggerName> and calls api->queryCatalog(&resultMsg, "ControlPanel", msg, 5000); based on the PanelResponse it posts <TriggerName>Success or <TriggerName>Failed with an Error string (Option Unknown / Panel Busy / Access Denied / PIN Required / Catalog Query Timeout…). This convention-based response posting is a reusable pattern for bridging query/reply into pub/sub anywhere:

<module name="PanelController1">
  <query name="ControlPanel" source="Panel1" />
  <trigger name="Reset"        type="Console.Command.Reset" />
  <trigger name="OptionSelect" type="Console.Command.Select" />
  <trigger name="EnterPIN"     type="Console.Command.EnterPIN" />
  <crank name="PanelController1" function="PsySystem::ControlPanelController" />
  <post name="OptionSelectSuccess"  type="Console.Command.Select.Success" />
  <post name="OptionSelectUnknown"  type="Console.Command.Select.Unknown" />
  <post name="OptionSelectPIN"      type="Console.Command.Select.EnterPIN" />
  <post name="EnterPINSuccess"      type="Console.Command.EnterPIN.Success" />
</module>

<catalog name="Panel1" type="ControlPanel">
  <parameter type="String" name="inifile" value="panel.json" />
  <parameter type="String" name="print"   value="yes" />
</catalog>

The same spec scripts a full panel interaction with a MessageScript crank — a <setup><track> of timed <post timems="+5000" name="OptionSelect" key="OptionName" value="status_screen"/> entries — Psyclone’s built-in way to script test interactions. The PsyProbe element (elements/controlpanel.html) is a thin shell that polls query=html and injects the panel’s own rendered HTML; clicks post back OptionSelect/EnterPIN/Reset. So across the four catalogs you see three PsyProbe integration styles: canvas drawing (CCM), charting (TimeSeries), and catalog-rendered HTML (ControlPanel).

What to steal for your own systems

• Catalogs are cranks with state. All four are plain exported functions; the <setup> XML arrives as the componentsetup parameter. Start from any module and add CTRL_QUERY handling.
• One query loop, two audiences. Serve module queryCatalog calls and PsyProbe HTTP (detect HTTP_OPERATION, answer with a JSON entry) from the same code path — the pattern behind every custom PsyProbe tab.
• Runtime subscriptions (api->addSubscription) let data files, not just the psySpec, define dataflow.
• Same-name proxy catalogs give transparent multi-system federation — slave modules don’t know the CCM is remote.
• Convention-based response posting (<Trigger>Success/<Trigger>Failed) bridges query/reply into pub/sub.
• Variables + <include> keep one spec family portable across master/slave/virtual/replay variants.