CMLabs · Psyclone AIOS

1. Administering Psyclone

Command lines, system parameters, logging levels and command-line variable overrides — everything an operator sets before and while a system runs.

Starting Psyclone

Psyclone is started from the command line with a PsySpec file. On Windows: Psyclone.exe spec=mysystem.xml; on UNIX/Linux: ./Psyclone spec=mysystem.xml. Started with no parameters, Psyclone runs in satellite mode: an idle Node waiting to be claimed by a master system on another computer (see Distributed Admin).

# Run a system
./Psyclone spec=pingpong.xml

# Run a satellite node on a non-default port
./Psyclone port=11000

# Full production-style invocation
./Psyclone spec=robot.xml port=9000 html=/opt/psyclone/html verbose=2 debug=0

The classic convenience parameters are:

ParameterMeaningDefault
spec=<file>PsySpec XML file to loadnone (satellite mode)
port=<n>Main network port (inter-system comms and PsyProbe URL)10000
html=<dir>PsyProbe HTML directoryauto-detected ./html
verbose=<0-9> / debug=<0-9>System-wide log levels1 / 1
verbose-<topic>= / debug-<topic>=Per-topic log levels (see below)inherit

Command-line variable overrides Shipped

Any key=value argument on the command line is fed into the PsySpec variable substitution table before the spec is expanded. The command line wins over the spec's <variable> default — the spec provides sane defaults, the operator overrides per deployment without editing XML.

<psySpec>
  <variable name="nodename" value="devrig" />
  <variable name="RecordingDir" value="./recordings" />
  <module name="Status-%nodename%">
    ...
  </module>
</psySpec>
# Same spec, production values — cmdline overrides the <variable> defaults
./Psyclone spec=robot.xml port=9000 nodename=rig RecordingDir=/data/rec

The reserved parameters (spec=, port=, html=, verbose=, debug= and the verbose-<topic>=/debug-<topic>= forms) keep their established meanings and are not treated as spec variables. Substitution is a literal string replacement of %name% anywhere in the spec — names, cranks, trigger types, paths — so the result must remain valid XML.

Tip. Use variables for everything deployment-specific: ports, hostnames, data directories, credentials files. One spec, many rigs — the diff between machines lives entirely on the command line (or in a launcher script).

Logging: verbose and debug levels

Psyclone separates verbose output (operational narrative) from debug output (developer detail). Both range from 0 (errors only) to 9 (maximum detail); the default for all topics is 1. Levels can be set system-wide or per topic:

# System-wide
./Psyclone spec=sys.xml verbose=5 debug=0

# Per topic
./Psyclone spec=sys.xml verbose-network=5 debug-memory=0
TopicCovers
SystemStartup/shutdown, overall system lifecycle
NetworkConnections, interfaces, node-to-node traffic
MemoryShared memory and allocation activity
ProcessProcess spaces, external processes
SyncNode synchronisation (IDs, components, subscriptions)
NodeNode lifecycle and management
SpaceProcess space creation, restart, monitoring
ComponentComponent creation and lifecycle
SubscriptionsSubscription registration and matching
TriggersTrigger firing and dispatch
TimingTimers, intervals, delays

Each topic accepts the verbose-<topic>= and debug-<topic>= command-line forms (lower case, e.g. verbose-subscriptions=7).

Note. The reference manual also shows system-wide levels on the root element (<psySpec verbose="5" debug="0" logfile="../log.txt">). In the current code base the root-level attributes are not wired — use the command line for system-wide levels, and per-component attributes (below) for targeted logging. Roadmap

Per-module verbose, debug and logfile Shipped

Each component can carry its own logging configuration, independently of the system-wide levels. This is the tool of choice when one module misbehaves in a large system: crank up that module alone, and optionally route its output to a dedicated file, without drowning in output from everything else.

<module name="Simple" verbose="3" debug="2" logfile="../log2.txt">
  <trigger name="in" type="data.raw" />
  <crank name="c1" function="MyLib::Process" />
  <post name="out" type="data.cooked" />
</module>
  • verbose / debug — per-module levels (0–9), overriding the node-wide levels for that module's output.
  • logfile — routes the module's log lines to its own file instead of (only) the node log.

These attributes appeared in the original manual but were historically parsed without effect; they are now fully wired into central logging. They work on all component kinds — modules, whiteboards, catalogs.

Main port

The main port carries inter-system communication and the PsyProbe URL. Set it on the command line (port=6789) or in the spec root: <psySpec port="6789">. PsyProbe is then at http://<hostname>:6789/. Multiple protocols (HTTP, Telnet, Message) can share the main port via protocol autodetection — see Custom Interfaces.

PsyProbe HTML location and port

PsyProbe's web root is auto-detected: an html directory near the executable, checking the current and parent directories. Override it with html=<dir>/html on the command line or in the spec:

<psyprobe location="some/dir/html" port="6789" />

Normally PsyProbe shares the main port (protocol autodetect); the port attribute adds a dedicated PsyProbe port, and defaultport="no" removes PsyProbe from the main port entirely. The <psyprobe> element can also hold URL aliases and clickable test posts — covered in PsyProbe Configuration.

Operating recipes and bakes

A recipe is a PsySpec submitted to a running node's Builder and executed step-by-step at runtime, rather than only at startup. Submission is asynchronous: the node allocates a bake-ID, runs the bake on its own thread and returns immediately, so a bake never blocks the node. As an administrator you mostly care about three things — how many can run, how to watch one, and how to stop one.

LimitValueWhat happens at the edge
Concurrent bakes per node8Over-cap submissions are rejected outright (no queue). A rejected submit returns no bake-ID.
Terminal session retention60 sA finished bake's session is kept this long so a submitter can still read the outcome, then reaped.
Progress queue high-water64 msgsBeyond this the bake coalesces interim progress (below).
Lost-subscriber grace5 sQueue full continuously for this long ⇒ subscriber judged gone.
Live CLI output ring32 linesPer step; oldest lines drop first and rings never cross step boundaries.
Pause safety cap300 sA paused bake will not sit parked forever — see below.
Interactive idle timeout600 s defaultOverridable per recipe via idletimeout (ms).

Watching a bake in flight

Bakes publish Psyclone.BakeProgress on the normal message bus, so monitoring needs no special channel — subscribe like any other type. The bake-ID appears both in the message reference field and as a bakeid user entry, because filters match named user entries rather than header fields:

<trigger name="BP" type="Psyclone.BakeProgress">
  <filter type="haskey" key="bakeid" />
</trigger>

Messages carry a kind: start, step, interim (live %-progress, one per CLI output line) and final. Under load the bake protects what matters: start/step/final are never dropped, while interim messages coalesce so the newest percentage replaces the previous one. You therefore always see the framing and the latest progress, but not necessarily every intermediate percentage — that is by design, not packet loss.

Do not build alerting that assumes arrival order. Progress is delivered through shared memory and drained by a subscriber crank polling on its own schedule, so the order messages are recorded may differ from the order they were published. A 20% interim can legitimately be recorded after a 100% one. The guarantee is that the latest interim is not discarded — not that it arrives last. Key your dashboards off the highest % seen and the final message, never off “the last row in the log”.

Steering and stopping a bake

Send Psyclone.BakeControl with the bake-ID in the reference field. Verbs: start/resume, pause, terminate, skip, retry.

  • terminate takes effect immediately — including killing an in-flight <cli> child process. It does not wait for the current step to finish. This is the reliable way to stop a runaway bake.
  • pause parks the step loop, but not indefinitely: after the 300 s safety cap the configured expiry action fires — TERMINATE by default, or RESUME. A forgotten paused bake therefore cleans itself up rather than holding a slot forever.
  • retry only works on steps that opted in with retryable="yes". A retry aimed at a non-opted step is refused and logged. This is deliberate: re-running a side-effecting command is usually worse than failing.
  • skip advances past the current step.

Interactive bakes

A recipe carrying an <interactive> section stays alive after its last step, awaiting sender data, until released by a terminate or until the idle timeout expires (default 600 s, overridable). Status: the grammar and the keep-alive/timeout/subscriber rules described here are enforced today, but the actual bidirectional data channel (pushing stdin to the session, streaming ordered terminal output back) ships in phase 2.7. Two operational consequences:

  • An interactive submit with no listening subscriber is rejected at submit time. Only a sender can release such a bake, so with nobody attached it would occupy a slot until timeout. If interactive submissions are being refused, check the subscriber — not the recipe.
  • When a subscriber disappears mid-bake, policy differs by type: a non-interactive bake terminates (nobody is watching, so finish it), an interactive one pauses so a human can reattach and resume. A bake found sitting in pause with no subscriber is usually this, not a hang.
Capacity note. Because interactive bakes hold their slot while idle and the cap is 8 per node, a deployment that uses interactive recipes heavily should either lower idletimeout or plan for the slots. Ordinary bakes release their slot as soon as they finish.

Administration checklist

  • Pin the main port= explicitly in production; do not rely on the 10000 default.
  • Move deployment-specific values into <variable> entries and override them on the command line.
  • Run production at verbose=1 debug=0; raise levels per topic or per module when investigating (see Debugging & Traces).
  • Building the SSL variant and locking down interfaces is covered in Security & SSL.
  • If the deployment submits recipes, remember the per-node cap of 8 concurrent bakes and that interactive bakes hold a slot while idle — see Operating recipes and bakes above.
Psyclone AIOS · CMLabs