10. Testing (CMSDK & PsyTest)

Two suites, two jobs

The CMSDK unit-test framework

All unit tests are registered into one unified framework and driven from the Psyclone binary itself. The basics:

# run the full default suite
./bin/linux64/Psyclone test=cmsdk

# list every registered test, grouped by category, with descriptions
./bin/linux64/Psyclone test=list

# run a single test in isolation
./bin/linux64/Psyclone test=network_sslverify
./bin/linux64/Psyclone test=psy_configvars

(test=all and test=sdk are accepted aliases for test=cmsdk.) A test is any function bool Func() registered by name; the suite prints one status line per test and exits 0 only if every test passed — ready for CI.

Options

Interpreting the output

Each test prints a [ PASS ] or [ FAIL ] line; failures include the reason set by the test (unittest::fail("...")). While a test runs on a terminal, a live progress line shows percent and current action (unittest::progress(pct, action)); piped/redirected output stays clean. Tests can also record named performance metrics (unittest::metric(name, value, unit)) which appear on the PASS line and land in the performance JSON used by compare=.

Tests worth knowing by name

PsyTest: the whole-system function + performance test

PsyTest is a self-checking Psyclone system: an ordinary PsySpec (Examples/psytest.xml) whose modules (cranks in Examples/src/PsyTest.cpp, library Examples) exercise the main single-node features in a chain of stages, each printing one pass/fail line with performance data:

./bin/linux64/Psyclone spec=Examples/psytest.xml

The stage chain runs sequentially, each stage activating the next via a Done post to a sibling Test.* context:

Intro → TimeTest → PingTest → PingTestUDP → Signals → Contexts → Catalogs → Done

Tuning the run

The spec exposes its knobs as <variable> definitions (Nodes, PingCount, PingCycles, SignalCount, ContextRounds…), all overridable from the command line. The ping stages also take a PayloadSize parameter (bytes of message payload per ball; 0 = empty), so you can measure throughput and latency at realistic message sizes:

# heavier ping load with 4 KB payloads
./bin/linux64/Psyclone spec=Examples/psytest.xml PingCount=100 PingCycles=100 PayloadSize=4096

Tail latency

The ping masters collect a per-message latency sample for every ball and report tail-latency percentiles alongside the averages, formatted as:

p50=42.1us p90=61.3us p95=70.8us p99=112.5us p99.9=413.0us

Averages hide stalls; p99/p99.9 expose scheduling hiccups, queue buildup and GC-like pauses. Track the percentiles across builds if latency matters to your application.

Reading the output

• Each stage prints exactly one [ PASS ]/[ FAIL ] line (log level 0) with its performance data; per-message chatter is at verbose level ≥ 2 (verbose=2 to see it).
• Stall watchdogs: the ping/signal masters detect a stalled ring (no progress for ~3 s) and report [ FAIL ] … stalled instead of hanging forever.
• During the context-switching stage, Msg filtered in getTriggersForMsg, filter 1 lines are expected and benign: they are the inactive responder’s pings being correctly filtered — proof the switch worked.
• A process-global result table aggregates per-stage results for the final verdict (all modules run in one process on a single node).
• PsyProbe runs throughout at http://localhost:10000 — watching the Activity and Performance tabs during a PsyTest run is an excellent way to learn both.

Adding your own tests

A new CMSDK unit test

Write a bool function using the unittest:: helpers and register it (engine-level tests live in Psyclone/src/PsycloneTests.cpp; SDK tests register next to the object they test, e.g. the SSL tests in NetworkManager.cpp):

static bool UnitTest_MyThing() {
    unittest::progress(10, "setting up");
    // ... exercise the object ...
    if (bad) { unittest::fail("expected X, got Y"); return false; }
    unittest::metric("ops_per_s", ops, "ops/s");
    return true;
}
// at registration time:
UnitTestRunner::instance().registerTest("my_thing", UnitTest_MyThing,
    "one-line description", "category", true /* in default run */);

It is then runnable as test=my_thing, appears in test=list, and (with the last argument true) joins the default test=cmsdk suite. Use unittest::detail(...) for verbose-only diagnostics and call unittest::progress() at phase boundaries so a timeout can report where a hang occurred.

A new PsyTest stage

Add a crank to Examples/src/PsyTest.cpp (declared extern "C" in include/PsyTest.h), declare its module in Examples/psytest.xml under a new sibling Test.* context, chain it in by having the previous stage’s Done post activate your context (and yours activate the next), report into the shared result table, and print a single [ PASS ]/[ FAIL ] line at level 0 with a stall watchdog if your stage forms a ring. The existing stages are short and idiomatic — copy the nearest one.