

//_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
//_/_/
//_/_/  This file is part of CMSDK version 2.1.1.260726 (26 July 2026)
//_/_/  
//_/_/  developed by
//_/_/  
//_/_/  Communicative Machines Ltd. 
//_/_/    http://cmlabs.com/
//_/_/  Copyright (c) 2026
//_/_/  
//_/_/  in collaboration with 
//_/_/  
//_/_/  Icelandic Institute for Intelligent Machines ses
//_/_/    Menntavegur 1
//_/_/    101 Reykjavik
//_/_/    ICELAND
//_/_/    http://www.iiim.is/
//_/_/
//_/_/  and is released under the following
//_/_/  
//_/_/  EXTENDED BSD License
//_/_/
//_/_/  Redistribution and use in source and binary forms, with or without
//_/_/  modification, is permitted provided that the following conditions 
//_/_/  are met:
//_/_/
//_/_/  - Redistributions of source code must retain the above copyright 
//_/_/    and collaboration notice, this list of conditions and the following 
//_/_/    disclaimer.
//_/_/
//_/_/  - Redistributions in binary form must reproduce the above copyright 
//_/_/    notice, this list of conditions and the following disclaimer in the 
//_/_/    documentation and/or other materials provided with the distribution.
//_/_/
//_/_/  - Neither the name of its copyright holders nor the names of its 
//_/_/    contributors may be used to endorse or promote products derived from 
//_/_/    this software without specific prior written permission.
//_/_/
//_/_/  - CADIA Clause: The license granted in and to the software under this 
//_/_/    agreement is a limited-use license. The software may not be used in 
//_/_/    furtherance of: (i) intentionally causing bodily injury or severe 
//_/_/    emotional harm to any person; (ii) invading the personal privacy or 
//_/_/    violating the human rights of any person; or (iii) committing or 
//_/_/    preparing for any act of war. 
//_/_/
//_/_/  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
//_/_/  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
//_/_/  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 
//_/_/  PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER 
//_/_/  OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 
//_/_/  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 
//_/_/  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 
//_/_/  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 
//_/_/  LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
//_/_/  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 
//_/_/  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//_/_/
//_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
/*! @file MemoryManager.h
    @brief Central shared-memory manager for a Psyclone node: master segment, per-subsystem shared maps and the page-based legacy allocator.

    This header is the hub of the CMSDK shared-memory core. A Psyclone system runs as a
    collection of separate OS processes ("spaces"), each hosting one or more components.
    All inter-process state — component registries, process/queue tables, data maps and the
    temporal message store — lives in named shared-memory segments so that any process can
    crash without corrupting the others (crash isolation), and can re-attach on restart.

    The MemoryManager singleton owns/attaches five segments per node instance (keyed by port):
    - the *master* segment (MasterMemory / MemoryMasterStruct): sizes, serials and time sync,
    - TemporalMemory: slot/bin allocator for time-limited DataMessages,
    - ProcessMemory: process table plus per-process command/message/signal/request queues,
    - ComponentMemory: component registry with parameters, stats and private data,
    - DataMapsMemory: type/context/tag/crank/request name-to-id maps.

    On Windows the segments are named file mappings (CreateFileMapping/MapViewOfFile);
    on POSIX systems (Linux, macOS) they are shm_open/mmap regions — see the shared-memory
    helpers in Utils for the actual primitive. The structs in this file are the raw
    on-segment layouts.

    @warning ABI/layout sensitivity: every struct in this header is mapped simultaneously
    into multiple processes. Their size, field order and alignment are a de-facto ABI.
    If you change any of these structs you MUST rebuild every binary that maps the segment
    (`make clean && make`); mixing binaries built from different layouts silently corrupts
    shared memory across processes (an ODR/layout violation with no compiler diagnostic).

    @defgroup Memory Shared-Memory Subsystem
    @brief Shared-memory segments, maps, queues and allocators used for inter-process communication in Psyclone.

    @par Why shared memory
    Psyclone deliberately runs as multiple cooperating OS processes rather than one big
    multithreaded program: a crash in one component's process cannot take down the rest
    of the system (crash isolation), processes can be restarted and re-attach to the
    live state, and external tools can inspect a running node without stopping it. The
    price is that all shared state must live in named shared-memory segments instead of
    ordinary heap objects — and this group provides everything needed to make that
    practical: the MemoryManager singleton that creates/attaches the segments, typed
    name→id maps (MemoryMaps.h, DataMapsMemory.h), ring-buffer message queues
    (MemoryQueues.h, ProcessMemory.h), the temporal message store (TemporalMemory.h)
    and the page-based legacy allocator.

    @par Why not plain OS APIs or Boost.Interprocess
    The raw primitives differ per platform — CreateFileMapping/MapViewOfFile on
    Windows, shm_open/mmap on Linux/macOS, with different naming, permission and
    cleanup semantics — and neither the OS nor std provides message queues, typed maps
    or a temporal store on top of them. CMSDK wraps the primitives once (in
    @ref utils, used by MemoryManager) and builds the higher-level containers in a way
    that integrates directly with @ref Messaging — a DataMessage's flat layout means it
    can be memcpy'd into a queue slot with no serialisation, and named semaphores give
    cross-process blocking waits.

    @par How to use it: producer/consumer via a shared queue
    Unrelated processes rendezvous on a queue by global name; each message is copied
    into shared memory on add and copied back out on receive, so no pointers ever
    cross the process boundary.
    @code
    // Process A (producer)
    uint32 qid;
    MessageQueue::CreateMessageQueue(qid, "vision.in");   // create/attach by name
    DataMessage* msg = new DataMessage(myType, myComponentID);
    msg->setString("camera", "front-left");
    MessageQueue::AddMessageToQueue(qid, msg);            // copied into the segment
    delete msg;                                           // producer keeps ownership

    // Process B (consumer) — blocks up to 100 ms on a named semaphore
    DataMessage* in = MessageQueue::WaitForMessageQueue(qid, 100);
    if (in) {
        const char* cam = in->getString("camera");
        // ... process ...
        delete in;                                        // heap copy owned by caller
    }
    @endcode
    For keyed lookup instead of FIFO delivery, use the shared maps (GenericMemoryMap
    and the registries in DataMapsMemory.h), which map names/ids to entries inside a
    segment the same way.

    @warning Struct-layout stability is the cardinal rule of this group. Every struct
    stored in a segment is mapped simultaneously into several processes, so its size,
    field order and alignment form a silent ABI. After changing ANY such struct,
    rebuild everything (`make clean && make`); running old and new binaries against
    the same segment corrupts memory across processes with no compiler or runtime
    diagnostic.

    @note Segments are keyed by node port, so several Psyclone instances can coexist
    on one machine; the machine-global instance index (PsycloneIndexHeader) lets them
    discover each other.

    @see MemoryManager, MessageQueue, GenericMemoryMap, TemporalMemory.h, @ref Messaging
    @{
*/

#if !defined(_MEMORYMANAGER_H_)
#define _MEMORYMANAGER_H_

#include "MemoryController.h"
#include "Utils.h"
#include "ThreadManager.h"
#include "DataMessage.h"

namespace cmlabs {

/*! @name Node lifecycle status values (PsycloneIndexEntry::status)
    @{ */
#define PSYCLONE_STATUS_NONE	0	/*!< No instance recorded in this slot. */
#define PSYCLONE_STATUS_ENDED	1	/*!< Instance has shut down (or crashed and was reaped). */
#define PSYCLONE_STATUS_INIT	2	/*!< Instance is starting up, not yet serving. */
#define PSYCLONE_STATUS_READY	3	/*!< Instance is fully operational. */
/*! @} */

/*! Fixed byte size of the per-message draft slots used in the recent-message rings of
    ComponentStats / ProcessStats. Messages larger than this are truncated when recorded. */
#define DRAFTMSGSIZE		4096

/*! @brief Header of the machine-global Psyclone instance index shared-memory segment.

    One well-known segment per machine records every Psyclone node instance currently or
    recently running on this computer, so tools and new instances can discover each other
    by port. The header is immediately followed in memory by PSYCLONE_INDEX_MAXCOUNT
    PsycloneIndexEntry records.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct PsycloneIndexStruct {
	uint32 size;	/*!< Total size of the index segment in bytes (header + entries). */
	uint32 cid;		/*!< Check/magic id used to validate the segment on attach. */
	uint32 count;	/*!< Number of entry slots that have ever been used. */
	// PsycloneIndexEntries
};

/*! @brief One slot in the machine-global Psyclone instance index.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct PsycloneIndexEntry {
	uint16 port;		/*!< TCP port the instance listens on; doubles as the instance key. */
	uint8 status;		/*!< One of the PSYCLONE_STATUS_* values. */
	uint64 heartbeat;	/*!< Last heartbeat timestamp (µs since epoch); staleness implies a dead instance. */
	uint64 instanceID;	/*!< Unique id of the owning instance, used to detect port reuse. */
};

/*! Maximum number of concurrently tracked Psyclone instances per machine. */
#define PSYCLONE_INDEX_MAXCOUNT	128
/*! Total byte size of the instance-index shared-memory segment. */
#define PSYCLONE_INDEX_SIZE		(sizeof(PsycloneIndexStruct) + (PSYCLONE_INDEX_MAXCOUNT * sizeof(PsycloneIndexEntry)))

/*! @brief Accessor for the machine-global instance index segment.
    @ingroup Memory

    Each running node creates one PsycloneIndex, registers its port/instance id via init(),
    and keeps its status/heartbeat fresh via setStatus(). Other processes query liveness
    with the static GetStatus() without needing an object of their own.
    Thread-safety: updates are simple word writes into shared memory; concurrent readers
    may observe a slightly stale status, which is acceptable for liveness polling. */
class PsycloneIndex {
public:
	/*! Look up the status of the instance registered on @a port.
	    @param port Node port to query.
	    @param status Receives the PSYCLONE_STATUS_* value.
	    @param heartbeat Receives the last heartbeat timestamp (µs).
	    @return true if an entry for @a port exists in the index. */
	static bool GetStatus(uint16 port, uint8& status, uint64& heartbeat);

	PsycloneIndex();
	~PsycloneIndex();
	/*! Attach to (or create) the index segment and claim/refresh the slot for this instance.
	    @param port Port this node listens on. @param instID Unique instance id.
	    @return true on success. */
	bool init(uint16 port, uint64 instID);
	/*! Update this instance's status and heartbeat in the shared index.
	    @param status New PSYCLONE_STATUS_* value. @return true if the entry was written. */
	bool setStatus(uint8 status);
private:
	PsycloneIndexStruct* index;	/*!< Mapped view of the shared index segment (not owned heap memory). */
	PsycloneIndexEntry* entry;	/*!< This instance's slot within the mapped segment. */
};

/*! @brief Raw per-component performance counters sampled inside shared memory.
    @ingroup Memory
    A snapshot of a component's resource usage and message throughput, written by the
    hosting process and read by the node for scheduling/monitoring decisions.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct PerfStats {
	uint32 compID;	/*!< Component id these stats belong to. */
	uint16 spaceID;	/*!< Id of the space (process slot) hosting the component. */
	uint16 nodeID;	/*!< Id of the node the component runs on. */
	uint32 osID;	/*!< OS process id of the hosting process. */
	uint8 type;		/*!< Component type (Whiteboard, Catalog, Module, ...). */
	uint8 status;    // Component status (idle, running, etc.)

	uint64 currentCPUTicks;		/*!< Cumulative CPU ticks consumed so far. */

	uint64 currentMemoryBytes;	/*!< Current memory footprint in bytes. */

	uint64 totalInputBytes;		/*!< Total bytes received as input messages. */
	uint64 totalInputCount;		/*!< Total number of input messages. */
	uint64 totalOutputBytes;	/*!< Total bytes posted as output messages. */
	uint64 totalOutputCount;	/*!< Total number of output messages. */

	uint64 runCount;			/*!< Number of triggered runs executed. */
	uint64 totalQueueBytes;		/*!< Bytes currently waiting in the component's queues. */
	uint32 totalQueueCount;		/*!< Messages currently waiting in the component's queues. */

	uint64 firstRunStartTime;	/*!< Timestamp (µs) of the very first run. */
	uint64 currentRunStartTime;	/*!< Timestamp (µs) the current/last run started. */
	uint64 totalCycleCount;		/*!< Total processing cycles across all runs. */
	uint64 totalRunCount;		/*!< Total runs across the component's lifetime. */
	uint32 migrationCount;		/*!< How many times the component migrated between spaces/nodes. */
};

/*! @brief Sliding-window averages derived from successive PerfStats snapshots.
    @ingroup Memory
    Each metric is kept for three averaging windows whose lengths (ms) are given in
    periodsMS[0..2] (typically short/medium/long). Stored in shared memory next to the
    component/process it describes; serialized for monitoring UIs via toXML().
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct AveragePerfStats {
	uint64 time;			/*!< Timestamp (µs) of the last update. */
	uint32 periodsMS[3];	/*!< The three averaging window lengths in milliseconds. */

	PerfStats lastStats;	/*!< Most recent raw snapshot used for delta computation. */

	double percentOfSystemCPU[3];		/*!< Share of the whole Psyclone system's CPU use, per window. */
	double percentOfOSCPU[3];			/*!< Share of the hosting OS process's CPU use, per window. */
	double localSystemCPUUsage[3];		/*!< CPU usage of the local Psyclone system, per window. */
	double localSystemMemoryUsage[3];	/*!< Memory usage of the local Psyclone system, per window. */
	double computerCPUUsage[3];			/*!< Whole-machine CPU usage, per window. */
	double computerMemoryUsage[3];		/*!< Whole-machine memory usage, per window. */

	uint64 maxMemoryBytes;			/*!< Peak memory footprint observed. */
	double memoryBytesAverage[3];	/*!< Average memory footprint, per window. */

	double dataInputBytesPerSec[3];		/*!< Input throughput in bytes/s, per window. */
	double dataInputCountPerSec[3];		/*!< Input message rate in msgs/s, per window. */
	double dataOutputBytesPerSec[3];	/*!< Output throughput in bytes/s, per window. */
	double dataOutputCountPerSec[3];	/*!< Output message rate in msgs/s, per window. */

	double dataQueueBytesAverage[3];	/*!< Average queued bytes, per window. */
	uint64 dataQueueCountAverage[3];	/*!< Average queued message count, per window. */

	double cycleCountPerSec[3];			/*!< Processing cycles per second, per window. */

	/*! Render all averaged metrics as a single \<performance/\> XML element for monitoring.
	    @return XML fragment; never fails. */
	std::string toXML() {
		return utils::StringFormat("<performance time=\"%llu\" ms1=\"%u\" ms2=\"%u\" ms3=\"%u\" maxmemory=\"%llu\" "
			"percentofsystemcpu1=\"%f\" percentofsystemcpu2=\"%f\" percentofsystemcpu3=\"%f\" "
			"percentofoscpu1=\"%f\" percentofoscpu2=\"%f\" percentofoscpu3=\"%f\" "
			"memorybytesaverage1=\"%f\" memorybytesaverage2=\"%f\" memorybytesaverage3=\"%f\" "
			"datainputbytespersec1=\"%f\" datainputbytespersec2=\"%f\" datainputbytespersec3=\"%f\" "
			"datainputcountpersec1=\"%f\" datainputcountpersec2=\"%f\" datainputcountpersec3=\"%f\" "
			"dataoutputbytespersec1=\"%f\" dataoutputbytespersec2=\"%f\" dataoutputbytespersec3=\"%f\" "
			"dataoutputcountpersec1=\"%f\" dataoutputcountpersec2=\"%f\" dataoutputcountpersec3=\"%f\" "
			"dataqueuebytesaverage1=\"%f\" dataqueuebytesaverage2=\"%f\" dataqueuebytesaverage3=\"%f\" "
			"dataqueuecountaverage1=\"%f\" dataqueuecountaverage2=\"%f\" dataqueuecountaverage3=\"%f\" "
			"cyclecountpersec1=\"%f\" cyclecountpersec2=\"%f\" cyclecountpersec3=\"%f\" "
			"localsystemcpuusage1=\"%f\" localsystemcpuusage2=\"%f\" localsystemcpuusage3=\"%f\" "
			"localsystemmemoryusage1=\"%f\" localsystemmemoryusage2=\"%f\" localsystemmemoryusage3=\"%f\" "
			"computercpuusage1=\"%f\" computercpuusage2=\"%f\" computercpuusage3=\"%f\" "
			"computermemoryusage1=\"%f\" computermemoryusage2=\"%f\" computermemoryusage3=\"%f\" "
			"/>\n",
			time, periodsMS[0], periodsMS[1], periodsMS[2], maxMemoryBytes,
			percentOfSystemCPU[0], percentOfSystemCPU[1], percentOfSystemCPU[2],
			percentOfOSCPU[0], percentOfOSCPU[1], percentOfOSCPU[2],
			memoryBytesAverage[0], memoryBytesAverage[1], memoryBytesAverage[2],
			dataInputBytesPerSec[0], dataInputBytesPerSec[1], dataInputBytesPerSec[2],
			dataInputCountPerSec[0], dataInputCountPerSec[1], dataInputCountPerSec[2],
			dataOutputBytesPerSec[0], dataOutputBytesPerSec[1], dataOutputBytesPerSec[2],
			dataOutputCountPerSec[0], dataOutputCountPerSec[1], dataOutputCountPerSec[2],
			dataQueueBytesAverage[0], dataQueueBytesAverage[1], dataQueueBytesAverage[2],
			dataQueueCountAverage[0], dataQueueCountAverage[1], dataQueueCountAverage[2],
			cycleCountPerSec[0], cycleCountPerSec[1], cycleCountPerSec[2],
			localSystemCPUUsage[0], localSystemCPUUsage[1], localSystemCPUUsage[2],
			localSystemMemoryUsage[0], localSystemMemoryUsage[1], localSystemMemoryUsage[2],
			computerCPUUsage[0], computerCPUUsage[1], computerCPUUsage[2],
			computerMemoryUsage[0], computerMemoryUsage[1], computerMemoryUsage[2]
			);
	}
};


/*! @brief Root header of the node's *master* shared-memory segment.
    @ingroup Memory
    The master segment is the rendezvous point for all other segments of a node instance:
    it stores their current sizes and *serial numbers*. Whenever a segment is resized it is
    re-created under a new serial; attached processes compare their cached serial against
    the value here (see the CHECK*MEMORYSERIAL macros) and re-open the segment when stale.
    It also carries node-wide time synchronisation and aggregate usage counters.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct MemoryMasterStruct {
	uint64 size;			/*!< Total size of this segment in bytes. */
	uint32 cid;				/*!< Check/magic id used to validate the segment on attach. */
	uint16 nodeID;			/*!< Cluster-wide id of this node. */
	uint16 status;			/*!< Node readiness, 0-10; 10 means fully ready. */
	uint64 createdTime;		/*!< Creation timestamp of the segment (µs). */
	// uint64 timesync;		// time sync
	uint64 currentTMC;			/*!< Current time-master clock value (µs) for cluster time sync. */
	int64 localSyncAdjustment;	/*!< Signed offset applied to local clock to match the time master. */
	uint64 memAlloc;		/*!< Total bytes currently allocated across all segments. */
	uint64 memUsage;		/*!< Total bytes currently in use across all segments. */
	uint64 currentCPUTicks;		/*!< Aggregate CPU ticks consumed by the node. */
	uint32 componentCount;	/*!< Number of registered components. */
	uint32 throughputCount;	/*!< Total messages passed through the node. */
	uint64 throughputSize;	/*!< Total message bytes passed through the node. */
	uint32 dynamicShmemSerial;		/*!< Serial of the dynamic (temporal) segment; bumped on every resize. */
	uint32 processShmemSerial;		/*!< Serial of the process segment; bumped on every resize. */
	uint32 componentShmemSerial;	/*!< Serial of the component segment; bumped on every resize. */
	uint32 datamapsShmemSerial;		/*!< Serial of the data-maps segment; bumped on every resize. */
	uint32 dynamicShmemLocks;		/*!< Diagnostic lock counter for the dynamic segment. */
	uint32 processShmemLocks;		/*!< Diagnostic lock counter for the process segment. */
	uint32 componentShmemLocks;		/*!< Diagnostic lock counter for the component segment. */
	// uint32 datamapsShmemLocks; // nothing ever locks in here beyond the simple mutex
	uint64 dynamicShmemSize;	/*!< Current byte size of the dynamic (temporal) segment. */
	uint64 processShmemSize;	/*!< Current byte size of the process segment. */
	uint64 componentShmemSize;	/*!< Current byte size of the component segment. */
	uint64 datamapsShmemSize;	/*!< Current byte size of the data-maps segment. */
};

class MemoryManager;
/*! @brief Handle to the node's master shared-memory segment (MemoryMasterStruct).
    @ingroup Memory
    Implements the MemoryController interface used by the subsystem memories
    (TemporalMemory, ProcessMemory, ComponentMemory, DataMapsMemory) to read/bump the
    resize serials and sizes of their segments. Exactly one process per node calls
    create(); all others attach via open(). All getters/setters operate directly on the
    mapped struct under a named cross-process mutex; calls may block briefly on that
    mutex but never for long. The mapped memory is owned by the OS, not this object. */
class MasterMemory : public MemoryController {
	friend class MemoryManager;
public:
	MasterMemory();
	~MasterMemory();
	/*! Create the master segment for the node instance listening on @a port.
	    @param port Node port; used to derive the segment's global name. @return true on success. */
	bool create(uint16 port);
	/*! Attach to an existing master segment created by another process.
	    @param port Node port identifying the segment. @return true if found and mapped. */
	bool open(uint16 port);

	/*! @return The cluster-wide node id stored in the master segment. */
	uint16 getNodeID();
	/*! Set the cluster-wide node id. @param id New node id. @return true on success. */
	bool setNodeID(uint16 id);
	/*! @return Creation timestamp (µs) of the master segment. */
	uint64 getCreatedTime();

	/*! @name Segment resize serials
	    Each subsystem segment has a serial that is incremented whenever the segment is
	    re-created (resized). Attached processes compare a cached serial to detect that
	    their mapping is stale and must be re-opened.
	    @{ */
	uint32 getDynamicShmemSerial();			/*!< @return Current serial of the dynamic (temporal) segment. */
	uint32 getProcessShmemSerial();			/*!< @return Current serial of the process segment. */
	uint32 getComponentShmemSerial();		/*!< @return Current serial of the component segment. */
	uint32 getDataMapsShmemSerial();		/*!< @return Current serial of the data-maps segment. */
	uint32 incrementDynamicShmemSerial();	/*!< Bump the dynamic segment serial after resize. @return New serial. */
	uint32 incrementProcessShmemSerial();	/*!< Bump the process segment serial after resize. @return New serial. */
	uint32 incrementComponentShmemSerial();	/*!< Bump the component segment serial after resize. @return New serial. */
	uint32 incrementDataMapsShmemSerial();	/*!< Bump the data-maps segment serial after resize. @return New serial. */
	/*! @} */

	/*! @name Segment sizes
	    Current byte sizes of the four subsystem segments, kept here so any process can
	    map the right amount of memory when (re-)attaching.
	    @{ */
	uint64 getDynamicShmemSize();			/*!< @return Byte size of the dynamic (temporal) segment. */
	uint64 getProcessShmemSize();			/*!< @return Byte size of the process segment. */
	uint64 getComponentShmemSize();			/*!< @return Byte size of the component segment. */
	uint64 getDataMapsShmemSize();			/*!< @return Byte size of the data-maps segment. */
	bool setDynamicShmemSize(uint64 size);	/*!< Record a new dynamic segment size. @param size Bytes. @return true on success. */
	bool setProcessShmemSize(uint64 size);	/*!< Record a new process segment size. @param size Bytes. @return true on success. */
	bool setComponentShmemSize(uint64 size);/*!< Record a new component segment size. @param size Bytes. @return true on success. */
	bool setDataMapsShmemSize(uint64 size);	/*!< Record a new data-maps segment size. @param size Bytes. @return true on success. */
	/*! @} */

	bool setID(uint16 id);	/*!< MemoryController interface: set the node id. @param id New id. @return true on success. */
	uint16 getID();			/*!< MemoryController interface: @return the node id. */

	uint16 port;	/*!< Node port this master segment belongs to (names all related segments). */

private:
	utils::Mutex* mutex;			/*!< Named cross-process mutex guarding the master struct. */
	MemoryMasterStruct* master;		/*!< Mapped view of the master segment (owned by the OS). */
};

class TemporalMemory;
class ProcessMemory;
class ComponentMemory;
class DataMapsMemory;

/*! @brief Top-level facade of the shared-memory subsystem for one process.
    @ingroup Memory
    Every Psyclone process creates exactly one MemoryManager (exposed via ::Singleton).
    The node's master process calls create() to lay out all segments; every other space
    process calls connect() to attach to them. After that, the four public subsystem
    pointers (temporalMemory, processMemory, componentMemory, dataMapsMemory) give typed
    access to the shared state. A background management thread (MemoryManagement) runs in
    the master process to expire end-of-life data and perform maintenance.

    Ownership: the MemoryManager owns the subsystem accessor objects (deleted in the
    destructor); the shared segments themselves outlive any single process and are owned
    by the OS. Thread-safety: all subsystem calls take their own cross-process mutexes;
    the facade itself is safe to use from multiple threads.

    @code
    // Master process:
    MemoryManager* mm = new MemoryManager();
    mm->create(port);                       // create all shared segments
    // Any space process:
    MemoryManager* mm2 = new MemoryManager();
    mm2->connect(port, false);              // attach to existing segments
    uint64 id;
    mm2->insertMessage(msg, id);            // publish a message into shared memory
    DataMessage* copy = mm->getCopyOfMessage(id); // retrieve it from any process
    @endcode */
class MemoryManager {
public:
	static MemoryManager* Singleton;	/*!< Per-process singleton instance, set by the constructor. */

	//static bool GetMemoryUsage(uint32& total, uint32& usage, uint32& sysTotal, uint32& sysUsage, uint32& staticTotal, uint32& staticUsage, uint32& dynamicTotal, uint32& dynamicUsage);
	//static bool InsertMemoryBlock(const char* data, uint32 size, uint64 eol, uint64& id);
	//static bool OverwriteMemoryBlock(uint64 id, const char* data, uint32 size, uint64 eol);
	//static char* GetAndLockMemoryBlock(uint64 id, uint32& size, uint64& eol);
	//static bool UnlockMemoryBlock(uint64 id);
	//static char* GetCopyMemoryBlock(uint64 id, uint32& size, uint64& eol);

	/*! Self-test of the shared-memory manager. @return true if all checks pass. */
	static bool UnitTest();
	/*! Self-test of the raw shared-memory primitives. @return true if all checks pass. */
	static bool ShmUnitTest();

	friend THREAD_RET THREAD_FUNCTION_CALL MemoryManagement(THREAD_ARG arg);

	MemoryManager();
	~MemoryManager();

	/*! Report allocation/usage figures for the system and data segments.
	    @param sysAlloc Receives bytes allocated for system segments.
	    @param sysUsage Receives bytes in use in system segments.
	    @param dataAlloc Receives bytes allocated for data (temporal) memory.
	    @param dataUsage Receives bytes in use in data memory.
	    @return true on success. */
	bool getMemoryUsage(uint64& sysAlloc, uint64& sysUsage, uint64& dataAlloc, uint64& dataUsage);

	/*! Get the node status from the master segment.
	    @param lastseen Receives the last heartbeat timestamp (µs).
	    @param createdTime Receives the segment creation time (µs).
	    @return Node status value (0-10, 10 = ready). */
	uint8 getNodeStatus(uint64& lastseen, uint64& createdTime);

	// Connect to existing PageMaster in shared memory
	/*! Attach this process to an existing node's shared segments.
	    @param sysID Node port/system id whose segments to open.
	    @param isMaster true if this process should run the management thread.
	    @return true if all segments were found and mapped. */
	bool connect(uint16 sysID, bool isMaster);

	// Create a new PageMaster in shared memory
	/*! Create all shared segments for a new node instance (master process only).
	    @param sysID Node port/system id used to name the segments.
	    @param slotCount Number of message slots in temporal memory.
	    @param binCount Number of block-size bins in temporal memory.
	    @param minBlockSize Smallest bin block size in bytes.
	    @param maxBlockSize Largest bin block size in bytes.
	    @param initSize Initial temporal segment size in bytes.
	    @param maxSize Maximum temporal segment size in bytes.
	    @param force If true, replace any stale existing segments.
	    @return true on success.
	    @warning Only one process per node may call this; all others must use connect(). */
	bool create(uint16 sysID, uint32 slotCount = 100000, uint16 binCount = 2, uint32 minBlockSize = 1024, uint32 maxBlockSize = 64*1024, uint64 initSize = 50000000L, uint64 maxSize = 1000000000L, bool force = false);
		//uint16 sysID, uint32 slotCount = 100, uint32 slotDuration = 100000, uint64 initSize = 10000000, uint64 maxSize = 100000000, uint32 growSteps = 8, bool force = false);

	// Insert new block of memory and return full id
	/*! Publish a DataMessage into shared (temporal) memory.
	    @param msg Message to copy into shared memory; caller retains ownership of @a msg.
	    @param id Receives the cluster-unique message id (node|serial|slot encoded).
	    @return true on success. May block briefly on the temporal-memory mutex. */
	bool insertMessage(DataMessage* msg, uint64& id);
	// Get copy of block of memory
	/*! Fetch a private copy of a previously published message.
	    @param id Message id returned by insertMessage() (possibly from another process).
	    @return Newly allocated DataMessage owned by the caller (delete it), or NULL if the
	    id is unknown or the message has expired (end-of-life reached). */
	DataMessage* getCopyOfMessage(uint64 id);

	std::string typeToText(PsyType type);			/*!< Resolve a type id to its registered name. @param type Type id. @return Name or empty. */
	std::string subTypeToText(uint16 subtype);		/*!< Resolve a subtype id to its registered name. @param subtype Subtype id. @return Name or empty. */
	std::string contextToText(PsyContext context);	/*!< Resolve a context id to its registered name. @param context Context id. @return Name or empty. */
	std::string subContextToText(uint16 subcontext);/*!< Resolve a subcontext id to its registered name. @param subcontext Subcontext id. @return Name or empty. */
	std::string getComponentName(uint32 compID);	/*!< Resolve a component id to its name. @param compID Component id. @return Name or empty. */

	uint16 getNodeID();			/*!< @return This node's cluster-wide id (from the master segment). */
	bool setNodeID(uint16 id);	/*!< Set this node's cluster-wide id. @param id New id. @return true on success. */

	/*! Enumerate all live Psyclone system ids/ports on this machine (via the PsycloneIndex).
	    @return Newly allocated message listing the systems; caller owns and must delete it. */
	DataMessage* retrieveAllSystemIDs();


	/*! Read the cluster time-sync values from the master segment.
	    @param tmc Receives the time-master clock (µs). @param adjust Receives the local offset.
	    @return true on success. */
	bool getTimeSyncData(uint64& tmc, int64& adjust);
	/*! Write the cluster time-sync values into the master segment.
	    @param tmc Time-master clock (µs). @param adjust Local clock offset. @return true on success. */
	bool setTimeSyncData(uint64 tmc, int64 adjust);

	TemporalMemory* temporalMemory;		/*!< Accessor for time-limited message storage. Owned by this object; backing store is shared memory. */
	ProcessMemory* processMemory;		/*!< Accessor for the process table and per-process queues. Owned; backed by shared memory. */
	ComponentMemory* componentMemory;	/*!< Accessor for the component registry. Owned; backed by shared memory. */
	DataMapsMemory* dataMapsMemory;		/*!< Accessor for the type/context/tag/crank/request maps. Owned; backed by shared memory. */

private:
	// Continuous management of eol pages
	/*! Body of the background management thread: expires EOL data and updates stats.
	    Runs only in the master process; loops until shouldContinue is cleared. @return Exit code. */
	uint32 runManager();

	// Resize the EOL Page
	/*! Legacy: resize the end-of-life bookkeeping page of the page allocator. */
	bool resizeEOLPage(uint32 slotCount, uint32 pagesPerSlot, uint32 slotDuration);

	PsycloneIndex* psycloneIndex;		/*!< Machine-global instance index accessor (owned). */
	MasterMemory* masterMemory;			/*!< Master segment accessor (owned). */


	bool shouldContinue;		/*!< Management-thread run flag; cleared to request shutdown. */
	bool isRunning;				/*!< True while the management thread is alive. */
	bool isLocalMaster;			/*!< True if this process created the segments and runs maintenance. */
	uint32 managementThreadID;	/*!< ThreadManager id of the management thread. */
	uint16 thisInstance;		/*!< Instance number of this process on the local machine. */
};














/*! Maximum number of system ids storable in a CMRegister block. */
#define MAXREGISTERCOUNT 64

/*! @brief Legacy registry block listing the Psyclone system ids present on this machine.
    @ingroup Memory
    Header followed in memory by up to MAXREGISTERCOUNT uint16 system ids.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct CMRegister {
	uint16 size; // size of memory allocation
	uint16 count; // number of entries
	// MAXREGISTERCOUNT x uint16 sysIDs
};

/*! Background maintenance thread entry point for MemoryManager / MemoryManagerX.
    @param arg The manager instance (cast internally). @return Thread exit code. */
THREAD_RET THREAD_FUNCTION_CALL MemoryManagement(THREAD_ARG arg);

/*! Reserved process id of the node process itself. */
#define NODE_PROCESS_ID		1
/*! Maximum Psyclone process instances per machine. */
#define MAXINSTANCES		256


/*! @brief Decomposed form of a 64-bit global memory-block id (legacy page allocator).
    @ingroup Memory
    Packed/unpacked via the MP_CalcID / MP_Get*ID macros.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct MemoryID {
	uint32 pageID; // contains [8] PageSerial [24] PageLoc
	uint16 blockID; // index containing block offset
	uint16 sysID; // global node id
};

/*! @brief Size prefix of a raw block inside a MemoryPage (legacy allocator).
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct MemoryBlock {
	uint32 size;
//	char* data;
};

/*! @brief Header of the page-pool structure holding reusable, currently idle pages (legacy allocator).
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct PagePoolHeader {
	uint32 size; // total size of pool structure in bytes
	// uint32 size, uint64 time, uint32 pageID
};

/*! @brief One recyclable page recorded in the page pool (legacy allocator).
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct PagePoolEntry {
	uint32 dataSize; // page data size
	uint64 time; // time of entry into pool
	uint32 pageID; // ID of page
};

/*! @brief Header of the end-of-life (EOL) time-wheel that schedules page expiry (legacy allocator).
    Pages holding time-limited data are hashed into time slots; the manager thread walks
    slots as time passes and reclaims expired pages.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct EOLHeader {
	uint32 slotCount;		/*!< Number of time slots in the wheel. */
	uint32 pagesPerSlot;	/*!< Capacity (page entries) of each slot. */
	uint32 startSlot;		/*!< Index of the slot representing startTime. */
	uint64 startTime;		/*!< Timestamp (µs) of the wheel's first slot. */
	uint32 slotDuration;	/*!< Duration covered by one slot (µs). */
	// slot0: page0,eol0 , page1,eol1 ..., pageN,eolN
	// slot1: page0,eol0 , page1,eol1 ..., pageN,eolN
};

/*! @brief One page/expiry pair inside an EOL time-wheel slot (legacy allocator).
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct EOLEntry {
	uint32 pageID;	/*!< Page scheduled for expiry. */
	uint64 eol;		/*!< Absolute end-of-life timestamp (µs). */
};

/*! @name Legacy page-allocator address helpers
    Pointer arithmetic over the raw in-segment page layout: a MemoryPage header is followed
    by its block-offset index table, then the packed block data.
    @{ */
#define MP_GetBlockIndexStartLoc(p) ((char*)p + sizeof(MemoryPage))
#define MP_GetBlockDataStartLoc(p) (MP_GetBlockIndexStartLoc(p) + (p->blockTableSize*sizeof(uint32)) )
//#define MP_GetBlockOffsetLoc(p,n) (uint32*)(MP_GetBlockIndexStartLoc(p) + (n * sizeof(uint32)))
//#define MP_GetBlockOffset(p,n) (*(MP_GetBlockOffsetLoc(p,n)))
//#define MP_GetBlockLoc(p,n) MP_GetBlockDataStartLoc(p) + MP_GetBlockOffset(p,n)
/*! @} */

/*! @brief In-segment header of one memory page holding packed data blocks (legacy allocator).
    Followed in memory by a uint32 block-offset table (blockTableSize entries) and the
    block data area. Address helpers: MP_GetBlockIndexStartLoc / MP_GetBlockDataStartLoc.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct MemoryPage {
	uint32 size; // total size of page in bytes
	uint64 eol; // current number of blocks in use -> first free location
	uint32 dataSize; // total size of data space in bytes
	uint32 dataUsage; // current usage of page in bytes
	uint16 count; // current number of blocks in use -> first free location
	uint16 blockTableSize; // maximum entries in index table
//	uint16 blockTable[1024]; // index table containing block offsets
//	char* data; // MemoryBlock0, MemoryBlock1, ..., MemoryBlockN
};

/*! @brief Page-table slot describing one page's identity, size and lock state (legacy allocator).
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct MemoryPageEntry {
	uint8 pageSerial; // running serial number for this page location (0-255)
	uint8 cacheSerial; // running serial number for this page cache (0-255)
	uint32 pageSize;
	uint32 pageID;
	uint64 lockID;
	uint64 lockTime;
	uint16 lockCount;
};

/*! @brief Header of a process-local (non-shared) cache of mapped page addresses (legacy allocator).
    Shared segments may map at different addresses per process, so each process caches its
    own page pointers keyed by page id and validated by serial numbers. */
struct MemoryPageCache {
	uint32 size; // total size of structure in bytes
	uint32 count; // total number of cache entries allocated
	// MemoryPageCacheEntries...
};

/*! @brief One process-local cached page mapping; stale when serials mismatch the page table (legacy allocator). */
struct MemoryPageCacheEntry {
	uint8 pageSerial; // running serial number for this page location (0-255)
	uint8 cacheSerial; // running serial number for this page cache (0-255)
	uint32 pageSize;
	uint32 pageID;
	MemoryPage* page;
};

/*! @name Legacy 64-bit memory-id packing macros
    A global block id packs pageID (32 bits), blockID (16 bits) and nodeID (16 bits);
    these macros compose and extract the fields without shifting at runtime where possible.
    @{ */
#define MP_SYSMEMID MP_CalcID(0, 0, pageMaster->nodeID)
#define MP_CalcID(pid,bid,nid) ((uint64)pid | (((uint64)bid) << 32) | (((uint64)nid) << 48))
#define MP_SetBlockID(id,var) var |= (((uint64)id) << 32)
#define MP_SetNodeID(id,var) var |= (((uint64)id) << 48)
#define MP_SetPageID(id,var) var |= ((uint64)id)
#define MP_GetBlockID(id) (*(uint16*)((char*)&id + sizeof(uint32)))
#define MP_GetNodeID(id) (*(uint16*)((char*)&id + sizeof(uint32) + sizeof(uint16)))
#define MP_GetPageID(id) (*(uint32*)(char*)&id)
#define MP_GetBitFieldLoc(m) ((char*)m + sizeof(MemoryPageMaster))
#define MP_GetPageTableLoc(m) (MP_GetBitFieldLoc(m) + m->bitFieldSize )
#define MP_GetPageEntryLoc(m,pid) (MP_GetPageTableLoc(m) + (pid * sizeof(MemoryPageEntry)) )
#define MP_GetPageLoc(m,pid) ((MemoryPageEntry*)MP_GetPageEntryLoc(m,pid))->page
/*! @} */

/*! @name Legacy page-type classification
    @{ */
#define MP_SYSTEMPAGE 0		/*!< Reserved page holding allocator/system structures. */
#define MP_STATICPAGE 1		/*!< Page with non-expiring data. */
#define MP_DYNAMICPAGE 2	/*!< Page with end-of-life (expiring) data. */
/*! Classify a page by id and eol: reserved ids are system pages; eol > 0 means dynamic. */
#define MP_GetPageType(id,eol) ( ((id < RESERVEDPAGECOUNT) ? MP_SYSTEMPAGE : (eol > 0 ? MP_DYNAMICPAGE : MP_STATICPAGE) ) )
/*! @} */

/*! @brief Root header of the legacy page-allocator segment ("PageMaster").
    Followed in memory by a page-usage bitfield and the page table (MemoryPageEntry[]).
    Superseded by the MemoryMasterStruct-based design but still used by MemoryManagerX.
    @warning Shared-memory layout; see the file-level ABI warning before changing. */
struct MemoryPageMaster {
	uint32 size; // total size of PageMaster in bytes
	uint16 sysID; // global node id
	uint16 status; // 0-10, where 10 means ready
	uint64 createdTime; // time of creation
	uint8 instance; // Instance number on each computer
	uint64 timesync; // time sync
	uint32 pagesSize; // total size of all pages in bytes {system, static, EOL}
	uint32 dataSize[3]; // total size of all pages data spaces in bytes {system, static, EOL}
	uint32 dataUsage[3]; // current usage of all pages in bytes {system, static, EOL}
	uint32 count[3]; // current number of pages in use, for info only {system, static, EOL}
	uint32 lastCreatedStaticPage; // id of last created static page
	uint32 bitFieldSize; // number of bytes in the bitfield
	uint32 pageTableSize; // maximum entries in page table and number of bits in bit field
	// char* pageBitfield; // which pages are in use -> 111010001110101010....011010
	// MemoryPageEntry* pageTable; // {ID24 offset -> address of MemoryPageEntry}
};

// SystemBlock for system data

// structure for managing ttl pages

/*! @brief Legacy page-based shared-memory allocator (predecessor of MemoryManager).
    @ingroup Memory
    Manages a single "PageMaster" segment divided into system, static and dynamic (EOL)
    pages, with block-level allocation inside pages and a time-wheel for expiry. Retained
    for compatibility and its unit tests; new code should use MemoryManager/TemporalMemory.
    Blocks are addressed by 64-bit ids (see MP_CalcID). get-and-lock calls must be paired
    with the corresponding unlock; returned pointers point directly into shared memory and
    are only valid while locked. getCopy* variants return heap copies owned by the caller. */
class MemoryManagerX {
public:
	static MemoryManagerX* Singleton;	/*!< Per-process singleton instance. */

	/*! Map and lock the system page @a pageID; pair with UnlockSystemBlock().
	    @param pageID Reserved system page id. @param size Receives block size.
	    @return Pointer into shared memory, valid only while locked; NULL on failure. */
	static char* GetAndLockSystemBlock(uint32 pageID, uint32& size);
	/*! Release the lock taken by GetAndLockSystemBlock(). @return true on success. */
	static bool UnlockSystemBlock(uint32 pageID);
	// Creates a new System Page with the next id
	static char* CreateAndLockNewSystemPage(uint32 size, uint32& id);
	// Resize an existing System Page
	static char* ResizeSystemPage(uint32 pageID, uint32 newSize);
	static bool GetMemoryUsage(uint32& total, uint32& usage, uint32& sysTotal, uint32& sysUsage, uint32& staticTotal, uint32& staticUsage, uint32& dynamicTotal, uint32& dynamicUsage);
	// Destroys an exising System Page by id
	static bool DestroySystemPage(uint32 pageID);

	/*! Copy @a data into a new shared block. @param data Source bytes (caller keeps ownership).
	    @param size Byte count. @param eol Absolute expiry time (0 = static).
	    @param id Receives the global block id. @return true on success. */
	static bool InsertMemoryBlock(const char* data, uint32 size, uint64 eol, uint64& id);
	/*! Replace the contents of an existing block. @return true on success. */
	static bool OverwriteMemoryBlock(uint64 id, const char* data, uint32 size, uint64 eol);
	/*! Lock a block and return a pointer into shared memory; pair with UnlockMemoryBlock().
	    @return In-segment pointer valid only while locked; NULL if unknown/expired. */
	static char* GetAndLockMemoryBlock(uint64 id, uint32& size, uint64& eol);
	/*! Release the lock taken by GetAndLockMemoryBlock(). @return true on success. */
	static bool UnlockMemoryBlock(uint64 id);
	/*! Return a heap copy of a block; caller owns and must free it. @return Copy or NULL. */
	static char* GetCopyMemoryBlock(uint64 id, uint32& size, uint64& eol);

	/*! Self-test of the legacy page allocator. @return true if all checks pass. */
	static bool UnitTest();

	friend THREAD_RET THREAD_FUNCTION_CALL MemoryManagement(THREAD_ARG arg);

	MemoryManagerX();
	~MemoryManagerX();

	// Connect to existing PageMaster in shared memory
	bool connect(uint16 sysID);

	// Create a new PageMaster in shared memory
	bool create(uint16 sysID, uint32 staticPageTableSize = 100, uint32 slotCount = 360, uint32 slotDuration = 60000000, uint32 pagesPerSlot = 10, bool force = false);

	// Either look up location of or create a new PageMaster in shared memory
	// bool init(uint16 nodeID, uint32 pageTableSize, uint32 slotCount = 600, uint32 slotDuration = 1000000, uint32 bufferSlots = 10, uint32 pagesPerSlot = 1024);

	// Insert new block of memory and return full id
	bool insertMemoryBlock(const char* data, uint32 size, uint64 eol, uint64& id);

	// Overwrite existing block of memory
	bool overwriteMemoryBlock(uint64 id, const char* data, uint32 size, uint64 eol);

	char* getAndLockMemoryBlock(uint64 id, uint32& size, uint64& eol);
	char* getLockedMemoryBlock(MemoryPage* page, uint64 id, uint32& size, uint64& eol);
	bool unlockMemoryBlock(uint64 id);
	char* getCopyMemoryBlock(uint64 id, uint32& size, uint64& eol);

	// Return a system memory block
	char* getSystemBlock(uint32 pageID, uint32& size);

	// Return a system memory block	and lock it
	char* getAndLockSystemBlock(uint32 pageID, uint32& size);
	char* getLockedSystemBlock(uint32 pageID, uint32& size);
	char* getLockedSystemBlock(MemoryPage* page, uint32& size);

	// Unlock System Block
	bool unlockSystemBlock(uint32 pageID);

	// Creates a new System Page with the next id
	char* createAndLockNewSystemPage(uint32 size, uint32& id);
	// Resize an existing System Page
	char* resizeSystemPage(uint32 pageID, uint32 size);

private:
	// Create new MemoryPage
	MemoryPage* createPage(uint32 size, uint16 tableSize, uint64 eol, uint32& pageID); // if either parameter is 0, the default values will be used

	// Insert new block of memory into a specific page and return full id
	bool insertMemoryBlockIntoPage(uint32 pageID, const char* data, uint32 size, uint64 eol, uint64& id);

	// Create a new Page Mutex
	// Mutex createPageMutex(uint32 id);
	// Get an existing Page Mutex
	// bool getPageMutex(uint32 id, NMutex &mutex);
	// Destroy a Page Mutex
	// bool destroyPageMutex(uint32 id);

//	utils::Semaphore* getPageSemaphore(uint32 pageID);
//	utils::Mutex* getPageMutex(uint32 pageID);

	bool lockPage(uint32 pageID);
	bool unlockPage(uint32 pageID);
//	bool lockPage(MemoryPage* page);
//	bool unlockPage(MemoryPage* page);
	bool lockPage(MemoryPageEntry* entry);
	bool unlockPage(MemoryPageEntry* entry);

	// Find an available page appropriate for the block size and ttl
	MemoryPage* findPageForBlock(uint32 size, uint64 eol, uint32& id);

	// Free up a page and put it into the pool
	bool freePageIntoPool(uint32 pageID);

	// Remove page from stats, but do not delete
	bool removePageFromStats(uint32 pageID, uint32& dataSize);

	// Insert a page into the idle pool
	bool insertPageIntoPool(uint32 pageID, uint32 dataSize);

	// Get page with required size from the idle pool
	bool getPageFromPool(uint32 size, uint32& pageID);

	// Destroy all Memory Pages
	bool destroyAllPages();

	// Destroy a Memory Page
	bool destroyPage(uint32 pageID, bool updateStats = true);

	// Continuous management of eol pages
	uint32 runManager();

	// Return string representing the bitfield
	char* getBitFieldAsString(uint32& size);
	// Print string representing the bitfield
	bool printBitFieldAsString(char* title);

	// Get the percentage in use of a page
	bool getPageUsage(uint32 pageID, double& val);
	// Get the percentage in use of a page
	double getPageUsage(uint32 pageID);
	// Get the number of bytes free of a page
	bool getPageFree(uint32 pageID, uint32& bytes);
	// Get the total size in bytes a page
	bool getPageSize(uint32 pageID, uint32& bytes);

	// Creates a System Page
	char* createSystemPage(uint32 id, uint32 size, uint64& memID);

	// Create the Manager System Pages
	bool createSystemPages();

	// Initialise the System Page
	bool initSystemPage(char* data, uint32 size);

	// Initialise the Page Pool
	bool initPagePool(char* data, uint32 size);

	// Resize the EOL Page
	bool resizeEOLPage(uint32 slotCount, uint32 pagesPerSlot, uint32 slotDuration, uint32 bufferSlots);

	// Insert page into EOL Structure
	bool insertPageIntoEOL(uint32 pageID, uint64 eol, uint64& slotEOL);

	// Initialise the Queue Page
	bool initQueuePage(char* data, uint32 size);

	// Calculate the new page size based on incoming block size
	static uint16 CalcBlockTableSize(uint32 size);

	MemoryPage* getCachedPage(uint32 pageID);
	MemoryPage* getCachedPage(MemoryPageEntry *entry);
	bool updateCachedPage(MemoryPageEntry *entry, MemoryPage *page);

	MemoryPageMaster* pageMaster; // Shared memory location of PageMaster, looked up or created
	utils::Mutex *pageMasterMutex; // Handle to global mutex for PageMaster access
	MemoryPageCache* pageCache; // Non-shared memory location of cached page addresses
	bool shouldContinue;
	bool isRunning;
	bool isMaster;
	uint32 managementThreadID;
	uint16 thisInstance;
};

} // namespace cmlabs

/*! @} */ // end of defgroup Memory (file-level)

#include "TemporalMemory.h"
#include "ComponentMemory.h"
#include "DataMapsMemory.h"
#include "ProcessMemory.h"

#endif //_MEMORYMANAGER_H_

