xtc/docs/AUDIO_ENGINE.md

Audio engine requirements

Real-time callback rules

The callback must not allocate/free memory, lock blocking mutexes, perform filesystem/network/console I/O, log, load plugins or samples, call GUI APIs, or wait for another thread.

Use preallocated buffers and bounded work. Publish parameter and graph changes through a documented real-time-safe mechanism. Verify atomics are lock-free where required on the target platform.

Processing model

Process a bounded number of frames per block. Do not assume a fixed block size unless the backend guarantees it. Define maximum frame and event counts and safe behavior when those limits are reached.

The initial milestone is offline processing, then a test oscillator -> gain -> meter/output graph, then a hardware backend.

Clock and transport

Audio sample position is the authoritative playback clock. Use integer or rational timeline representations to avoid cumulative floating-point drift. Define tempo-map conversion and behavior at tempo changes before implementing tempo automation.

Phase 1 offline API

xtc_graph_create accepts 1..256 nodes; IDs are monotonic stable identifiers, never reused within a graph's lifetime. Enumeration indexes are dense and distinct from IDs. Supported nodes are gain (any finite multiplier), sine oscillator (finite nonnegative Hz), pass-through peak meter and audio output. Ports carry mono float audio: input 0, output 1. Oscillators have no input and output nodes have no output. Port lookup reports direction, signal type and channel count. MIDI/control ports and channel conversion are not implemented; invalid ports/directions, duplicate edges and cycles are rejected before an edit becomes visible. Fan-out copies a source to consumers; fan-in sums inputs without normalization. Failed additions leave IDs and graph contents unchanged.

xtc_graph_compile(graph, sample_rate, max_frames, &plan) runs on the control thread. Empty graphs compile to silence. Sample rate must be finite and positive, oscillators must be at or below Nyquist, and max_frames is 1..65536. Compilation allocates and copies topology/parameters, builds a topological schedule and input adjacency, and allocates one max_frames float buffer per node. The maximum buffer storage is 64 MiB per plan. Compilation may fail with an allocation error; output plan is then NULL. Destroy a plan on the control thread.

xtc_plan_process uses only prepared state: no allocation/free, sorting, logging, I/O or locks. Each call accepts up to max_frames; oversized calls return capacity errors without modifying output or state. Output storage must hold frame_count floats. NULL input means silence; input/output may alias. Zero frames leaves all state unchanged. No plan operations may run concurrently with processing, including reset, meter reads or destruction.

Unconnected non-oscillator nodes receive external input. Connected inputs receive only the sum of their upstream buffers. Explicit output nodes are summed to the caller's output. If none exist, terminal nodes are summed for compatibility with the original gain-chain API, unless xtc_graph_require_output_nodes(graph, 1) is set. The GUI uses that policy; no output nodes means silence. Disconnected components cannot contribute when explicit outputs exist, but they are still processed. Floating point overflow and nonfinite input become silence per sample. No clipping or normalization is applied to finite samples.

Oscillators start at phase zero and carry phase continuously across blocks; reset clears phases and meter peaks. Meter reads return the last nonempty block's absolute peak. These are same-thread reads, not a thread-safe GUI meter bridge. Rendering is deterministic on the same build/platform regardless of block partitioning; libm results need not be bit-identical across platforms.

xtc_graph_process_offline remains a convenience helper, compiling at 48000 Hz and destroying the plan after one block. It accepts a finite external-input gain and requires non-NULL input/output. It now follows routing rather than multiplying every gain in insertion order. It allocates and is not suitable for an audio callback. Use a persistent plan for phase continuity and repeated blocks.

Tests in tests/unit/test_offline.c cover known oscillator samples, gain and meter output, reset and partition equivalence, fan-out/fan-in, reverse insertion order, disconnected nodes, in-place processing, nonfinite input, invalid ports, cycles, capacity limits, plan lifetime and repeated compilation/cleanup. The offline API remains single-owner. The realtime/backend APIs below add publication and JACK processing; offline tests alone do not qualify hardware.

Phase 2 backend and realtime bridge

Use xtc_backend.h on one serialized control thread:

1. Open a config with XTC_BACKEND_JACK, a client name, channel_count 1, and max_frames 1..65536. It must accommodate the server's current buffer size. The actual rate and buffer size are returned through get_status. Open does not start the server or acquire devices directly. Invalid config returns invalid argument, missing build support returns unsupported, and JACK setup failures return I/O error. If cleanup itself fails, a non-NULL output handle is preserved for close retry; no other operation is allowed on that handle.
2. Submit a graph (copied/compiled at the server rate), then start. Starting an already running client is an invalid-state error; stopping a stopped client succeeds. With no submitted graph, callbacks produce silence.
3. Enumerate available audio ports with visit_ports. Names are borrowed only during the visitor; do not reenter backend operations from the visitor. Connect the client's input to a remote output or its output to a remote input explicitly. JACK maps device selection to server/device port routes; ALSA device identifiers are not accepted here. The server controls rate, period and hardware configuration outside this API. Names/ports can disappear between enumeration and connection; route failures are returned as I/O errors.
4. During playback, submit edited graphs and periodically collect retired resources. Capacity means one pending graph is still awaiting adoption; keep the current graph and retry later. Stopped submissions instead compile a replacement engine off-thread and swap only after success, while callbacks are quiescent. Its block/revision counters restart and oscillator phase resets; no edit is blocked behind a plan that cannot be adopted until playback. get_status is a nonblocking snapshot of rate, frames, running/server-loss/rate-change flags, xruns, processed blocks and last process result. No active DSP memory is exposed.
5. Stop pauses the retained DSP state after deactivation. Restart resumes it. Close removes the JACK client, synchronizes all callbacks, then destroys plans. Stop/close failures preserve the handle for recovery/retry. No plan is freed on the audio thread. Sample-rate change or server loss requires close/reopen and a fresh submission; there is no automatic reconnect.

Buffers can change size. In-limit changes process normally; oversized blocks are silent and report capacity errors until the size is supported again. Rate changes latch silence and invalid-state processing results. A missing input buffer is treated as unavailable input (silence). A missing output buffer is reported as an I/O process error. Shutdown callbacks only set the loss flag; the control thread closes the client. No JACK or GUI types cross the C API.

The standalone xtc_realtime.h API allows hardware-independent consumers and stress tests. Submit compiles on the producer; process adopts at a block boundary without allocation/free, I/O, locks or waits. All process calls must come from one consumer thread, and destruction requires that consumer stopped and joined. Release/acquire publication, retirement backpressure and atomic requirements are specified in decision 0004. Ordinary project processing and JACK notification callbacks were audited for prohibited operations; JACK's own internal realtime guarantees depend on the installed implementation/server.

Validation commands and results are recorded in BUILD.md and the integration test README. Current limitations: mono input/output, no parameter ramping or crossfaded graph changes, no realtime meter bridge, no automatic reconnect, no physical-device or long-duration endurance qualification. Phase 3 connects the FLTK shell to backend controls and status; see GUI.md. Its editors remain placeholders except the Phase 4 modular editor described below.

Phase 4 graph snapshots and edits

The control-thread API exposes copies through get_counts, get_node(index) and get_connection(index), including node IDs/types/values and port endpoints. No mutable graph/engine storage is exposed. Node IDs increase monotonically; removing a node compacts private storage and adjusts private edge indexes but leaves surviving IDs unchanged. Exhausted ID space returns capacity. Compiled meter lookup resolves stable IDs too.

xtc_graph_apply_edit(source, edit, &candidate, &added_id) accepts ADD_NODE, REMOVE_NODE, CONNECT or DISCONNECT. It copies the source, validates the edit, and returns a caller-owned candidate on success. On failure candidate is NULL, source and added_id are unchanged. added_id is optional and written only for ADD_NODE. ADD_NODE validates type and the used parameter (finite gain, or finite nonnegative oscillator Hz). Meter/output do not use the value field. Compile additionally checks the oscillator against the actual sample rate. Removing a node removes its incident connections; disconnecting a nonexistent edge returns not-found. Missing nodes/ports, reversed directions, duplicate edges, cycles and unknown command/type values are rejected without visible edits.

The GUI keeps a 64-node graph requiring explicit output nodes. It creates a candidate, submits that exact candidate to the backend, then swaps its owned graph and refreshes the canvas only if submission succeeds. Failed compile or pending-slot pressure destroys the candidate and retains both the displayed graph and previously submitted engine plan. When disconnected at startup, edits can be stored locally and are submitted on reconnect. Lost/changed active backends reject edits until reconnect. Reconnect submits the retained graph.

Submitted/active revision counters expose adoption without exposing plan pointers. A submission increments the submitted revision; only the audio consumer acknowledges the revision when adopting the plan at a block boundary. The editor shows prepared while stopped, queued while an older plan is active, and active when those revisions agree. Atomic fields remain independent snapshots and wrap modulo UINT_MAX + 1. Rebuilding a stopped engine or reopening a backend starts a new counter epoch. See decision 0004 for slot ordering.

Movement, selection and scroll position are GUI-only state; they never compile or publish audio plans. Empty graphs and graphs without explicit outputs are silent under the GUI policy. Terminal fallback remains the default for earlier offline/standalone callers. Input on unconnected nodes retains the existing external-input contract; the GUI does not connect external JACK ports.

Phase 5 transport

Backend status includes a uint64 sample_position. Each successful processing callback advances it by its exact frame count (including silence); errors do not advance it. The counter uses compiler atomics, verified always lock-free before backend setup, with overflow detected rather than wrapped. No extra allocation, I/O, mutex or GUI work occurs in callbacks. Fields remain individually coherent snapshots.

Stop pauses the clock. Start resumes; stopped and running graph submissions preserve position even when plan/revision counters reset. Reopen starts at zero. xtc_backend_seek changes position only on a healthy stopped backend; it does not rewind oscillator phase. Project tick/sample conversions use integer floor arithmetic at constant tempo, with explicit capacity errors and rates 1..768000. The GUI observes the counter every 100 ms, so visual updates are block-quantized and delayed by polling. No external JACK transport synchronization is provided.

Silent clip metadata is control-thread data, never traversed by the callback. Audio/MIDI scheduling will be added in later phases through compiled event/clip plans rather than direct callback access to the editable project.

Phase 6 audio source and recording

xtc_audio.h provides control-thread import and immutable preloaded clip plans. The backend prepares a project only while stopped: build audio candidate, compile/submit its graph, then replace the old audio plan without a later fallible step. Preparation failures retain both old plans. Missing/invalid assets produce explicit issue metadata and silence; OOM/capacity fails. Graph-only edits still use the existing pending/retired mechanism while running.

Each callback renders the plan into a preallocated mono timeline buffer using the authoritative sample position. xtc_realtime_process_audio passes that buffer into the compiled graph; XTC_NODE_AUDIO (output port only) reads it, independently from external JACK input. The old process APIs remain wrappers with a silent timeline. Import creates a source/output route if necessary. No samples or GUI/project arrays are shared with editable state. Clip mixing, rate conversion and graph processing are bounded; see AUDIO_FILES.md for limits.

A borrowed recorder can be attached/detached only while stopped. Raw JACK input is copied to its SPSC ring before graph processing. Producer and consumer index release/acquire operations protect payload ownership; all atomics are checked always lock-free. The control thread drains to a temporary float32 WAV, then finalizes only after successful stop/close has synchronized the callback. An overflowing ring or unavailable input creates a sticky error; later input is dropped, reported and never silently joined into a continuous-looking take. A partial overflow take is finalized for manual recovery. File errors retain the previous destination and remove unfinished temporaries.

GUI recording uses a two-second ring and 100 ms control-thread draining, with no worker thread. A GUI stall can overflow it. No callback allocation/free, file I/O, logging, locks or FLTK calls were added. Backend stop failure preserves the borrowed recorder; the caller must not free it before successful quiescence. Live software source/capture acceptance and concurrent ring tests pass; physical recording, latency compensation and disk endurance remain unverified.

Phase 7 MIDI source

The C MIDI plan prepares bounded source-relative notes into sample spans and renders a separate instrument buffer. xtc_plan_process_sources and xtc_realtime_process_sources distinguish external audio, WAV timeline and instrument input; older wrappers supply silence for omitted sources. Graph instrument nodes consume the shared mono instrument buffer. The backend owns prepared MIDI runtime state exclusively while active, with lock-free panic requests and no callback allocation/I/O. Stop/seek clear live voices; scheduled notes chase their absolute phase. See MIDI.md for event ordering, panic semantics, capacities and test coverage. No new dependency is required.