xtc/docs/ROADMAP.md

Roadmap

Phase 0 — Baseline

Inspect repository/toolchain, verify Makefile and minimal executable, document dependencies. No system changes without permission.

Baseline implementation: headless startup/shutdown executable, core tests, separate debug build, optional FLTK shell build, and header dependencies. See BUILD.md for toolchain, commands, licensing and limitations.

Phase 1 — C core and offline processing

Graph data structures, validation, cycle detection, topological ordering, preallocated buffers, test oscillator, gain and offline render tests.

Implemented: mono ports, transactional cycle rejection, independent compiled plans, precomputed input routing, oscillator/gain/meter/output, deterministic variable-block tests and lifecycle/error coverage. See AUDIO_ENGINE.md for API limits and compatibility behavior.

Phase 2 — Real-time backend

Choose and implement a Linux backend. Document callback semantics, start/stop, device errors, shutdown, and safe graph publication.

Implemented: optional JACK-compatible mono backend, explicit audio port routing, format/xrun/loss status, start/stop/reopen, and bounded pending/retired plan handoff using lock-free atomics. Hardware-independent failure/concurrency tests and a 30-second software-loopback run pass. See decision 0004 and AUDIO_ENGINE.md. Physical hardware playback/endurance and live server-loss recovery remain unverified; GUI engine controls are implemented in Phase 3.

Phase 3 — FLTK shell

Main window, transport controls, view switching, backend status, clean startup and shutdown through the C API.

Implemented: dark FLTK shell, Play/Stop/Reconnect, view/menu shortcuts, backend state/format/xrun/process status, persistent command errors, timer-based refresh and retirement collection, and callback-safe close/retry. Mock-backend GUI tests and live GUI/JACK start/stop/reconnect/close tests pass under Xvfb. Sequencer, mixer and project-file actions remain placeholders; initial graph is silent. See GUI.md.

Phase 4 — Modular editor

Draw nodes/ports/edges; create, move and connect nodes; reject invalid edges; safely submit edits to the C core.

Implemented: scrollable node/port/wire canvas; add/select/move/delete nodes; drag connections, select/delete wires, and show invalid-edit feedback. C owns stable IDs, read-only snapshots and transactional edits. Exact accepted graph snapshots are submitted to the backend; prepared/queued/active state is exposed through revision acknowledgements. Core/render, GUI rollback, mouse smoke and live adoption tests pass. See GUI.md and AUDIO_ENGINE.md. Preset parameters, mono-only processing, fixed canvas and no persistence/undo/zoom remain limits.

Phase 5 — Project and sequencer

Versioned persistence, stable IDs, tracks, clips, timeline, selection, zoom, scrolling, and transport synchronization.

Implemented: bounded version-1 project persistence with transactional loads and temporary-file/fsync/rename saves; stable track/clip/node IDs; silent regions; track and region creation/deletion, selection, move/trim, timeline zoom/scroll; sample-counted playhead, pause/resume and stopped seek. Project/graph arrangement roundtrips, malformed-input, GUI rollback/gesture and live transport tests pass. See PROJECT_FORMAT.md and GUI.md. Constant tempo/4/4, silent metadata regions, no undo, asset playback, MIDI or persisted GUI layout are explicit limits.

Phase 6 — Audio clips and recording

Audio import/playback, recording through bounded buffers, missing-asset handling, no file I/O in callbacks.

Implemented: bounded PCM16/24/32 and float32 WAV import (mono/stereo downmix), preloaded overlap-summed clip plans with linear rate conversion, and an audio source node using ordinary graph routing. A lock-free SPSC ring transfers JACK input to a control-thread WAV writer; Stop finalizes and adds successful takes. Version 2 metadata stores external asset references and reads v1. Missing assets remain silent with reporting and manual relinking. Core, callback, GUI and live software record/save/reopen/play acceptance tests pass. See AUDIO_FILES.md for memory/format limits, overflow recovery and deferred physical-device validation. No new dependency or callback I/O was introduced.

Phase 7 — MIDI and instrument

MIDI input/events/clips, simple internal instrument, note-off/panic handling, then a basic piano roll.

Implemented: bounded source-relative note events with stable IDs; sample-offset JACK MIDI input; a velocity-scaled sine instrument and separate graph source; panic, note-off, stop and seek handling; and a piano roll for drawing, moving, resizing and deleting notes. Version 3 saves notes and reads v1/v2 projects. Core, fault-injection, GUI and live software MIDI/save/reopen/play tests pass. Acceptance uses drawn notes; MIDI recording and SMF import are deferred. See MIDI.md for capacities, immediate note-off, routing and hardware limits.

Phase 8 — Mixer and editing

Gain/pan, meters, mute/solo, automation, undo/redo, keyboard workflow.

Implemented: selectable per-track channel strips with gain/pan and mute/solo; stereo graph/JACK output and output peak meters; editable gain/pan automation lanes with sample interpolation; bounded 32-edit C snapshot undo/redo and mixer keyboard controls. Format v4 persists mixing/automation and reads v1–v3. Core, GUI rollback/gesture/history and live stereo software playback tests pass. See MIXER.md for routing, mono compatibility, pan law, history behavior and limits. Track inserts/sends, live fader adoption, MIDI capture, per-track meters and advanced automation remain deferred. No project-file editing is needed for the implemented composition/mixing workflow.

Phase 9 — Extensibility and release

Native node API, evaluate plugin hosting, licensing review, packaging, installation tests, performance and recovery tests.

Implemented: versioned native C processor lifecycle and graph source/effect API, explicit instrument-buffer integration, example and error/retirement tests; external-hosting evaluation (deferred); dependency/license review; deterministic source/release targets; core/GUI staged install/uninstall and installed C/C++ consumer tests; headless version/help/metadata validation; offline performance and process-crash/write-failure recovery tests. Source format stays v4, reading v1–v3. See NATIVE_API.md, RELEASING.md, LINUX.md, LICENSES.md, PERFORMANCE.md and RECOVERY.md. Native graphs remain session-only, with unsupported saves preserving files. No stable binary plugin ABI, external host, system package, autosave or physical-device/endurance qualification is claimed.

Phase 10 — Engine Correctness and Reliability

Goal: Verify that the existing implementation behaves correctly under realistic workloads before adding more features.

Tasks

• Audit graph compilation, topological ordering, port compatibility, and cycle detection.
• Test graph replacement while audio is running, including safe reclamation of old processing graphs.
• Verify that audio-thread operations are bounded and non-blocking.
• Test repeated initialization, start/stop cycles, device failures, shutdown, and partial initialization.
• Run AddressSanitizer, UndefinedBehaviorSanitizer, leak checks, and static analysis where available.
• Add regression tests for every bug discovered.
• Document known limitations and thread ownership for shared resources.

Acceptance criteria

• Reproducible tests cover engine lifecycle, graph changes, error handling, and DSP output.
• No known critical real-time safety defects remain.
• Resource ownership and cleanup behavior are documented.

Implemented: graph/DSP and ownership audit; stereo source-alias correction with exact-output regression tests; maximum-size reverse-order graph/cycle rollback; allocation-failure sweeps and callback allocation/free guards; 2000 concurrent native-plan replacements with control-thread destruction and balanced lifetimes; repeatable ASan/UBSan/leak and Clang static-analysis targets. Core, no-JACK, GUI/mock/live JACK, recovery and TSan publication/recorder tests pass. See RELIABILITY.md for ownership, synchronization, bounded-work analysis, commands and evidence. Physical-device/endurance validation and trusted native callback behavior remain explicit limits; no critical realtime safety defect is known within the documented contracts.

Phase 11 — Complete Minimal Music-Making Workflow

Goal: Ensure a user can create, save, reopen, and play a small musical project without manually editing project files.

Tasks

• Verify project creation, track creation, clip arrangement, routing, and transport as one integrated workflow.
• Ensure play, pause, stop, seek, and return-to-start behavior are consistent.
• Save, close, reopen, and continue editing projects.
• Provide useful defaults and actionable error messages.
• Keep the sequencer as the default workspace, with the modular graph available when needed.
• Test that GUI state and engine state remain synchronized.

Acceptance criteria

• A user can create and reopen a small project and reproduce the same arrangement and routing.
• Common operations do not require manual project-file edits.
• Transport and GUI state remain consistent during normal playback and error conditions.

Implemented: explicit Pause/F7 and To start/Home controls, position-preserving Stop compatibility, recording guards, sequencer selection on New/Open, first-use composition/routing guidance, and GUI status refresh after failed installation. A real GUI/backend workflow test creates MIDI and audio, arranges/routes, saves, closes/destroys the window, reopens in a fresh window, compares exact stereo rendering and stable clip/routing metadata, then plays/seeks/pauses/resumes, continues editing with undo/redo and saves/reopens again. Mock failure tests and live JACK/GUI/software MIDI tests pass. See WORKFLOW.md for instructions, transport semantics, test commands and limits. External playback routing remains explicit; physical-device listening and endurance remain unqualified.

Phase 12 — Audio Quality and Transport Correctness

Goal: Make timing and audio output predictable and musically reliable.

Tasks

• Define precise behavior for tempo and meter changes.
• Verify conversion between musical positions and sample positions without accumulated timing drift.
• Test clips and events that cross processing-block boundaries.
• Prevent clicks at clip boundaries and during playback start/stop where appropriate.
• Implement gain ramps and parameter smoothing where abrupt changes could create artifacts.
• Test silence, clipping, denormal values, channel layouts, and unusual supported buffer sizes.
• Document latency reporting and identify whether latency compensation is required by the current architecture.
• Add deterministic tests for sample-accurate event scheduling where supported.

Acceptance criteria

• Automated tests verify timing conversions and expected sample output.
• Tests cover events and clips at block boundaries and tempo changes.
• Listening tests cover transitions, start/stop behavior, and representative projects.
• Known limitations involving latency and synchronization are documented.

Implemented: documented constant-tempo/fixed-4/4 snapshot policy; absolute timing and sample-offset regression matrix; configurable audio/scheduled-note boundary fades; single-owner gain ramps; GUI 5 ms start and asynchronous Pause/Stop fades with control-thread acknowledgement/deactivation; finite/subnormal output policy; exact endpoint, block-size, stereo, overload and failure/retry tests. Latency and synchronization limits are documented in AUDIO_QUALITY.md. Normal/no-JACK, sanitizer/static-analysis, TSan, GUI/workflow/live JACK and install checks pass. make quality-preview generates matched listening WAVs. Human listening and physical-device acceptance remain pending; immediate safety/recording/seek paths and live graph swaps may still click. No tempo/meter map, latency compensation, limiter or project-format change is claimed.

Phase 13 — Recording and Recovery

Goal: Protect recorded material and handle recording failures predictably.

Tasks

• Test long recordings and recording-buffer saturation.
• Handle disk-full conditions, write errors, interrupted recording, and unavailable destinations.
• Keep file I/O outside the real-time audio callback.
• Define and implement recovery or clear reporting for incomplete recordings.
• Make recording destinations and supported file formats explicit.
• Protect existing files from accidental overwrites.
• Test reopening projects when media files are missing, incomplete, or inaccessible.
• Ensure shutdown and project closure do not silently discard pending recorded data.

Acceptance criteria

• Recording failures are reported clearly and do not cause unsafe audio-thread behavior.
• Valid recorded material is preserved whenever possible.
• Incomplete recordings and missing media are detected and explained.
• Recording and recovery behavior is covered by automated or repeatable integration tests.

Implemented: no-clobber recording destinations and race-safe publication; explicit opt-in replacement API; retained failed temporaries and recovery-path reporting; streaming xtc-check --recover-recording TEMP NEW.wav salvage; quiescent destructor draining and exceptional-shutdown reporting. Virtual 30-minute recording, saturation/concurrency, destination races, disk-write failure/prefix recovery, killed writers and missing/incomplete media tests pass. See AUDIO_FILES.md and RECOVERY.md. Physical endurance, power-loss guarantees and playback of assets above the existing preload limit remain unqualified.

Phase 14 — Native Plugin, Instrument, and Effect Architecture

Goal: Provide a stable internal extension interface before hosting external plugins.

Tasks

• Finalize the native C node API, including initialization, processing, reset, and destruction.
• Define parameter metadata, defaults, valid ranges, and automation behavior.
• Specify audio channel layouts and MIDI/event handling.
• Define node state serialization and restoration requirements.
• Establish rules for node creation, failure, reconfiguration, and destruction.
• Ensure the GUI uses documented public APIs rather than internal engine structures.
• Provide a small example instrument and effect implemented against the public interface.
• Document ABI/API compatibility expectations for native extensions.

Acceptance criteria

• A developer can implement a small native instrument or effect using documented interfaces.
• Invalid parameters, unsupported layouts, and failed initialization are handled safely.
• Node state can be saved and restored where required.
• No C++ or FLTK types leak into the public C API.

Phase 15 — External Plugin Hosting

Goal: Add external plugin support only after the internal node lifecycle is reliable.

Tasks

• Evaluate candidate plugin formats for licensing, dependencies, maintenance, and Linux support.
• Record the selected format and rationale in an architecture decision document.
• Implement plugin discovery and loading outside the real-time callback.
• Map plugin audio ports, parameters, MIDI/events, and channel layouts to the engine's node model.
• Support plugin state save/load and clear reporting for missing or incompatible plugins.
• Test plugins with different channel counts, parameter sets, and processing requirements.
• Define how plugin failures are reported and how the engine recovers.
• Document crash-isolation limitations; an in-process plugin must not be assumed safe merely because the host is safe.

Acceptance criteria

• A small, documented set of compatible third-party plugins can be loaded, connected, saved, reopened, and played.
• Missing or incompatible plugins do not silently corrupt the project.
• Plugin loading and destruction do not occur in the real-time callback.
• The chosen format's dependencies and licensing are documented.

Phase 16 — Automation and Expressive Control

Goal: Make automation and MIDI control reliable, sample-timed, and usable.

Tasks

• Support automation for common parameters such as volume, pan, instrument parameters, and effect parameters.
• Define interpolation modes and behavior at clip boundaries.
• Evaluate automation at the correct time and sample offsets for each processing block.
• Smooth parameter changes where needed to avoid zipper noise or audio artifacts.
• Add MIDI controller mapping where appropriate.
• Handle note-off, panic, seek, stop, project teardown, and device disconnect correctly.
• Define deterministic ordering for MIDI and automation events that occur at the same sample position.
• Test overlapping notes, repeated note-ons, and interrupted playback.

Acceptance criteria

• Automation and MIDI control reproduce consistent results across repeated playback and project reloads.
• Held notes are released safely on stop, panic, seek, and relevant failure paths.
• Event ordering and interpolation behavior are documented and tested.

Phase 17 — Editing Workflow, Undo/Redo, and Usability

Goal: Make common editing tasks efficient and protect users from accidental changes.

Tasks

• Make common operations undoable, including create, delete, move, trim, duplicate, and connect.
• Define undo behavior for recording and destructive operations.
• Add consistent keyboard shortcuts and context menus for frequently used actions.
• Implement predictable snapping, grid settings, zoom, scrolling, and selection behavior.
• Improve clip labels, track organization, and visual feedback.
• Make destructive actions difficult to trigger accidentally.
• Keep the GUI responsive while audio is running.
• Ensure undo/redo updates project state and the active engine graph consistently.
• Test keyboard focus and accessibility of essential controls where practical.

Acceptance criteria

• Common editing workflows can be completed without unnecessary dialogs or manual workarounds.
• Undo/redo restores the intended project state and corresponding engine state.
• GUI operations do not block the real-time audio thread.
• Interaction behavior is documented and covered by relevant tests.

Phase 18 — Project Format, Compatibility, and Portability

Goal: Keep project files valid, recoverable, and maintainable over time.

Tasks

• Version the project format and implement explicit migration logic when the format changes.
• Validate project data before constructing engine objects from it.
• Handle malformed files, unknown node types, missing assets, and unavailable plugins.
• Define project-folder behavior and reliable media path handling.
• Make saving robust against interrupted writes, for example by writing a temporary file and replacing the destination safely where supported.
• Test save/load round trips and migration from every format version that is supported.
• Keep the format documented and preferably human-inspectable.
• Avoid silently discarding unknown data during load/save unless the format policy explicitly requires it.

Acceptance criteria

• Supported older projects load correctly or produce clear, actionable compatibility errors.
• Malformed project files do not crash the application or trigger unsafe allocations.
• Missing media and plugins are reported clearly.
• Save/load round trips and format migrations have automated tests.

Phase 19 — Performance, Scalability, and Stress Testing

Goal: Measure performance and establish predictable behavior as projects grow.

Tasks

• Create repeatable benchmarks for increasing node counts and audio channel counts.
• Measure processing time at supported sample rates and block sizes.
• Profile graph compilation separately from real-time audio processing.
• Identify unnecessary buffer copies, allocations, and redundant processing.
• Test large arrangements, many clips, automation events, and active meters.
• Define bounded processing limits and meaningful overload diagnostics.
• Test under CPU pressure and during frequent GUI redraws.
• Optimize only after profiling identifies a meaningful bottleneck.
• Record baseline benchmark results so regressions can be detected.

Acceptance criteria

• Performance targets and test conditions are documented.
• Repeatable benchmarks cover representative small and large projects.
• Overload behavior is defined and does not cause unbounded work in the callback.
• Significant performance regressions can be detected through repeatable tests.

Phase 20 — Release Engineering, Packaging, and Maintenance

Goal: Make the application straightforward to build, install, use, diagnose, and maintain.

Tasks

• Provide reproducible release builds and clean build instructions.
• Add Makefile targets for building, testing, installing, and uninstalling.
• Package the application and required resources for the initial supported Linux target.
• Review licenses and attribution for direct and bundled dependencies.
• Document supported audio backends, required system packages, and configuration.
• Add useful crash and error diagnostics without compromising real-time processing.
• Test fresh installations, upgrades, missing dependencies, and removal.
• Publish a basic user manual, developer documentation, and a known-limitations list.
• Define a release checklist and a policy for handling regressions and project-format compatibility.

Acceptance criteria

• A person other than the primary developer can build or install the application using the documented steps.
• The installed application can complete the supported core workflow.
• Installation and removal are tested on the initial target environment.
• Dependencies, licensing, limitations, and troubleshooting steps are documented.
• A release can be reproduced from a clean checkout with recorded build instructions.

---

Recommended Execution Order

Reorder the phases when the audit reveals a blocking dependency. Do not mark a phase complete merely because code exists: its acceptance criteria must be demonstrated by builds, tests, or documented manual verification.

Definition of Done for Each Phase

A phase is complete only when:

• The project builds with warnings enabled.
• Relevant automated tests pass, and manual tests are documented where automation is impractical.
• Error paths, resource ownership, and thread boundaries have been reviewed.
• Real-time audio constraints remain satisfied.
• Documentation is updated alongside API or behavior changes.
• No unrelated functionality was silently removed.
• Untested behavior and known limitations are reported honestly.
• The phase's acceptance criteria are explicitly checked off.

Do not skip phases without documenting why and preserving acceptance criteria.