xtc/docs/ROADMAP.md

# Roadmap

## Phase 0 — Baseline
Inspect repository/toolchain, verify Makefile and minimal executable, document
dependencies. No system changes without permission.

Baseline implementation: headless startup/shutdown executable, core tests,
separate debug build, optional FLTK shell build, and header dependencies.
See [BUILD.md](BUILD.md) for toolchain, commands, licensing and limitations.

## Phase 1 — C core and offline processing
Graph data structures, validation, cycle detection, topological ordering,
preallocated buffers, test oscillator, gain and offline render tests.

Implemented: mono ports, transactional cycle rejection, independent compiled
plans, precomputed input routing, oscillator/gain/meter/output, deterministic
variable-block tests and lifecycle/error coverage. See AUDIO_ENGINE.md for API
limits and compatibility behavior.

## Phase 2 — Real-time backend
Choose and implement a Linux backend. Document callback semantics, start/stop,
device errors, shutdown, and safe graph publication.

Implemented: optional JACK-compatible mono backend, explicit audio port routing,
format/xrun/loss status, start/stop/reopen, and bounded pending/retired plan
handoff using lock-free atomics. Hardware-independent failure/concurrency tests
and a 30-second software-loopback run pass. See decision 0004 and AUDIO_ENGINE.md.
Physical hardware playback/endurance and live server-loss recovery remain
unverified; GUI engine controls are implemented in Phase 3.

## Phase 3 — FLTK shell
Main window, transport controls, view switching, backend status, clean startup
and shutdown through the C API.

Implemented: dark FLTK shell, Play/Stop/Reconnect, view/menu shortcuts, backend
state/format/xrun/process status, persistent command errors, timer-based refresh
and retirement collection, and callback-safe close/retry. Mock-backend GUI tests
and live GUI/JACK start/stop/reconnect/close tests pass under Xvfb. Sequencer,
mixer and project-file actions remain placeholders; initial graph is silent. See GUI.md.

## Phase 4 — Modular editor
Draw nodes/ports/edges; create, move and connect nodes; reject invalid edges;
safely submit edits to the C core.

Implemented: scrollable node/port/wire canvas; add/select/move/delete nodes;
drag connections, select/delete wires, and show invalid-edit feedback. C owns
stable IDs, read-only snapshots and transactional edits. Exact accepted graph
snapshots are submitted to the backend; prepared/queued/active state is exposed
through revision acknowledgements. Core/render, GUI rollback, mouse smoke and
live adoption tests pass. See GUI.md and AUDIO_ENGINE.md. Preset parameters,
mono-only processing, fixed canvas and no persistence/undo/zoom remain limits.

## Phase 5 — Project and sequencer
Versioned persistence, stable IDs, tracks, clips, timeline, selection, zoom,
scrolling, and transport synchronization.

Implemented: bounded version-1 project persistence with transactional loads and
temporary-file/fsync/rename saves; stable track/clip/node IDs; silent regions;
track and region creation/deletion, selection, move/trim, timeline zoom/scroll;
sample-counted playhead, pause/resume and stopped seek. Project/graph arrangement
roundtrips, malformed-input, GUI rollback/gesture and live transport tests pass.
See PROJECT_FORMAT.md and GUI.md. Constant tempo/4/4, silent metadata regions,
no undo, asset playback, MIDI or persisted GUI layout are explicit limits.

## Phase 6 — Audio clips and recording
Audio import/playback, recording through bounded buffers, missing-asset
handling, no file I/O in callbacks.

Implemented: bounded PCM16/24/32 and float32 WAV import (mono/stereo downmix),
preloaded overlap-summed clip plans with linear rate conversion, and an audio
source node using ordinary graph routing. A lock-free SPSC ring transfers JACK
input to a control-thread WAV writer; Stop finalizes and adds successful takes.
Version 2 metadata stores external asset references and reads v1. Missing assets
remain silent with reporting and manual relinking. Core, callback, GUI and live
software record/save/reopen/play acceptance tests pass. See AUDIO_FILES.md for
memory/format limits, overflow recovery and deferred physical-device validation.
No new dependency or callback I/O was introduced.

## Phase 7 — MIDI and instrument
MIDI input/events/clips, simple internal instrument, note-off/panic handling,
then a basic piano roll.

Implemented: bounded source-relative note events with stable IDs; sample-offset
JACK MIDI input; a velocity-scaled sine instrument and separate graph source;
panic, note-off, stop and seek handling; and a piano roll for drawing, moving,
resizing and deleting notes. Version 3 saves notes and reads v1/v2 projects.
Core, fault-injection, GUI and live software MIDI/save/reopen/play tests pass.
Acceptance uses drawn notes; MIDI recording and SMF import are deferred.
See MIDI.md for capacities, immediate note-off, routing and hardware limits.

## Phase 8 — Mixer and editing
Gain/pan, meters, mute/solo, automation, undo/redo, keyboard workflow.

Implemented: selectable per-track channel strips with gain/pan and mute/solo;
stereo graph/JACK output and output peak meters; editable gain/pan automation
lanes with sample interpolation; bounded 32-edit C snapshot undo/redo and mixer
keyboard controls. Format v4 persists mixing/automation and reads v1–v3.
Core, GUI rollback/gesture/history and live stereo software playback tests pass.
See MIXER.md for routing, mono compatibility, pan law, history behavior and limits.
Track inserts/sends, live fader adoption, MIDI capture, per-track meters and
advanced automation remain deferred. No project-file editing is needed for the
implemented composition/mixing workflow.

## Phase 9 — Extensibility and release
Native node API, evaluate plugin hosting, licensing review, packaging,
installation tests, performance and recovery tests.

Implemented: versioned native C processor lifecycle and graph source/effect API,
explicit instrument-buffer integration, example and error/retirement tests;
external-hosting evaluation (deferred); dependency/license review; deterministic
source/release targets; core/GUI staged install/uninstall and installed C/C++
consumer tests; headless version/help/metadata validation; offline performance
and process-crash/write-failure recovery tests. Source format stays v4, reading
v1–v3. See NATIVE_API.md, RELEASING.md, LINUX.md, LICENSES.md, PERFORMANCE.md and
RECOVERY.md. Native graphs remain session-only, with unsupported saves preserving
files. No stable binary plugin ABI, external host, system package, autosave or
physical-device/endurance qualification is claimed.

## Phase 10 — Engine Correctness and Reliability

**Goal:** Verify that the existing implementation behaves correctly under realistic workloads before adding more features.

### Tasks

- Audit graph compilation, topological ordering, port compatibility, and cycle detection.
- Test graph replacement while audio is running, including safe reclamation of old processing graphs.
- Verify that audio-thread operations are bounded and non-blocking.
- Test repeated initialization, start/stop cycles, device failures, shutdown, and partial initialization.
- Run AddressSanitizer, UndefinedBehaviorSanitizer, leak checks, and static analysis where available.
- Add regression tests for every bug discovered.
- Document known limitations and thread ownership for shared resources.

### Acceptance criteria

- Reproducible tests cover engine lifecycle, graph changes, error handling, and DSP output.
- No known critical real-time safety defects remain.
- Resource ownership and cleanup behavior are documented.

Implemented: graph/DSP and ownership audit; stereo source-alias correction with
exact-output regression tests; maximum-size reverse-order graph/cycle rollback;
allocation-failure sweeps and callback allocation/free guards; 2000 concurrent
native-plan replacements with control-thread destruction and balanced lifetimes;
repeatable ASan/UBSan/leak and Clang static-analysis targets. Core, no-JACK,
GUI/mock/live JACK, recovery and TSan publication/recorder tests pass. See
RELIABILITY.md for ownership, synchronization, bounded-work analysis, commands
and evidence. Physical-device/endurance validation and trusted native callback
behavior remain explicit limits; no critical realtime safety defect is known
within the documented contracts.

## Phase 11 — Complete Minimal Music-Making Workflow

**Goal:** Ensure a user can create, save, reopen, and play a small musical project without manually editing project files.

### Tasks

- Verify project creation, track creation, clip arrangement, routing, and transport as one integrated workflow.
- Ensure play, pause, stop, seek, and return-to-start behavior are consistent.
- Save, close, reopen, and continue editing projects.
- Provide useful defaults and actionable error messages.
- Keep the sequencer as the default workspace, with the modular graph available when needed.
- Test that GUI state and engine state remain synchronized.

### Acceptance criteria

- A user can create and reopen a small project and reproduce the same arrangement and routing.
- Common operations do not require manual project-file edits.
- Transport and GUI state remain consistent during normal playback and error conditions.

Implemented: explicit Pause/F7 and To start/Home controls, position-preserving
Stop compatibility, recording guards, sequencer selection on New/Open, first-use
composition/routing guidance, and GUI status refresh after failed installation.
A real GUI/backend workflow test creates MIDI and audio, arranges/routes, saves,
closes/destroys the window, reopens in a fresh window, compares exact stereo
rendering and stable clip/routing metadata, then plays/seeks/pauses/resumes,
continues editing with undo/redo and saves/reopens again. Mock failure tests and
live JACK/GUI/software MIDI tests pass. See WORKFLOW.md for instructions,
transport semantics, test commands and limits. External playback routing remains
explicit; physical-device listening and endurance remain unqualified.

## Phase 12 — Audio Quality and Transport Correctness

**Goal:** Make timing and audio output predictable and musically reliable.

### Tasks

- Define precise behavior for tempo and meter changes.
- Verify conversion between musical positions and sample positions without accumulated timing drift.
- Test clips and events that cross processing-block boundaries.
- Prevent clicks at clip boundaries and during playback start/stop where appropriate.
- Implement gain ramps and parameter smoothing where abrupt changes could create artifacts.
- Test silence, clipping, denormal values, channel layouts, and unusual supported buffer sizes.
- Document latency reporting and identify whether latency compensation is required by the current architecture.
- Add deterministic tests for sample-accurate event scheduling where supported.

### Acceptance criteria

- Automated tests verify timing conversions and expected sample output.
- Tests cover events and clips at block boundaries and tempo changes.
- Listening tests cover transitions, start/stop behavior, and representative projects.
- Known limitations involving latency and synchronization are documented.

Implemented: documented constant-tempo/fixed-4/4 snapshot policy; absolute timing
and sample-offset regression matrix; configurable audio/scheduled-note boundary
fades; single-owner gain ramps; GUI 5 ms start and asynchronous Pause/Stop fades
with control-thread acknowledgement/deactivation; finite/subnormal output policy;
exact endpoint, block-size, stereo, overload and failure/retry tests. Latency and
synchronization limits are documented in AUDIO_QUALITY.md. Normal/no-JACK,
sanitizer/static-analysis, TSan, GUI/workflow/live JACK and install checks pass.
`make quality-preview` generates matched listening WAVs. Human listening and
physical-device acceptance remain pending; immediate safety/recording/seek paths
and live graph swaps may still click. No tempo/meter map, latency compensation,
limiter or project-format change is claimed.

## Phase 13 — Recording and Recovery

**Goal:** Protect recorded material and handle recording failures predictably.

### Tasks

- Test long recordings and recording-buffer saturation.
- Handle disk-full conditions, write errors, interrupted recording, and unavailable destinations.
- Keep file I/O outside the real-time audio callback.
- Define and implement recovery or clear reporting for incomplete recordings.
- Make recording destinations and supported file formats explicit.
- Protect existing files from accidental overwrites.
- Test reopening projects when media files are missing, incomplete, or inaccessible.
- Ensure shutdown and project closure do not silently discard pending recorded data.

### Acceptance criteria

- Recording failures are reported clearly and do not cause unsafe audio-thread behavior.
- Valid recorded material is preserved whenever possible.
- Incomplete recordings and missing media are detected and explained.
- Recording and recovery behavior is covered by automated or repeatable integration tests.

Implemented: no-clobber recording destinations and race-safe publication;
explicit opt-in replacement API; retained failed temporaries and recovery-path
reporting; streaming `xtc-check --recover-recording TEMP NEW.wav` salvage;
quiescent destructor draining and exceptional-shutdown reporting. Virtual
30-minute recording, saturation/concurrency, destination races, disk-write
failure/prefix recovery, killed writers and missing/incomplete media tests pass.
See AUDIO_FILES.md and RECOVERY.md. Physical endurance, power-loss guarantees
and playback of assets above the existing preload limit remain unqualified.

## Phase 14 — Native Plugin, Instrument, and Effect Architecture

**Goal:** Provide a stable internal extension interface before hosting external plugins.

### Tasks

- Finalize the native C node API, including initialization, processing, reset, and destruction.
- Define parameter metadata, defaults, valid ranges, and automation behavior.
- Specify audio channel layouts and MIDI/event handling.
- Define node state serialization and restoration requirements.
- Establish rules for node creation, failure, reconfiguration, and destruction.
- Ensure the GUI uses documented public APIs rather than internal engine structures.
- Provide a small example instrument and effect implemented against the public interface.
- Document ABI/API compatibility expectations for native extensions.

### Acceptance criteria

- A developer can implement a small native instrument or effect using documented interfaces.
- Invalid parameters, unsupported layouts, and failed initialization are handled safely.
- Node state can be saved and restored where required.
- No C++ or FLTK types leak into the public C API.

Implemented: ABI 2 parameter-only C processor lifecycle with copied descriptors,
metadata/default/range validation, sample-offset parameter/MIDI ordering,
transactional portable configuration restoration, and public gain/instrument
examples. Lifecycle, invalid input, event ordering, in-place output, reset,
callback failure and state compatibility tests pass. See NATIVE_API.md. ABI 1
graph compatibility is retained; native graph persistence and opaque plugin
state are explicitly outside this parameter-only interface.

## Phase 15 — External Plugin Hosting

**Goal:** Add external plugin support only after the internal node lifecycle is reliable.

### Tasks

- Evaluate candidate plugin formats for licensing, dependencies, maintenance, and Linux support.
- Record the selected format and rationale in an architecture decision document.
- Implement plugin discovery and loading outside the real-time callback.
- Map plugin audio ports, parameters, MIDI/events, and channel layouts to the engine's node model.
- Support plugin state save/load and clear reporting for missing or incompatible plugins.
- Test plugins with different channel counts, parameter sets, and processing requirements.
- Define how plugin failures are reported and how the engine recovers.
- Document crash-isolation limitations; an in-process plugin must not be assumed safe merely because the host is safe.

### Acceptance criteria

- A small, documented set of compatible third-party plugins can be loaded, connected, saved, reopened, and played.
- Missing or incompatible plugins do not silently corrupt the project.
- Plugin loading and destruction do not occur in the real-time callback.
- The chosen format's dependencies and licensing are documented.

Implemented: bounded LADSPA 1.1 C host with explicit file/index discovery,
control-thread loading/lifecycle, mono source/effect graph mapping, independent
stereo instances, metadata/range checks and v5 identity/control persistence.
Public-API GUI load/configure commands support ordinary routing/save/reopen and
undo. Fixture/error/retirement/GUI tests and installed amp/inverter/sine acceptance
pass. See decision 0009, PLUGIN_HOSTING.md and LICENSES.md. MIDI, true multichannel
layouts, opaque state, editors, latency compensation and crash isolation are
unsupported; plugin realtime behavior and physical listening are unqualified.

## Phase 16 — Automation and Expressive Control

**Goal:** Make automation and MIDI control reliable, sample-timed, and usable.

### Tasks

- Support automation for common parameters such as volume, pan, instrument parameters, and effect parameters.
- Define interpolation modes and behavior at clip boundaries.
- Evaluate automation at the correct time and sample offsets for each processing block.
- Smooth parameter changes where needed to avoid zipper noise or audio artifacts.
- Add MIDI controller mapping where appropriate.
- Handle note-off, panic, seek, stop, project teardown, and device disconnect correctly.
- Define deterministic ordering for MIDI and automation events that occur at the same sample position.
- Test overlapping notes, repeated note-ons, and interrupted playback.

### Acceptance criteria

- Automation and MIDI control reproduce consistent results across repeated playback and project reloads.
- Held notes are released safely on stop, panic, seek, and relevant failure paths.
- Event ordering and interpolation behavior are documented and tested.

Implemented: persisted sample-evaluated LADSPA parameter lanes with linear/step
modes, range/discrete validation and public GUI editing; absolute automation
position through seek and plan handoff; fixed live CC7/10/64 mappings with volume/
pan ramps, sustain/retrigger safety, and exact block-partition regressions.
Existing track Gain/Pan lanes cover audio and scheduled instrument volume.
Format v6 migrates v1–v5. See CONTROL.md. MIDI capture/learn, pitch bend, new
instrument timbres and physical listening remain outside the supported scope.

## Phase 17 — Editing Workflow, Undo/Redo, and Usability

**Goal:** Make common editing tasks efficient and protect users from accidental changes.

### Tasks

- Make common operations undoable, including create, delete, move, trim, duplicate, and connect.
- Define undo behavior for recording and destructive operations.
- Add consistent keyboard shortcuts and context menus for frequently used actions.
- Implement predictable snapping, grid settings, zoom, scrolling, and selection behavior.
- Improve clip labels, track organization, and visual feedback.
- Make destructive actions difficult to trigger accidentally.
- Keep the GUI responsive while audio is running.
- Ensure undo/redo updates project state and the active engine graph consistently.
- Test keyboard focus and accessibility of essential controls where practical.

### Acceptance criteria

- Common editing workflows can be completed without unnecessary dialogs or manual workarounds.
- Undo/redo restores the intended project state and corresponding engine state.
- GUI operations do not block the real-time audio thread.
- Interaction behavior is documented and covered by relevant tests.

## Phase 18 — Project Format, Compatibility, and Portability

**Goal:** Keep project files valid, recoverable, and maintainable over time.

### Tasks

- Version the project format and implement explicit migration logic when the format changes.
- Validate project data before constructing engine objects from it.
- Handle malformed files, unknown node types, missing assets, and unavailable plugins.
- Define project-folder behavior and reliable media path handling.
- Make saving robust against interrupted writes, for example by writing a temporary file and replacing the destination safely where supported.
- Test save/load round trips and migration from every format version that is supported.
- Keep the format documented and preferably human-inspectable.
- Avoid silently discarding unknown data during load/save unless the format policy explicitly requires it.

### Acceptance criteria

- Supported older projects load correctly or produce clear, actionable compatibility errors.
- Malformed project files do not crash the application or trigger unsafe allocations.
- Missing media and plugins are reported clearly.
- Save/load round trips and format migrations have automated tests.

## Phase 19 — Performance, Scalability, and Stress Testing

**Goal:** Measure performance and establish predictable behavior as projects grow.

### Tasks

- Create repeatable benchmarks for increasing node counts and audio channel counts.
- Measure processing time at supported sample rates and block sizes.
- Profile graph compilation separately from real-time audio processing.
- Identify unnecessary buffer copies, allocations, and redundant processing.
- Test large arrangements, many clips, automation events, and active meters.
- Define bounded processing limits and meaningful overload diagnostics.
- Test under CPU pressure and during frequent GUI redraws.
- Optimize only after profiling identifies a meaningful bottleneck.
- Record baseline benchmark results so regressions can be detected.

### Acceptance criteria

- Performance targets and test conditions are documented.
- Repeatable benchmarks cover representative small and large projects.
- Overload behavior is defined and does not cause unbounded work in the callback.
- Significant performance regressions can be detected through repeatable tests.

## Phase 20 — Release Engineering, Packaging, and Maintenance

**Goal:** Make the application straightforward to build, install, use, diagnose, and maintain.

### Tasks

- Provide reproducible release builds and clean build instructions.
- Add Makefile targets for building, testing, installing, and uninstalling.
- Package the application and required resources for the initial supported Linux target.
- Review licenses and attribution for direct and bundled dependencies.
- Document supported audio backends, required system packages, and configuration.
- Add useful crash and error diagnostics without compromising real-time processing.
- Test fresh installations, upgrades, missing dependencies, and removal.
- Publish a basic user manual, developer documentation, and a known-limitations list.
- Define a release checklist and a policy for handling regressions and project-format compatibility.

### Acceptance criteria

- A person other than the primary developer can build or install the application using the documented steps.
- The installed application can complete the supported core workflow.
- Installation and removal are tested on the initial target environment.
- Dependencies, licensing, limitations, and troubleshooting steps are documented.
- A release can be reproduced from a clean checkout with recorded build instructions.

---

## Recommended Execution Order

Reorder the phases when the audit reveals a blocking dependency. Do not mark a phase complete merely because code exists: its acceptance criteria must be demonstrated by builds, tests, or documented manual verification.

## Definition of Done for Each Phase

A phase is complete only when:

- The project builds with warnings enabled.
- Relevant automated tests pass, and manual tests are documented where automation is impractical.
- Error paths, resource ownership, and thread boundaries have been reviewed.
- Real-time audio constraints remain satisfied.
- Documentation is updated alongside API or behavior changes.
- No unrelated functionality was silently removed.
- Untested behavior and known limitations are reported honestly.
- The phase's acceptance criteria are explicitly checked off.

Do not skip phases without documenting why and preserving acceptance criteria.