xtc/tests/integration

Integration tests

Configure the root project first with cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug, then cmake --build build --parallel. Run ctest --test-dir build --output-on-failure for registered tests. Live tests require a separate configure with -DXTC_ENABLE_LIVE_TESTS=ON and an existing JACK server; CTest serializes them. See BUILD.md for presets and feature options. Older validation records below describe tests performed at that time; commands use the CMake workflow and assume the relevant configuration has already completed.

cmake --build build --target check is hardware/server independent: offline render tests, concurrent plan-replacement stress, and (when JACK headers are present) JACK ABI fault injection. The fake exports JACK entry points against the installed declarations; the CMake core target carries the optional JACK link dependency. It does not claim to reproduce JACK scheduling; the separate threaded stress test exercises publication/reclamation.

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-jack explicitly runs a five-second live JACK loopback test. Requires an existing JACK or PipeWire JACK-compatible server; does not start a server, change its sample rate/block size, or connect physical outputs. Source output is connected only to the test's capture client, then disconnected and removed. It checks finite/bounded captured samples at two gains, active plan replacement, status and ten restart cycles. For a longer run:

cmake --build build --target test_jack
./build/test_jack 30

On the inspected system the 30-second test passed at 48000 Hz (initial 1024 frames): 704 engine blocks, 716 probe blocks, both gains observed, zero xruns, and ten restart cycles. This is a limited software-loopback result, not an endurance or physical-device qualification. No live server termination or global format change is performed on the shared server. Those paths, missing server, port/callback setup failure, activation/deactivation/close failure, null buffers, invalid configuration, oversize buffers, and reopen are tested with injected JACK callbacks/results in tests/unit/test_backend.c.

Sequencer component regression checks

test_gui verifies the actual FLTK hierarchy: a master sequencer owns its ruler and track groups; tracks own clips, velocity lanes and header groups; headers own instrument selectors. It checks view identity through track reordering and undo, child membership after cross-track moves, deferred deletion and project replacement during an active gesture. Clip and MIDI velocity gestures also run through Fl::handle at the window boundary to exercise native event dispatch and capture, alongside the existing direct-widget gesture tests.

cmake --build build --target test_gui
ctest --test-dir build -R '^gui-sequencer$' --output-on-failure
ctest --test-dir build -R '^gui$' --output-on-failure

Run GUI variants sequentially (ctest --test-dir build -L gui -j 1) because several existing workflows use the same fixed temporary project filenames.

FLTK shell tests

cmake --build build --target gui test-gui
# Optional, requires a running JACK-compatible server:
cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-gui-jack

These targets require FLTK development files, xvfb-run and Xvfb. No additional GUI test library is needed. test_gui.cpp uses real widgets, menus and timers with a fake public C backend. It covers Play/Stop/Reconnect, view switching, persistent startup/control errors, server loss/rate changes, partial handles, close-failure retry and timer removal. All 18 opened handles are closed. test_gui_jack.cpp uses the real backend, repeats start/stop nine times across three windows, reconnects, and closes each window while running. No audio ports are connected. Both tests passed on the inspected host.

A separate xdotool smoke run against build/xtc under a 1280x800 Xvfb display exercised F5/F8/F6, Space, Ctrl+1/2/3 and Ctrl+Q. The app exited successfully; visual inspection confirmed running status and mixer view switching.

Clang ASan/UBSan found no application errors in the GUI tests. The unsuppressed run reports exit-time allocations originating in the installed fontconfig library used by FLTK/Pango. lsan.supp narrowly suppresses that library; leak checking stays enabled for application allocations. This is not a claim that the third-party font stack is leak-free. Apply only to GUI sanitizer tests:

LSAN_OPTIONS=suppressions=tests/integration/lsan.supp xvfb-run -a -s '-screen 0 1280x800x24' ./build/gui-sanitize/test_gui

Build that binary in a separate CMake directory with -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DXTC_SANITIZE=ON. CMake propagates sanitizer compile/link options and filters FLTK discovery flags so they do not override the chosen compiler or optimization settings. No suppressions are used for core/backend/realtime sanitizer tests.

Phase 4 graph editing coverage

The same cmake --build build --target test-gui target now adds nodes through toolbar callbacks, connects the exact rendered graph, validates expected oscillator/gain samples in the submitted backend plan, checks cycle/direction/duplicate rejection, checks backend I/O/pending-capacity rollback, deletes wires and incident edges, and verifies stable IDs, preserved positions, prepared/queued/active feedback and empty-graph silence. The fake backend compiles submitted C snapshots rather than accepting an unrelated GUI model. Move-only actions do not submit plans.

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-gui-jack exercises the same widget against real JACK: it builds a graph while stopped, starts it, adds/deletes nodes during active processing, waits for revision acknowledgement, and rejects cycles without changing the model. Start/stop/reconnect and callback-safe active close remain covered. No routes are created by this test. Core tests separately check known sample changes after deleting/reconnecting nodes and meter lookup after ID compaction.

A live xdotool mouse run under a 1280x800 Xvfb display created oscillator → gain → meter → output by dragging ports, observed a rejected feedback cycle, moved a source with its wire attached, deleted a wire and node, and closed while running. Visual inspection confirmed actual wires and active-state feedback, with zero xruns reported during that smoke run. Zoom, multiselection, editable parameters, project save/load, realtime meter visualization and hardware routing controls are outside this phase. Node layout currently lasts only for a session.

Phase 5 project and transport coverage

test_gui adds a track and region, injects FLTK mouse gestures for movement and left trimming, checks zoom, saves/reopens the arrangement, and rejects missing files and simulated backend failures without replacing the document. A sample-status update drives the cursor to an exact tick; a graph edit retains that position. test_gui_jack checks actual playhead advancement and a stable paused cursor across repeated starts/stops. No routes are created. test_project additionally covers serialization, exact graph samples/IDs, truncation/mutation input, capacity, save failures and cleanup (part of CTest). The core/backend tests remain unsuppressed under ASan/UBSan; GUI checks retain the narrowly scoped fontconfig-only suppression described above.

Phase 6 audio acceptance

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-jack-audio requires a running JACK-compatible server. It creates only software routes between its probe and xtc backend: known 0.25 source, input capture through the recorder ring, WAV finalization, project save/reopen, then clip source graph playback observed at the probe input. All test ports, files and clients are removed. It never connects hardware or changes server configuration.

test_gui covers record/Stop, exact take playback, saving/reopening, failed import, missing assets, and blocked edits/file actions/close while recording. The live test_gui_jack records a silent take, finalizes and saves/reopens it, then starts playback before callback-safe close. Decoder and producer/consumer concurrency checks are part of CTest; run test_audio and test_realtime under TSan as in BUILD.md. Physical input/output and latency compensation are not tested.

Phase 7 verification

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --parallel && cmake --build build --target check gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
cmake -S . -B build/nojack -DXTC_WITH_JACK=OFF && cmake --build build/nojack --parallel && cmake --build build/nojack --target check gui test-gui

test_midi adds exact sample scheduling, trim/seek/panic, block independence, voice capacities, stable notes and v3 malformed-file/roundtrip/v2 migration coverage. Backend fault injection checks event offsets, dropped/malformed input, disconnection, xruns, missing output buffers and stop/restart cleanup. GUI tests draw/move/resize notes, save/reopen and render them. The live MIDI target connects only software clients and checks on/off, panic, disconnect, and saved-note playback/seek. It needs an existing JACK-compatible server. No hardware routes or system changes are made. MIDI recording/SMF and physical devices are deferred.

Phase 8 verification

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --parallel && cmake --build build --target check gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
cmake -S . -B build/nojack -DXTC_WITH_JACK=OFF && cmake --build build/nojack --parallel && cmake --build build/nojack --target check gui test-gui

test_mixer covers stereo gain/pan, mute/solo, sample automation, block independence, WAV and graph routing, v4 persistence failures and bounded history. GUI tests add automation mouse edits, mixer keys, history rollback/branching and save/reopen. The existing live MIDI test now uses two software capture ports and checks hard-right saved-note playback and output meter snapshots; mono audio record/play acceptance remains covered. A native Xvfb mouse smoke check confirms sliders and automation drawing. See MIXER.md for capacities and deferred features.

The fresh Phase 8 verification build, no-JACK core/GUI suite and Clang ASan/UBSan core/GUI suite passed without compiler warnings. GUI leak checking retains only the existing fontconfig suppression. ThreadSanitizer also passed the recorder concurrency and realtime publication tests. Physical devices and long-running stereo performance remain unverified.

Phase 9 release preparation

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --parallel && cmake --build build --target check gui debug test-gui test-gui-jack test-jack test-jack-audio test-jack-midi
cmake --build build --target test-install test-recovery test-performance
cmake --build build --target test-release

Core test now includes native ABI/lifecycle/retirement and the native-gain example. Optional release tests use Python 3.12+ standard library, Git for indexed source exports, GNU/LLVM deterministic ar and temporary directories. No live prefix is installed. test-install checks core/GUI files, a prefix containing spaces, headless CLI, pkg-config C/C++ consumers, exact/idempotent uninstall and preservation of unrelated files. test-release compares two clean exported optimized builds and repacked archives byte-for-byte. test-recovery kills private child writers and forces a child-local write limit, checking complete finalized destinations and explicit orphan temporaries. Performance timings are reported, not used as a universal realtime guarantee. See RELEASING.md, RECOVERY.md and PERFORMANCE.md.

Phase 9 verification passed a fresh warning-enabled core/debug/GUI build, all software JACK targets, staged install/uninstall, recovery and performance tests. No-JACK core/GUI/install checks and Clang ASan/UBSan core/GUI checks also passed (the GUI retains only its existing fontconfig leak suppression). TSan recorder and realtime-handoff checks passed. Two independent optimized source exports and their repacked source archives were byte-identical on this toolchain.

Phase 10 reliability audit

Run cmake --build build --target test-reliability for the core suites, allocation-failure sweep, callback allocation/free guard and native reclamation checks. Run cmake --build build --target test-sanitize for Clang ASan/UBSan/leak checks and cmake --build build --target analyze for Clang static analysis of all core sources. These need installed development tools; the allocation guard uses a GNU-compatible linker. See RELIABILITY.md for ownership, audit findings, full verification commands and remaining limits.

Phase 11 composition workflow

cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-workflow builds and runs test_workflow.cpp under Xvfb with the real JACK-compatible backend. It requires an already running server. It creates and arranges MIDI and imported audio through GUI commands, saves/closes/destroys the window, reopens in another window, compares stereo rendering exactly, verifies playhead/meters and pause/resume/seek/home, and continues editing/saving. Mock GUI tests inject failed stop/seek/project preparation and recording guards. See WORKFLOW.md for acceptance details and limitations.

Phase 12 transitions

Backend fault tests verify start/stop ramp samples, repeated requests, callback completion, silent/held blocks before collection and deactivation error/retry. GUI tests keep immediate mocked stop semantics; real workflow/live GUI tests wait for asynchronous fade acknowledgement. test_quality in the unit suite checks exact source envelopes/timing and variable blocks, including tempo snapshot changes. See AUDIO_QUALITY.md for commands, latency limitations and the pending listening checklist.

playback-example renders the bundled four-bar song offline and checks finite, bounded stereo audio in every bar. test_playback exercises real window events for Play/Stop/Space and verifies stereo audio with a private JACK software probe while saved MIDI devices are absent. Run from the repository root:

cmake --build build --target test_playback
xvfb-run -a ./build/test_playback

It requires an existing JACK/PipeWire server, changes no user preferences and connects no physical outputs. Enable XTC_ENABLE_LIVE_TESTS to include playback in CTest. The fake-backend gui-midi-workflows test also checks absent input/output/both, returning devices and genuine backend failures.

The optional JUCE workstation builds the shared GUI regression source against src/gui/JUCE as test_juce_gui; juce-gui-* CTests cover the same retained commands, fault injection, dialogs, hierarchy and gestures without FLTK. Build all test fixtures before running them. The real-backend test_juce_playback delivers JUCE MouseEvents to native transport buttons, resumes with a JUCE KeyPress, and checks finite nonzero stereo samples through a private JACK probe with missing saved MIDI devices. It writes no user preferences and connects no physical ports. Run it from the repository root under Xvfb after building:

cmake --build build/juce-gui --parallel 4
ctest --test-dir build/juce-gui -R juce-gui --output-on-failure -j1
xvfb-run -a ./build/juce-gui/test_juce_playback

XTC_ENABLE_LIVE_TESTS registers the latter as juce-playback. An optional XTC_JUCE_SCREENSHOT absolute output path captures the actual workstation component during that live regression.

Linux qualification used JUCE 8.0.15: all 12 JUCE GUI workflows passed (with focused reruns after native focus/blur fixes), alongside all 57 existing FLTK/core tests. Native Play/Stop/Space, expanded-track rendering and finite stereo output passed with deliberately unavailable saved MIDI devices. Bare Xvfb can emit JUCE X11 BadAtom diagnostics without a window manager; these do not indicate a failed assertion in the workflow checks.