158 lines
8.4 KiB
Markdown
158 lines
8.4 KiB
Markdown
# Integration tests
|
|
|
|
`make test` is hardware/server independent: offline render tests, concurrent
|
|
plan-replacement stress, and (when JACK headers are present) JACK ABI fault
|
|
injection. The fake links against installed declarations without linking
|
|
libjack. It does not claim to reproduce JACK scheduling; the separate
|
|
threaded stress test exercises publication/reclamation.
|
|
|
|
`make test-jack` explicitly runs a five-second live JACK loopback test. Requires
|
|
an existing JACK or PipeWire JACK-compatible server; does not start a server,
|
|
change its sample rate/block size, or connect physical outputs. Source output
|
|
is connected only to the test's capture client, then disconnected and removed.
|
|
It checks finite/bounded captured samples at two gains, active plan replacement,
|
|
status and ten restart cycles. For a longer run:
|
|
|
|
```sh
|
|
make build/test_jack
|
|
./build/test_jack 30
|
|
```
|
|
|
|
On the inspected system the 30-second test passed at 48000 Hz (initial 1024
|
|
frames): 704 engine blocks, 716 probe blocks, both gains observed, zero xruns,
|
|
and ten restart cycles. This is a limited software-loopback result, not an
|
|
endurance or physical-device qualification. No live server termination or global
|
|
format change is performed on the shared server. Those paths, missing server,
|
|
port/callback setup failure, activation/deactivation/close failure, null buffers,
|
|
invalid configuration, oversize buffers, and reopen are tested with injected
|
|
JACK callbacks/results in `tests/unit/test_backend.c`.
|
|
|
|
## FLTK shell tests
|
|
|
|
```sh
|
|
make gui test-gui
|
|
# Optional, requires a running JACK-compatible server:
|
|
make test-gui-jack
|
|
```
|
|
|
|
These targets require FLTK development files, `xvfb-run` and Xvfb. No additional
|
|
GUI test library is needed. `test_gui.cpp` uses real widgets, menus and timers
|
|
with a fake public C backend. It covers Play/Stop/Reconnect, view switching,
|
|
persistent startup/control errors, server loss/rate changes, partial handles,
|
|
close-failure retry and timer removal. All 18 opened handles are closed.
|
|
`test_gui_jack.cpp` uses the real backend, repeats start/stop nine times across
|
|
three windows, reconnects, and closes each window while running. No audio ports
|
|
are connected. Both tests passed on the inspected host.
|
|
|
|
A separate xdotool smoke run against `build/xtc` under a 1280x800 Xvfb display
|
|
exercised F5/F8/F6, Space, Ctrl+1/2/3 and Ctrl+Q. The app exited successfully;
|
|
visual inspection confirmed running status and mixer view switching.
|
|
|
|
Clang ASan/UBSan found no application errors in the GUI tests. The unsuppressed
|
|
run reports exit-time allocations originating in the installed fontconfig
|
|
library used by FLTK/Pango. `lsan.supp` narrowly suppresses that library; leak
|
|
checking stays enabled for application allocations. This is not a claim that
|
|
the third-party font stack is leak-free. Apply only to GUI sanitizer tests:
|
|
|
|
```sh
|
|
LSAN_OPTIONS=suppressions=tests/integration/lsan.supp xvfb-run -a -s '-screen 0 1280x800x24' ./build/gui-sanitize/test_gui
|
|
```
|
|
|
|
Build that binary with Clang/Clang++, `-fsanitize=address,undefined` in both
|
|
CFLAGS/CXXFLAGS and LDFLAGS, using the baseline warning/include/standard flags
|
|
from BUILD.md. Keep FLTK's `-I` and `-D` flags from `fltk-config --cxxflags`,
|
|
but exclude the distribution's GCC-specific LTO/object-code flags for Clang.
|
|
No suppressions are used for core/backend/realtime sanitizer tests.
|
|
|
|
## Phase 4 graph editing coverage
|
|
|
|
The same `make test-gui` target now adds nodes through toolbar callbacks,
|
|
connects the exact rendered graph, validates expected oscillator/gain samples
|
|
in the submitted backend plan, checks cycle/direction/duplicate rejection,
|
|
checks backend I/O/pending-capacity rollback, deletes wires and incident edges,
|
|
and verifies stable IDs, preserved positions, prepared/queued/active feedback
|
|
and empty-graph silence. The fake backend compiles submitted C snapshots rather
|
|
than accepting an unrelated GUI model. Move-only actions do not submit plans.
|
|
|
|
`make test-gui-jack` exercises the same widget against real JACK: it builds a
|
|
graph while stopped, starts it, adds/deletes nodes during active processing,
|
|
waits for revision acknowledgement, and rejects cycles without changing the
|
|
model. Start/stop/reconnect and callback-safe active close remain covered. No
|
|
routes are created by this test. Core tests separately check known sample
|
|
changes after deleting/reconnecting nodes and meter lookup after ID compaction.
|
|
|
|
A live xdotool mouse run under a 1280x800 Xvfb display created oscillator → gain
|
|
→ meter → output by dragging ports, observed a rejected feedback cycle, moved
|
|
a source with its wire attached, deleted a wire and node, and closed while
|
|
running. Visual inspection confirmed actual wires and active-state feedback,
|
|
with zero xruns reported during that smoke run. Zoom, multiselection, editable
|
|
parameters, project save/load, realtime meter visualization and hardware routing
|
|
controls are outside this phase. Node layout currently lasts only for a session.
|
|
|
|
|
|
## Phase 5 project and transport coverage
|
|
|
|
test_gui adds a track and region, injects FLTK mouse gestures for movement and
|
|
left trimming, checks zoom, saves/reopens the arrangement, and rejects missing
|
|
files and simulated backend failures without replacing the document.
|
|
A sample-status update drives the cursor to an exact tick; a graph edit retains
|
|
that position. test_gui_jack checks actual playhead advancement and a stable
|
|
paused cursor across repeated starts/stops. No routes are created.
|
|
test_project additionally covers serialization, exact graph samples/IDs,
|
|
truncation/mutation input, capacity, save failures and cleanup (part of make test).
|
|
The core/backend tests remain unsuppressed under ASan/UBSan; GUI checks retain
|
|
the narrowly scoped fontconfig-only suppression described above.
|
|
|
|
|
|
## Phase 6 audio acceptance
|
|
|
|
make test-jack-audio requires a running JACK-compatible server. It creates only
|
|
software routes between its probe and xtc backend: known 0.25 source, input
|
|
capture through the recorder ring, WAV finalization, project save/reopen,
|
|
then clip source graph playback observed at the probe input. All test ports,
|
|
files and clients are removed. It never connects hardware or changes server
|
|
configuration.
|
|
|
|
test_gui covers record/Stop, exact take playback, saving/reopening, failed import,
|
|
missing assets, and blocked edits/file actions/close while recording. The live
|
|
test_gui_jack records a silent take, finalizes and saves/reopens it, then starts
|
|
playback before callback-safe close. Decoder and producer/consumer concurrency
|
|
checks are part of make test; run test_audio and test_realtime under TSan as in
|
|
BUILD.md. Physical input/output and latency compensation are not tested.
|
|
|
|
## Phase 7 verification
|
|
|
|
```sh
|
|
make all test gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
|
|
make BUILD_DIR=build/nojack PKG_CONFIG=false all test gui test-gui
|
|
```
|
|
|
|
`test_midi` adds exact sample scheduling, trim/seek/panic, block independence,
|
|
voice capacities, stable notes and v3 malformed-file/roundtrip/v2 migration
|
|
coverage. Backend fault injection checks event offsets, dropped/malformed input,
|
|
disconnection, xruns, missing output buffers and stop/restart cleanup. GUI tests
|
|
draw/move/resize notes, save/reopen and render them. The live MIDI target connects
|
|
only software clients and checks on/off, panic, disconnect, and saved-note
|
|
playback/seek. It needs an existing JACK-compatible server. No hardware routes
|
|
or system changes are made. MIDI recording/SMF and physical devices are deferred.
|
|
|
|
## Phase 8 verification
|
|
|
|
```sh
|
|
make all test gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
|
|
make BUILD_DIR=build/nojack PKG_CONFIG=false all test gui test-gui
|
|
```
|
|
|
|
`test_mixer` covers stereo gain/pan, mute/solo, sample automation, block
|
|
independence, WAV and graph routing, v4 persistence failures and bounded history.
|
|
GUI tests add automation mouse edits, mixer keys, history rollback/branching and
|
|
save/reopen. The existing live MIDI test now uses two software capture ports and
|
|
checks hard-right saved-note playback and output meter snapshots; mono audio
|
|
record/play acceptance remains covered. A native Xvfb mouse smoke check confirms
|
|
sliders and automation drawing. See MIXER.md for capacities and deferred features.
|
|
|
|
The fresh Phase 8 verification build, no-JACK core/GUI suite and Clang
|
|
ASan/UBSan core/GUI suite passed without compiler warnings. GUI leak checking
|
|
retains only the existing fontconfig suppression. ThreadSanitizer also passed
|
|
the recorder concurrency and realtime publication tests. Physical devices and
|
|
long-running stereo performance remain unverified.
|