hanez/xtc
2
1
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Migrated from Make to CMake.

Browse source
This commit is contained in:
Johannes Findeisen 2026-11-06 19:05:14 +01:00
commit b6ba093000
52 changed files with 1276 additions and 1210 deletions
Download patch file Download diff file Expand all files Collapse all files

60
tests/integration/README.md
View file

@ -1,12 +1,20 @@
# Integration tests
`make test` is hardware/server independent: offline render tests, concurrent
Configure the root project first with `cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug`,
then `cmake --build build --parallel`. Run `ctest --test-dir build
--output-on-failure` for registered tests. Live tests require a separate configure
with `-DXTC_ENABLE_LIVE_TESTS=ON` and an existing JACK server; CTest serializes them.
See [BUILD.md](../../docs/BUILD.md) for presets and feature options. Older validation
records below describe tests performed at that time; commands use the CMake
workflow and assume the relevant configuration has already completed.
`cmake --build build --target check` is hardware/server independent: offline render tests, concurrent
plan-replacement stress, and (when JACK headers are present) JACK ABI fault
injection. The fake links against installed declarations without linking
libjack. It does not claim to reproduce JACK scheduling; the separate
injection. The fake exports JACK entry points against the installed declarations;
the CMake core target carries the optional JACK link dependency. It does not claim to reproduce JACK scheduling; the separate
threaded stress test exercises publication/reclamation.
`make test-jack` explicitly runs a five-second live JACK loopback test. Requires
`cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-jack` explicitly runs a five-second live JACK loopback test. Requires
an existing JACK or PipeWire JACK-compatible server; does not start a server,
change its sample rate/block size, or connect physical outputs. Source output
is connected only to the test's capture client, then disconnected and removed.
@ -14,7 +22,7 @@ It checks finite/bounded captured samples at two gains, active plan replacement,
status and ten restart cycles. For a longer run:
```sh
make build/test_jack
cmake --build build --target test_jack
./build/test_jack 30
```
@ -30,9 +38,9 @@ JACK callbacks/results in `tests/unit/test_backend.c`.
## FLTK shell tests
```sh
make gui test-gui
cmake --build build --target gui test-gui
# Optional, requires a running JACK-compatible server:
make test-gui-jack
cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-gui-jack
```
These targets require FLTK development files, `xvfb-run` and Xvfb. No additional
@ -58,15 +66,15 @@ the third-party font stack is leak-free. Apply only to GUI sanitizer tests:
LSAN_OPTIONS=suppressions=tests/integration/lsan.supp xvfb-run -a -s '-screen 0 1280x800x24' ./build/gui-sanitize/test_gui
```
Build that binary with Clang/Clang++, `-fsanitize=address,undefined` in both
CFLAGS/CXXFLAGS and LDFLAGS, using the baseline warning/include/standard flags
from BUILD.md. Keep FLTK's `-I` and `-D` flags from `fltk-config --cxxflags`,
but exclude the distribution's GCC-specific LTO/object-code flags for Clang.
Build that binary in a separate CMake directory with
`-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DXTC_SANITIZE=ON`.
CMake propagates sanitizer compile/link options and filters FLTK discovery flags
so they do not override the chosen compiler or optimization settings.
No suppressions are used for core/backend/realtime sanitizer tests.
## Phase 4 graph editing coverage
The same `make test-gui` target now adds nodes through toolbar callbacks,
The same `cmake --build build --target test-gui` target now adds nodes through toolbar callbacks,
connects the exact rendered graph, validates expected oscillator/gain samples
in the submitted backend plan, checks cycle/direction/duplicate rejection,
checks backend I/O/pending-capacity rollback, deletes wires and incident edges,
@ -74,7 +82,7 @@ and verifies stable IDs, preserved positions, prepared/queued/active feedback
and empty-graph silence. The fake backend compiles submitted C snapshots rather
than accepting an unrelated GUI model. Move-only actions do not submit plans.
`make test-gui-jack` exercises the same widget against real JACK: it builds a
`cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-gui-jack` exercises the same widget against real JACK: it builds a
graph while stopped, starts it, adds/deletes nodes during active processing,
waits for revision acknowledgement, and rejects cycles without changing the
model. Start/stop/reconnect and callback-safe active close remain covered. No
@ -99,14 +107,14 @@ A sample-status update drives the cursor to an exact tick; a graph edit retains
that position. test_gui_jack checks actual playhead advancement and a stable
paused cursor across repeated starts/stops. No routes are created.
test_project additionally covers serialization, exact graph samples/IDs,
truncation/mutation input, capacity, save failures and cleanup (part of make test).
truncation/mutation input, capacity, save failures and cleanup (part of CTest).
The core/backend tests remain unsuppressed under ASan/UBSan; GUI checks retain
the narrowly scoped fontconfig-only suppression described above.
## Phase 6 audio acceptance
make test-jack-audio requires a running JACK-compatible server. It creates only
cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-jack-audio requires a running JACK-compatible server. It creates only
software routes between its probe and xtc backend: known 0.25 source, input
capture through the recorder ring, WAV finalization, project save/reopen,
then clip source graph playback observed at the probe input. All test ports,
@ -117,14 +125,14 @@ test_gui covers record/Stop, exact take playback, saving/reopening, failed impor
missing assets, and blocked edits/file actions/close while recording. The live
test_gui_jack records a silent take, finalizes and saves/reopens it, then starts
playback before callback-safe close. Decoder and producer/consumer concurrency
checks are part of make test; run test_audio and test_realtime under TSan as in
checks are part of CTest; run test_audio and test_realtime under TSan as in
BUILD.md. Physical input/output and latency compensation are not tested.
## Phase 7 verification
```sh
make all test gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
make BUILD_DIR=build/nojack PKG_CONFIG=false all test gui test-gui
cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --parallel && cmake --build build --target check gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
cmake -S . -B build/nojack -DXTC_WITH_JACK=OFF && cmake --build build/nojack --parallel && cmake --build build/nojack --target check gui test-gui
```
`test_midi` adds exact sample scheduling, trim/seek/panic, block independence,
@ -139,8 +147,8 @@ or system changes are made. MIDI recording/SMF and physical devices are deferred
## Phase 8 verification
```sh
make all test gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
make BUILD_DIR=build/nojack PKG_CONFIG=false all test gui test-gui
cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --parallel && cmake --build build --target check gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
cmake -S . -B build/nojack -DXTC_WITH_JACK=OFF && cmake --build build/nojack --parallel && cmake --build build/nojack --target check gui test-gui
```
`test_mixer` covers stereo gain/pan, mute/solo, sample automation, block
@ -160,9 +168,9 @@ long-running stereo performance remain unverified.
## Phase 9 release preparation
```sh
make all test gui debug test-gui test-gui-jack test-jack test-jack-audio test-jack-midi
make test-install test-recovery test-performance
make test-release
cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --parallel && cmake --build build --target check gui debug test-gui test-gui-jack test-jack test-jack-audio test-jack-midi
cmake --build build --target test-install test-recovery test-performance
cmake --build build --target test-release
```
Core test now includes native ABI/lifecycle/retirement and the native-gain example.
@ -185,9 +193,9 @@ and their repacked source archives were byte-identical on this toolchain.
## Phase 10 reliability audit
Run `make test-reliability` for the core suites, allocation-failure sweep,
Run `cmake --build build --target test-reliability` for the core suites, allocation-failure sweep,
callback allocation/free guard and native reclamation checks. Run
`make test-sanitize` for Clang ASan/UBSan/leak checks and `make analyze` for
`cmake --build build --target test-sanitize` for Clang ASan/UBSan/leak checks and `cmake --build build --target analyze` for
Clang static analysis of all core sources. These need installed development
tools; the allocation guard uses a GNU-compatible linker. See
[RELIABILITY.md](../../docs/RELIABILITY.md) for ownership, audit findings,
@ -195,7 +203,7 @@ full verification commands and remaining limits.
## Phase 11 composition workflow
`make test-workflow` builds and runs `test_workflow.cpp` under Xvfb with the real
`cmake -S . -B build -DXTC_ENABLE_LIVE_TESTS=ON && cmake --build build --target test-workflow` builds and runs `test_workflow.cpp` under Xvfb with the real
JACK-compatible backend. It requires an already running server. It creates and
arranges MIDI and imported audio through GUI commands, saves/closes/destroys the
window, reopens in another window, compares stereo rendering exactly, verifies

52
tests/integration/test_install.py
View file

@ -27,20 +27,39 @@ def main():
sentinel = root / 'lib/keep-unrelated.txt'
sentinel.parent.mkdir(parents=True)
sentinel.write_text('keep\n')
make = shlex.split(os.environ.get('MAKE', 'make'))
# MAKEFLAGS jobserver descriptors do not cross Python subprocesses.
env = dict(os.environ, MAKEFLAGS='', MFLAGS='')
variables = [f'BUILD_DIR={args.build_dir}', f'PREFIX={prefix}', f'DESTDIR={stage}']
# Generate the real CMake install driver against the already-built inputs.
# The temporary cache keeps the caller's build configuration untouched.
installer = stage / 'cmake-driver'
cache = (args.build_dir / 'CMakeCache.txt').read_text()
generator = next(line.partition('=')[2] for line in cache.splitlines()
if line.startswith('CMAKE_GENERATOR:INTERNAL='))
jack = 'OFF' if 'XTC_WITH_JACK:BOOL=OFF' in cache else 'ON'
env = dict(os.environ)
variables = ['-DBUILD_TESTING=OFF', f'-DXTC_BUILD_GUI={"ON" if args.gui else "OFF"}',
f'-DCMAKE_INSTALL_PREFIX={prefix}',
f'-DXTC_INSTALL_BUILD_DIR={args.build_dir.resolve()}',
f'-DXTC_WITH_JACK={jack}', '-DXTC_INSTALL_GUI=OFF', '-DCMAKE_INSTALL_DOCDIR=share/doc/xtc']
if args.custom_layout:
variables += [f'exec_prefix={prefix}/runtime', f'libdir={prefix}/lib64',
f'includedir={prefix}/headers', f'datarootdir={prefix}/resources',
f'docdir={prefix}/manual', f'pkgconfigdir={prefix}/metadata']
variables += ['-DCMAKE_INSTALL_BINDIR=runtime/bin', '-DCMAKE_INSTALL_LIBDIR=lib64',
'-DCMAKE_INSTALL_INCLUDEDIR=headers', '-DCMAKE_INSTALL_DATAROOTDIR=resources',
'-DCMAKE_INSTALL_DOCDIR=manual', '-DXTC_INSTALL_PKGCONFIGDIR=metadata']
def install_command(operation, options, **kwargs):
config = subprocess.run(['cmake', '-G', generator, '-S', str(source), '-B', str(installer),
*options], env=env, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True)
if config.returncode:
raise RuntimeError(config.stdout)
command = (['cmake', '--build', str(installer), '--target', 'uninstall']
if operation == 'uninstall' else ['cmake', '--install', str(installer)])
return subprocess.run(command, env=dict(env, DESTDIR=str(stage)), **kwargs)
binaries = root / ('runtime/bin' if args.custom_layout else 'bin')
headers = root / ('headers' if args.custom_layout else 'include')
resources = root / ('resources' if args.custom_layout else 'share')
docs = root / ('manual' if args.custom_layout else 'share/doc/xtc')
metadata = root / ('metadata' if args.custom_layout else 'lib/pkgconfig')
run(make + ['install'] + variables, env=env)
install_command('install', variables, check=True)
assert not (binaries / 'xtc').exists()
assert (resources / 'licenses/xtc/LICENSE').read_bytes() == (source / 'LICENSE').read_bytes()
assert (docs / 'PROJECT_FORMAT.md').is_file()
@ -64,7 +83,7 @@ def main():
(binaries / 'xtc-check').symlink_to(protected)
# Upgrade/repair repeats the same owned-file install and keeps unrelated data.
(headers / 'xtc/xtc.h').write_text('old installation marker\n')
run(make + ['install'] + variables, env=env)
install_command('install', variables, check=True)
assert (headers / 'xtc/xtc.h').read_bytes() == (source / 'include/xtc/xtc.h').read_bytes()
assert sentinel.read_text() == 'keep\n'
assert protected.read_text() == 'keep unrelated target\n'
@ -79,8 +98,7 @@ def main():
pkg_env = dict(env, PKG_CONFIG_PATH=str(metadata) + ':' + env.get('PKG_CONFIG_PATH', ''),
PKG_CONFIG_SYSROOT_DIR=str(stage))
pkg = shlex.split(os.environ.get('PKG_CONFIG', 'pkg-config'))
# A no-JACK test can be invoked with PKG_CONFIG=false; package consumer
# discovery itself still uses the installed pkg-config tool.
# The build may disable JACK; consumer discovery still uses pkg-config.
if pkg == ['false']:
pkg = ['pkg-config']
flags = shlex.split(subprocess.check_output(pkg + ['--cflags', '--libs', '--static', 'xtc'], env=pkg_env, text=True))
@ -110,7 +128,7 @@ int main(int argc, char **argv) {
run(command + [f'-std={standard}', '-Wall', '-Wextra', '-Werror', str(consumer)] + flags + ['-o', str(output)])
run([str(output), str(stage / ("installed-" + compiler + ".xtc"))])
if args.gui:
run(make + ['install-gui'] + variables, env=env)
install_command('install', variables + ['-DXTC_INSTALL_GUI=ON'], check=True)
gui_env = dict(env, DISPLAY='', WAYLAND_DISPLAY='')
assert subprocess.check_output([str(binaries / 'xtc'), '--version'], env=gui_env, text=True) == version
run([str(binaries / 'xtc'), '--help'], env=gui_env)
@ -120,9 +138,9 @@ int main(int argc, char **argv) {
with tempfile.TemporaryDirectory(prefix='xtc-install-outside-') as outside:
escape = root / 'escape'
escape.symlink_to(outside, target_is_directory=True)
redirected = variables + [f'docdir={prefix}/escape/docs']
redirected = variables + [f'-DCMAKE_INSTALL_DOCDIR={prefix}/escape/docs']
for operation in ('install', 'uninstall'):
refusal = subprocess.run(make + [operation] + redirected, env=env,
refusal = install_command(operation, redirected,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
assert refusal.returncode != 0 and not list(Path(outside).iterdir())
assert (headers / 'xtc/xtc.h').is_file()
@ -131,13 +149,13 @@ int main(int argc, char **argv) {
saved_binary = stage / 'saved-check'
(binaries / 'xtc-check').rename(saved_binary)
(binaries / 'xtc-check').mkdir()
refusal = subprocess.run(make + ['uninstall'] + variables, env=env,
refusal = install_command('uninstall', variables + ['-DCMAKE_INSTALL_DOCDIR=manual' if args.custom_layout else '-DCMAKE_INSTALL_DOCDIR=share/doc/xtc'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
assert refusal.returncode != 0 and (headers / 'xtc/xtc.h').is_file()
(binaries / 'xtc-check').rmdir()
saved_binary.rename(binaries / 'xtc-check')
run(make + ['uninstall'] + variables, env=env)
run(make + ['uninstall'] + variables, env=env)
install_command('uninstall', variables, check=True)
install_command('uninstall', variables, check=True)
remaining = sorted(p.relative_to(root).as_posix() for p in root.rglob('*') if p.is_file() or p.is_symlink())
assert remaining == ['lib/keep-unrelated.txt'], remaining
assert sentinel.read_text() == 'keep\n'

17
tests/integration/test_release.py
View file

@ -4,7 +4,6 @@ import argparse
import hashlib
import os
from pathlib import Path
import shlex
import subprocess
import tarfile
import tempfile
@ -18,6 +17,7 @@ def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('archive', type=Path)
parser.add_argument('--gui', action='store_true')
parser.add_argument('--generator', default=os.environ.get('CMAKE_GENERATOR'))
args = parser.parse_args()
archive = args.archive.resolve()
assert archive.with_name(archive.name + '.sha256').read_text().split()[0] == digest(archive)
@ -33,16 +33,23 @@ def main():
assert all('__pycache__' not in Path(n).parts and '.git' not in Path(n).parts for n in names)
for name in ('first', 'different-path'):
source.extractall(root / name, filter='data')
env = dict(os.environ, SOURCE_DATE_EPOCH=str(epoch), MAKEFLAGS='', MFLAGS='')
env = dict(os.environ, SOURCE_DATE_EPOCH=str(epoch))
if args.generator:
env['CMAKE_GENERATOR'] = args.generator
builds = []
for name in ('first', 'different-path'):
project = root / name / top
log = root / (name + '.log')
make = shlex.split(env.get('MAKE', 'make'))
with log.open('w') as output:
try:
subprocess.run(make + ['BUILD_DIR=build', 'release'], cwd=project, env=env, stdout=output,
stderr=subprocess.STDOUT, check=True)
subprocess.run(['cmake', '-S', '.', '-B', 'build/release',
'-DCMAKE_BUILD_TYPE=Release', '-DXTC_REPRODUCIBLE=ON',
f'-DXTC_BUILD_GUI={"ON" if args.gui else "OFF"}'],
cwd=project, env=env, stdout=output, stderr=subprocess.STDOUT, check=True)
subprocess.run(['cmake', '--build', 'build/release', '--parallel', '3'],
cwd=project, env=env, stdout=output, stderr=subprocess.STDOUT, check=True)
subprocess.run(['ctest', '--test-dir', 'build/release', '--output-on-failure'],
cwd=project, env=env, stdout=output, stderr=subprocess.STDOUT, check=True)
except subprocess.CalledProcessError:
print(log.read_text()[-12000:])
raise