Docs file normalization.
This commit is contained in:
parent
dd32143bf8
commit
619c2d0808
46 changed files with 3587 additions and 2579 deletions
|
|
@ -1,48 +1,54 @@
|
|||
# Performance and realtime qualification
|
||||
# Performance and Realtime Qualification
|
||||
|
||||
The original `test-performance` ran a hardware-independent 48 kHz, 128-frame stereo
|
||||
workload: 64 graph nodes, 4096 MIDI notes (32 simultaneous), 16 tracks and gain
|
||||
automation. Preparation and block timing use CLOCK_MONOTONIC outside the engine;
|
||||
callback code itself does not clock or log. The test checks finite output and
|
||||
reports mean, p99, maximum and CPU-time/audio-duration ratio. A run is a workload
|
||||
measurement, not a universal realtime threshold. Run more blocks with:
|
||||
The original `test-performance` ran a hardware-independent 48 kHz, 128-frame
|
||||
stereo workload: 64 graph nodes, 4096 MIDI notes (32 simultaneous), 16 tracks
|
||||
and gain automation. Preparation and block timing use CLOCK_MONOTONIC outside
|
||||
the engine; callback code itself does not clock or log. The test checks finite
|
||||
output and reports mean, p99, maximum and CPU-time/audio-duration ratio. A run
|
||||
is a workload measurement, not a universal realtime threshold. Run more blocks
|
||||
with:
|
||||
|
||||
```sh
|
||||
./build/test_performance 10000
|
||||
```
|
||||
|
||||
Initial measurement on an AMD Ryzen Threadripper 1920X, GCC 16.2.1, -O2,
|
||||
2000 blocks: preparation 3.045 ms, mean 308.301 us, p99 422.153 us, maximum
|
||||
665.096 us, versus a 2666.667 us block deadline; CPU/audio ratio 0.1156. These are
|
||||
one host/run's observed values and depend on scheduling/load/compiler. The test
|
||||
Initial measurement on an AMD Ryzen Threadripper 1920X, GCC 16.2.1, -O2, 2000
|
||||
blocks: preparation 3.045 ms, mean 308.301 us, p99 422.153 us, maximum 665.096
|
||||
us, versus a 2666.667 us block deadline; CPU/audio ratio 0.1156. These are one
|
||||
host/run's observed values and depend on scheduling/load/compiler. The test
|
||||
excludes WAV disk preparation, external/native plugin complexity, GUI redraw,
|
||||
server scheduling and actual hardware latency. Native callbacks must provide
|
||||
their own bounded realtime implementation.
|
||||
|
||||
Existing software JACK loopback, stereo MIDI/save/reopen, record/reopen,
|
||||
restart/close and concurrent graph handoff tests remain available. Runtime
|
||||
fault injection covers loss/rate/xrun/null-buffer and lifecycle failures without
|
||||
restart/close and concurrent graph handoff tests remain available. Runtime fault
|
||||
injection covers loss/rate/xrun/null-buffer and lifecycle failures without
|
||||
altering the shared server. ASan/UBSan validate memory/error paths; TSan checks
|
||||
recorder/handoff races, not timing. Sanitized runs are not latency qualification.
|
||||
recorder/handoff races, not timing. Sanitized runs are not latency
|
||||
qualification.
|
||||
|
||||
Current worst-case storage includes up to 256 MiB preloaded mono WAV samples,
|
||||
paired graph buffers (128 MiB at 256 nodes/65536 frames; GUI limit 32 MiB), bounded
|
||||
MIDI/mix metadata and up to 33 metadata/graph history snapshots. Pending/retired
|
||||
plans can retain additional graph memory. Native preparation can allocate more;
|
||||
the API cannot enforce an extension's memory budget. There is no CPU overload
|
||||
scheduler, disk streaming, latency compensation or callback allocator audit of
|
||||
arbitrary third-party code. High-density interactive projects, physical devices,
|
||||
long-duration stereo/recording and live server termination remain unqualified.
|
||||
paired graph buffers (128 MiB at 256 nodes/65536 frames; GUI limit 32 MiB),
|
||||
bounded MIDI/mix metadata and up to 33 metadata/graph history snapshots.
|
||||
Pending/retired plans can retain additional graph memory. Native preparation can
|
||||
allocate more; the API cannot enforce an extension's memory budget. There is no
|
||||
CPU overload scheduler, disk streaming, latency compensation or callback
|
||||
allocator audit of arbitrary third-party code. High-density interactive
|
||||
projects, physical devices, long-duration stereo/recording and live server
|
||||
termination remain unqualified.
|
||||
|
||||
## Matrix, capacity and contention checks
|
||||
---
|
||||
|
||||
## Matrix, Capacity and Contention Checks
|
||||
|
||||
`make benchmark` records 24 warmed graph cases: 2/16/64/256 nodes, mono/stereo,
|
||||
44.1k/31, 48k/128 and 96k/1024. Each reports control-thread compile time separately
|
||||
from 300 process calls, mean/p99/max, block deadline, mean load, and paired graph
|
||||
buffer bytes. Timing, sorting, allocation and logging happen outside processing.
|
||||
Metered gain chains test graph scaling; they do not represent expensive plugins.
|
||||
The checked-in `performance-baseline.json` records CPU/compiler/flags/kernel and
|
||||
all measurements. Copy a new run only after reviewing changes and conditions.
|
||||
44.1k/31, 48k/128 and 96k/1024. Each reports control-thread compile time
|
||||
separately from 300 process calls, mean/p99/max, block deadline, mean load, and
|
||||
paired graph buffer bytes. Timing, sorting, allocation and logging happen
|
||||
outside processing. Metered gain chains test graph scaling; they do not
|
||||
represent expensive plugins. The checked-in `performance-baseline.json` records
|
||||
CPU/compiler/flags/kernel and all measurements. Copy a new run only after
|
||||
reviewing changes and conditions.
|
||||
|
||||
```sh
|
||||
make benchmark test-performance test-scale test-stress
|
||||
|
|
@ -60,37 +66,42 @@ shared-machine contention still affect results. No real-time scheduling/pinning
|
|||
or external profiler is required. CPU-pressure runs are correctness checks, not
|
||||
comparable timing baselines. Tests/benchmark outputs live below BUILD_DIR.
|
||||
|
||||
Initial matrix (Threadripper 1920X, GCC 16.2.1, -O2, 300 blocks): 256-node stereo
|
||||
48k/128 mean 353.214 us, p99 396.495 us, load .132; 96k/1024 mean 2478.520 us,
|
||||
p99 3092.434 us, load .232. The full JSON is the evidence. These met the local
|
||||
investigation target of mean load <.5 for ordinary graph chains; this is a
|
||||
host-specific development target, not a supported maximum CPU or latency promise.
|
||||
Initial matrix (Threadripper 1920X, GCC 16.2.1, -O2, 300 blocks): 256-node
|
||||
stereo 48k/128 mean 353.214 us, p99 396.495 us, load .132; 96k/1024 mean
|
||||
2478.520 us, p99 3092.434 us, load .232. The full JSON is the evidence. These
|
||||
met the local investigation target of mean load <.5 for ordinary graph chains;
|
||||
this is a host-specific development target, not a supported maximum CPU or
|
||||
latency promise.
|
||||
|
||||
The mixed 64-node/4096-note workload measured 340–357 us before removing unused
|
||||
live-controller processing and 313 us after (mean over 2000 blocks); p99 varied
|
||||
with system load. The measured hot-path change skips live voices/controller ramps
|
||||
when neither is active and avoids finished-ramp calls. Exact CC/odd-block tests
|
||||
continue to pass. No other graph optimization was justified by this measurement.
|
||||
with system load. The measured hot-path change skips live voices/controller
|
||||
ramps when neither is active and avoids finished-ramp calls. Exact CC/odd-block
|
||||
tests continue to pass. No other graph optimization was justified by this
|
||||
measurement.
|
||||
|
||||
`test-scale` constructs all 64 tracks, 1024 clips, 1024 automation points and
|
||||
4096 notes, compiles a standalone 256-node metered graph, renders stereo odd-sized
|
||||
blocks and rejects capacity excesses. Project/GUI graphs remain capped at 64
|
||||
nodes. `test-stress` runs publication, allocation/retirement, capacity and optional
|
||||
GUI/timer/redraw tests while two ordinary CPU workers compete; subprocesses have
|
||||
timeouts and workers are always terminated. Existing callback guards verify no
|
||||
host allocation/free/loading/unloading. This does not qualify arbitrary plugins.
|
||||
4096 notes, compiles a standalone 256-node metered graph, renders stereo
|
||||
odd-sized blocks and rejects capacity excesses. Project/GUI graphs remain capped
|
||||
at 64 nodes. `test-stress` runs publication, allocation/retirement, capacity and
|
||||
optional GUI/timer/redraw tests while two ordinary CPU workers compete;
|
||||
subprocesses have timeouts and workers are always terminated. Existing callback
|
||||
guards verify no host allocation/free/loading/unloading. This does not qualify
|
||||
arbitrary plugins.
|
||||
|
||||
Callback work is bounded by documented node/frame/clip/note/event/point capacities.
|
||||
Exceeding model/plan/mailbox capacities returns CAPACITY transactionally; invalid
|
||||
or overlarge processing buffers are silenced and reported, JACK xruns are counted.
|
||||
There is no CPU scheduler or preemption inside a DSP callback: an expensive trusted
|
||||
plugin can miss a deadline or hang. Automated LADSPA lanes currently use run(1)
|
||||
and bounded per-sample lane searches and may be costly; they are excluded from
|
||||
the gain-chain baseline. Large WAV memory limits remain those in AUDIO_FILES.md.
|
||||
Physical-device endurance, CPU-overload sound quality and a full GUI latency
|
||||
benchmark remain unqualified.
|
||||
Callback work is bounded by documented node/frame/clip/note/event/point
|
||||
capacities. Exceeding model/plan/mailbox capacities returns CAPACITY
|
||||
transactionally; invalid or overlarge processing buffers are silenced and
|
||||
reported, JACK xruns are counted. There is no CPU scheduler or preemption inside
|
||||
a DSP callback: an expensive trusted plugin can miss a deadline or hang.
|
||||
Automated LADSPA lanes currently use run(1) and bounded per-sample lane searches
|
||||
and may be costly; they are excluded from the gain-chain baseline. Large WAV
|
||||
memory limits remain those in AUDIO_FILES.md. Physical-device endurance,
|
||||
CPU-overload sound quality and a full GUI latency benchmark remain unqualified.
|
||||
|
||||
## Qualification workloads
|
||||
---
|
||||
|
||||
## Qualification Workloads
|
||||
|
||||
The current `test-performance` emits one JSON record (the old prose output is
|
||||
retired). Default: 2,000 stereo blocks at 48 kHz/128, 64 graph nodes, 16 tracks,
|
||||
|
|
@ -101,10 +112,10 @@ for this revision. `benchmark-project` runs nine cases: 1/16/64 tracks,
|
|||
Two long notes per first 16 tracks remain audible. Other notes are scheduled
|
||||
later to remain within the 32-voice plan limit. The position wraps every 96,000
|
||||
samples to keep long runs active. Future notes exercise storage/preparation and
|
||||
bounded scans rather than additional simultaneous synthesis. Each track has
|
||||
16 gain points; graph processing uses the same 64-node chain in every case.
|
||||
The graph matrix separately scales nodes and chain connections from 2 to 256.
|
||||
These synthetic cases do not qualify every possible dense graph or plugin chain.
|
||||
bounded scans rather than additional simultaneous synthesis. Each track has 16
|
||||
gain points; graph processing uses the same 64-node chain in every case. The
|
||||
graph matrix separately scales nodes and chain connections from 2 to 256. These
|
||||
synthetic cases do not qualify every possible dense graph or plugin chain.
|
||||
|
||||
```sh
|
||||
make benchmark benchmark-project
|
||||
|
|
@ -122,22 +133,25 @@ exceedances, not actual JACK xruns: there is no server scheduling in the timing
|
|||
loop. Graph compilation or project construction/preparation is timed separately.
|
||||
`process_cpu_ms` measures CLOCK_PROCESS_CPUTIME_ID over the complete measured
|
||||
loop, including timing and finite-output checks; it excludes preparation and
|
||||
sorting. Project `cpu_audio_ratio` divides that CPU time by simulated audio time.
|
||||
`peak_rss_kib` is Linux getrusage process high-water RSS, including model, plans,
|
||||
measurement storage and executable/runtime overhead; it is not current heap size
|
||||
or a leak test. Graph buffer bytes remain a separate exact buffer-size estimate.
|
||||
Timing probes, memory queries, sorting and output occur outside engine calls.
|
||||
sorting. Project `cpu_audio_ratio` divides that CPU time by simulated audio
|
||||
time. `peak_rss_kib` is Linux getrusage process high-water RSS, including model,
|
||||
plans, measurement storage and executable/runtime overhead; it is not current
|
||||
heap size or a leak test. Graph buffer bytes remain a separate exact buffer-size
|
||||
estimate. Timing probes, memory queries, sorting and output occur outside engine
|
||||
calls.
|
||||
|
||||
Checked-in `project-performance-baseline.json` provides initial project cases;
|
||||
the original graph baseline is retained. `graph-performance-baseline.json` adds
|
||||
a current graph report with CPU/RSS/deadline fields. Gates use the existing
|
||||
1.5x mean-time budget and 1 us noise allowance, with p99/max/misses and memory
|
||||
a current graph report with CPU/RSS/deadline fields. Gates use the existing 1.5x
|
||||
mean-time budget and 1 us noise allowance, with p99/max/misses and memory
|
||||
reported for investigation. There is no universal hardware-independent deadline
|
||||
or RSS pass threshold. `test-benchmark-tools` verifies same-context success,
|
||||
regression rejection, missing cases and host/workload mismatch without relying
|
||||
on machine timing. Do not replace a failing baseline without investigating.
|
||||
|
||||
## Endurance, faults and callback audit
|
||||
---
|
||||
|
||||
## Endurance, Faults and Callback Audit
|
||||
|
||||
```sh
|
||||
make test-soak test-stress test-stress-plugins
|
||||
|
|
@ -153,24 +167,26 @@ quiescent position/rebuild cycles. It runs faster than wall time and models
|
|||
position wrap/seek and restart publication; it is not a real device start/stop
|
||||
or hardware endurance test. The project benchmark separately wraps active MIDI.
|
||||
|
||||
`test-stress` defaults to three rounds under two ordinary CPU workers. Each round
|
||||
runs concurrent publication, allocation/lifetime guards, maximum model capacity,
|
||||
project relocation/reload and missing-media retention, save-fault diagnostics,
|
||||
and the soak test. With JACK headers present it also repeats the deterministic
|
||||
backend fake's start/stop/seek, rate/buffer/server-loss and failure/retry cases.
|
||||
Optional GUI tests run after those rounds under the same CPU load.
|
||||
`test-stress-plugins` adds the original LADSPA fixture's identity/layout/range,
|
||||
instantiation-error and invalid-metadata cases each round; it needs LADSPA headers.
|
||||
No command changes a physical device, kills the shared server or loads unknown
|
||||
plugins. Subprocess deadlines bound each test and workers are cleaned up on exit.
|
||||
`test-stress` defaults to three rounds under two ordinary CPU workers. Each
|
||||
round runs concurrent publication, allocation/lifetime guards, maximum model
|
||||
capacity, project relocation/reload and missing-media retention, save-fault
|
||||
diagnostics, and the soak test. With JACK headers present it also repeats the
|
||||
deterministic backend fake's start/stop/seek, rate/buffer/server-loss and
|
||||
failure/retry cases. Optional GUI tests run after those rounds under the same
|
||||
CPU load. `test-stress-plugins` adds the original LADSPA fixture's
|
||||
identity/layout/range, instantiation-error and invalid-metadata cases each
|
||||
round; it needs LADSPA headers. No command changes a physical device, kills the
|
||||
shared server or loads unknown plugins. Subprocess deadlines bound each test and
|
||||
workers are cleaned up on exit.
|
||||
|
||||
The reliability executable now interposes fopen/fread/fwrite/read/write,
|
||||
pthread_mutex_lock and pthread_cond_wait in addition to malloc/calloc/realloc/free.
|
||||
Guarded host graph/realtime render calls fail immediately on these operations.
|
||||
This complements the existing native lifetime and plugin callback guards and
|
||||
control/callback ownership review in RELIABILITY.md. It does not intercept every
|
||||
libc entry point, raw syscalls, hidden dynamic-module calls, or GUI code by magic;
|
||||
third-party DSP remains trusted. No known host callback safety violation was found.
|
||||
pthread_mutex_lock and pthread_cond_wait in addition to
|
||||
malloc/calloc/realloc/free. Guarded host graph/realtime render calls fail
|
||||
immediately on these operations. This complements the existing native lifetime
|
||||
and plugin callback guards and control/callback ownership review in
|
||||
RELIABILITY.md. It does not intercept every libc entry point, raw syscalls,
|
||||
hidden dynamic-module calls, or GUI code by magic; third-party DSP remains
|
||||
trusted. No known host callback safety violation was found.
|
||||
|
||||
`check-development` combines the ordinary build/tests, capacity/soak checks,
|
||||
benchmark gate tests, static analysis and ASan/UBSan. The sanitizer target now
|
||||
|
|
@ -178,16 +194,18 @@ includes maximum capacity and virtual endurance. These are opt-in developer
|
|||
checks requiring installed Clang/Python; no runtime dependency was added.
|
||||
Sanitized/contended/profiled runs must not become timing baselines.
|
||||
|
||||
## Profiling evidence and remaining costs
|
||||
---
|
||||
|
||||
## Profiling Evidence and Remaining Costs
|
||||
|
||||
Before considering DSP changes, a 10,000-block, 64-track/128-frame project was
|
||||
sampled with GCC `-O2 -g -pg` and GNU gprof on the documented baseline host.
|
||||
Flat samples attributed about 30% to mixer automation `value`, 29% to MIDI
|
||||
`segment`, and 11% to finite-value handling. Instrumentation and inlining affect
|
||||
attribution; this identifies investigation candidates, not exact production
|
||||
cost or evidence that checks should be removed. MIDI scans bounded stored-note
|
||||
arrays and evaluates per-voice, per-sample gain/pan lanes. No DSP optimization
|
||||
was introduced without a separate correctness/performance justification.
|
||||
attribution; this identifies investigation candidates, not exact production cost
|
||||
or evidence that checks should be removed. MIDI scans bounded stored-note arrays
|
||||
and evaluates per-voice, per-sample gain/pan lanes. No DSP optimization was
|
||||
introduced without a separate correctness/performance justification.
|
||||
|
||||
Reproduce in an isolated build (GNU gprof must already be installed):
|
||||
|
||||
|
|
@ -209,10 +227,10 @@ CPU/audio .142 and peak process RSS 19,296 KiB. The 256-node stereo/128 graph
|
|||
mean was 286.398 us, p99 484.712 us, compile .300 ms. These case runs observed
|
||||
zero deadline exceedances. See JSON for all cases and host metadata.
|
||||
|
||||
The project comparison passed. Repeated graph comparisons flagged variable
|
||||
small cases despite unchanged graph processing code; a 3,000-block run against
|
||||
the retained baseline still flagged 16-node mono/31-frame mean 5.089 us
|
||||
above its 1.5x budget. The new graph report is evidence, not a replacement chosen
|
||||
to make that comparison pass. This shared-host timing alert remains an
|
||||
investigation item; no CPU/latency guarantee is inferred from the correctness
|
||||
suite. Longer comparisons can use `BENCHMARK_BLOCKS=3000`.
|
||||
The project comparison passed. Repeated graph comparisons flagged variable small
|
||||
cases despite unchanged graph processing code; a 3,000-block run against the
|
||||
retained baseline still flagged 16-node mono/31-frame mean 5.089 us above its
|
||||
1.5x budget. The new graph report is evidence, not a replacement chosen to make
|
||||
that comparison pass. This shared-host timing alert remains an investigation
|
||||
item; no CPU/latency guarantee is inferred from the correctness suite. Longer
|
||||
comparisons can use `BENCHMARK_BLOCKS=3000`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue