Phase 28: add project diagnostics, dependency inspection and durability regressions

This commit is contained in:
Johannes Findeisen 2026-10-27 16:58:26 +01:00
commit ca6c869b0b
16 changed files with 495 additions and 42 deletions

View file

@ -1,6 +1,6 @@
# Project format v10 (reads v1–v9)
Phases 5–26 use a bounded text format implemented in C99 without a parser library.
Phases 5–28 use a bounded text format implemented in C99 without a parser library.
The extension is `.xtc`. File operations run exclusively on the control thread.
The project owns its graph, tracks and silent/audio arrangement regions. Audio samples remain in external WAV
files; MIDI notes are stored inline.
@ -104,8 +104,8 @@ node types 1..4, migrating their clips to empty asset paths. Version 2 accepts
node types 1..5 and ASSET records. Both migrate to MIDI=0 and no notes.
Version 3 accepts instrument nodes and MIDI/NOTES records. Versions 1–3 migrate
all tracks to unity gain, centered pan, mute/solo off and no automation.
Saving always writes v7. Unknown versions are rejected without replacing the destination; older
applications cannot read v7. Migration and maximum-path roundtrips are tested.
Saving always writes v10. Unknown versions are rejected without replacing the destination; older
applications without v10 support cannot read current saves. Migration and maximum-path roundtrips are tested.
## Ownership and failures
@ -114,8 +114,9 @@ the destination. Any failure retains the complete previous project. Saving
validates the project, creates a mode-0600 temporary in the destination
directory, writes, flushes, fsyncs, closes and renames over the destination.
Failures before rename retain the previous destination and remove the temporary.
The parent directory is not fsynced: survival of the rename through sudden power
loss is not guaranteed. Save replaces symlinks themselves and does not preserve
After rename, the parent directory is fsynced. A directory-sync failure reports
I/O error even though the new complete destination may already be present.
Storage and filesystem behavior still determine power-loss durability. Save replaces symlinks themselves and does not preserve
existing file permissions. POSIX I/O is private to the Linux-first persistence
module; public headers contain no POSIX types.
@ -123,7 +124,8 @@ Saved data reproduces the arrangement, graph, track mix and automation, includin
ID allocation state. Undo history is not saved. See MIXER.md for pan/interpolation.
Graph node positions, timeline zoom/scroll/selection, transport position and DSP
oscillator phase are GUI/session state and are not saved. Asset references are
external absolute paths, not copied on Save As. Metadata loading preserves
absolute in memory; saves use relative paths for media inside the project folder,
otherwise absolute paths. Media are not copied on Save As. Metadata loading preserves
missing references; audio-plan preparation reports unreadable/invalid assets
and renders them silently. Relinking replaces a selected clip's path without
changing its IDs or arrangement. See AUDIO_FILES.md for recovery and limitations.
@ -139,7 +141,7 @@ writer and unfinished-recording behavior. Metadata validation is available via
## Phase 15: version 5 plugin metadata
The writer now emits `XTC_PROJECT 5`; versions 1–4 still load. All v4 records and
Phase 15 introduced `XTC_PROJECT 5`; its reader also supported versions 1–4. All v4 records and
order remain the same. Node type 8 is a LADSPA plugin; native pointer-bearing type
7 is still unsupported in files. Immediately after each type-8 `NODE id 8
00000000` record come:
@ -163,7 +165,7 @@ stored. See PLUGIN_HOSTING.md and decision 0009 for supported processors/limits.
## Phase 16: version 6
Writer emits v6 and readers accept v1–v6. After each plugin's CONTROL records,
Phase 16 introduced v6 and its reader accepted v1–v6. After each plugin's CONTROL records,
`PLUGIN_AUTOMATION count` is followed by exactly count (<=32) records:
`PARAM tick port mode float32_hex_bits`. Mode is 0 linear or 1 step. Duplicate
port/tick pairs, nonfinite values, unknown control ports and ticks beyond the
@ -205,7 +207,7 @@ the selected document. In-memory models and the existing destination are preserv
on load failure. No automatic unknown-field stripping or media deletion occurs.
`make test-portability test test-plugins test-recovery test-recording` covers checked-in
v1–v6 fixtures (including missing v5/v6 plugins), v7 exact roundtrips, moved-folder
v1–v10 fixtures (including missing v5/v6 plugins), current exact roundtrips, moved-folder
playback from a different cwd, missing media retention, unknown records, every
truncation and deterministic byte mutations, malformed plugin metadata, interrupted
writers and write-limit failures. ASan/UBSan/static analysis additionally audit the
@ -240,7 +242,7 @@ format change (`make test test-portability test-audio-editing`).
## Phase 24: version 9 mix routing
The writer emits v9. Immediately after each TRACK's MIX record it writes
Phase 24 introduced v9. Immediately after each TRACK's MIX record it writes
`ROUTE is_bus bus_id`. `is_bus` is 0 or 1; `bus_id` is an existing bus track ID,
or decimal UINT64_MAX (18446744073709551615) for master. Buses must route to
master and cannot contain clips. Invalid destinations, flags and nested routing
@ -266,6 +268,72 @@ bindings and invalid interpolation modes reject the complete candidate. v1–v9
supply default 7/10/64 bindings on every channel. No event history/current live
controller values are serialized. Older readers reject v10.
Fixtures cover v1–v9 migration. Exact roundtrips, malformed metadata, step/linear
Fixtures cover v1–v10 loading and migration. Exact roundtrips, malformed metadata, step/linear
renders and remapped live events cover the new writer/reader. Public clients
must rebuild and initialize the appended `xtc_automation_point.step` field.
## Phase 28 compatibility and diagnostic contract
The current reader accepts versions 1–10 and the writer always emits version 10.
There is no format change in Phase 28. Saving an older document upgrades it;
retain an original backup if an older application must still open it. APP is
informational; compatibility depends on the format version, not the producer's
application version. A future incompatible schema requires a new format number.
Public C/C++ consumers must rebuild when public structs change.
| Version | Added representation | Defaults when loading earlier versions |
| --- | --- | --- |
| 1 | Graph, tracks, silent regions | Baseline |
| 2 | External WAV references | Empty asset references |
| 3 | MIDI clips and notes | Non-MIDI clips, no notes |
| 4 | Track mix and automation | Unity gain, center pan, mute/solo off, no points |
| 5 | LADSPA metadata | No plugin nodes |
| 6 | Plugin automation | Empty plugin lanes |
| 7 | Relative media references | Earlier media paths must be absolute |
| 8 | Clip gain and fades | Unity gain, zero explicit fades |
| 9 | Bus routing | Ordinary tracks routed directly to master |
| 10 | Track interpolation and controller bindings | Linear points; CC 7/10/64 on every channel |
The grammar has no optional extension namespace. Unknown records, extra fields,
node types or trailing data cannot be preserved by this model and are rejected
rather than silently removed. Supported older records migrate in a separate
candidate and pass complete model validation before adoption. Failed parsing,
reference validation or close retains the existing destination model. GUI Open
also prepares the candidate before replacing the live document; missing or
incompatible plugins retain the current document and identify the module.
Missing media references remain available for relinking and render silently.
`xtc_project_load_diagnostic` adds a copied result, parsed format version (zero
if unavailable), expected record/stage and message. Syntax locations are 1-based;
line zero denotes I/O, size, path resolution or whole-model validation. MODEL
errors do not identify the original line of a cross-reference. The original
`xtc_project_load` remains available with identical transactional semantics.
`xtc-check --validate PROJECT` checks bounded metadata without loading dependencies.
`xtc-check --inspect PROJECT` additionally lists resolved media and module paths,
plugin labels, unique IDs and descriptor indices. It checks readable regular-file
availability only. Exit status is 0 when references are available, 2 when valid
metadata has missing/unreadable references, and 1 for invalid metadata or I/O.
Neither command executes plugin code, decodes WAV data, connects JACK or verifies
hardware. A readable file can still be an invalid WAV or incompatible plugin;
actual preparation performs those checks. Paths and labels are escaped for
terminal output. There is no search-path substitution or automatic plugin relink.
Portability currently targets Linux/POSIX. The text representation is independent
of byte order and locale and requires IEEE binary32 floats. IDs used by the graph
must fit the receiving host's size_t; a 64-bit file is not guaranteed to fit a
32-bit build. Paths follow POSIX semantics; Windows drive letters and separators
are not translated. Relative media can travel with their project folder, while
absolute references and symlink targets must be restored separately. LADSPA
modules are absolute installed references: match architecture, ABI, descriptor
identity, ports and legal controls on the destination. Plugin binaries and state
are not bundled. JACK availability, sample rate, block limits and device routing
are runtime dependencies; loading metadata does not restore device connections.
`make test` includes v1–v10 migration, malformed/truncated/mutated/oversized input,
retained-state diagnostics, CLI dependency inspection, and deterministic save
faults at flush, file sync, close, rename and directory sync. Pre-rename faults
preserve the old bytes and remove the temporary. After rename, a sync failure
reports I/O while leaving a complete new document; inspect it before retrying.
Fault wrappers are linked into tests only. These tests complement killed-writer
coverage in `make test-recovery`; they do not qualify power-cut behavior.