hanez/xtc
2
1
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
xtc/docs/PROJECT_FORMAT.md

11 KiB
Raw Blame History

Project format v7 (reads v1–v6)

Phases 5–15 use a bounded text format implemented in C99 without a parser library. The extension is .xtc. File operations run exclusively on the control thread. The project owns its graph, tracks and silent/audio arrangement regions. Audio samples remain in external WAV files; MIDI notes are stored inline.

Schema

Records occur in this exact order, separated by LF, including the final record. Fields use ASCII spaces. Integers are unsigned decimal, without signs. Counts determine the number of subsequent records:

XTC_PROJECT 7
APP 0 1 0
TEMPO 120000
IDS 1 1
GRAPH 1 2 2 1
NODE 0 1 3e800000
NODE 1 4 00000000
EDGE 0 1
TRACKS 1
TRACK 0 547261636b
MIX 3f800000 00000000 0 0
CLIPS 1
CLIP 0 0 3840 2880 960 3840 526567696f6e
ASSET -
MIDI 0
NOTES 0 0
AUTOMATION 0
END
  • APP: producer major/minor/patch; informational, never executed.
  • TEMPO: constant milli-BPM (20000..300000, default 120000), fixed 4/4.
  • IDS: next track ID and next clip ID. IDs are uint64, monotonic and never reused; UINT64_MAX is reserved. Track and clip ID namespaces are separate.
  • GRAPH: output-node policy (0/1), next node ID, node count, edge count. Nodes retain stable IDs (limited to the host's size_t range). No pointers or compiled state are serialized.
  • NODE: ID, type (1 gain, 2 oscillator, 3 meter, 4 output, 5 audio source, 6 instrument, 8 LADSPA plugin), and exactly eight hexadecimal digits encoding the IEEE binary32 parameter bits, independent of locale and host byte order. Nonfinite values are rejected.
  • EDGE: source and destination node IDs. Port directions are implied by node type; endpoints, duplicates and cycles are validated by the graph API.
  • TRACK: ID and hex-encoded name bytes. In v4/v5 each is immediately followed by MIX gain_bits pan_bits mute solo: finite binary32 gain 0..4, pan -1..1 encoded as eight hex digits, mute and solo unsigned 0/1.
  • CLIP: ID, track ID, start tick, length tick, source offset tick, source length tick, hex-encoded name bytes. Trimming preserves source bounds, so a region can re-extend only within its original length. Overlaps are allowed and audio clips mix through the audio source node.
  • ASSET: required after each CLIP in v2/v3/v4/v5; "-" means no audio asset, otherwise hex-encoded absolute path bytes, <=511 bytes after decoding, no embedded NUL or ASCII control characters. Each clip directly references a WAV; no sample data, pointers or DSP state are embedded. Paths can contain spaces.
  • MIDI: required after each ASSET in v3/v4/v5; 0 for silent/audio regions, 1 for MIDI clips. MIDI clips must have no asset.
  • NOTES: next stable note ID and global note count, followed by that many NOTE id clip_id start_tick length_tick pitch velocity channel records. Notes reference MIDI clips; start/length are source-relative, positive length within source bounds. Pitch is 0..127, velocity 1..127, channel 0..15. Note IDs are unique, monotonic uint64 IDs with UINT64_MAX reserved.
  • AUTOMATION: global point count, after the last NOTE and before END in v4/v5. Follow with POINT track_id parameter tick value_bits for each point. Parameter 0 is gain (0..4), 1 pan (-1..1); values use finite binary32 hex. Track must exist; tick <= MAX_TICK; duplicate track/parameter/tick is invalid. Record order is preserved; rendering sorts a separate prepared copy.
  • Names decode to nonempty valid UTF-8 of at most 63 bytes; ASCII control characters and embedded NULs are forbidden. Hex avoids quoting ambiguity.

Ticks use 960 per quarter note, 3840 per bar. All region endpoints are bounded by 38400000 ticks (10000 bars). Limits: 64 tracks, 1024 clips, 64 graph nodes, 4096 edges, 4096 notes, 1024 automation points, 4 MiB input, 1535 bytes per record, 12 tokens per record. The reader rejects invalid references, duplicate IDs, inconsistent next IDs, ranges, truncated records, raw control/non-ASCII bytes, and trailing data.

Unsupported format versions return UNSUPPORTED. Other malformed data returns FORMAT. The reader explicitly accepts v1 records without ASSET lines, with node types 1..4, migrating their clips to empty asset paths. Version 2 accepts node types 1..5 and ASSET records. Both migrate to MIDI=0 and no notes. Version 3 accepts instrument nodes and MIDI/NOTES records. Versions 1–3 migrate all tracks to unity gain, centered pan, mute/solo off and no automation. Saving always writes v7. Unknown versions are rejected without replacing the destination; older applications cannot read v7. Migration and maximum-path roundtrips are tested.

Ownership and failures

Load builds and validates a separate candidate, closes the input, then replaces the destination. Any failure retains the complete previous project. Saving validates the project, creates a mode-0600 temporary in the destination directory, writes, flushes, fsyncs, closes and renames over the destination. Failures before rename retain the previous destination and remove the temporary. The parent directory is not fsynced: survival of the rename through sudden power loss is not guaranteed. Save replaces symlinks themselves and does not preserve existing file permissions. POSIX I/O is private to the Linux-first persistence module; public headers contain no POSIX types.

Saved data reproduces the arrangement, graph, track mix and automation, including ID allocation state. Undo history is not saved. See MIXER.md for pan/interpolation. Graph node positions, timeline zoom/scroll/selection, transport position and DSP oscillator phase are GUI/session state and are not saved. Asset references are external absolute paths, not copied on Save As. Metadata loading preserves missing references; audio-plan preparation reports unreadable/invalid assets and renders them silently. Relinking replaces a selected clip's path without changing its IDs or arrangement. See AUDIO_FILES.md for recovery and limitations.

Phase 9 extension and recovery policy

Phase 9 kept format v4. Native processor descriptors are process-local and have no serialized representation; saving a native graph returns UNSUPPORTED and retains the existing destination. Unknown file versions/types remain rejected. See NATIVE_API.md for the extension boundary and RECOVERY.md for write-failure, killed writer and unfinished-recording behavior. Metadata validation is available via xtc-check --validate PROJECT without a GUI or server; it does not load assets.

Phase 15: version 5 plugin metadata

The writer now emits XTC_PROJECT 5; versions 1–4 still load. All v4 records and order remain the same. Node type 8 is a LADSPA plugin; native pointer-bearing type 7 is still unsupported in files. Immediately after each type-8 NODE id 8 00000000 record come:

PLUGIN unique_id descriptor_index mono_inputs control_count hex_label
MODULE hex_absolute_module_path
CONTROL port_index float32_hex_bits

There are exactly control_count CONTROL records. Label length is 1–63 bytes, module path 1–511 bytes (absolute), both hex-encoded with no embedded NUL. Index is <1024, mono_inputs 0/1, controls <=16, control port indices <64 strictly increasing, and values finite. Unique ID fits uint32. The one output is implicit. Structural validation never opens libraries. On graph compilation the host checks identity, layout, input control port IDs and rate-adjusted legal ranges against the module. Unknown nodes/versions fail without mutating the current project. Missing modules remain preserved by metadata load/save; GUI installation rejects them with a repair message. No transient DSP state, pointer or plugin binary is stored. See PLUGIN_HOSTING.md and decision 0009 for supported processors/limits.

Phase 16: version 6

Writer emits v6 and readers accept v1–v6. After each plugin's CONTROL records, PLUGIN_AUTOMATION count is followed by exactly count (<=32) records: PARAM tick port mode float32_hex_bits. Mode is 0 linear or 1 step. Duplicate port/tick pairs, nonfinite values, unknown control ports and ticks beyond the project limit are rejected. Loaded plugin tempo comes from TEMPO. Range and discrete-mode compatibility are checked on control-thread compilation. Versions 1–5 migrate with empty plugin lanes. See CONTROL.md for timing/ordering.

Phase 18: version 7 and project folders

Current writer emits v7; readers migrate v1–v6. Record ordering and field bounds are otherwise unchanged from v6. v7 ASSET may contain a relative path, interpreted against the actual containing directory of the named project file (resolved on the control thread), never the launch working directory. Opening a symlink to a project uses the resolved target project directory; Save replaces the named entry as before. After loading, all core asset references are absolute, including missing media. Media inside the save folder are written relative; references outside remain absolute. Save As resolves against its new folder and preserves media identity rather than rebasing it by accident. Copy/move a project folder together with its referenced media to retain portability. Saving does not copy assets: import/record into that folder first. Plugin binaries remain trusted absolute module references, not bundled resources.

Empty assets still use -. Paths exclude embedded NUL/control bytes and must fit 511 bytes after resolution; overly long resolved references fail transactionally. Relative references can name outside the folder using ordinary POSIX semantics; this is not a sandbox. Symlinks follow filesystem behavior, so portability requires moving their targets too. Old formats continue to require absolute media paths. No unknown version/node/record is silently discarded; unsupported versions/types or malformed/trailing data reject the candidate and retain the current project. Parsing never opens media/plugins or constructs an audio engine.

Save validates the complete model, writes/fsyncs a same-folder temporary, closes it, atomically renames it, then fsyncs the containing directory. Failures before rename retain the old destination. A directory sync failure after rename reports I/O error but the new complete file may already be present: inspect/reopen it; there is no rollback across that durability boundary. Filesystem/hardware power loss guarantees still depend on the underlying storage. Recording destinations have their separate no-clobber policy; ordinary project Save intentionally replaces the selected document. In-memory models and the existing destination are preserved on load failure. No automatic unknown-field stripping or media deletion occurs.

make test-portability test test-plugins test-recovery test-recording covers checked-in v1–v6 fixtures (including missing v5/v6 plugins), v7 exact roundtrips, moved-folder playback from a different cwd, missing media retention, unknown records, every truncation and deterministic byte mutations, malformed plugin metadata, interrupted writers and write-limit failures. ASan/UBSan/static analysis additionally audit the bounded parser. This is regression/fault-injection coverage, not exhaustive fuzzing or a claim of power-cut qualification.