Phase 32: add MIDI track CLAP, VST2 and VST3 instruments

This commit is contained in:
Johannes Findeisen 2026-10-27 17:51:48 +01:00
commit fe484537f9
41 changed files with 1138 additions and 65 deletions

View file

@ -22,7 +22,7 @@
plus baseline and at most one reserved metadata snapshot; GUI edits reserve
history before mutation/publication. Media is
never removed by undo/delete. Broad screen-reader/accessibility testing is pending.
- Format v11 reads v1–v10 and uses relative local media references. External absolute
- Format v12 reads v1–v11 and uses relative local media references. External absolute
media/module paths need manual repair after relocation. Saving does not bundle
media; missing media stays referenced, missing plugins reject GUI installation.
Unknown data is rejected rather than stripped. Post-rename directory-sync errors
@ -81,3 +81,34 @@ workers, sidechains or true stereo/multichannel buses are implemented. Plugin
bundles must be opened at their actual Linux shared-library binary. Only original
ABI fixtures are qualified; arbitrary third-party modern plugins remain untested.
See PLUGIN_HOSTING.md and decision 0010 for lifecycle and licensing boundaries.
## Phase 32 instrument support
MIDI tracks can bind CLAP, VST2 and VST3 instruments. In the Track menu, select
**Load instrument plugin** for the selected sequencer track; **Use built-in
instrument** removes the binding. Assignment also selects the track for live
MIDI. **Live MIDI to selected instrument** changes that target; **Live MIDI to
built-in sine** restores the legacy live source. MIDI clip notes always go to
their own track's instrument. Independent instances feed track gain/pan,
mute/solo, bus routing and automation. Bindings/live target save in v12.
Supply the actual trusted Linux binary (including the inner .so of a VST3
bundle). Supported instruments have zero audio inputs, one mono output,
float32 processing, zero latency and one note/event input. Default configuration
only: no custom editor, opaque state/preset restoration, modern parameter
editing/automation, stereo/multichannel instruments or MIDI output. No arbitrary
third-party instrument is qualified by the original ABI fixture tests.
CLAP accepts MIDI or native notes; VST2 uses MIDI events; VST3 uses IEventList
note events. Live CC reaches CLAP MIDI/VST2; native CLAP/VST3 have no controller
mapping (including sustain). Live MIDI goes to exactly one selected target.
Seeks/loops retrigger notes crossing the destination with fresh envelopes;
plugin oscillator/opaque DSP state cannot be reconstructed. Panic delivers
all-off (VST3 held-key note-offs); plugin release tails may remain. Processing
uses preallocated 256-frame chunks and 256 incoming events per instance/chunk;
overflow or processing failure silences the block and schedules reset events.
The existing 32 simultaneous scheduled-note and 4096 stored-note bounds remain.
Do not unload/replace modules during playback; candidate plans are prepared off
the callback and old instances are reclaimed on the control thread. Missing
references remain inspectable without loading; failed GUI preparation retains
the current project. Save a backup for applications limited to v11 or earlier.

View file

@ -136,3 +136,34 @@ Phase 26: CC7/10/64 are defaults, configurable per channel through Audio → MID
controller mapping. Disabled/unmapped CCs have no effect; CC120/123 stay panic.
Decode accepts valid CC0..127 messages within the existing event budget. See
CONTROL.md for the exact bindings, ramp, sustain and automation precedence rules.
## Phase 32 instrument support
MIDI tracks can bind CLAP, VST2 and VST3 instruments. In the Track menu, select
**Load instrument plugin** for the selected sequencer track; **Use built-in
instrument** removes the binding. Assignment also selects the track for live
MIDI. **Live MIDI to selected instrument** changes that target; **Live MIDI to
built-in sine** restores the legacy live source. MIDI clip notes always go to
their own track's instrument. Independent instances feed track gain/pan,
mute/solo, bus routing and automation. Bindings/live target save in v12.
Supply the actual trusted Linux binary (including the inner .so of a VST3
bundle). Supported instruments have zero audio inputs, one mono output,
float32 processing, zero latency and one note/event input. Default configuration
only: no custom editor, opaque state/preset restoration, modern parameter
editing/automation, stereo/multichannel instruments or MIDI output. No arbitrary
third-party instrument is qualified by the original ABI fixture tests.
CLAP accepts MIDI or native notes; VST2 uses MIDI events; VST3 uses IEventList
note events. Live CC reaches CLAP MIDI/VST2; native CLAP/VST3 have no controller
mapping (including sustain). Live MIDI goes to exactly one selected target.
Seeks/loops retrigger notes crossing the destination with fresh envelopes;
plugin oscillator/opaque DSP state cannot be reconstructed. Panic delivers
all-off (VST3 held-key note-offs); plugin release tails may remain. Processing
uses preallocated 256-frame chunks and 256 incoming events per instance/chunk;
overflow or processing failure silences the block and schedules reset events.
The existing 32 simultaneous scheduled-note and 4096 stored-note bounds remain.
Do not unload/replace modules during playback; candidate plans are prepared off
the callback and old instances are reclaimed on the control thread. Missing
references remain inspectable without loading; failed GUI preparation retains
the current project. Save a backup for applications limited to v11 or earlier.

View file

@ -19,7 +19,7 @@ plugin recovery policy, preparation failure/partial cleanup tests, bounded event
and parameter publication, latency/tail handling, callback-safe unload after
retirement, a GUI/main-thread boundary and an explicit crash-containment decision.
Native ABI tests cover some ownership mechanics but do not prove third-party
code safe. Event graph ports and custom-instrument GUI selection remain gaps.
code safe. Event graph ports remain a gap; Phase 32 adds track instrument selection.
Choose one format with concrete user needs after those contracts are specified;
do not introduce a general multi-format framework now. See decision 0008.
@ -113,8 +113,7 @@ Physical listening and plugin CPU/endurance qualification remain pending.
Discovery remains explicit-file only, bounded to 1024 descriptor indices; no
recursive scan, plugin database or new dependency is introduced. Unsupported
layouts are skipped with counts; source/effect roles describe audio ports, not
MIDI instrument capabilities. Built-in MIDI synthesis remains the only GUI MIDI
instrument. LADSPA sources can be identified, edited, replaced and removed in
MIDI instrument capabilities. Phase 32 adds modern track instruments (see below). LADSPA sources can be identified, edited, replaced and removed in
the graph but cannot receive MIDI events.
**Edit → Insert LADSPA on selected wire** replaces one existing connection with
@ -224,7 +223,7 @@ prefers CLAP, then VST2, then VST3, otherwise LADSPA; the C API also provides
The modern subset is **one mono audio input and one mono output, float32, default
configuration, zero reported latency**. Stereo graph playback uses independent
instances. True stereo/multichannel/sidechain buses, instruments, MIDI/events,
instances. For the Phase 31 effect subset, true stereo/multichannel/sidechain buses, instruments, MIDI/events,
custom editors, parameter changes/automation, programs/presets, opaque state,
transport/time information, workers and latency compensation are not implemented
for these three formats. Modern metadata with controls/automation is rejected,
@ -257,3 +256,34 @@ CLAP start/process failures, graph replacement rollback, 50 plan retirements per
format, missing modules, v11 save/reopen and GUI add/browser/undo/reopen. Host
allocation/loading/unloading guards remain active while rendering. This evidence
qualifies the subset above, not arbitrary external plugins or physical audio.
## Phase 32 instrument support
MIDI tracks can bind CLAP, VST2 and VST3 instruments. In the Track menu, select
**Load instrument plugin** for the selected sequencer track; **Use built-in
instrument** removes the binding. Assignment also selects the track for live
MIDI. **Live MIDI to selected instrument** changes that target; **Live MIDI to
built-in sine** restores the legacy live source. MIDI clip notes always go to
their own track's instrument. Independent instances feed track gain/pan,
mute/solo, bus routing and automation. Bindings/live target save in v12.
Supply the actual trusted Linux binary (including the inner .so of a VST3
bundle). Supported instruments have zero audio inputs, one mono output,
float32 processing, zero latency and one note/event input. Default configuration
only: no custom editor, opaque state/preset restoration, modern parameter
editing/automation, stereo/multichannel instruments or MIDI output. No arbitrary
third-party instrument is qualified by the original ABI fixture tests.
CLAP accepts MIDI or native notes; VST2 uses MIDI events; VST3 uses IEventList
note events. Live CC reaches CLAP MIDI/VST2; native CLAP/VST3 have no controller
mapping (including sustain). Live MIDI goes to exactly one selected target.
Seeks/loops retrigger notes crossing the destination with fresh envelopes;
plugin oscillator/opaque DSP state cannot be reconstructed. Panic delivers
all-off (VST3 held-key note-offs); plugin release tails may remain. Processing
uses preallocated 256-frame chunks and 256 incoming events per instance/chunk;
overflow or processing failure silences the block and schedules reset events.
The existing 32 simultaneous scheduled-note and 4096 stored-note bounds remain.
Do not unload/replace modules during playback; candidate plans are prepared off
the callback and old instances are reclaimed on the control thread. Missing
references remain inspectable without loading; failed GUI preparation retains
the current project. Save a backup for applications limited to v11 or earlier.

View file

@ -1,4 +1,4 @@
# Project format v11 (reads v1–v10)
# Project format v12 (reads v1–v11)
Phases 5–31 use a bounded text format implemented in C99 without a parser library.
The extension is `.xtc`. File operations run exclusively on the control thread.
@ -104,8 +104,8 @@ node types 1..4, migrating their clips to empty asset paths. Version 2 accepts
node types 1..5 and ASSET records. Both migrate to MIDI=0 and no notes.
Version 3 accepts instrument nodes and MIDI/NOTES records. Versions 1–3 migrate
all tracks to unity gain, centered pan, mute/solo off and no automation.
Saving always writes v11. Unknown versions are rejected without replacing the destination; older
applications without v11 support cannot read current saves. Migration and maximum-path roundtrips are tested.
Saving always writes v12. Unknown versions are rejected without replacing the destination; older
applications without v12 support cannot read current saves. Migration and maximum-path roundtrips are tested.
## Ownership and failures
@ -274,7 +274,7 @@ must rebuild and initialize the appended `xtc_automation_point.step` field.
## Phase 28 compatibility and diagnostic contract
The current reader accepts versions 1–11 and the writer always emits version 11.
The current reader accepts versions 1–12 and the writer always emits version 12.
Phase 28 made no format change; Phase 31 adds plugin format identity in v11. Saving an older document upgrades it;
retain an original backup if an older application must still open it. APP is
informational; compatibility depends on the format version, not the producer's
@ -367,3 +367,25 @@ publishing. Missing module metadata survives C load/save; GUI preparation retain
the existing document on failure. No automatic path search or plugin substitution
occurs. Current and historical migration fixtures, malformed fields, identity
mismatch, missing-module and exact-render roundtrips cover this extension.
## Phase 32: version 12 track instruments
After CONTROL_MAPS/MAP records and before END:
```
TRACK_INSTRUMENTS count
INSTRUMENT track_id format descriptor_index unique_id hex_label
MODULE hex_absolute_binary_path
IDENTITY hex_stable_id_or_dash
LIVE_INSTRUMENT track_id_or_18446744073709551615
END
```
Each binding comprises INSTRUMENT/MODULE/IDENTITY. Formats 1/2/3 mean
CLAP/VST2/VST3. Bindings require distinct existing non-bus tracks and compatible
instrument metadata (zero audio inputs). The live target must have a binding;
UINT64_MAX selects built-in sine. Empty bindings are valid. Older documents
migrate to no bindings and built-in live sine. Parsing/inspection never executes
modules; preparing playback verifies binary identity and configuration. Deleting
tracks removes bindings; reordering preserves stable track identities. Undo
snapshots include bindings. Public clients must rebuild for the new spec field.

View file

@ -1,6 +1,6 @@
# Persistence and failure recovery
The project format is v11. Readers migrate v1–v10, validate bounded input
The project format is v12. Readers migrate v1–v11, validate bounded input
and atomically install a candidate only after complete success. Existing tests
cover exact migration/roundtrip, truncations, malformed references, counts,
IDs, nonfinite values and failed GUI/backend installation. Native processors
@ -87,7 +87,7 @@ rename and directory-sync errors. Pre-rename failures retain the previous bytes
and clean temporary files. Post-rename sync errors leave a complete new file and
return failure; there is no rollback after replacement. Load diagnostics identify
the expected record and syntax line, or line zero for whole-model/I/O stages.
Unknown versions report the supported range (1–11); unknown records/fields are
Unknown versions report the supported range (1–12); unknown records/fields are
rejected without partially changing the current project.
Modern CLAP/VST2/VST3 module identity is preserved in v11 metadata even when the
@ -96,3 +96,34 @@ CLAP restart and VST2 I/O-change requests fail/silence subsequent blocks; stop
and rebuild/reload rather than attempting callback-time reconfiguration. Modern
instances restart at default configuration; no edited opaque state is exposed
or saved by the initial adapters. Trusted plugin crashes cannot be contained.
## Phase 32 instrument support
MIDI tracks can bind CLAP, VST2 and VST3 instruments. In the Track menu, select
**Load instrument plugin** for the selected sequencer track; **Use built-in
instrument** removes the binding. Assignment also selects the track for live
MIDI. **Live MIDI to selected instrument** changes that target; **Live MIDI to
built-in sine** restores the legacy live source. MIDI clip notes always go to
their own track's instrument. Independent instances feed track gain/pan,
mute/solo, bus routing and automation. Bindings/live target save in v12.
Supply the actual trusted Linux binary (including the inner .so of a VST3
bundle). Supported instruments have zero audio inputs, one mono output,
float32 processing, zero latency and one note/event input. Default configuration
only: no custom editor, opaque state/preset restoration, modern parameter
editing/automation, stereo/multichannel instruments or MIDI output. No arbitrary
third-party instrument is qualified by the original ABI fixture tests.
CLAP accepts MIDI or native notes; VST2 uses MIDI events; VST3 uses IEventList
note events. Live CC reaches CLAP MIDI/VST2; native CLAP/VST3 have no controller
mapping (including sustain). Live MIDI goes to exactly one selected target.
Seeks/loops retrigger notes crossing the destination with fresh envelopes;
plugin oscillator/opaque DSP state cannot be reconstructed. Panic delivers
all-off (VST3 held-key note-offs); plugin release tails may remain. Processing
uses preallocated 256-frame chunks and 256 incoming events per instance/chunk;
overflow or processing failure silences the block and schedules reset events.
The existing 32 simultaneous scheduled-note and 4096 stored-note bounds remain.
Do not unload/replace modules during playback; candidate plans are prepared off
the callback and old instances are reclaimed on the control thread. Missing
references remain inspectable without loading; failed GUI preparation retains
the current project. Save a backup for applications limited to v11 or earlier.

View file

@ -15,7 +15,7 @@ undo/redo and a generic LADSPA editor. JACK is the only live audio/MIDI backend;
core validation and offline rendering work without it. Native/LADSPA processors
are trusted in-process code. No proprietary energyXT resources are included.
Project format v11 reads versions 1–11 transactionally. Syntax diagnostics identify
Project format v12 reads versions 1–12 transactionally. Syntax diagnostics identify
records and lines. `xtc-check --inspect` lists dependency paths without executing
plugins. Phase 29 adds timing/CPU/RSS reports, repeatable stress runs and virtual
endurance. Phase 30 adds configurable install directories, shared install/remove
@ -24,7 +24,7 @@ inventory, installed MIDI examples and source archive checksum sidecars.
## Upgrading and removal
Back up documents and external media before upgrading. Opening an older project
does not rewrite it; saving upgrades it to v11, which older readers may reject.
does not rewrite it; saving upgrades it to v12, which older readers may reject.
Use Save As for a separate migrated copy. Moving a project folder preserves media
stored inside it via relative references. Absolute media and plugin module paths
must be restored separately. There is no bundled plugin/media migration or saved
@ -57,3 +57,7 @@ GUI discovery/insertion/replacement and v11 format identity. Pinned MIT CLAP and
BSD-3-Clause VST3 C headers ship with notices; VST2 uses original compatibility
declarations. Modern instruments, parameters/state/editors and multichannel
layouts remain unsupported. No third-party modern plugin has been qualified.
Phase 32 adds CLAP/VST2/VST3 mono instruments bound to MIDI tracks, sample-offset
notes, a saved live target, track mixing, GUI assignment/undo and v12 persistence.
See PLUGIN_HOSTING.md for compatibility and controller/state limits.

View file

@ -901,6 +901,36 @@ These require separate future work; they are not claimed as implemented by loadi
---
## Phase 32 — MIDI Track Plugin Instruments
### Goal
Play CLAP, VST2 and VST3 synthesizer/instrument plugins from MIDI clips and live MIDI.
### Tasks and acceptance criteria
- [x] Discover and initialize compatible zero-input, mono float32, zero-latency instruments.
- [x] Bind independent instances to tracks; preserve built-in sine for unassigned tracks.
- [x] Deliver sample-offset notes and bounded event queues without callback allocations.
- [x] Apply track/bus mixing and provide an explicit saved live-MIDI target.
- [x] Handle discontinuity note chase, panic, teardown and failed preparation.
- [x] Save bindings in v12; migrate v1–v11 and retain missing module references.
- [x] Add GUI load/clear/live-target commands with undo and candidate preparation.
- [x] Test original fixtures for all three formats and document the supported subset.
### Implementation status
Implemented with track-owned prepared instances and 256-frame processing chunks.
Only original ABI fixtures are qualified; arbitrary third-party instruments and
physical MIDI/audio hardware remain unverified. Mono/default configuration only;
no custom editors, opaque state or modern parameter automation. Native-note CLAP
and VST3 carry notes only; CLAP MIDI and VST2 also receive live controllers.
Seeking retriggers held scheduled notes with fresh envelopes; plugin release tails
can remain after note-off/panic. See PLUGIN_HOSTING.md and decision 0011.
Validation: normal/minimal core and GUI builds, core/instrument/GUI fixtures,
JACK/workflow, reliability, ASan/UBSan core and GUI, static analysis, staged
installation, reproducible source releases and relocatable binary packages pass.
---
## Recommended Execution Order
Reorder the phases when the audit reveals a blocking dependency. Do not mark a phase complete merely because code exists: its acceptance criteria must be demonstrated by builds, tests, or documented manual verification.

View file

@ -166,5 +166,36 @@ CLAP/VST2/VST3 effect subset. Select the actual module binary (`.clap` or `.so`,
including a VST3 bundle's inner binary). Modern effects require one mono input
and output, float32 and zero latency; they use default configuration. Instruments,
modern parameter editing/automation, opaque state and custom editors are not
implemented. Projects now save v11 and read v1–v11. See PLUGIN_HOSTING.md for the
implemented. Projects now save v12 and read v1–v12. See PLUGIN_HOSTING.md for the
full contract and test/qualification limits.
## Phase 32 instrument support
MIDI tracks can bind CLAP, VST2 and VST3 instruments. In the Track menu, select
**Load instrument plugin** for the selected sequencer track; **Use built-in
instrument** removes the binding. Assignment also selects the track for live
MIDI. **Live MIDI to selected instrument** changes that target; **Live MIDI to
built-in sine** restores the legacy live source. MIDI clip notes always go to
their own track's instrument. Independent instances feed track gain/pan,
mute/solo, bus routing and automation. Bindings/live target save in v12.
Supply the actual trusted Linux binary (including the inner .so of a VST3
bundle). Supported instruments have zero audio inputs, one mono output,
float32 processing, zero latency and one note/event input. Default configuration
only: no custom editor, opaque state/preset restoration, modern parameter
editing/automation, stereo/multichannel instruments or MIDI output. No arbitrary
third-party instrument is qualified by the original ABI fixture tests.
CLAP accepts MIDI or native notes; VST2 uses MIDI events; VST3 uses IEventList
note events. Live CC reaches CLAP MIDI/VST2; native CLAP/VST3 have no controller
mapping (including sustain). Live MIDI goes to exactly one selected target.
Seeks/loops retrigger notes crossing the destination with fresh envelopes;
plugin oscillator/opaque DSP state cannot be reconstructed. Panic delivers
all-off (VST3 held-key note-offs); plugin release tails may remain. Processing
uses preallocated 256-frame chunks and 256 incoming events per instance/chunk;
overflow or processing failure silences the block and schedules reset events.
The existing 32 simultaneous scheduled-note and 4096 stored-note bounds remain.
Do not unload/replace modules during playback; candidate plans are prepared off
the callback and old instances are reclaimed on the control thread. Missing
references remain inspectable without loading; failed GUI preparation retains
the current project. Save a backup for applications limited to v11 or earlier.

View file

@ -0,0 +1,27 @@
# 0011 — Prepared MIDI track instruments
Accepted for Phase 32. Extend the modern C99 adapters with zero-audio-input,
mono-output instruments and one event input. Keep graph effects separate from
track bindings. Each assigned track owns one prepared instance; the existing
instrument source sums their mixed output and the sine output of unassigned
tracks. A stable saved live target prevents accidental broadcast/double playback.
Bindings are copied metadata in the project/history model, serialized in v12.
Loading metadata never executes modules. GUI edits prepare a candidate before
publication, preserving the previous project on missing/invalid binaries. Existing
quiescent plan retirement owns destruction; no module operation belongs in audio.
Preallocate 256-frame audio scratch and 256 incoming events per chunk. Stable
insertion orders reset, scheduled off/on and live events without callback sort or
allocation. VST3 translates into a bounded 2304-event IEventList (2048 possible
held keys plus 256 incoming events); panic expands held keys to note-offs. CLAP
prefers MIDI where supported, otherwise native note events; VST2 uses MIDI ABI
messages. Native CLAP and VST3 currently omit controller mapping. Existing
32 simultaneous scheduled notes and 4096 stored notes remain validation bounds.
Discontinuities send all-off and chase spanning notes with fresh note-ons.
Panic suppresses already-started scheduled notes. Neither operation promises
opaque DSP reconstruction or removal of release tails. Overflow/process errors
silence the block and arm reset for the following render. Hosts cannot guarantee
that third-party processing is realtime safe. Qualification covers original ABI
fixtures; arbitrary third-party plugins, physical audio and MIDI remain unverified.