xtc/docs/MIDI.md

# MIDI Clips and Instrument

Drawing persisted notes and live JACK MIDI input are supported. Record still
captures audio to WAV: MIDI capture and Standard MIDI File import are deferred.

---

## Workflow

Add/select a track in Sequencer, choose Add MIDI clip, then Piano roll (Ctrl+4).
Creation adds an instrument-to-output route if no instrument exists. Existing
instruments retain their routes; use Modular Graph to connect them as needed.
Scheduled MIDI clips now apply their track mix/automation before summing to
paired stereo instrument buffers; live input stays centered and unassigned. No
hardware ports are connected automatically. Connect a MIDI source to the
backend's `midi_in` JACK port, and route audio output explicitly to hear it
while Play is running.

The piano roll shows all 128 pitches and edits clip-source time. Click blank
space to draw a one-beat note (velocity 100, filtered channel or channel 0);
drag its body to move it, drag the right edge to resize, or right-click/Delete
to remove it. Edits snap to the configured grid (default 1/16). Escape cancels a
drag. Trimmed-out source notes remain editable and are preserved.
Arrangement/note edits stop playback and are blocked during audio recording.
Save/Open preserves MIDI flags, notes and stable IDs.

---

## Timing and Ownership

The C model owns up to 4096 notes per project, with source-relative ticks at 960
ticks/quarter and constant tempo. Control-thread preparation intersects notes
with clip trims and converts boundaries to integer sample positions. Scheduled
rendering uses absolute note age: block partitions do not change samples, and
seeking into a note chases its phase. Clip ends cut notes off. The instrument
source has its own buffer, separate from external audio input and WAV timeline
playback, and routes through the ordinary graph.

Prepared ranges are immutable; only their audio owner changes voice/panic state.
Preparation allocates; rendering uses bounded preallocated storage and performs
no allocation, freeing, I/O, locks, logging or GUI calls. Project preparation
builds audio and MIDI candidates before committing the stopped backend state.

JACK events have sample offsets within the current block. Supported messages are
complete note on/off (including velocity-zero note on) and CC120/123 on channels
0..15, plus CC7 volume, CC10 pan and CC64 sustain (see CONTROL.md). Equal-offset
events retain input order. Bend, program changes, running status and SysEx are
unsupported. At most 256 events are accepted per block. Malformed/overflowed
input discards that block's events and triggers panic, with a MIDI-drop status
counter.

---

## Recovery and Limits

Panic/F9 clears live voices and suppresses scheduled notes already started;
future timeline notes can still start. CC120/123 does this for its channel.
Stop/start and seek clear live voices and reset suppression, allowing timeline
notes to be chased again. MIDI port disconnection, xruns and callback processing
failures request panic. Control requests use lock-free generation counters;
voice state changes on the audio thread, or while stopped.

The sine instrument scales amplitude by 0.1 times velocity/127. It supports 32
simultaneous scheduled notes and a separate 32 live voices; live overflow steals
the oldest voice, and repeated channel/pitch retriggers it. Preparation rejects
more than 32 overlapping scheduled notes with CAPACITY. Such metadata can be
saved by the C model, but GUI Open/Play rejects it until overlap is reduced.
Immediate note-off/stealing can click; there is no envelope or limiter, and
summed voices can exceed unity. Above-Nyquist pitches are silent. There is no
per-track/channel instrument routing, MIDI recording, SMF import or piano-roll
zoom yet. Note edits participate in project snapshot undo/redo.

---

## Verification

`make test` checks exact onset/off samples, block partition independence, trim,
seek, panic, channel handling, voice/event limits, persistence migration and
malformed files, plus callback fault recovery. `make test-gui` draws, moves and
resizes notes with FLTK events, then saves/reopens and renders them.
`make test-jack-midi` uses only software MIDI/audio ports to verify live on/off,
panic, disconnection and saved/reopened scheduled playback. JACK port discovery
may lag activation under PipeWire; the test waits before connecting. Physical
MIDI devices, bridge/device loss behind an unchanged port, latency and
long-running performance remain unverified. Panic/Stop provides manual recovery.

Track mixing, automation and project undo/redo are available; see MIXER.md.

---

## Composition Commands

The piano-roll **Notes** menu exposes these commands. Shortcuts apply while the
piano roll has focus; use the main window Edit menu for undo/redo.

| Command | Shortcut | Behavior |
| --- | --- | --- |
| Grid | G | 1, 120, 160, 240, 320, 480 or 960 ticks |
| Filter | F | Channel -1 (all) or 0..15, inclusive pitch range 0..127 |
| Velocity and channel | V | Set either property; -1 preserves it |
| Quantize | Q | Snap starts to nearest configured grid; ties go later |
| Transpose | T / Up / Down | Semitone count / one semitone up / down |
| Repeat | Ctrl+D | Additional copies at a positive source-tick spacing |
| Select filtered | Ctrl+A | Select every note matching the current clip filter |
| Delete | Delete | Remove the single selected note or all filtered notes |

160 ticks is a sixteenth-note triplet; 320 is an eighth-note triplet. The grid
governs drawing, drag movement and minimum resize duration. At 1-tick snap the
visual grid remains 120 ticks to stay legible at the fixed zoom. The header
shows current grid, channel/pitch filter and selection scope. Velocity also
affects unselected note brightness. Preferences are session-only.

Click selects one visible note; Ctrl+A selects the whole filtered set. A click
or filter change clears that set selection. Hidden notes cannot be mouse-edited.
A channel filter also sets the channel for newly drawn notes. No lasso or
arbitrary disjoint multi-selection is provided. Moving/resizing uses the mouse;
V edits velocity/channel without changing timing, pitch or IDs. Quantization
changes starts only, preserving durations and other properties. Repetition keeps
originals and gives every copy a fresh stable ID; spacing need not equal note
length, so both overlapping patterns and rests are possible. The existing clip
source must have room for every result; extend it in Sequencer first if needed.
Source notes outside the visible arrangement trim remain editable.

All batch edits validate before mutation. Invalid pitch/range, copy counts,
capacity overflow or a note crossing the source boundary rejects the entire
operation; notes are never silently clamped, merged or shortened. Unmatched
notes, notes in other clips and unrelated properties are preserved. Each
successful command records one undo step and stops playback before model edits;
Play prepares the new engine plan. Undo/redo reinstalls the corresponding plan.
These edits use the existing persisted note fields; format remains v7.

Scheduled notes occupy [start,end), with ends counted before starts at an equal
sample for the 32-voice limit. Overlapping same-channel/pitch scheduled notes
are independent sine voices; adjacent repetitions restart phase. Live repeated
notes instead retrigger the matching channel/pitch voice, and a note-off
releases that voice (or defers release under sustain). Equal-sample live
on/off/control messages execute in caller/JACK order, with no implicit priority
reordering. The renderer rejects malformed/unsorted/out-of-range events before
touching output or runtime; raw velocity-zero note-ons decode to note-off.

`make test-composition` covers filters, property preservation, triplet rounding,
repeats/fresh IDs, rejected batch rollback, capacity, undo snapshots,
equal-offset live order, adjacent/overlapping notes, exact block partitions and
seek chase. `make test-gui` covers mouse edits and the new commands with
undo/redo and engine reinstallation. Listening and physical MIDI-device
validation remain pending.

CC7/10/64 are defaults, configurable per channel through Audio → MIDI controller
mapping. Disabled/unmapped CCs have no effect; CC120/123 stay panic. Decode
accepts valid CC0..127 messages within the existing event budget. See CONTROL.md
for the exact bindings, ramp, sustain and automation precedence rules.

---

## Instrument Support

MIDI tracks can bind CLAP, VST2 and VST3 instruments. In the Track menu, select
**Load instrument plugin** for the selected sequencer track; **Use built-in
instrument** removes the binding. Assignment also selects the track for live
MIDI. **Live MIDI to selected instrument** changes that target; **Live MIDI to
built-in sine** restores the legacy live source. MIDI clip notes always go to
their own track's instrument. Independent instances feed track gain/pan,
mute/solo, bus routing and automation. Bindings/live target save in v12.

Supply the actual trusted Linux binary (including the inner .so of a VST3
bundle). Supported instruments have zero audio inputs, one mono output, float32
processing, zero latency and one note/event input. Default configuration only:
no custom editor, opaque state/preset restoration, modern parameter
editing/automation, stereo/multichannel instruments or MIDI output. No arbitrary
third-party instrument is qualified by the original ABI fixture tests.

CLAP accepts MIDI or native notes; VST2 uses MIDI events; VST3 uses IEventList
note events. Live CC reaches CLAP MIDI/VST2; native CLAP/VST3 have no controller
mapping (including sustain). Live MIDI goes to exactly one selected target.
Seeks/loops retrigger notes crossing the destination with fresh envelopes;
plugin oscillator/opaque DSP state cannot be reconstructed. Panic delivers
all-off (VST3 held-key note-offs); plugin release tails may remain. Processing
uses preallocated 256-frame chunks and 256 incoming events per instance/chunk;
overflow or processing failure silences the block and schedules reset events.
The existing 32 simultaneous scheduled-note and 4096 stored-note bounds remain.
Do not unload/replace modules during playback; candidate plans are prepared off
the callback and old instances are reclaimed on the control thread. Missing
references remain inspectable without loading; failed GUI preparation retains
the current project. Save a backup for applications limited to v11 or earlier.

---

## Note Dynamics and Clip Editing

Double-click a MIDI clip to open its piano roll. The Note velocity slider and
fixed Velocity lane edit individual notes from 1 to 127, preserving timing,
pitch and channel. Chord notes at the same start can be selected individually in
the keys before editing their highlighted bar. Completed gestures are undoable
and save in the existing note velocity field. Velocity is not a separate dB
volume stage: the built-in sine scales amplitude, while plugins choose their own
patch-dependent response. The track header instrument dropdown reuses saved
project instruments or opens explicit loading; bus tracks cannot host
instruments. See WORKSPACE.md for grid/snap divisions, filters and control
behavior.