1
0
Fork 0
forked from hanez/xtc

Phase 7: Add MIDI clips, instrument and piano roll

This commit is contained in:
Johannes Findeisen 2026-10-27 00:53:41 +01:00
commit cacc086805
38 changed files with 1373 additions and 74 deletions

View file

@ -106,3 +106,15 @@ SPSC ring publishes complete payload blocks through lock-free indices. Only
the consumer writes WAV bytes; finalization follows successful callback stop.
No editable project pointer, GUI object or FILE is accessed by the producer.
See decision 0005 and AUDIO_FILES.md for format, limits and recovery contracts.
## Phase 7 MIDI source
The C MIDI plan prepares bounded source-relative notes into sample spans and
renders a separate instrument buffer. `xtc_plan_process_sources` and
`xtc_realtime_process_sources` distinguish external audio, WAV timeline and
instrument input; older wrappers supply silence for omitted sources. Graph
instrument nodes consume the shared mono instrument buffer. The backend owns
prepared MIDI runtime state exclusively while active, with lock-free panic
requests and no callback allocation/I/O. Stop/seek clear live voices; scheduled
notes chase their absolute phase. See MIDI.md for event ordering, panic semantics,
capacities and test coverage. No new dependency is required.

View file

@ -238,3 +238,15 @@ I/O, logging, locks or FLTK calls were added. Backend stop failure preserves the
borrowed recorder; the caller must not free it before successful quiescence.
Live software source/capture acceptance and concurrent ring tests pass; physical
recording, latency compensation and disk endurance remain unverified.
## Phase 7 MIDI source
The C MIDI plan prepares bounded source-relative notes into sample spans and
renders a separate instrument buffer. `xtc_plan_process_sources` and
`xtc_realtime_process_sources` distinguish external audio, WAV timeline and
instrument input; older wrappers supply silence for omitted sources. Graph
instrument nodes consume the shared mono instrument buffer. The backend owns
prepared MIDI runtime state exclusively while active, with lock-free panic
requests and no callback allocation/I/O. Stop/seek clear live voices; scheduled
notes chase their absolute phase. See MIDI.md for event ordering, panic semantics,
capacities and test coverage. No new dependency is required.

View file

@ -154,3 +154,24 @@ ASan/UBSan passes the core and GUI tests (GUI retains the existing fontconfig-on
LSan suppression). ThreadSanitizer passes the concurrent recorder and existing
realtime handoff tests. Hardware capture/playback, disk endurance and real server
loss during recording remain unverified. See AUDIO_FILES.md.
## Phase 7 verification
```sh
make all test gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
make BUILD_DIR=build/nojack PKG_CONFIG=false all test gui test-gui
```
`test_midi` adds exact sample scheduling, trim/seek/panic, block independence,
voice capacities, stable notes and v3 malformed-file/roundtrip/v2 migration
coverage. Backend fault injection checks event offsets, dropped/malformed input,
disconnection, xruns, missing output buffers and stop/restart cleanup. GUI tests
draw/move/resize notes, save/reopen and render them. The live MIDI target connects
only software clients and checks on/off, panic, disconnect, and saved-note
playback/seek. It needs an existing JACK-compatible server. No hardware routes
or system changes are made. MIDI recording/SMF and physical devices are deferred.
A fresh `build/verify` build passed all targets above with no compiler warnings.
The no-JACK core/GUI suite and Clang ASan/UBSan core/GUI suite also passed; GUI
leak checking uses only the existing fontconfig suppression documented in the
integration README. Physical MIDI and audio devices remain unverified.

View file

@ -124,7 +124,7 @@ A ruler click seeks while stopped (or positions the cursor when disconnected).
The 100 ms GUI refresh converts the backend's actual sample counter to ticks;
it never integrates wall-clock time. Play resumes and Stop pauses. Reconnect,
New and Open reset position. Add region creates silent metadata; imported WAV clips play through the
audio source node (Phase 6 below). MIDI content is not implemented. The C model
audio source node (Phase 6 below). MIDI clips and notes are described below. The C model
supports a constant tempo; a tempo map and scheduler are future work.
Tests cover C-model/persistence boundaries, GUI move/trim gestures and zoom,
@ -155,3 +155,14 @@ See AUDIO_FILES.md for supported WAV formats, memory limits, musical placement,
recording overflow/partial-take recovery, external asset portability and the
software-only acceptance tests. Waveform drawing, input monitoring, recording
latency compensation and per-track mixing are not implemented.
## Phase 7 piano roll
Add MIDI clip creates a one-bar clip on the selected/first existing track at the
cursor. Select it and choose Piano roll (View/Piano roll, Ctrl+4). Click to draw,
drag a body to move pitch/time, drag its right edge to resize, and right-click
or Delete to remove. Notes use source ticks with 1/16 snapping; trimmed source
content is retained. Edits stop transport and are blocked during audio recording.
Panic/F9 silences held notes; Record continues to mean WAV audio capture.
Modular Graph offers Add instrument for explicit routing. Status includes MIDI
drops. See MIDI.md for voice limits, unsupported messages and live input routing.

73
docs/MIDI.md Normal file
View file

@ -0,0 +1,73 @@
# MIDI clips and instrument
Phase 7 supports drawing persisted notes and live JACK MIDI input. Record still
captures audio to WAV: MIDI capture and Standard MIDI File import are deferred.
## Workflow
Add/select a track in Sequencer, choose Add MIDI clip, then Piano roll (Ctrl+4).
Creation adds an instrument-to-output route if no instrument exists. Existing
instruments retain their routes; use Modular Graph to connect them as needed.
All MIDI clips and live input share one mono instrument mix. No hardware ports
are connected automatically. Connect a MIDI source to the backend's `midi_in`
JACK port, and route audio output explicitly to hear it while Play is running.
The piano roll shows all 128 pitches and edits clip-source time. Click blank
space to draw a one-beat note (velocity 100, channel 0); drag its body to move
it, drag the right edge to resize, or right-click/Delete to remove it. Edits
snap to 1/16 notes. Escape cancels a drag. Trimmed-out source notes remain editable
and are preserved. Arrangement/note edits stop playback and are blocked during
audio recording. Save/Open preserves MIDI flags, notes and stable IDs.
## Timing and ownership
The C model owns up to 4096 notes per project, with source-relative ticks at
960 ticks/quarter and constant tempo. Control-thread preparation intersects
notes with clip trims and converts boundaries to integer sample positions.
Scheduled rendering uses absolute note age: block partitions do not change
samples, and seeking into a note chases its phase. Clip ends cut notes off.
The instrument source has its own buffer, separate from external audio input
and WAV timeline playback, and routes through the ordinary graph.
Prepared ranges are immutable; only their audio owner changes voice/panic state.
Preparation allocates; rendering uses bounded preallocated storage and performs
no allocation, freeing, I/O, locks, logging or GUI calls. Project preparation
builds audio and MIDI candidates before committing the stopped backend state.
JACK events have sample offsets within the current block. Supported messages
are complete note on/off (including velocity-zero note on) and CC120/123 on
channels 0..15. Equal-offset events retain input order. Other messages are
ignored; sustain, bend, program changes, running status and SysEx are unsupported.
At most 256 events are accepted per block. Malformed/overflowed input discards
that block's events and triggers panic, with a MIDI-drop status counter.
## Recovery and limits
Panic/F9 clears live voices and suppresses scheduled notes already started;
future timeline notes can still start. CC120/123 does this for its channel.
Stop/start and seek clear live voices and reset suppression, allowing timeline
notes to be chased again. MIDI port disconnection, xruns and callback processing
failures request panic. Control requests use lock-free generation counters;
voice state changes on the audio thread, or while stopped.
The sine instrument scales amplitude by 0.1 times velocity/127. It supports
32 simultaneous scheduled notes and a separate 32 live voices; live overflow
steals the oldest voice, and repeated channel/pitch retriggers it. Preparation
rejects more than 32 overlapping scheduled notes with CAPACITY. Such metadata
can be saved by the C model, but GUI Open/Play rejects it until overlap is
reduced. Immediate note-off/stealing can click; there is no envelope or limiter,
and summed voices can exceed unity. Above-Nyquist pitches are silent.
There is no per-track/channel instrument routing, velocity editor, MIDI recording,
SMF import, undo or piano-roll zoom yet.
## Verification
`make test` checks exact onset/off samples, block partition independence, trim,
seek, panic, channel handling, voice/event limits, persistence migration and
malformed files, plus callback fault recovery. `make test-gui` draws, moves and
resizes notes with FLTK events, then saves/reopens and renders them.
`make test-jack-midi` uses only software MIDI/audio ports to verify live on/off,
panic, disconnection and saved/reopened scheduled playback. JACK port discovery
may lag activation under PipeWire; the test waits before connecting.
Physical MIDI devices, bridge/device loss behind an unchanged port, latency and
long-running performance remain unverified. Panic/Stop provides manual recovery.

View file

@ -1,9 +1,9 @@
# Project format v2 (reads v1)
# Project format v3 (reads v1/v2)
Phases 5/6 use a bounded text format implemented in C99 without a parser library.
Phases 5–7 use a bounded text format implemented in C99 without a parser library.
The extension is `.xtc`. File operations run exclusively on the control thread.
The project owns its graph, tracks and silent/audio arrangement regions. Audio samples remain in external WAV
files; MIDI content is a subsequent phase.
files; MIDI notes are stored inline.
## Schema
@ -12,7 +12,7 @@ Fields use ASCII spaces. Integers are unsigned decimal, without signs.
Counts determine the number of subsequent records:
```text
XTC_PROJECT 2
XTC_PROJECT 3
APP 0 1 0
TEMPO 120000
IDS 1 1
@ -25,6 +25,8 @@ TRACK 0 547261636b
CLIPS 1
CLIP 0 0 3840 2880 960 3840 526567696f6e
ASSET -
MIDI 0
NOTES 0 0
END
```
@ -35,7 +37,7 @@ END
- `GRAPH`: output-node policy (0/1), next node ID, node count, edge count.
Nodes retain stable IDs (limited to the host's size_t range). No pointers or
compiled state are serialized.
- `NODE`: ID, type (1 gain, 2 oscillator, 3 meter, 4 output, 5 audio source), and exactly eight
- `NODE`: ID, type (1 gain, 2 oscillator, 3 meter, 4 output, 5 audio source, 6 instrument), and exactly eight
hexadecimal digits encoding the IEEE binary32 parameter bits, independent
of locale and host byte order. Nonfinite values are rejected.
- `EDGE`: source and destination node IDs. Mono port directions are implied
@ -45,24 +47,32 @@ END
length tick, hex-encoded name bytes. Trimming
preserves source bounds, so a region can re-extend only within its original
length. Overlaps are allowed and audio clips mix through the audio source node.
- `ASSET`: required after each CLIP in v2; "-" means silent, otherwise
- `ASSET`: required after each CLIP in v2/v3; "-" means no audio asset, otherwise
hex-encoded absolute path bytes, <=511 bytes after decoding, no embedded NUL
or ASCII control characters. Each clip directly references a WAV; no sample
data, pointers or DSP state are embedded. Paths can contain spaces.
- `MIDI`: required after each ASSET in v3; 0 for silent/audio regions, 1 for
MIDI clips. MIDI clips must have no asset.
- `NOTES`: next stable note ID and global note count, followed by that many
`NOTE id clip_id start_tick length_tick pitch velocity channel` records.
Notes reference MIDI clips; start/length are source-relative, positive length
within source bounds. Pitch is 0..127, velocity 1..127, channel 0..15.
Note IDs are unique, monotonic uint64 IDs with UINT64_MAX reserved.
- Names decode to nonempty valid UTF-8 of at most 63 bytes; ASCII control
characters and embedded NULs are forbidden. Hex avoids quoting ambiguity.
Ticks use 960 per quarter note, 3840 per bar. All region endpoints are bounded
by 38400000 ticks (10000 bars). Limits: 64 tracks, 1024 clips, 64 graph nodes,
4096 edges, 2 MiB input, 1535 bytes per record, 12 tokens per record. The reader
4096 edges, 4096 notes, 4 MiB input, 1535 bytes per record, 12 tokens per record. The reader
rejects invalid references, duplicate IDs, inconsistent next IDs, ranges,
truncated records, raw control/non-ASCII bytes, and trailing data.
Unsupported format versions return UNSUPPORTED. Other malformed data returns
FORMAT. The reader explicitly accepts v1 records without ASSET lines, with
node types 1..4, migrating their clips to empty asset paths. Saving always writes
v2. Unknown versions are rejected without replacing the destination; older
applications cannot read v2. Migration and maximum-path roundtrips are tested.
node types 1..4, migrating their clips to empty asset paths. Version 2 accepts
node types 1..5 and ASSET records. Both migrate to MIDI=0 and no notes.
Saving always writes v3. Unknown versions are rejected without replacing the destination; older
applications cannot read v3. Migration and maximum-path roundtrips are tested.
## Ownership and failures

View file

@ -80,6 +80,14 @@ No new dependency or callback I/O was introduced.
MIDI input/events/clips, simple internal instrument, note-off/panic handling,
then a basic piano roll.
Implemented: bounded source-relative note events with stable IDs; sample-offset
JACK MIDI input; a velocity-scaled sine instrument and separate graph source;
panic, note-off, stop and seek handling; and a piano roll for drawing, moving,
resizing and deleting notes. Version 3 saves notes and reads v1/v2 projects.
Core, fault-injection, GUI and live software MIDI/save/reopen/play tests pass.
Acceptance uses drawn notes; MIDI recording and SMF import are deferred.
See MIDI.md for capacities, immediate note-off, routing and hardware limits.
## Phase 8 — Mixer and editing
Gain/pan, meters, mute/solo, automation, undo/redo, keyboard workflow.

View file

@ -0,0 +1,16 @@
# 0006 — Bounded MIDI notes and internal instrument
Status: accepted, Phase 7.
Use the existing JACK backend for timestamped MIDI input, avoiding another
backend/dependency. Store source-relative notes in the C project and v3 format;
read v1/v2 without MIDI. Prepare sample spans on the control thread and render
a small sine instrument into its own graph source buffer. Use a piano roll to
meet draw/play/save/reopen acceptance before adding MIDI capture or SMF import.
Keep explicit bounds: 4096 project notes, 32 overlapping scheduled notes, 32 live
voices and 256 events/block. Reject scheduled overcapacity at preparation; steal
oldest live voices. Panic handles event loss, disconnects and callback failures;
Stop/seek resets live state. Immediate note-off and a fixed sine voice keep this
prototype small, at the cost of possible clicks and no expressive controllers.
See ../MIDI.md for ownership, detailed limits and validation.