Phase 7: Add MIDI clips, instrument and piano roll
This commit is contained in:
parent
8a37090b81
commit
cacc086805
38 changed files with 1373 additions and 74 deletions
|
|
@ -106,3 +106,15 @@ SPSC ring publishes complete payload blocks through lock-free indices. Only
|
|||
the consumer writes WAV bytes; finalization follows successful callback stop.
|
||||
No editable project pointer, GUI object or FILE is accessed by the producer.
|
||||
See decision 0005 and AUDIO_FILES.md for format, limits and recovery contracts.
|
||||
|
||||
## Phase 7 MIDI source
|
||||
|
||||
The C MIDI plan prepares bounded source-relative notes into sample spans and
|
||||
renders a separate instrument buffer. `xtc_plan_process_sources` and
|
||||
`xtc_realtime_process_sources` distinguish external audio, WAV timeline and
|
||||
instrument input; older wrappers supply silence for omitted sources. Graph
|
||||
instrument nodes consume the shared mono instrument buffer. The backend owns
|
||||
prepared MIDI runtime state exclusively while active, with lock-free panic
|
||||
requests and no callback allocation/I/O. Stop/seek clear live voices; scheduled
|
||||
notes chase their absolute phase. See MIDI.md for event ordering, panic semantics,
|
||||
capacities and test coverage. No new dependency is required.
|
||||
|
|
|
|||
|
|
@ -238,3 +238,15 @@ I/O, logging, locks or FLTK calls were added. Backend stop failure preserves the
|
|||
borrowed recorder; the caller must not free it before successful quiescence.
|
||||
Live software source/capture acceptance and concurrent ring tests pass; physical
|
||||
recording, latency compensation and disk endurance remain unverified.
|
||||
|
||||
## Phase 7 MIDI source
|
||||
|
||||
The C MIDI plan prepares bounded source-relative notes into sample spans and
|
||||
renders a separate instrument buffer. `xtc_plan_process_sources` and
|
||||
`xtc_realtime_process_sources` distinguish external audio, WAV timeline and
|
||||
instrument input; older wrappers supply silence for omitted sources. Graph
|
||||
instrument nodes consume the shared mono instrument buffer. The backend owns
|
||||
prepared MIDI runtime state exclusively while active, with lock-free panic
|
||||
requests and no callback allocation/I/O. Stop/seek clear live voices; scheduled
|
||||
notes chase their absolute phase. See MIDI.md for event ordering, panic semantics,
|
||||
capacities and test coverage. No new dependency is required.
|
||||
|
|
|
|||
|
|
@ -154,3 +154,24 @@ ASan/UBSan passes the core and GUI tests (GUI retains the existing fontconfig-on
|
|||
LSan suppression). ThreadSanitizer passes the concurrent recorder and existing
|
||||
realtime handoff tests. Hardware capture/playback, disk endurance and real server
|
||||
loss during recording remain unverified. See AUDIO_FILES.md.
|
||||
|
||||
## Phase 7 verification
|
||||
|
||||
```sh
|
||||
make all test gui debug test-gui test-gui-jack test-jack-audio test-jack-midi
|
||||
make BUILD_DIR=build/nojack PKG_CONFIG=false all test gui test-gui
|
||||
```
|
||||
|
||||
`test_midi` adds exact sample scheduling, trim/seek/panic, block independence,
|
||||
voice capacities, stable notes and v3 malformed-file/roundtrip/v2 migration
|
||||
coverage. Backend fault injection checks event offsets, dropped/malformed input,
|
||||
disconnection, xruns, missing output buffers and stop/restart cleanup. GUI tests
|
||||
draw/move/resize notes, save/reopen and render them. The live MIDI target connects
|
||||
only software clients and checks on/off, panic, disconnect, and saved-note
|
||||
playback/seek. It needs an existing JACK-compatible server. No hardware routes
|
||||
or system changes are made. MIDI recording/SMF and physical devices are deferred.
|
||||
|
||||
A fresh `build/verify` build passed all targets above with no compiler warnings.
|
||||
The no-JACK core/GUI suite and Clang ASan/UBSan core/GUI suite also passed; GUI
|
||||
leak checking uses only the existing fontconfig suppression documented in the
|
||||
integration README. Physical MIDI and audio devices remain unverified.
|
||||
|
|
|
|||
13
docs/GUI.md
13
docs/GUI.md
|
|
@ -124,7 +124,7 @@ A ruler click seeks while stopped (or positions the cursor when disconnected).
|
|||
The 100 ms GUI refresh converts the backend's actual sample counter to ticks;
|
||||
it never integrates wall-clock time. Play resumes and Stop pauses. Reconnect,
|
||||
New and Open reset position. Add region creates silent metadata; imported WAV clips play through the
|
||||
audio source node (Phase 6 below). MIDI content is not implemented. The C model
|
||||
audio source node (Phase 6 below). MIDI clips and notes are described below. The C model
|
||||
supports a constant tempo; a tempo map and scheduler are future work.
|
||||
|
||||
Tests cover C-model/persistence boundaries, GUI move/trim gestures and zoom,
|
||||
|
|
@ -155,3 +155,14 @@ See AUDIO_FILES.md for supported WAV formats, memory limits, musical placement,
|
|||
recording overflow/partial-take recovery, external asset portability and the
|
||||
software-only acceptance tests. Waveform drawing, input monitoring, recording
|
||||
latency compensation and per-track mixing are not implemented.
|
||||
|
||||
## Phase 7 piano roll
|
||||
|
||||
Add MIDI clip creates a one-bar clip on the selected/first existing track at the
|
||||
cursor. Select it and choose Piano roll (View/Piano roll, Ctrl+4). Click to draw,
|
||||
drag a body to move pitch/time, drag its right edge to resize, and right-click
|
||||
or Delete to remove. Notes use source ticks with 1/16 snapping; trimmed source
|
||||
content is retained. Edits stop transport and are blocked during audio recording.
|
||||
Panic/F9 silences held notes; Record continues to mean WAV audio capture.
|
||||
Modular Graph offers Add instrument for explicit routing. Status includes MIDI
|
||||
drops. See MIDI.md for voice limits, unsupported messages and live input routing.
|
||||
|
|
|
|||
73
docs/MIDI.md
Normal file
73
docs/MIDI.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
# MIDI clips and instrument
|
||||
|
||||
Phase 7 supports drawing persisted notes and live JACK MIDI input. Record still
|
||||
captures audio to WAV: MIDI capture and Standard MIDI File import are deferred.
|
||||
|
||||
## Workflow
|
||||
|
||||
Add/select a track in Sequencer, choose Add MIDI clip, then Piano roll (Ctrl+4).
|
||||
Creation adds an instrument-to-output route if no instrument exists. Existing
|
||||
instruments retain their routes; use Modular Graph to connect them as needed.
|
||||
All MIDI clips and live input share one mono instrument mix. No hardware ports
|
||||
are connected automatically. Connect a MIDI source to the backend's `midi_in`
|
||||
JACK port, and route audio output explicitly to hear it while Play is running.
|
||||
|
||||
The piano roll shows all 128 pitches and edits clip-source time. Click blank
|
||||
space to draw a one-beat note (velocity 100, channel 0); drag its body to move
|
||||
it, drag the right edge to resize, or right-click/Delete to remove it. Edits
|
||||
snap to 1/16 notes. Escape cancels a drag. Trimmed-out source notes remain editable
|
||||
and are preserved. Arrangement/note edits stop playback and are blocked during
|
||||
audio recording. Save/Open preserves MIDI flags, notes and stable IDs.
|
||||
|
||||
## Timing and ownership
|
||||
|
||||
The C model owns up to 4096 notes per project, with source-relative ticks at
|
||||
960 ticks/quarter and constant tempo. Control-thread preparation intersects
|
||||
notes with clip trims and converts boundaries to integer sample positions.
|
||||
Scheduled rendering uses absolute note age: block partitions do not change
|
||||
samples, and seeking into a note chases its phase. Clip ends cut notes off.
|
||||
The instrument source has its own buffer, separate from external audio input
|
||||
and WAV timeline playback, and routes through the ordinary graph.
|
||||
|
||||
Prepared ranges are immutable; only their audio owner changes voice/panic state.
|
||||
Preparation allocates; rendering uses bounded preallocated storage and performs
|
||||
no allocation, freeing, I/O, locks, logging or GUI calls. Project preparation
|
||||
builds audio and MIDI candidates before committing the stopped backend state.
|
||||
|
||||
JACK events have sample offsets within the current block. Supported messages
|
||||
are complete note on/off (including velocity-zero note on) and CC120/123 on
|
||||
channels 0..15. Equal-offset events retain input order. Other messages are
|
||||
ignored; sustain, bend, program changes, running status and SysEx are unsupported.
|
||||
At most 256 events are accepted per block. Malformed/overflowed input discards
|
||||
that block's events and triggers panic, with a MIDI-drop status counter.
|
||||
|
||||
## Recovery and limits
|
||||
|
||||
Panic/F9 clears live voices and suppresses scheduled notes already started;
|
||||
future timeline notes can still start. CC120/123 does this for its channel.
|
||||
Stop/start and seek clear live voices and reset suppression, allowing timeline
|
||||
notes to be chased again. MIDI port disconnection, xruns and callback processing
|
||||
failures request panic. Control requests use lock-free generation counters;
|
||||
voice state changes on the audio thread, or while stopped.
|
||||
|
||||
The sine instrument scales amplitude by 0.1 times velocity/127. It supports
|
||||
32 simultaneous scheduled notes and a separate 32 live voices; live overflow
|
||||
steals the oldest voice, and repeated channel/pitch retriggers it. Preparation
|
||||
rejects more than 32 overlapping scheduled notes with CAPACITY. Such metadata
|
||||
can be saved by the C model, but GUI Open/Play rejects it until overlap is
|
||||
reduced. Immediate note-off/stealing can click; there is no envelope or limiter,
|
||||
and summed voices can exceed unity. Above-Nyquist pitches are silent.
|
||||
There is no per-track/channel instrument routing, velocity editor, MIDI recording,
|
||||
SMF import, undo or piano-roll zoom yet.
|
||||
|
||||
## Verification
|
||||
|
||||
`make test` checks exact onset/off samples, block partition independence, trim,
|
||||
seek, panic, channel handling, voice/event limits, persistence migration and
|
||||
malformed files, plus callback fault recovery. `make test-gui` draws, moves and
|
||||
resizes notes with FLTK events, then saves/reopens and renders them.
|
||||
`make test-jack-midi` uses only software MIDI/audio ports to verify live on/off,
|
||||
panic, disconnection and saved/reopened scheduled playback. JACK port discovery
|
||||
may lag activation under PipeWire; the test waits before connecting.
|
||||
Physical MIDI devices, bridge/device loss behind an unchanged port, latency and
|
||||
long-running performance remain unverified. Panic/Stop provides manual recovery.
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
# Project format v2 (reads v1)
|
||||
# Project format v3 (reads v1/v2)
|
||||
|
||||
Phases 5/6 use a bounded text format implemented in C99 without a parser library.
|
||||
Phases 5–7 use a bounded text format implemented in C99 without a parser library.
|
||||
The extension is `.xtc`. File operations run exclusively on the control thread.
|
||||
The project owns its graph, tracks and silent/audio arrangement regions. Audio samples remain in external WAV
|
||||
files; MIDI content is a subsequent phase.
|
||||
files; MIDI notes are stored inline.
|
||||
|
||||
## Schema
|
||||
|
||||
|
|
@ -12,7 +12,7 @@ Fields use ASCII spaces. Integers are unsigned decimal, without signs.
|
|||
Counts determine the number of subsequent records:
|
||||
|
||||
```text
|
||||
XTC_PROJECT 2
|
||||
XTC_PROJECT 3
|
||||
APP 0 1 0
|
||||
TEMPO 120000
|
||||
IDS 1 1
|
||||
|
|
@ -25,6 +25,8 @@ TRACK 0 547261636b
|
|||
CLIPS 1
|
||||
CLIP 0 0 3840 2880 960 3840 526567696f6e
|
||||
ASSET -
|
||||
MIDI 0
|
||||
NOTES 0 0
|
||||
END
|
||||
```
|
||||
|
||||
|
|
@ -35,7 +37,7 @@ END
|
|||
- `GRAPH`: output-node policy (0/1), next node ID, node count, edge count.
|
||||
Nodes retain stable IDs (limited to the host's size_t range). No pointers or
|
||||
compiled state are serialized.
|
||||
- `NODE`: ID, type (1 gain, 2 oscillator, 3 meter, 4 output, 5 audio source), and exactly eight
|
||||
- `NODE`: ID, type (1 gain, 2 oscillator, 3 meter, 4 output, 5 audio source, 6 instrument), and exactly eight
|
||||
hexadecimal digits encoding the IEEE binary32 parameter bits, independent
|
||||
of locale and host byte order. Nonfinite values are rejected.
|
||||
- `EDGE`: source and destination node IDs. Mono port directions are implied
|
||||
|
|
@ -45,24 +47,32 @@ END
|
|||
length tick, hex-encoded name bytes. Trimming
|
||||
preserves source bounds, so a region can re-extend only within its original
|
||||
length. Overlaps are allowed and audio clips mix through the audio source node.
|
||||
- `ASSET`: required after each CLIP in v2; "-" means silent, otherwise
|
||||
- `ASSET`: required after each CLIP in v2/v3; "-" means no audio asset, otherwise
|
||||
hex-encoded absolute path bytes, <=511 bytes after decoding, no embedded NUL
|
||||
or ASCII control characters. Each clip directly references a WAV; no sample
|
||||
data, pointers or DSP state are embedded. Paths can contain spaces.
|
||||
- `MIDI`: required after each ASSET in v3; 0 for silent/audio regions, 1 for
|
||||
MIDI clips. MIDI clips must have no asset.
|
||||
- `NOTES`: next stable note ID and global note count, followed by that many
|
||||
`NOTE id clip_id start_tick length_tick pitch velocity channel` records.
|
||||
Notes reference MIDI clips; start/length are source-relative, positive length
|
||||
within source bounds. Pitch is 0..127, velocity 1..127, channel 0..15.
|
||||
Note IDs are unique, monotonic uint64 IDs with UINT64_MAX reserved.
|
||||
- Names decode to nonempty valid UTF-8 of at most 63 bytes; ASCII control
|
||||
characters and embedded NULs are forbidden. Hex avoids quoting ambiguity.
|
||||
|
||||
Ticks use 960 per quarter note, 3840 per bar. All region endpoints are bounded
|
||||
by 38400000 ticks (10000 bars). Limits: 64 tracks, 1024 clips, 64 graph nodes,
|
||||
4096 edges, 2 MiB input, 1535 bytes per record, 12 tokens per record. The reader
|
||||
4096 edges, 4096 notes, 4 MiB input, 1535 bytes per record, 12 tokens per record. The reader
|
||||
rejects invalid references, duplicate IDs, inconsistent next IDs, ranges,
|
||||
truncated records, raw control/non-ASCII bytes, and trailing data.
|
||||
|
||||
Unsupported format versions return UNSUPPORTED. Other malformed data returns
|
||||
FORMAT. The reader explicitly accepts v1 records without ASSET lines, with
|
||||
node types 1..4, migrating their clips to empty asset paths. Saving always writes
|
||||
v2. Unknown versions are rejected without replacing the destination; older
|
||||
applications cannot read v2. Migration and maximum-path roundtrips are tested.
|
||||
node types 1..4, migrating their clips to empty asset paths. Version 2 accepts
|
||||
node types 1..5 and ASSET records. Both migrate to MIDI=0 and no notes.
|
||||
Saving always writes v3. Unknown versions are rejected without replacing the destination; older
|
||||
applications cannot read v3. Migration and maximum-path roundtrips are tested.
|
||||
|
||||
## Ownership and failures
|
||||
|
||||
|
|
|
|||
|
|
@ -80,6 +80,14 @@ No new dependency or callback I/O was introduced.
|
|||
MIDI input/events/clips, simple internal instrument, note-off/panic handling,
|
||||
then a basic piano roll.
|
||||
|
||||
Implemented: bounded source-relative note events with stable IDs; sample-offset
|
||||
JACK MIDI input; a velocity-scaled sine instrument and separate graph source;
|
||||
panic, note-off, stop and seek handling; and a piano roll for drawing, moving,
|
||||
resizing and deleting notes. Version 3 saves notes and reads v1/v2 projects.
|
||||
Core, fault-injection, GUI and live software MIDI/save/reopen/play tests pass.
|
||||
Acceptance uses drawn notes; MIDI recording and SMF import are deferred.
|
||||
See MIDI.md for capacities, immediate note-off, routing and hardware limits.
|
||||
|
||||
## Phase 8 — Mixer and editing
|
||||
Gain/pan, meters, mute/solo, automation, undo/redo, keyboard workflow.
|
||||
|
||||
|
|
|
|||
16
docs/decisions/0006-midi-instrument-and-notes.md
Normal file
16
docs/decisions/0006-midi-instrument-and-notes.md
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
# 0006 — Bounded MIDI notes and internal instrument
|
||||
|
||||
Status: accepted, Phase 7.
|
||||
|
||||
Use the existing JACK backend for timestamped MIDI input, avoiding another
|
||||
backend/dependency. Store source-relative notes in the C project and v3 format;
|
||||
read v1/v2 without MIDI. Prepare sample spans on the control thread and render
|
||||
a small sine instrument into its own graph source buffer. Use a piano roll to
|
||||
meet draw/play/save/reopen acceptance before adding MIDI capture or SMF import.
|
||||
|
||||
Keep explicit bounds: 4096 project notes, 32 overlapping scheduled notes, 32 live
|
||||
voices and 256 events/block. Reject scheduled overcapacity at preparation; steal
|
||||
oldest live voices. Panic handles event loss, disconnects and callback failures;
|
||||
Stop/seek resets live state. Immediate note-off and a fixed sine voice keep this
|
||||
prototype small, at the cost of possible clicks and no expressive controllers.
|
||||
See ../MIDI.md for ownership, detailed limits and validation.
|
||||
Loading…
Add table
Add a link
Reference in a new issue