Audio track velocity editing in the sequencer.

This commit is contained in:
Johannes Findeisen 2026-11-06 21:10:48 +01:00
commit 896a7907d3
10 changed files with 292 additions and 6 deletions

View file

@ -140,6 +140,7 @@ cmake --build build --target clean
- The GUI now exposes left `output` and `output_right` JACK ports; route both explicitly for stereo. See [mixer workflow and limits](docs/MIXER.md).
- Use Add track/Add region, drag regions or their edges, and Zoom +/-; scrollbars pan.
- Click the small arrow beside a track name to expand its MIDI velocity lane downward. Drag a note bar’s cap vertically to set velocity (1–127); Escape cancels and Ctrl+Z undoes. Click the arrow again to collapse. See the [expanded-track screenshot](screenshots/track-velocity-expanded.png).
- Audio tracks open an **audio velocity** level envelope in the same lane. Click anywhere to add a gain point (0–400%), drag a point vertically to adjust it, or right-click its cap to remove it. Positions are independent of the snap grid; Escape cancels and undo/redo is supported. This is the track’s saved gain automation, shared with the mixer. See the [audio envelope screenshot](screenshots/audio-velocity-expanded.png).
- Ctrl+N/O/S creates, opens or saves a project; Save As chooses another path.
- Arrangement edits stop playback; graph edits remain live.
- The ruler seeks while stopped. Stop pauses; Reconnect and project load reset position. F5 starts, F8 stops, Space toggles, F6 reconnects, and Ctrl+1/2/3 switches views. Ctrl+Q closes the client and window. No physical ports are connected automatically.

View file

@ -3612,6 +3612,50 @@ checks, plus rendered screenshot inspection. See [the gallery](../screenshots/RE
---
## Phase 110 — Continuous Audio Velocity Editing in the Sequencer
### Goal
Extend the sequencer's downward-expanding velocity lanes to audio tracks, with
level editing at arbitrary timeline positions using the existing C gain
automation, playback, persistence and history contracts.
### Tasks and acceptance criteria
- [x] Display a connected audio level envelope beneath audio tracks using the
same disclosure arrows and row layout as MIDI velocity lanes.
- [x] Add points anywhere, independently of snap and note positions; drag caps
vertically to adjust gain between silence and 400%, and right-click to remove.
- [x] Preview edits locally, cancel with Escape, and commit once per gesture
through the arrangement edit hook and undo/redo publication path.
- [x] Share saved gain automation with the mixer, respecting existing Linear/Hold
segments and preserving interpolation mode when editing an existing point.
- [x] Verify off-grid point creation, edits, clamping, removal, cancellation,
undo/redo, save/reopen and rendered audio levels in GUI integration tests.
- [x] Capture the real audio envelope beside MIDI velocity bars, add it to the
complete screenshot gallery and document the workflow.
### Implementation status
Implemented on 2026-11-06 without engine, dependency or project-format changes.
Audio velocity controls track gain (0–4 linear, displayed as 0–400%); it is not
MIDI note velocity. Empty tracks, buses and tracks containing audio display the
gain envelope; tracks containing only MIDI retain their note velocity bars.
On mixed audio/MIDI tracks the gain envelope affects both sources, and individual
MIDI velocities remain editable in the piano roll. Automation replaces static
track gain and holds its first/last values beyond its endpoints. New points use
Linear interpolation; existing Hold segments remain supported. An initial point
therefore sets the whole track's level until further points shape the curve.
Points stay at their timeline position during vertical drags; add/remove points
to change timing. The existing project-wide 1024-point automation limit applies.
Envelope edits are saved; track disclosure state remains temporary GUI state.
Validation: warning-enabled CMake build, CTest checks including real FLTK
lane gestures and comparison of rendered audio against the unautomated source,
and inspection of the actual [audio envelope capture](../screenshots/audio-velocity-expanded.png).
---
## Recommended Execution Order
Reorder the phases when the audit reveals a blocking dependency. Do not mark a

View file

@ -37,6 +37,22 @@ Keys and Bass expand downward to reveal MIDI velocity bars while following track
---
## Shape audio velocity at any point
![Continuous audio velocity envelope beneath an expanded track](audio-velocity-expanded.png)
The audio track exposes a continuous level envelope with editable gain points at arbitrary timeline positions. The Keys track below retains its MIDI note velocity bars.
---
## Preview an audio level adjustment
![Dragging an audio velocity envelope point](audio-velocity-editing.png)
Dragging a point vertically previews the new envelope and displays its level in the track header. Release to commit the edit or press Escape to cancel.
---
## Start a new track
![Start a new track](create-track-menu.png)

View file

@ -32,6 +32,27 @@ APIs and GUI history/preparation paths. Original projects, media and user
configuration are not overwritten. Black margins are the virtual desktop;
floating views show the actual detached windows together.
## Continuous audio velocity editing (Phase 110)
![Audio velocity envelope beneath a waveform track](audio-velocity-expanded.png)
The expanded audio track shows a continuous gain envelope alongside the
Keys MIDI velocity lane. Points were created through actual sequencer gestures;
the screenshot is an unmodified application capture. Click anywhere in an audio
lane to add a point, drag vertically to adjust it, or right-click a cap to remove
it. The level range is 0–400%; points share the mixer's saved gain automation.
The [editing preview](audio-velocity-editing.png) shows an active vertical drag
with its level displayed in the track header.
Reproduce these two captures without replacing the other images:
```sh
cmake --build build --target capture_screenshots
XTC_CAPTURE_AUDIO_VELOCITY_ONLY=1 xvfb-run -a -s '-screen 0 1440x960x24' \
./build/capture_screenshots "$PWD/screenshots"
```
## Expandable track velocity lanes (Phase 109)
![Tracks expanded downward with editable MIDI velocity bars](track-velocity-expanded.png)

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

View file

@ -153,6 +153,58 @@ bool SequencerWidget::velocity_note(const xtc_note_info &note, uint64_t track, i
return true;
}
bool SequencerWidget::audio_velocity_lane(uint64_t track) const
{
size_t tracks=0, clips=0; xtc_project_get_counts(project_,&tracks,&clips);
bool midi=false;
for(size_t i=0;i<clips;++i) {
xtc_clip_info c{}; xtc_project_get_clip(project_,i,&c);
if(c.track_id!=track) continue;
if(!c.midi) return true;
midi=true;
}
return !midi;
}
float SequencerWidget::audio_velocity_value(int py) const
{
return std::clamp(static_cast<float>(velocity_top_+velocity_height_-8-py)*4.0f/(velocity_height_-28),0.0f,4.0f);
}
void SequencerWidget::draw_audio_velocity(const xtc_track_info &track, int top, int left, int right)
{
std::array<xtc_automation_point,XTC_MAX_AUTOMATION_POINTS+1> points{};
size_t count=0,total=0; xtc_project_automation_count(project_,&total);
for(size_t i=0;i<total;++i) {
xtc_automation_point a{}; xtc_project_get_automation(project_,i,&a);
if(a.track_id!=track.id || a.parameter!=XTC_AUTOMATION_GAIN) continue;
if(mode_==7 && track_==track.id && a.tick==audio_velocity_preview_.tick) continue;
points[count++]=a;
}
if(mode_==7 && track_==track.id) points[count++]=audio_velocity_preview_;
std::sort(points.begin(),points.begin()+count,[](const auto &a,const auto &b){return a.tick<b.tick;});
auto px=[&](uint64_t t){return x()+header_width_+static_cast<int>(t*static_cast<unsigned int>(scale_)/960U);};
auto py=[&](float v){return top+velocity_height_-8-static_cast<int>(v*(velocity_height_-28)/4.0f);};
fl_color(xtc_ui::track(track.id));
if(!count) fl_line(left,py(track.gain),right,py(track.gain));
else {
fl_line(left,py(points[0].value),px(points[0].tick),py(points[0].value));
for(size_t i=1;i<count;++i) {
const auto &a=points[i-1]; const auto &b=points[i];
if(a.step) {
fl_line(px(a.tick),py(a.value),px(b.tick),py(a.value));
fl_line(px(b.tick),py(a.value),px(b.tick),py(b.value));
} else fl_line(px(a.tick),py(a.value),px(b.tick),py(b.value));
}
fl_line(px(points[count-1].tick),py(points[count-1].value),right,py(points[count-1].value));
for(size_t i=0;i<count;++i) {
int point_x=px(points[i].tick);
if(point_x<left-4 || point_x>right+4) continue;
fl_rectf(point_x-4,py(points[i].value)-3,8,6);
}
}
}
unsigned int SequencerWidget::velocity_value(int py) const
{
return static_cast<unsigned int>(std::clamp((velocity_top_+velocity_height_-8-py)*127/(velocity_height_-28),1,127));
@ -391,8 +443,11 @@ void SequencerWidget::draw()
fl_push_clip(std::max(cx,header_left()+header_width_),top,cw,velocity_height_);
fl_color(xtc_ui::border()); fl_line(x()+header_width_,top,x()+w(),top);
fl_font(FL_HELVETICA,10); fl_color(xtc_ui::muted());
fl_draw("Drag a note bar to change velocity",std::max(x()+header_width_,header_left()+header_width_)+8,top+14);
for(size_t n=0;n<notes;++n) {
bool audio=audio_velocity_lane(t.id);
int lane_left=std::max(cx,header_left()+header_width_);
fl_draw(audio ? "Click anywhere to add a level point; drag vertically; right-click a point to remove" : "Drag a note bar to change velocity",lane_left+8,top+14);
if(audio) draw_audio_velocity(t,top,lane_left,cx+cw);
for(size_t n=0;!audio && n<notes;++n) {
xtc_note_info note{}; xtc_project_get_note(project_,n,&note); int px=0;
if(!velocity_note(note,t.id,px)) continue;
unsigned int value=mode_==6 && velocity_original_.id==note.id ? velocity_preview_ : note.velocity;
@ -453,9 +508,14 @@ void SequencerWidget::draw()
fl_pop_clip();
if(track_expanded(t.id)) {
fl_color(xtc_ui::muted()); fl_font(FL_HELVETICA,11);
fl_draw("VELOCITY 1..127",pinned+12,py+76);
bool audio=audio_velocity_lane(t.id);
fl_draw(audio ? "AUDIO VELOCITY" : "VELOCITY 1..127",pinned+12,py+76);
fl_font(FL_HELVETICA,10);
fl_draw(t.is_bus ? "No MIDI notes on buses" : "Drag bars to edit notes",pinned+12,py+94);
fl_draw(audio ? "Level envelope 0..400%" : "Drag bars to edit notes",pinned+12,py+94);
if(mode_==7 && t.id==track_) {
char value[32]; std::snprintf(value,sizeof(value),"Level: %.0f%%",audio_velocity_preview_.value*100);
fl_draw(value,pinned+12,py+114);
}
if(mode_==6 && t.id==track_) {
char value[24]; std::snprintf(value,sizeof(value),"Velocity: %u",velocity_preview_);
fl_draw(value,pinned+12,py+114);
@ -477,6 +537,25 @@ void SequencerWidget::draw()
*/
int SequencerWidget::handle(int event)
{
if(mode_==7) {
if(event==FL_KEYDOWN && Fl::event_key()==FL_Escape) { mode_=0; redraw(); return 1; }
if(event==FL_DRAG) { audio_velocity_preview_.value=audio_velocity_value(Fl::event_y()); redraw(); return 1; }
if(event==FL_RELEASE) {
mode_=0;
auto point=audio_velocity_preview_;
point.value=audio_velocity_value(Fl::event_y());
if(audio_velocity_existing_) {
size_t count=0; xtc_project_automation_count(project_,&count);
for(size_t i=0;i<count;++i) {
xtc_automation_point a{}; xtc_project_get_automation(project_,i,&a);
if(a.track_id==point.track_id && a.parameter==point.parameter && a.tick==point.tick && a.value==point.value) { redraw(); return 1; }
}
}
xtc_result r=before_edit ? before_edit(context_) : XTC_OK;
if(r==XTC_OK) r=xtc_project_set_automation(project_,&point);
changed(r); return 1;
}
}
if(mode_==6) {
if(event==FL_KEYDOWN && Fl::event_key()==FL_Escape) { mode_=0; redraw(); return 1; }
if(event==FL_DRAG) { velocity_preview_=velocity_value(Fl::event_y()); redraw(); return 1; }
@ -534,7 +613,33 @@ int SequencerWidget::handle(int event)
expand_track(t.id,!track_expanded(t.id)); return 1;
}
if(Fl::event_y()>=track_top(static_cast<size_t>(row))+56) {
if(Fl::event_button()!=FL_LEFT_MOUSE || Fl::event_x()<header_left()+header_width_) return 1;
if(Fl::event_x()<header_left()+header_width_) return 1;
velocity_top_=track_top(static_cast<size_t>(row))+56;
if(audio_velocity_lane(t.id)) {
track_=t.id; clip_=UINT64_MAX; selection_.clear();
uint64_t position=std::min(XTC_PROJECT_MAX_TICK,static_cast<uint64_t>(std::max(0,Fl::event_x()-x()-header_width_))*960/static_cast<unsigned int>(scale_));
audio_velocity_preview_={t.id,position,XTC_AUTOMATION_GAIN,audio_velocity_value(Fl::event_y()),0};
audio_velocity_existing_=false;
size_t count=0; xtc_project_automation_count(project_,&count); int distance=12;
for(size_t i=0;i<count;++i) {
xtc_automation_point a{}; xtc_project_get_automation(project_,i,&a);
if(a.track_id!=t.id || a.parameter!=XTC_AUTOMATION_GAIN) continue;
int px=x()+header_width_+static_cast<int>(a.tick*static_cast<unsigned int>(scale_)/960U);
int py=velocity_top_+velocity_height_-8-static_cast<int>(a.value*(velocity_height_-28)/4.0f);
int d=std::abs(Fl::event_x()-px)+std::abs(Fl::event_y()-py);
if(d<distance) { distance=d; audio_velocity_preview_=a; audio_velocity_existing_=true; }
}
if(Fl::event_button()==FL_RIGHT_MOUSE) {
if(audio_velocity_existing_) {
auto point=audio_velocity_preview_;
xtc_result r=before_edit ? before_edit(context_) : XTC_OK;
if(r==XTC_OK) r=xtc_project_remove_automation(project_,point.track_id,point.parameter,point.tick);
changed(r);
}
} else if(Fl::event_button()==FL_LEFT_MOUSE) mode_=7;
redraw(); return 1;
}
if(Fl::event_button()!=FL_LEFT_MOUSE) return 1;
size_t notes=0; xtc_project_note_count(project_,&notes);
int distance=10; bool hit=false;
velocity_top_=track_top(static_cast<size_t>(row))+56;

View file

@ -4,6 +4,7 @@
#include <FL/Fl_Choice.H>
#include "xtc/xtc_plugin.h"
#include "xtc/xtc_editing.h"
#include "xtc/xtc_mixer.h"
#include <array>
#include "xtc/xtc_audio.h"
#include <memory>
@ -206,6 +207,11 @@ class SequencerWidget : public Fl_Group {
xtc_note_info velocity_original_{};
unsigned int velocity_preview_ = 100;
int velocity_top_ = 0;
bool audio_velocity_lane(uint64_t track) const;
void draw_audio_velocity(const xtc_track_info &track, int top, int left, int right);
float audio_velocity_value(int py) const;
xtc_automation_point audio_velocity_preview_{};
bool audio_velocity_existing_ = false;
struct TrackChoice { Fl_Choice *widget = nullptr; uint64_t track = UINT64_MAX; };
std::array<TrackChoice,XTC_PROJECT_MAX_TRACKS> track_choices_{};
std::vector<xtc_plugin_spec> catalog_;

View file

@ -1214,6 +1214,69 @@ int main(int argc, char **argv)
unlink("/tmp/xtc-workspace.xtc"); window.do_callback();
}
reset();
{
MainWindow window(false,""); window.show(); auto *s=timeline(&window);
char wave[4096]; CHECK(realpath("examples/projects/original-pluck.wav",wave));
CHECK(window.create_track(false)==XTC_OK && window.import_audio(wave)==XTC_OK);
uint64_t track=s->selected_track(); CHECK(s->expand_track(track,true));
CHECK(s->set_zoom(96)==XTC_OK && s->grid(960)==XTC_OK);
float base_l[256]{},base_r[256]{},edited_l[256]{},edited_r[256]{};
xtc_audio_plan *plan=nullptr;
CHECK(xtc_audio_plan_create(s->project(),48000,&plan)==XTC_OK);
CHECK(xtc_audio_plan_render_stereo(plan,1000,base_l,base_r,256)==XTC_OK); xtc_audio_plan_destroy(plan);
// An arbitrary off-grid position, with no MIDI notes, creates an audio level point.
int px=s->header_width()+37, top=28+56;
gesture(s,px,top+65,0,-15); // 2x gain at tick 370.
size_t count=0; xtc_project_automation_count(s->project(),&count); CHECK(count==1);
xtc_automation_point point{}; xtc_project_get_automation(s->project(),0,&point);
CHECK(point.track_id==track && point.tick==370 && point.parameter==XTC_AUTOMATION_GAIN && point.value==2.0f && !point.step);
CHECK(xtc_audio_plan_create(s->project(),48000,&plan)==XTC_OK);
CHECK(xtc_audio_plan_render_stereo(plan,1000,edited_l,edited_r,256)==XTC_OK); xtc_audio_plan_destroy(plan);
for(int i=0;i<256;++i) CHECK(std::fabs(edited_l[i]-base_l[i]*2)<1e-5f && std::fabs(edited_r[i]-base_r[i]*2)<1e-5f);
CHECK(window.undo(false)==XTC_OK); xtc_project_automation_count(s->project(),&count); CHECK(count==0);
CHECK(window.undo(true)==XTC_OK && s->track_expanded(track));
gesture(s,px,top+50,0,500); // Existing point clamps to silence.
xtc_project_get_automation(s->project(),0,&point); CHECK(point.tick==370 && point.value==0);
gesture(s,px,top+80,0,-500); // Existing point clamps to 400%.
xtc_project_get_automation(s->project(),0,&point); CHECK(point.value==4);
gesture(s,s->header_width()+61,top+65,0,0);
xtc_project_automation_count(s->project(),&count); CHECK(count==2);
// A pre-existing Hold point keeps its interpolation mode when dragged.
xtc_project_get_automation(s->project(),1,&point); point.step=1;
CHECK(xtc_project_set_automation(s->project(),&point)==XTC_OK);
gesture(s,s->header_width()+61,top+65,0,-15);
xtc_project_get_automation(s->project(),1,&point); CHECK(point.tick==610 && point.value==2 && point.step==1);
// Escape cancels a point at another arbitrary position.
int ox=Fl::e_x,oy=Fl::e_y,ok=Fl::e_keysym;
Fl::e_x=s->x()+s->header_width()+91; Fl::e_y=s->y()+top+50;
Fl::e_keysym=FL_Button+FL_LEFT_MOUSE; CHECK(s->handle(FL_PUSH)==1);
Fl::e_y+=10; CHECK(s->handle(FL_DRAG)==1);
Fl::e_keysym=FL_Escape; CHECK(s->handle(FL_KEYDOWN)==1);
xtc_project_automation_count(s->project(),&count); CHECK(count==2);
CHECK(window.save_project("/tmp/xtc-audio-velocity.xtc")==XTC_OK);
CHECK(window.new_project()==XTC_OK && window.open_project("/tmp/xtc-audio-velocity.xtc")==XTC_OK);
CHECK(s->expand_track(track,true));
xtc_project_get_automation(s->project(),0,&point); CHECK(point.value==4 && point.tick==370);
CHECK(s->set_zoom(96)==XTC_OK);
// Delete a point with the real right mouse gesture; undo restores it.
Fl::e_x=s->x()+px; Fl::e_y=s->y()+top+20;
Fl::e_keysym=FL_Button+FL_RIGHT_MOUSE; CHECK(s->handle(FL_PUSH)==1);
Fl::e_x=ox; Fl::e_y=oy; Fl::e_keysym=ok;
xtc_project_automation_count(s->project(),&count); CHECK(count==1);
CHECK(window.undo(false)==XTC_OK); xtc_project_automation_count(s->project(),&count); CHECK(count==2);
// Pinned headers and horizontal scrolling still map to timeline positions.
auto *scroll=dynamic_cast<Fl_Scroll *>(s->parent()); CHECK(scroll);
scroll->scroll_to(80,0); pump();
int screen_x=s->header_left()+s->header_width()+100;
uint64_t expected=static_cast<uint64_t>(screen_x-s->x()-s->header_width())*960/static_cast<unsigned int>(s->zoom_level());
gesture(s,screen_x-s->x(),top+50,0,0);
xtc_project_automation_count(s->project(),&count); CHECK(count==3);
bool found=false;
for(size_t i=0;i<count;++i) {xtc_project_get_automation(s->project(),i,&point); if(point.tick==expected && point.value==2) found=true;}
CHECK(found);
unlink("/tmp/xtc-audio-velocity.xtc"); window.do_callback();
}
reset();
{
MainWindow window(false,""); window.show(); auto *s=timeline(&window);
CHECK(window.create_track(true)==XTC_OK);

View file

@ -64,6 +64,35 @@ int main(int argc,char **argv) {
MainWindow window(false,"");main_window=&window;window.resize(0,0,1400,900);window.show();check(window.open_project(project.c_str()));pump();
auto *s=first<SequencerWidget>(window);if(!s)throw std::runtime_error("Sequencer missing");
auto reset=[&](){window.reset_layout();check(window.open_project(project.c_str()));window.resize(0,20,1400,900);tab("Sequencer");s->select(0,0);s->position(0);check(window.set_loop(3840,7680,true));window.fit_song();pump();};
auto audio_velocity_capture=[&]() {
reset();
char source[4096]; if(!realpath("examples/projects/original-pluck.wav",source)) throw std::runtime_error("Missing audio source");
check(window.create_track(false)); check(window.import_audio(source));
uint64_t track=s->selected_track();
check(s->rename_selection("Pluck audio")); check(s->repeat_selection(7));
for(int i=0;i<5;++i) check(s->move_track(-1));
check(s->set_zoom(192)); s->expand_track(track,true); s->expand_track(0,true);
pump();
// Use the real lane gestures, including its undo/publication callbacks.
for(auto point:{std::pair<int,int>{0,65},{48,50},{96,72},{192,58},{288,65},{384,50},{576,68},{768,65}}) {
int ox=Fl::e_x,oy=Fl::e_y,ok=Fl::e_keysym;
Fl::e_x=s->x()+s->header_width()+point.first; Fl::e_y=s->track_top(0)+56+point.second;
Fl::e_keysym=FL_Button+FL_LEFT_MOUSE;
s->handle(FL_PUSH); s->handle(FL_RELEASE);
Fl::e_x=ox; Fl::e_y=oy; Fl::e_keysym=ok;
}
pump(); capture("audio-velocity-expanded");
int ox=Fl::e_x,oy=Fl::e_y,ok=Fl::e_keysym;
Fl::e_x=s->x()+s->header_width()+48; Fl::e_y=s->track_top(0)+56+50;
Fl::e_keysym=FL_Button+FL_LEFT_MOUSE; s->handle(FL_PUSH);
Fl::e_y+=10; s->handle(FL_DRAG);
pump(); capture("audio-velocity-editing");
Fl::e_keysym=FL_Escape; s->handle(FL_KEYDOWN);
Fl::e_x=ox; Fl::e_y=oy; Fl::e_keysym=ok;
};
if(std::getenv("XTC_CAPTURE_AUDIO_VELOCITY_ONLY")) {
audio_velocity_capture(); window.do_callback(); return 0;
}
if(std::getenv("XTC_CAPTURE_VELOCITY_ONLY")) {
reset(); capture("compact-workspace"); capture("sequencer-loop");
check(s->set_zoom(96));
@ -145,7 +174,8 @@ int main(int argc,char **argv) {
}
reset();capture("overview");capture("sequencer-loop");capture("arrangement-markers");
check(s->set_zoom(96)); s->expand_track(0,true); s->expand_track(1,true);
pump(); capture("track-velocity-expanded"); reset();
pump(); capture("track-velocity-expanded");
audio_velocity_capture(); reset();
s->position(4800);check(window.loop_section());capture("section-navigation");
dialog("marker-edit",[&]{s->marker_dialog();});
dialog("ripple-time",[&]{s->ripple_dialog(false);});