1
0
Fork 0
forked from hanez/xtc
xtc/docs/AUDIO_FILES.md

218 lines
13 KiB
Markdown

# Audio clips and recording (Phase 6)
## Import and playback
File/Import WAV (Ctrl+I) loads a WAV into the selected track at the playhead.
With no selected track it creates an Audio track. Add region still creates a
silent placeholder. Imported clips use the existing move/trim controls.
Import stops playback, validates a candidate project, and commits only after
successful backend preparation (or locally when disconnected).
Supported input: little-endian RIFF/WAVE, PCM16/24/32 and IEEE float32, one or
two channels, 1..768000 Hz. Stereo is averaged to mono. Compressed WAV, RF64,
WAVE_FORMAT_EXTENSIBLE, PCM8, other containers and multichannel files are
unsupported. Nonfinite float data and malformed/truncated files are rejected.
There is no new dependency: the bounded C decoder/writer is covered by known
sample, truncation and invalid-codec tests. See decision 0005.
Audio is preloaded before playback: maximum 64 distinct paths per plan,
16777216 mono frames per asset (64 MiB), 67108864 total frames (256 MiB).
Import rejects files over 256 MiB or over 4096 chunks. No streaming decoder,
waveform cache, peak drawing or background load worker exists yet. Preloading
can briefly block the GUI. Candidate compilation retains the old plan until
success; temporary build memory can include both old and candidate buffers.
The callback renders only overlapping clip ranges from memory. Clips on all
tracks are summed through their mix/automation settings into the graph's
**Audio clips** source node.
Import creates that source and connects it to an existing/new output if no
audio source exists. Subsequent routing is explicit: disconnect it or route it
through gain/meter nodes in Modular Graph. Removing the output silences editor
graphs. This source does not monitor JACK input; unconnected ordinary graph
inputs retain their earlier external-input behavior. Track faders/pan/mute
are implemented by the Phase 8 track mixer (see MIXER.md); individual
track graph routing remains deferred.
Linear interpolation converts source sample rate to backend rate. There is no
antialiasing filter, time stretching, looping or fades. Musical durations round
up to a tick on import; playback boundaries and trims use floor tick-to-sample
conversion. A rounded tail pads with silence past the source, and a recording's
placement can differ by less than one tick from the capture start sample.
Changing tempo changes the musical span, not the source playback rate.
Arrange edits stop playback. Play preloads and adopts the current arrangement;
Stop pauses the sample clock. Graph-only edits can still be submitted during
playback. Graph recompilation can reset oscillator phase as before. Playback
does not stop automatically at the arrangement end.
## Assets and recovery
Version 2 projects store external absolute canonical paths, not sample data.
Save the project AND keep its referenced WAV files. Save As changes only the
metadata path; it does not copy assets. Moving/deleting a WAV makes its regions
silent, preserves their IDs/positions and reports the path and unique issue
count on Open/Play. An already loaded plan keeps playing even if its file is
removed; the next preparation detects it. Unreadable, malformed and unsupported
assets are also reported, rather than substituted with unrelated files.
Capacity/OOM is a preparation failure and retains the old plans.
Select the affected clip, then File/Relink selected audio to choose a replacement.
Relinking retains its original timeline/source bounds. The replacement is
validated on the next Play; a shorter replacement pads beyond its end with
silence. Saving retains unresolved references for later repair. Missing assets
are also checked on Open without a working backend, using a temporary plan.
Automatic asset search, content hashes and portable project bundles are deferred.
## Recording
Record (or Audio/Record WAV, Shift+F5) opens a destination chooser, stops playback,
prepares the project, opens a temporary mono float32 WAV and starts JACK.
Connect a source to xtc's JACK input using an external patchbay; no physical or
software route is created automatically. Without a route the take contains
silence. Recording captures raw mono JACK input, not the output/effect mix.
A preallocated two-second SPSC ring transfers complete callback blocks to the
GUI/control thread, which drains it every 100 ms. The producer uses verified
lock-free atomics and bounded copies only. It never allocates, logs, takes a
mutex, opens/writes files or calls FLTK. A slow disk/blocked GUI can overflow
the ring; capture then stops accepting input with a sticky, visible capacity
error rather than hiding gaps or compressing time.
Stop first synchronizes the callback, detaches the recorder, drains remaining
samples, writes the final WAV header, flushes/fsyncs/closes and renames the
same-directory temporary using an atomic no-clobber link; an existing destination is refused. A successful nonempty take is
imported at its capture position on the selected track (or a new Recording
track), then the document becomes unsaved. Save the project after Stop. Seeking
back to the take and pressing Play auditions it. Recording continues existing
arrangement playback, but there is no automatic input monitoring, punch, loop,
latency compensation, multitrack arming or overdub alignment compensation.
Graph/arrangement edits, project file actions, reconnect and close are blocked
until Stop finishes the take. Deactivation failure retains the recorder for a
safe retry. On overflow a finalized partial WAV is retained for manual recovery
and is not automatically added. On import/preparation failure the completed
take remains at its chosen path with an explicit recovery message. I/O errors
are reported; failed temporary data is retained and its recovery path is reported. Recordings
are RIFF-limited to 1073741814 frames, but the smaller playback/import asset
limit still applies: longer takes may need external splitting before import.
There is no crash-recovery journal. The file is mode 0600; parent-directory
fsync and preservation of previous permissions are not implemented.
## Verification
make test includes deterministic WAV/graph samples, resampling/overlap/trim,
malformed inputs, missing/relinked assets, SPSC concurrency/overflow/cleanup,
and record -> save/reopen -> exact playback through the backend callback.
make test-gui covers the workflow and rollback with a fake backend.
make test-gui-jack records/reopens a silent take using real JACK callbacks.
make test-jack-audio routes only two software clients: known source -> recorded
WAV -> saved/reopened project -> graph playback -> capture assertion.
Physical input/output, device latency, slow-disk endurance and server loss
during a real recording remain unverified.
## Phase 13 recording protection
The default recorder and GUI require a new destination, including rejecting
symlinks. Publication also refuses files created during recording. The explicit
`xtc_recorder_create_replace` API retains authorized replacement for callers
that deliberately need it; the GUI never uses it. After a failed finish, copy
`xtc_recorder_recovery_path` before destruction. Disk-write/finalization failures
retain the temporary. Destruction after producer quiescence attempts to drain
and finalize accepted data; normal code should call finish and inspect errors.
The GUI refuses closure while recording and prints the retained/finalized path
on exceptional destruction.
Use `xtc-check --recover-recording TEMP NEW.wav` to copy complete float32 frames
from an interrupted xtc WAV into a new WAV. Source and existing destinations are
never modified. A trailing partial frame is ignored. This handles xtc's canonical
44-byte mono float32 header only, not arbitrary damaged WAVs. Recovery is explicit
and control-thread only; no orphan scan runs during playback.
`make test-recording` simulates 30 minutes at 8 kHz with repeated ring wrapping
and draining, destination races/symlinks, a child-local disk-size limit, prefix
recovery and incomplete/missing asset reopening. `test-recovery` retains process
kill coverage; `test_audio` covers saturation and concurrent drain. This is
virtual-duration testing, not a wall-clock hardware endurance qualification.
Recordings remain RIFF mono float32, bounded to roughly 4 GiB; the preloaded
player still limits each asset to 16M frames. Longer valid takes may require
external splitting before import. No directory fsync/power-loss guarantee is
claimed. WAV destinations are chosen explicitly by the user.
## Phase 22: non-destructive clip editing
Move clips by dragging their body; drag the left/right edge to trim within the
existing source length. Ctrl+D duplicates adjacent to the selected clip. Delete
removes the reference. With an audio clip selected, click the ruler while stopped
to position the cursor strictly inside it, then choose **Edit / Split audio at
cursor**, press S with timeline focus, or use the clip context menu. The right
piece becomes selected. The four-pixel minimum draw width is also selectable for
very short clips; zoom in for precise edge gestures.
**Edit / Audio clip gain and fades** (A with timeline focus, also in the context
menu) takes three numbers: linear gain 0..4, fade-in ticks, fade-out ticks.
Defaults are `1 0 0`; 0 fade disables that explicit fade. Each duration can be up
to the original source length, even after trimming; overlapping long fades
attenuate a short clip without extending it. MIDI clips reject these operations;
silent metadata regions support them for later audio relinking.
These operations are one undo step each, stop playback before model edits and
are refused while recording. Main Edit / Undo and Redo restore the model and
prepare its matching stopped engine plan; Play uses the latest settings.
Splitting and duplication allocate metadata IDs only: no media copies. All
operations preserve original WAV bytes, including Delete and undoing recording.
No destructive audio processing or file deletion command is supplied.
A split retains the original source length/path/name/gain on both pieces. Its
right offset advances by the left length, keeping trims reversible within the
same source. Left keeps its ID and fade-in; right gets a fresh ID and fade-out.
Inner explicit fades are cleared. Gain/fades are retained on duplication.
Moving does not alter offsets; trimming changes only timeline/source bounds and
makes explicit fades follow the new audible boundaries.
The renderer maps tick boundaries by the existing absolute floor conversion,
linearly resamples the source and never reads beyond its samples. Its audible
end is the earlier of the trimmed end and actual WAV end (imports round metadata
length up to a tick). A fade uses `age / fade_in_frames` at the start and
`remaining_frames / fade_out_frames` at the end, capped at 1; overlapping fades
use the smaller value. The first/last audible samples are zero when their
respective fade has a positive sample duration. Durations rounding to zero
samples are disabled. A one-sample clip with any active fade is silent. Explicit
fade gain multiplies the existing optional safety envelope, then clip gain,
then track gain/pan/automation. No limiter is added; gains/overlaps may clip.
The GUI retains its 5 ms boundary safety fades, including new split edges, so
cuts can introduce a brief dip rather than an abrupt discontinuity. The raw C
plan still defaults to zero safety fade for exact-sample/offline consumers;
callers requesting raw cuts accept possible clicks. Tick quantization can place
an edit up to one output sample away from a separately rounded boundary; this
is not sample-resolution audio editing. Manual listening remains pending.
Time-stretching, independent pitch shifting and crossfade tools are future work;
no suitable algorithm/quality target or new dependency has been selected.
### Cached waveform display
Timeline audio clips show mono min/max peaks from an immutable C summary,
shared by reference path across split/duplicated clips. Up to 64 paths each
cache at most 1024 bins (fewer for short files). It uses playback's WAV parser,
downmix and file/frame bounds; creation temporarily loads one WAV then frees
its sample storage. Bins cover every source sample, including a final partial
bin. The display follows source offsets and current tempo; it shows **raw source
peaks before clip/track gain and fades**, clamped visually to +/-1. It is an
arrangement aid, not a sample editor or loudness meter.
Loading/rebuilding is synchronous on the GUI/control thread during project
binding or edits that introduce media paths. Large files/slow disks can delay
that UI operation. Draw/redraw, scrolling and playback never load a WAV or build
peaks; the audio callback does not use this cache at all. Missing/invalid assets
show no waveform and keep the existing playback issue reporting. Failures are
cached, removed paths are evicted, and Open rebuilds caches, including changed
external files. No persistent disk cache, file watcher or background loader is
claimed. Source files should remain unchanged during a session; reopen after
external replacement. Cache allocation failure omits previews without changing
music metadata.
`make test-audio-editing` verifies exact split/gain/fade playback, odd blocks,
seeks, shortened media tails, one-frame files, source/project limits, capacity
rollback, duplication/fade inheritance, persistence/invalid metadata, undo, peak
bins after source removal and byte-identical source preservation. GUI tests
cover short-clip selection, shared previews, split/duplicate/delete and undo.