# Browser instrument separation (inactive experiment)

The active audio-to-MIDI pipeline uses the same Basic Pitch model for a bass-reduced upper-note pass and an original-audio bass pass. It does not load these files or download the Demucs model. The implementation below is retained for reference.

The experimental pipeline loads `browser-separation.js`, which probes WebGPU, resamples audio
to stereo 44.1 kHz and runs `separation-worker.js` in a module worker. The worker
returns four aligned stereo stems. Basic Pitch transcribes vocals, bass and
accompaniment separately; the existing percussion detector processes drums.
This does not classify piano versus other solo instruments, and accompaniment
is not guaranteed to contain only one instrument.

## Dependencies and provenance

- `processor.js`, `fft.js`, `constants.js`: timcsy/demucs-web, MIT,
  commit `617385fbf5883b04a2e6f474be63ed4d93f4e44c` (package version 1.0.2).
  https://github.com/timcsy/demucs-web
- Local processor changes: fix segment progress for short clips and final chunks,
  preserve the first sample's overlap weight, validate both model output branches,
  and dispose input/output tensors between segments.
- ONNX Runtime Web 1.23.0: loaded from jsDelivr, with one WASM thread and WebGPU
  selected for inference. No cross-origin isolation headers are required for this
  single-thread setup. See `ONNXRUNTIME-LICENSE.txt` (MIT).
- Four-stem HTDemucs ONNX weights: 180,534,758 bytes, hosted at
  https://huggingface.co/timcsy/demucs-web-onnx/tree/92e33df61cfc9eb820272aaa62d2ef6dcf4d950d
  and based on Meta's Demucs. Upstream Demucs is MIT licensed; see
  https://github.com/adefossez/demucs and `DEMUCS-LICENSE.txt`.
  No weights from bakkot/demucs-js are used.

The runtime version and model revision are pinned in `separation-worker.js`.
The model is fetched directly with CORS, checked for expected byte length, then
cached in Cache Storage when available. A cache/quota failure does not prevent
inference. Models are not included in the repository or deployment artifact.
Only library/model downloads use the network; audio is never sent for separation.
The existing, separately chosen sample donation remains an upload.

## Limits and fallback

- Five minutes maximum for separation, limiting the full-length stem buffers.
  Inference runs on overlapping 7.8-second segments in a worker; full output
  buffers still grow with recording duration.
- Missing WebGPU (including a missing adapter), model/runtime failure, processing
  failure, an idle stage lasting three minutes, or unusable stem transcription
  falls back to the existing Basic Pitch conversion of the original audio.
- A persistent notice states the reason and explains that results may be less
  accurate, especially for full songs. A new conversion clears the old notice.
- No slow CPU-only neural separation fallback. Workers are terminated after each
  attempt to free their GPU/runtime resources before Basic Pitch runs.
- Dropped files are processed sequentially to avoid concurrent model allocation.

Deploy these files alongside the updated audio-to-MIDI script and Razor view.
Serve over HTTPS (or localhost). CDN and model hosts must be reachable. Conversion
quality and real-device GPU performance have not been benchmarked in this change.