# NeuroTunes — Model Card

> **Open source, closed weights.** The code in this repository is released under
> Apache-2.0. The **trained model weights are not published here.** This card
> documents the models, how they are trained and evaluated, and how weights are
> supplied to a running deployment.
>
> **NeuroTunes — Patent Pending.** This is a **research platform for
> educational and research use only. It is not a medical device and does not
> provide medical advice, diagnosis, or treatment.**

---

## 1. Overview

NeuroTunes generates personalized, physiologically-informed therapeutic music
from a short description of a person's current state (e.g. mood, stress, energy,
therapy goal). It is built for two audiences: clinicians / music therapists
running consent-gated studies, and ML researchers who want a programmatic,
reproducible platform.

The system is composed of three trained components plus a deterministic music
renderer:

| Component | File | Role |
|---|---|---|
| **EmotionNet** | `mlServer/neurotunes_model.py` | Maps a preprocessed profile to a 4-dim affective state `[arousal, valence, focus, calm]` (served v4 `MODEL_INFO.json` `outputs`), then to clinical music parameters. |
| **RewardNet** | `mlServer/reward_model.py` | RLHF reward model trained on user/clinician feedback; drives periodic retraining. |
| **Federated / site models** | `mlServer/site_client.py`, `federation/` | Local site models aggregated by a federated coordinator with secure aggregation + differential privacy. |
| **Music generator** | `mlServer/music_generator.py` | Deterministic renderer: binaural-beat encoding + parameterized synthesis (not a learned generative audio model). |

## 2. Intended use

- **In scope:** research and education; generating candidate therapeutic audio
  for study; collecting RLHF feedback; federated / multi-site ML research;
  reproducible experiments via the `/api/v1` API and Python SDK.
- **Out of scope / prohibited:** clinical diagnosis or treatment; unsupervised
  use as a medical intervention; any use implying regulatory clearance. There is
  no FDA/CE clearance and none is claimed.

## 3. Architecture (shapes are fixed in code)

- **EmotionNet:** MLP `input → 64 → 32 → 4`, ReLU hidden, `tanh` output
  (bounded affective descriptors).
- **RewardNet:** MLP `input → 128 → 64 → 32 → 1`, ReLU + dropout (0.3).
- **Distillation (research):** a compact student is distilled from a larger
  teacher — teacher ≈ **202,972** parameters → student ≈ **4,196** parameters
  (see `research-papers/distillation/distill.py`).

## 4. Training data

- **Public affective-music / physiological datasets** used for pretraining and
  transfer: DEAM, PMEmo, EmoMusic, DEAP, combined with pretrained audio
  embeddings (MERT / CLAP / OpenL3).
- **Synthetic profile data** (`data/synthetic_patient_data.csv`) for the
  bootstrap EmotionNet when no trained artifact is present.
- **RLHF feedback** collected through the platform (de-identified) via the
  `rlhf_training_data` view; retraining requires a minimum of **50** feedback
  records before a run is eligible.

> **Data-leakage note (honest):** an earlier synthetic generator produced a
> degenerate target correlation (|corr| ≈ 1.00). This was fixed by
> `research-papers/distillation/generate_dataset_v2.py`, which reduces max feature/target
> correlation to ≈ 0.80. Reported metrics use the corrected data.

## 5. Evaluation (honest, reproducible)

- **Reward model:** held-out **R²** and **MSE** plus **5-fold cross-validation**
  (mean ± std) are computed by `reward_model.py` and persisted to
  `reward_metrics.json`.
- **Promotion gating (champion/challenger):** a retrained challenger is promoted
  only if its held-out R² ≥ champion R² − tolerance; otherwise the champion is
  restored (rollback).
- **Distillation result:** student trained on hard labels reached R² ≈ **0.917**;
  a distilled student reached R² ≈ **0.987**. This **+7.6% gain was obtained only
  via transfer / pretraining on public data** — plain self-distillation showed no
  gain. Reported as a research result, not a clinical claim.
- **Federated privacy:** secure aggregation (HE + Shamir) and differential
  privacy (ε = 1.0, δ = 1e-5) are implemented and verified **single-node /
  simulated**. Real cross-site empirical validation is **future work** and
  requires partner institutions.

Numbers above are reproducible from the code and the `research-papers/distillation/`
scripts; they are **not** to be read as validated clinical efficacy.

## 6. Safety & limitations

- A parameter **safety validator** (`mlServer/safety_boundaries.py`) hard-clamps
  tempo, binaural frequency, duration, and unit-interval descriptors, and emits
  recommended-range warnings, before any track is returned.
- A basic **IRB / informed-consent gate** (`verify_consent()` +
  `/generate_music_irb`) records a protocol reference with consent-gated
  sessions. In production research deployments this is required; the public demo
  console makes it optional so the platform is easy to try.
- Limitations: small models; synthetic-data bootstrap; no validated clinical
  outcomes; music renderer is parametric (not a learned audio generator);
  condition-specific adapters are not yet trained (see `docs/ROADMAP.md`).

### 6a. Clinical parameter tables and deployed-model scope (added Oct 2026)
- **Parameter tables are unvalidated defaults.** The goal tempo windows, diagnosis→binaural bands, melodic range/movement and instrument tables in `music_generator.py` are *literature-informed engineering defaults, pending review by a board-certified music therapist (MT-BC/NMT)*. They are not a validated clinical protocol.
- **Precedence today (`mapping_mode = affect_unbounded`).** The affect map (from the EmotionNet state) sets tempo, mode, instruments, dynamics and note density. It **overrides** the goal tempo window, the diagnosis instrument table and the severity/age dynamics. Every response reports this per track in `music_params.provenance`. Diagnosis still acts through the binaural band, melodic range and melodic movement.
- **Before Oct 3 2026 (commit dc8474b), diagnosis tables never fired for UI diagnosis strings** (key mismatch, e.g. `"Parkinson's"` vs `parkinson`). All tracks generated before that commit, including the Stage-2 critic grid, were diagnosis-blind on binaural band, melodic range and movement.
- **The deployed server model is not the paper's 89-feature EmotionNet v3.** Since Oct 2026 the server loads several selectable EmotionNet versions from the `netrai/neurotunes-weights` repo (`WEIGHTS_ROOT`). The production default is **v4.2**: the app's 8 inputs plus optional `mood`, 14 conditions, trained on the paper's 291K synthetic set. The paper's v3-research model (89 features, about 8k parameters, R² 0.938) is in `research/dataset_v3/paper_final/` and is **not** served. Unseen categorical inputs are reported in `music_params.model_unknown_inputs`. (Before Oct 2026 the server loaded a single 29-feature, 4-condition model from `mlServer/models/`.)

### 6b. Calm → music mapping: coefficient, thresholds, measured behaviour (added Oct 2026)

Since KPNS #5 the renderer uses EmotionNet's 4th output (**calm**) in `MusicGenerator._apply_calm_mapping()`. Every value below is an **engineering default, pending MT-BC review**. None comes from a published protocol, and none is validated on listeners.

| Parameter | Rule (`c_eff` = effective calm, range −1…+1) | Basis |
|---|---|---|
| Reverb wetness (MIDI CC91) | linear: 0.10 at `c_eff = −1` → 0.45 at `+1` | engineering default |
| Articulation | staccato `< −0.30` (durations ×0.75); legato `> +0.30` (×1.25); else normal | engineering default (symmetric band around neutral) |
| Binaural hint | alpha (8–13 Hz) `> +0.40`; beta (13–30 Hz) `< −0.40`; else none | thresholds: engineering default. Band edges: standard EEG band definitions |
| Safety | staccato skipped when an arousal cap is active; hint skipped when the band is locked by a diagnosis protocol or override; never changes `render_arousal` / `arousal_cap` | design rule, covered by `test_calm_mapping.py` |

**Calm↔arousal partial-out coefficient (`CALM_AROUSAL_PARTIAL_R`).** KPNS #5 (commit `27082eb`) shipped `c_eff = c + 0.7·a` for v2/v4. The **−0.7** was an *assumed* value. It was not in the project spec and was not estimated from NeuroTunes data. It has now been **measured** on the served weights with `mlServer/research/calm_arousal_check.py` (n = 20,000 random UI-reachable inputs, seed 42). Training labels are not shipped with the weights, so this measures model *outputs*:

| Model | r(calm, arousal) | OLS slope | calm p5 / p50 / p95 | legato | staccato | alpha hint | beta hint |
|---|---|---|---|---|---|---|---|
| v2 | +0.18 | +0.21 | −0.83 / 0.07 / 0.92 | 41.1% | 36.2% | 36.9% | 32.1% |
| v4 | +0.30 | +0.34 | −0.78 / −0.08 / 0.77 | 30.1% | 38.2% | 24.6% | 32.1% |
| v4.1 | −0.85 | −0.56 | 0.15 / 0.49 / 0.91 | 79.2% | 0.0% | 64.2% | 0.0% |
| v4.2 (default) | −0.89 | −0.53 | 0.16 / 0.56 / 0.96 | 83.2% | 0.0% | 70.9% | 0.0% |

*(Percentages are how often each threshold fires before diagnosis locks and caps apply. They are computed with the current constant, so `c_eff = c` for every model.)*

**Conclusions:**
1. For **v2/v4**, calm is only weakly and *positively* related to arousal. A −0.7 partial-out therefore *created* a strong coupling: r rose to +0.58 / +0.68. **`CALM_AROUSAL_PARTIAL_R` is set to 0.0 (disabled).** Re-run the script before changing it.
2. For **v4.1/v4.2** (music-target models, no partial-out by design), calm is strongly anti-correlated with arousal (r ≈ −0.85 to −0.89). The calm mapping therefore mostly *reinforces* the arousal-driven parameters rather than adding an independent dimension. For music *targets* this is coherent (calm music is low-arousal music), but it should be stated.
3. **The thresholds barely discriminate for the default model.** v4.2 calm almost never goes negative, so staccato and beta never fire, legato fires for ~83% of inputs and the alpha hint for ~71%. Retuning (e.g. per-model percentile thresholds) is a clinical design decision. It is left to MT-BC review and is not changed here.

**Reproduce:** `WEIGHTS_ROOT=/path/to/neurotunes-weights python3 mlServer/research/calm_arousal_check.py 20000`.

## 7. Weights: packaging & distribution

Weights are **not** committed to this public repository. They are maintained
separately and supplied to a deployment at runtime.

- **Storage:** the private repository `netrai/neurotunes-weights` holds the
  trained artifacts. Weights are distributed as **release assets** or via
  **Git LFS** — never as raw blobs in the public repo's history.
- **Layout of the weights repo:** `vX/` = EmotionNet versions (`emotion_net.pth`,
  `preprocessor.pkl`, `columns.pkl`, `MODEL_INFO.json`); `reward/v1/` = RewardNet
  seed (`reward_model.pth`, `reward_scaler.pkl`, `reward_encoders.pkl`,
  `reward_columns.pkl`); `legacy/` = archived files, not loaded.
- **Wiring into a deployment** (`docker-compose-ml.yml`; see `.env.ml.example`):
  - **`NEUROTUNES_WEIGHTS_DIR`** (required, host path): the weights clone, mounted
    read-only at `/weights`. The server sets `WEIGHTS_ROOT=/weights` and offers
    every `vX/` in the UI model picker.
  - **`DEFAULT_MODEL`**: the version used when a request doesn't pick one
    (default `v4.2`; rollback with `v4`).
  - **`NEUROTUNES_DATA_DIR`** (required, host path): writable runtime state.
    `models/` is mounted at `/app/models` (RewardNet, rewritten by retraining),
    `output/` at `/app/output` (generated tracks), and `logs/` at `/app/logs`.
    Seed it once with `scripts/init_neurotunes_data_dir.sh`, which copies
    `reward/v1` and never overwrites existing files.
  - Example:
    ```bash
    export NEUROTUNES_WEIGHTS_DIR=$HOME/projects/netrai/neurotunes-weights
    export NEUROTUNES_DATA_DIR=$HOME/neurotunes-data
    scripts/init_neurotunes_data_dir.sh
    docker compose -f docker-compose-ml.yml up -d --build
    ```
  - `WEIGHTS_PATH` (legacy single-directory override) should stay empty.
- If no weights are present, EmotionNet bootstraps a fresh model from synthetic
  data so the platform still runs; RewardNet is trained on the first eligible
  retraining run.

## 8. License & citation

- **Code:** Apache-2.0 (see `LICENSE`, `NOTICE`).
- **Weights:** closed; access by arrangement with NETR.AI.
- **Patent:** Patent Pending (U.S. provisional application on file).

If you use this platform in academic work, please cite the NeuroTunes Research
Platform (NETR.AI, 2026). A formal citation will be added with the accompanying
workshop paper.