# Concept 7: Generative Audio System — Cascade Sound Architecture

**Status:** Canonical spec v1  
**Date:** 2026-05-20  
**Purpose:** Define the full generative audio pipeline from lattice state to on-device synthesis

---

## The Design Premise

Cascade's emoji-native UI means audio is the system's voice. There are no text notifications, no spoken labels — sound replaces language. Every event in the lattice has a sonic signature. The ambient layer (R6) is always playing, shifting with your constellation's state.

All audio is **synthesized on-device** using the Web Audio API. No streaming. No royalties. No CDN latency. The audio is as local and personal as the cart itself.

---

## Audio Engine Architecture

### Core Stack

```
Web Audio API (browser-native)
├── AudioContext
├── OscillatorNode (tone generation)
├── GainNode (volume/envelope)
├── BiquadFilterNode (frequency shaping)
├── ConvolverNode (reverb/space)
├── DynamicsCompressorNode (dynamics)
└── AnalyserNode (visualization data)
```

No external libraries. Pure Web Audio API, all synthesis.

### Three Audio Layers

| Layer | Description | When Active |
|-------|-------------|-------------|
| **Ambient** | Continuous generative score, shifts with lattice state | Always on (can be muted) |
| **Event** | Momentary sounds for ring events, signals, seeds | Triggered by specific events |
| **Feedback** | Micro-confirmations for UI interactions | Every tap, swipe, select |

---

## Lattice State → Musical Parameters

The user's constellation state maps to a set of musical parameters that drive the generative engine.

### Parameter Mapping Table

| Lattice State | Musical Parameter | Range | Effect |
|---------------|-------------------|-------|--------|
| Ring positions filled | Base frequency | 110Hz (sparse) → 440Hz (full) | Deeper = fewer bonds, Higher = more |
| R1 count (0-6) | Chord complexity | Triad → 7th chord | More bonds = richer harmony |
| R2 archetypes active | Scale mode | Dorian → Mixolydian → Lydian | Varies per archetype energy |
| R3 vector count | Rhythmic density | Sparse (1 vector) → driving (5+) | More vectors = more rhythmic pulse |
| Cart composition | Timbre | Warm (food-heavy) → bright (activity) → deep (habitat) | Filters applied to oscillators |
| Music items in cart | Melody direction | Ascending / Descending / Static | Melody contour from music category |
| Time of day | Reverb depth | Morning = dry, Evening = spacious | Convolution reverb wet/dry |
| Season | Key center shift | Winter = minor, Summer = major | Root note + mode |
| R6 ambient warmth | Drone presence | Cool = absent, Warm = present | Sub-bass drone layer |

### Implementation: `LatticeToAudio` Function

```typescript
interface LatticeState {
  r1Count: number;        // 0-6
  r2Active: number;      // 0-12
  r3Vectors: number;     // 0-5
  cartWeights: {
    food: number;        // 0-1
    activity: number;    // 0-1
    habitat: number;     // 0-1
    music: number;       // 0-1
  };
  ambientWarmth: number;  // 0-1 (R6)
  hour: number;          // 0-23
  dayOfYear: number;     // 0-365
}

function latticeToAudioParams(state: LatticeState): AudioParams {
  const baseFreq = mapRange(state.r1Count, 0, 6, 110, 440);
  const chordComplexity = mapRange(state.r1Count, 0, 6, 3, 7);
  const rhythmDensity = mapRange(state.r3Vectors, 0, 5, 0.2, 1.2);
  const warmth = state.cartWeights.food * 0.5 + state.cartWeights.habitat * 0.3;
  const brightness = state.cartWeights.activity * 0.7 + state.cartWeights.music * 0.3;
  const reverbWet = mapRange(state.hour, 6, 20, 0.1, 0.4);
  const keyMode = state.dayOfYear > 150 && state.dayOfYear < 270 ? 'major' : 'minor';

  return {
    baseFreq,
    chordComplexity,
    rhythmDensity,
    warmth,
    brightness,
    reverbWet,
    keyMode,
    dronePresence: state.ambientWarmth > 0.5,
  };
}
```

---

## Ambient Layer — The Continuous Score

### Structure

The ambient layer is a continuously evolving generative piece, built from four sub-layers:

| Sub-layer | Description | Instrument |
|-----------|-------------|------------|
| **Drone** | Sustained root note + fifth, shifts with warmth | 2× saw oscillator, heavy filter |
| **Harmony** | Slow chord progression, 4-8 bar cycles | Triangle oscillators, soft attack |
| **Melody** | Sparse note events, contour from music cart | Sine oscillator, long release |
| **Rhythm** | Very subtle pulse, dense when R3 vectors are high | Filtered noise, extreme low-pass |

### Generative Rules

**Chord progression:**
- Root → IV → V → I (classic cycle, adjustable)
- Each chord held for 4-8 bars (randomized within range)
- Transitions are slow cross-fades (2 bar cross-fade)

**Melody events:**
- One note every 8-16 bars, randomized
- Note selection from current scale, weighted toward chord tones
- Octave range limited to one octave above root to one below
- Long release envelope (2-4 seconds) so notes linger

**Rhythmic pulse:**
- BPM: 60-90 (slower when sparse, faster when vectors are high)
- Pulse is filtered noise, not a click — feels like breathing, not a drum machine
- Syncopation added when R1 count is high (more people, more rhythmic interest)

### R6 Ambient Mapping

| R6 Warmth Level | Ambient Sound |
|-----------------|---------------|
| 🔵 Cool | Silence (no drone, only sparse melody) |
| 🟠 Warming | Sub-bass enters, barely perceptible |
| 🟡 Warm | Drone present + sparse melody |
| 💜 Confirmed | Full ambient — drone + harmony + melody |

The R6 warmth indicator IS the ambient volume knob.

---

## Event Sounds

### Ring Confirmation Sounds

| Event | Sound | Synthesis |
|-------|-------|-----------|
| R1 claimed | Warm ascending tone | Sine osc, 220→440Hz over 0.3s, release 0.5s |
| R2 archetype linked | Harmonic cluster | 3× sine at root + third + fifth, slight stagger |
| R3 vector declared | Directional whoosh | Filtered noise sweep, direction encoded in stereo pan |
| R4 Echo confirmed | Resonant bloom | Sine at root + feedback delay, exponential decay 1.5s |
| R5 Wave confirmed | Wave crest | 4 layered sine tones rising in sequence, 0.8s total |
| R6 ambient active | Temperature pulse | Single sustained tone, warmth = higher pitch within range |

### Signal Sounds

| Signal | Sound | Synthesis |
|--------|-------|-----------|
| 💜 Heart detected | Crystalline ping | Sine at 880Hz, very short (50ms), slight reverb |
| ⭐ Star (category match) | Bright two-note | Sine at root then major third, 100ms total |
| 🌙 Moon (complement) | Low warmth note | Triangle at 165Hz, soft attack, 300ms |
| ⏳ Hourglass (time pattern) | Tick-tock feel | Paired pings at 0.3s interval, 440Hz then 330Hz |
| 🎈 Balloon (rare) | Ascending shimmer | 5 sine tones ascending rapidly, sparkle effect |

### Feedback Sounds (UI)

| Interaction | Sound |
|-------------|-------|
| Tap / select | Soft click (10ms noise burst) |
| Swipe | Short noise sweep in direction |
| Confirm | Short ascending two-note |
| Dismiss | Short descending two-note |
| Error | Low buzz (100ms, 80Hz) |

---

## Cascade-Processed Content Audio

When a Fractal Seed carries Cascade-processed media (avatar, cart item image), the audio layer processes it:

### Audio Processing Pipeline

```
Raw file submitted
    ↓
Cascade Server
    ↓ (validates, sanitizes, transforms)
Generated artifact
    ↓
Audio layer receives asset_ref
    ↓
AnalyserNode examines artifact characteristics
    ↓
Resonant note generated to "match" the visual
    ↓
Played when artifact is viewed
```

**Example:** A user's avatar has dominant colors warm orange and deep purple. The audio engine generates a note with harmonic content that complements those colors — warmer timbres for orange, higher frequencies for purple. This is cross-modal matching.

---

## Audio Visualization

The ambient layer drives a visual element — a subtle waveform or spectral display that lives behind the constellation view. It is not foregrounded, but it is always present, showing the user that sound is happening.

```typescript
// Web Audio AnalyserNode for visualization
const analyser = audioContext.createAnalyser();
analyser.fftSize = 256;

function getAudioData(): Uint8Array {
  const data = new Uint8Array(analyser.frequencyBinCount);
  analyser.getByteFrequencyData(data);
  return data;
}

// Use in render loop
// data maps to visual waveform intensity
```

---

## On-Device Synthesis

### Why On-Device

1. **No latency** — events trigger instantly, no network round-trip
2. **No CDN cost** — audio is generated, not streamed
3. **Privacy** — audio state stays on device, never sent to server
4. **Offline** — ambient layer works without connection (events require connection but use cached synthesis parameters)

### Performance Budget

| Scenario | CPU | Memory |
|----------|-----|--------|
| Ambient only | <5% | <20MB |
| Ambient + event | <10% | <25MB |
| Peak (multiple events) | <15% | <30MB |

Web Audio API is highly optimized in modern browsers. The ambient layer is intentionally light — slow attack, low frequencies, sparse events.

---

## Implementation Phases

### Phase 1 — Core (MVP)
- Ambient layer with basic parameter mapping
- 3 event sounds (R1, signal ping, seed received)
- Basic feedback sounds

### Phase 2 — Rich
- Full event sound set (all 6 ring events + all 5 signals)
- Audio visualization
- Cross-modal content processing

### Phase 3 — Sophisticated
- Full chord progression engine
- Melody generation from music cart
- Seasonal key/mode shifts

---

## Status

**Ready for:** Audio engine implementation, sound design brief, Web Audio API prototype

*Concept 7 — Generative Audio System — complete*