feat(music): integrate Flow Music & Google Labs MusicFX generation across CLI, API, MCP, and Chrome extension

[Veer376]

Summary

Extends Flow Agent with full Text-to-Music (T2M) and soundtrack generation capabilities powered by Flow Music (flowmusic.app) and Google Labs MusicFX (aisandbox-pa.googleapis.com).

Music generation is now natively accessible across all Flow Agent interfaces: the unified CLI (flow music), OpenAI-compatible HTTP API (/v1/audio/generations and /v1/music/generations), MCP server (generate_flow_music), and Chrome extension bridge.

---

What Changed

1. Chrome Extension Bridge (flow-extension/)

• Host Permissions & Matching: Added host permissions and content-script matches for flowmusic.app and www.flowmusic.app.
• Token Capture: Added automatic token extraction (getOrCaptureFlowMusicToken) from active Flow Music browser tabs via localStorage.
• SSE Stream Processing: Implemented server-sent events (SSE) stream consumption for /__api/messages/{job_id}/stream to reliably capture asynchronous clip_id events.
• API Routing: Routes Flow Music API requests (conversation, stream, clips) through the extension with appropriate origin and bearer credentials.

2. Flow Engine & Generator (flow_engine/generators/t2m.py, bridge.py, config.py)

• T2M Generator (t2m.py): Submits conversation requests, processes streamed generation events, fetches clip metadata (audio URL, WAV URL, title, duration), and downloads the resulting audio tracks.
• Fallback Ladder: Automatically falls back to Google Labs MusicFX endpoints (/v1:runMusicFx, /v1:runSoundDemo, /v1/sound:generate, /v1/music:batchGenerateMusic) if Flow Music is unavailable or if explicitly requested.
• ReCAPTCHA & Session Refresh: Re-authenticates and forces a tab reload when encountering reCAPTCHA or unauthenticated responses.
• Config & Constants: Added music endpoint definitions and supported duration presets (MUSIC_DURATIONS = [10, 30, 50, 70], default: 30s).

3. HTTP API & Request Models (flow_server/)

• Endpoints: Added POST /v1/audio/generations and alias POST /v1/music/generations.
• Request Model: Added MusicGenerationRequest with prompt, duration, loop, n (1-4 variations), seed, and response_format (url or b64_json).
• Idempotency & History: Full integration with Idempotency-Key headers to prevent duplicate generation requests, and automatic tracking into persistent history.json.

4. Unified CLI (main.py)

• Added flow music command:

  flow music "a cinematic ambient synth soundtrack" --duration 30
  flow music "relaxing lo-fi hip hop beat" --loop --output bgm.mp3
  flow music "epic orchestral intro" --duration 50 --count 2 --seed 42 -o ./soundtrack.mp3

5. MCP Server (flow_server/mcp_server.py)

• Added generate_flow_music tool enabling AI assistants (Claude, Cursor, Windsurf, Antigravity) to generate music tracks and ambient soundscapes with configurable durations, looping, count, and seeds.

6. Media Sniffing & History Registry (media_types.py, media_history.py)

• Extended signature-based byte sniffing to detect MP3 (ID3 tags and sync frames), WAV, M4A, FLAC, OGG, and AAC containers so output filenames and MIME types match audio payload bytes.

7. Documentation (README.md, .gitignore)

• Updated README with flow music CLI examples, generate_flow_music MCP documentation, and HTTP API endpoint details.
• Added audio extensions (*.mp3, *.m4a, *.wav, *.ogg, *.flac, *.aac) to .gitignore.

---

Verification & Tests

• Unit Test Suite: Added comprehensive test suite in tests/test_music_generation.py (13 tests) covering:
  • Audio MIME and signature sniffing for all common audio formats
  • Multi-step Flow Music streaming pipeline resolution
  • Automatic fallback on error
  • Base64 and URL audio downloads
  • API route validation and MCP tool schema registration
• Regression Suite: All 72 tests passing (uv run pytest -k "not test_ext_http_api").
• Linter: Clean uvx ruff check . --select F,E9 with 0 warnings or errors.
• Manifest & Extension Syntax: manifest.json and background.js syntax verified.
• Backward Compatibility: Fully backward-compatible; existing image and video generation pipelines are completely unaffected.

[dickyindra]

+1