Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,55 @@ for i in range(len(result.audio)):
default) that's the same single-entry list as before this option existed, and
the top-level `result.title` stays an alias for `result.audio[0].title`.

### Stems (async)

`stems=True` also splits the generated music into four stems — **drums**,
**bass**, **vocals** and **other** — alongside the normal output. It is free
of charge, and an async-only option like `output_format`: `submit()` /
`generate_async()` accept it on both `text_to_music` and `video_to_music`
(an explicit non-async `mode` alongside it raises `SoniloError` locally,
same as the other async-only options). On `video_to_music` it splits the
**generated** music, never the video's own audio.

```python
result = client.text_to_music.generate_async(
prompt="cinematic orchestral score",
duration=60,
stems=True,
timeout=2400, # separation can run long — see below
)
result.save("track.m4a")
entry = result.stems_for(0) # look up by stream_index, never list position
if entry is not None:
result.save_stem("drums.m4a", which="drums")
result.save_stem("bass.m4a", which="bass", stream_index=0)
if result.stems_error:
print("separation incomplete:", result.stems_error)
```

The result gains two **independent** fields:

- `result.stems` — one entry per stream that separated successfully, each
carrying its `stream_index` and the four stems as media objects
(`url` / `content_type` / `file_size`). Look entries up by `stream_index`,
never by list position — the list can be shorter than `audio` when some
streams failed to separate; `result.stems_for(stream_index)` does the
lookup, and `save_stem(path, which=..., stream_index=...)` downloads one
stem. The stems normally follow `output_format`; each stem's
`content_type` reports what was actually delivered.
- `result.stems_error` — present when separation failed wholly or in part,
or was skipped. It can appear **alongside** a partial `stems` list, so
never treat it as "no stems" — check `stems` itself for what did come
back.

Separation runs **after** generation: it typically adds 2-6 minutes to the
wait and gives up after 30. The SDK's default wait is `DEFAULT_WAIT_TIMEOUT`
(600 seconds), which a legitimate stems task can outlive — pass a longer
`timeout` explicitly (2400 seconds covers the separation ceiling on top of a
normal generation, and is what the CLI uses), or prefer `submit()` plus your
own `client.tasks.wait(...)`, the same advice as for dubbing. A timed-out
wait only stops waiting; the task keeps running server-side.

### Prompt influence

`prompt_influence` (0-1, API default `0.5`) sets how strongly the generated
Expand Down
2 changes: 2 additions & 0 deletions context7.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
"Read the API key from the SONILO_API_KEY environment variable by default; pass api_key= explicitly only when the caller has a reason to override it.",
"text-to-music and video-to-music: use generate() (streaming) for short synchronous tracks. It does NOT support output_format=\"wav\", isolate_vocals, preserve_speech or ducking.",
"For those options use generate_async(), or submit() + client.tasks.wait(parser=parse_music_result). isolate_vocals/preserve_speech/ducking are video-to-music only; text-to-music takes output_format alone.",
"stems=True on text_to_music / video_to_music submit()/generate_async() splits the GENERATED music (never a video's own audio) into drums/bass/vocals/other. Free, async-only; separation adds 2-6 min (30-min cap) — pass timeout=2400, not the 600s default.",
"Look MusicResult.stems entries up by stream_index (stems_for(i), save_stem(which=, stream_index=)), never list position — the list can be shorter than audio. stems_error can accompany a PARTIAL stems list; never treat it as \"no stems\".",
"text-to-sfx and video-to-sfx are always async: submit() returns a task and generate() (or tasks.wait()) polls it to completion. Neither has a streaming variant.",
"client.video_to_sound and client.video_to_video_sound score one clip with a music bed AND sound effects in a single call \u2014 prefer them over chaining video-to-music with video-to-sfx, which is charged twice.",
"Both sound endpoints are async-only and take identical options: music_prompt and sfx_prompt (NOT a single prompt), preserve_speech, ducking. video_to_sound returns mixed audio; video_to_video_sound returns the video with that audio muxed in.",
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "sonilo"
version = "0.14.0"
version = "0.15.0"
description = "Official Python client for the Sonilo API"
readme = "README.md"
license = "MIT"
Expand Down
25 changes: 23 additions & 2 deletions sonilo-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,8 @@ production sign-in coexist without overwriting each other.
### Notes

- `text-to-music` / `video-to-music` stream a short `.m4a` by default. `--format wav`,
`--preserve-speech`, `--variants` above 1, and the legacy alias `--isolate-vocals` each switch
to the async submit-and-poll path.
`--preserve-speech`, `--variants` above 1, `--stems`, and the legacy alias `--isolate-vocals`
each switch to the async submit-and-poll path.
- `text-to-sfx` / `video-to-sfx` are always async; `--format` accepts `wav|mp3|aac|flac`.
- Output defaults to `./output.<ext>`; override with `--output`.

Expand Down Expand Up @@ -157,6 +157,27 @@ not sent at all and the API's own 0.5 default applies; `--prompt-influence 0` is

sonilo video-to-music --video clip.mp4 --prompt "tense synths" --prompt-influence 0.8

### Music stems

`--stems` on `text-to-music` and `video-to-music` also splits the generated music into four
stems — drums, bass, vocals, other — saved next to the main output with the stem name spliced
before the extension (`take.m4a` → `take.drums.m4a`, and per variant with `--variants` above 1:
`take.0.drums.m4a`). It is free of charge, and forces the async path. On `video-to-music` it
splits the *generated* music, never the video's own audio.

sonilo text-to-music --prompt "warm lo-fi piano" --duration 60 --stems --output take.m4a
# writes take.m4a, take.drums.m4a, take.bass.m4a, take.vocals.m4a, take.other.m4a

- Separation runs **after** generation and typically adds 2-6 minutes to the wait; the CLI
waits up to 2400 seconds on these runs (covering the separation service's own 30-minute
ceiling, the way dubbing's `--timeout` covers its backend's). If the wait still times out,
the task keeps running server-side — resume it with `sonilo tasks wait <task-id>`.
- Separation can also come up short without failing the run: streams that did not separate are
reported on stderr (the API's `stems_error`), while the stems that did come back are still
saved — a partial result is not an error exit, and the main output is always written.
- Not the same flag as `--stem` on the sound commands, which saves layers those endpoints
already return; `--stems` *requests* a separation the API would not otherwise run.

### Scored video

`video-to-video-music` and `video-to-video-sfx` are the video-out counterparts of `video-to-music`
Expand Down
4 changes: 2 additions & 2 deletions sonilo-cli/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ build-backend = "hatchling.build"

[project]
name = "sonilo-cli"
version = "0.13.0"
version = "0.14.0"
description = "Command-line interface for the Sonilo API: generate music and sound effects from text or video"
readme = "README.md"
license = "MIT"
requires-python = ">=3.9"
authors = [{ name = "Sonilo AI" }]
dependencies = ["sonilo>=0.14.0,<0.15"]
dependencies = ["sonilo>=0.15.0,<0.16"]
keywords = ["sonilo", "cli", "music", "sfx", "text-to-music", "video-to-music", "ai"]

[project.urls]
Expand Down
2 changes: 1 addition & 1 deletion sonilo-cli/src/sonilo_cli/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
__version__ = "0.13.0"
__version__ = "0.14.0"

__all__ = ["__version__"]
74 changes: 72 additions & 2 deletions sonilo-cli/src/sonilo_cli/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

from sonilo import Sonilo
from sonilo.errors import APIError, SoniloError
from sonilo.resources.tasks import DEFAULT_WAIT_TIMEOUT

from sonilo_cli import __version__, credentials
from sonilo_cli.login import LoginError, cmd_login, cmd_logout, cmd_whoami
Expand Down Expand Up @@ -291,10 +292,53 @@ def _save_music_variants(result: Any, out: str) -> None:
_wrote(path, path.stat().st_size)


# Matched to the separation service's own ceiling on top of the generation
# wait: separation runs after generation, typically adds 2-6 minutes and gives
# up after 30 (1800s), so the SDK's generic DEFAULT_WAIT_TIMEOUT of 600s would
# abandon a stems task the user has already been charged the generation for —
# the same reasoning as DUBBING_WAIT_TIMEOUT below.
STEMS_WAIT_TIMEOUT = 2400.0

_MUSIC_STEMS = ("drums", "bass", "vocals", "other")


def _save_music_stems(result: Any, out: str) -> None:
"""Save the separated stems of an async music result next to `out`,
`take.m4a` becoming `take.drums.m4a` etc. — the same transform --stem
applies on the sound commands. With several variants the stem files pick
up the entry's own stream_index (`take.0.drums.m4a`), matching the
indexed file _save_music_variants wrote for that stream.

Entries are matched by stream_index, never list position — the list can
be shorter than `audio`. `stems_error` is a warning, not a failure, and
never suppresses the partial stems that did come back: the music itself
succeeded and separation is free, so incomplete stems must not turn the
whole command into an error exit."""
if result.stems_error:
sys.stderr.write(f"sonilo: stem separation incomplete: {result.stems_error}\n")
entries = result.stems or []
if not entries:
if not result.stems_error:
sys.stderr.write("sonilo: task succeeded but returned no stems\n")
return
multi = len(result.audio or []) > 1
for entry in entries:
base = _variant_path(out, entry.stream_index) if multi else out
for stem in _MUSIC_STEMS:
media = getattr(entry, stem, None)
if media is None:
continue
path = result.save_stem(
_stem_path(base, stem, media),
which=stem, stream_index=entry.stream_index,
)
_wrote(path, path.stat().st_size)


def cmd_text_to_music(client: Sonilo, args: argparse.Namespace) -> None:
fmt = args.format
multi = args.variants is not None and args.variants > 1
use_async = args.use_async or fmt != "m4a" or multi
use_async = args.use_async or fmt != "m4a" or multi or args.stems
out = _music_output(args, fmt)
segments = _segments(args)
if use_async:
Expand All @@ -304,8 +348,12 @@ def cmd_text_to_music(client: Sonilo, args: argparse.Namespace) -> None:
segments=segments,
output_format=fmt if fmt != "m4a" else None,
variants_num=args.variants,
stems=True if args.stems else None,
timeout=STEMS_WAIT_TIMEOUT if args.stems else DEFAULT_WAIT_TIMEOUT,
)
_save_music_variants(result, out)
if args.stems:
_save_music_stems(result, out)
else:
track = client.text_to_music.generate(
prompt=args.prompt, duration=args.duration, segments=segments
Expand All @@ -318,7 +366,8 @@ def cmd_video_to_music(client: Sonilo, args: argparse.Namespace) -> None:
fmt = args.format
multi = args.variants is not None and args.variants > 1
use_async = (
args.use_async or fmt != "m4a" or args.isolate_vocals or args.preserve_speech or multi
args.use_async or fmt != "m4a" or args.isolate_vocals or args.preserve_speech
or multi or args.stems
)
out = _music_output(args, fmt)
segments = _segments(args)
Expand All @@ -333,8 +382,12 @@ def cmd_video_to_music(client: Sonilo, args: argparse.Namespace) -> None:
output_format=fmt if fmt != "m4a" else None,
variants_num=args.variants,
prompt_influence=args.prompt_influence,
stems=True if args.stems else None,
timeout=STEMS_WAIT_TIMEOUT if args.stems else DEFAULT_WAIT_TIMEOUT,
)
_save_music_variants(result, out)
if args.stems:
_save_music_stems(result, out)
else:
# prompt_influence rides the streaming path too — it is a generation
# parameter, not a finalize-time one, so it never forces async.
Expand Down Expand Up @@ -675,6 +728,21 @@ def _add_variants(parser: argparse.ArgumentParser) -> None:
)


def _add_stems(parser: argparse.ArgumentParser) -> None:
# Only the two music-generation commands take this — the API accepts it
# nowhere else. Not to be confused with --stem on the sound commands,
# which saves layers those endpoints already return; --stems *requests*
# a separation the API would not otherwise run.
parser.add_argument(
"--stems", action="store_true",
help="Also split the generated music into drums/bass/vocals/other "
"stems, saved next to the output (output.drums.m4a, ...). Free "
"of charge. Forces async, and separation typically adds 2-6 "
"minutes to the wait. Streams that fail to separate are "
"reported on stderr; the rest are still saved.",
)


def build_parser() -> argparse.ArgumentParser:
parser = _Parser(prog="sonilo", description="Command-line interface for the Sonilo API")
parser.add_argument("--version", action="version", version=__version__)
Expand Down Expand Up @@ -728,6 +796,7 @@ def build_parser() -> argparse.ArgumentParser:
p_t2m.add_argument("--async", dest="use_async", action="store_true",
help="Submit and poll instead of streaming.")
_add_variants(p_t2m)
_add_stems(p_t2m)
p_t2m.set_defaults(func=cmd_text_to_music)

p_v2m = sub.add_parser("video-to-music", help="Generate music matched to a video")
Expand All @@ -750,6 +819,7 @@ def build_parser() -> argparse.ArgumentParser:
help="Submit and poll instead of streaming.")
_add_variants(p_v2m)
_add_prompt_influence(p_v2m)
_add_stems(p_v2m)
p_v2m.set_defaults(func=cmd_video_to_music)

p_t2s = sub.add_parser("text-to-sfx", help="Generate a sound effect from a text prompt")
Expand Down
Loading
Loading