diff --git a/developer-guide/resources/coding-agents.mdx b/developer-guide/resources/coding-agents.mdx index f453002e..44c8a74b 100644 --- a/developer-guide/resources/coding-agents.mdx +++ b/developer-guide/resources/coding-agents.mdx @@ -76,7 +76,7 @@ Once installed, ask your agent in plain language; it will use the correct client ## Or connect via MCP -The [Fish Audio MCP server](/overview/mcp) at `https://api.fish.audio/mcp` gives your agent tools instead of docs: it can search the voice library, generate speech, and transcribe audio with your Fish Audio account (OAuth sign-in, no API key). See the [MCP server guide](/overview/mcp) for client setup and the tool list. +The [Fish Audio MCP server](/overview/mcp) at `https://api.fish.audio/mcp` gives your agent tools instead of docs: it can search the voice library, clone voices, generate speech, transcribe audio, create images and videos, and produce Story Studio projects with your Fish Audio account (OAuth sign-in, no API key). See the [MCP server guide](/overview/mcp) for client setup and the [tool list](/overview/mcp/tools) for what the agent can call. This site also exposes [llms.txt](https://docs.fish.audio/llms.txt) and [llms-full.txt](https://docs.fish.audio/llms-full.txt) for agents that fetch docs directly. diff --git a/docs.json b/docs.json index 201fe319..a61f9b61 100644 --- a/docs.json +++ b/docs.json @@ -67,7 +67,11 @@ "overview/platform/create-voice/voice-design" ] }, - "overview/mcp" + { + "group": "MCP Server", + "icon": "plug", + "pages": ["overview/mcp", "overview/mcp/tools"] + } ] }, { diff --git a/llms.txt b/llms.txt index 43c2c725..98d48939 100644 --- a/llms.txt +++ b/llms.txt @@ -7,6 +7,8 @@ - [Coding Assistant Quickstart](https://docs.fish.audio/developer-guide/resources/agent-quickstart.md): Minimal-noise entry point for AI coding assistants with retrieval order, canonical URLs, and task routing. - [Quick Start](https://docs.fish.audio/developer-guide/getting-started/quickstart.md): Generate your first AI voice with Fish Audio in under 5 minutes. - [AI Coding Assistants](https://docs.fish.audio/developer-guide/resources/coding-agents.md): Install the Fish Audio skill so your coding agent writes correct SDK and API code. +- [MCP Server](https://docs.fish.audio/overview/mcp.md): Client setup, credits, and plan requirements for connecting an MCP client to your Fish Audio account with OAuth. +- [MCP Tools](https://docs.fish.audio/overview/mcp/tools.md): What each Fish Audio MCP server tool does, grouped by area. ## API Specs diff --git a/overview/mcp.mdx b/overview/mcp.mdx index 37a567b4..b93365fc 100644 --- a/overview/mcp.mdx +++ b/overview/mcp.mdx @@ -1,16 +1,17 @@ --- title: "MCP Server" -description: "Connect Claude, Cursor, or any MCP client to your Fish Audio account: search voices, generate speech, transcribe audio, and create images and videos" +sidebarTitle: "Overview" +description: "Connect Claude, Cursor, or another MCP client to your Fish Audio account for voices, speech, images, video, and Story Studio" icon: "plug" --- import { PromptCard } from "/snippets/prompt-card.jsx"; -The Fish Audio MCP server gives AI agents direct access to your Fish Audio account: they can browse the voice library, generate speech, transcribe audio, and generate or edit images and videos for you. +The Fish Audio MCP server gives AI agents direct access to your Fish Audio account: they can browse the voice library, clone a voice, generate speech, transcribe audio, generate or edit images and videos, and produce multi-speaker [Story Studio](https://fish.audio/app/story-studio) projects for you. - **Endpoint**: `https://api.fish.audio/mcp` (streamable HTTP) - **Authentication**: OAuth. Your MCP client opens a browser window, you sign in with your Fish Audio account, and choose the team the connection works in. -- **Billing**: usage draws from your plan's package credits, exactly like the [web app](https://fish.audio/app). It does not consume developer API credits. Image and video generation is quoted first, and the agent asks you to confirm the price before it generates. +- **Billing**: usage is billed the same way as the [web app](https://fish.audio/app), not through the developer API. See [Credits](#credits) for what each tool costs. ## Connect @@ -46,11 +47,10 @@ The Fish Audio MCP server gives AI agents direct access to your Fish Audio accou ```bash - codex mcp add fish-audio https://api.fish.audio/mcp - codex mcp login fish-audio + codex mcp add fish-audio --url https://api.fish.audio/mcp ``` - `codex mcp login` opens the browser sign-in. Verify with `codex mcp list`. + Codex opens the browser sign-in when it adds the server. If it doesn't, run `codex mcp login fish-audio`. Verify with `codex mcp list`. @@ -121,17 +121,56 @@ Once connected, ask in plain language and the agent picks the right tools. Click icon="film" prompt="Turn the image you just generated into a short video with a slow push-in, and give me the video URL and what was charged." /> + + In coding agents like Claude Code and Codex, tools compose with the shell: the agent can download generated audio, images, and videos with `curl`, upload local recordings for transcription or reference images for editing, or batch-generate a directory of scripts. +## Tools + +The [tools reference](/overview/mcp/tools) lists what each tool does, by area: + +- [Voices](/overview/mcp/tools#voices): search the voice library and clone voices +- [Speech and transcription](/overview/mcp/tools#speech-and-transcription): text to speech and speech to text +- [File uploads](/overview/mcp/tools#file-uploads): local files for transcription, cloning, and generation +- [Images and video](/overview/mcp/tools#images-and-video): models, estimates, and generation +- [Story Studio](/overview/mcp/tools#story-studio): long-form, multi-speaker audio projects +- [Asset library](/overview/mcp/tools#asset-library): saved files, folders, and avatars +- [Account and feedback](/overview/mcp/tools#account-and-feedback): credits, workspaces, and feedback to Fish Audio + + + The tools the server exposes, with the exact tool names and what each one + does. + + +## Credits + +| Tool or area | Cost | +| ------------------------------------------------------- | ---------------------------------------------------------------------- | +| Speech (`text_to_speech`) | 1 credit per UTF-8 byte of text, audio tags included | +| Story Studio audio (`studio_generate`, `studio_export`) | 1 credit per UTF-8 byte of text; blocks with up-to-date audio are free | +| Transcription (`speech_to_text`) | By length of audio; `flash` costs less than `pro` | +| Images and video | The price in the estimate | + +In UTF-8, English letters, digits, and basic punctuation are one byte each, and most Chinese, Japanese, and Korean characters are three. Voice search and cloning, uploads, Story Studio import and editing, estimates and quotes, and the asset library are free. Speech and Story Studio audio in another team's [professional voice clone](/overview/platform/create-voice/professional-voice-clone) cost double, and failed generations are refunded. + ## Good to know -- Add audio tags in square brackets inside the text to control delivery: `[whispering]`, `[excited]`, `[laughing]`. Tags are performed, never spoken. +- Add [audio tags](/developer-guide/core-features/emotions) in square brackets inside the text to control delivery, such as `[whispering]`, `[excited]`, or `[laughing]`. - If no voice is specified, a curated default voice for the requested language is used. -- Local files can be transcribed too. The agent uploads them to a temporary slot that is deleted automatically after 7 days. -- Before an image or video is generated, the agent requests a credit estimate and asks you to confirm it. The server re-checks the price when the job is submitted and never charges more than the amount you approved. -- Video generation needs a Plus plan or higher. Free plans have a daily image allowance and a smaller image count per request; the estimate reports these limits before you are asked to approve anything. -- Reference images for editing or image-to-video are JPEG, PNG, or WebP files up to 20 MB. The agent uploads them for you, and an image it just generated can be reused as input without downloading it. -- Generated audio, images, and videos are returned as permanent URLs; failed generations are refunded. -- A connection is bound to the team you chose at sign-in. To work in another team, add the server again and choose that team. +- Local files for transcription or cloning go to temporary storage that is deleted after 7 days, while files you upload as inputs for image or video generation are kept in your workspace. +- The agent asks you to approve an image or video estimate before it generates; if the credit price changes before it submits, nothing is charged and the agent asks you again. +- Every account sees the same tools, and plan requirements are checked when a tool runs. +- Video (including lip sync and talking avatars) needs a Plus plan or higher, and Story Studio export needs a paid plan. +- Free plans limit how many images you can generate, and your [plan](https://fish.audio/app/billing) sets how many private or unlisted voices you can keep. +- The estimate or quote tells you when a request is over your plan, before anything is charged. +- New voice clones are unlisted by default, so anyone with the voice ID can use them; ask for a private voice to keep one to your workspace. +- Generated audio, images, and videos are returned as permanent links that anyone with the link can open. +- A connection is bound to the team you chose at sign-in; to work in another team, add the server again and choose that team. +- Voice, speech, transcription, and Story Studio tools use your team's default workspace, while image, video, and asset library tools use the workspace you choose (`list_my_workspaces` lists them). +- Tools can change between releases; reconnect your client to refresh its tool list. diff --git a/overview/mcp/tools.mdx b/overview/mcp/tools.mdx new file mode 100644 index 00000000..fccf2c62 --- /dev/null +++ b/overview/mcp/tools.mdx @@ -0,0 +1,104 @@ +--- +title: "MCP Tools" +sidebarTitle: "Tools" +description: "What each tool on the Fish Audio MCP server does, grouped by area" +icon: "wrench" +--- + +These are the tools the [Fish Audio MCP server](/overview/mcp) exposes. Naming one in a prompt helps when you want to steer a step, for example "List my own voices with `search_voices`." + +The agent gets each tool's arguments from the server when your client connects. Costs are in [Credits](/overview/mcp#credits), and plan requirements in [Good to know](/overview/mcp#good-to-know). + +## Voices + +| Tool | What it does | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `search_voices` | Searches the public voice library, browses popular voices for a language, or lists your own workspace's voices (`self`). | +| `get_voice` | Shows one voice's description, languages, tags, popularity, and a sample. | +| `create_voice_clone` | Creates a voice from recordings of one speaker ([what makes a good sample](/features/voice-cloning)) once you confirm you have the rights to it. Free. | +| `verify_voice_clone` | Finishes a clone that needs speaker verification, from a recording of the speaker reading a short passage Fish Audio provides. | + +## Speech and transcription + +Long text may come back as several clips. How much text one call can take depends on your plan, and `get_credit_balance` shows the limit. + +| Tool | What it does | +| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `text_to_speech` | Speaks text with a voice, or the default voice for the language, and returns a permanent audio link. | +| `speech_to_text` | Transcribes audio from a direct HTTPS link to the file, such as one from `get_upload_url`; `pro` is the default and accepts a language hint, and `flash` costs less. | + +## File uploads + +Both tools return an upload link, and the agent then uploads the file to it. In clients that cannot upload files themselves, such as claude.ai, the agent gives you a link to upload the file from your browser instead. Anyone with the link can upload to it until it expires, so don't share it. + +| Tool | What it does | +| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `get_upload_url` | Creates a link for uploading a local file to temporary storage, so the agent can pass the file to another tool, for example a recording to transcribe or samples to clone. | +| `create_media_upload` | Creates a link for uploading an image, video, or audio file to your workspace as a generation input; `get_media_model` lists the files each model accepts. | + +## Images and video + +Jobs run in the background, and finished results also appear in your [image and video history](https://fish.audio/app/image-video/history/). The agent can use a result from the same workspace as the input to another generation, for example to animate an image it made or to lip-sync a video to `text_to_speech` audio, without downloading it. + +| Tool | What it does | +| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `list_media_models` | Lists the image and video models, what each can do (for example text-to-image, image edit, upscale, text-to-video, image-to-video, lip sync, or talking avatar), and their prices. | +| `get_media_model` | Shows one model's options, required inputs, and accepted files. | +| `estimate_image_generation` | Prices and checks an image request, including your plan's limits. Free. | +| `estimate_image_edit` | Prices and checks an image edit or upscale, including your plan's limits. Free. | +| `estimate_video_generation` | Prices and checks a video request, including your plan's limits. Free. | +| `generate_image` | Submits the image request you approved. | +| `generate_image_edit` | Submits the image edit or upscale you approved. | +| `generate_video` | Submits the video request you approved. | +| `get_generation_status` | Checks running jobs; finished jobs include their results. | +| `get_generation_result` | Shows the results and charges of one finished job. | + +## Story Studio + +A project holds chapters, and chapters hold blocks (a paragraph or heading), each with its own voice. These tools edit the same projects as the [Story Studio editor](https://fish.audio/app/story-studio), and edits show up in an open editor. Importing and editing are free; audio [costs credits](/overview/mcp#credits), and the agent quotes it first. + +A block needs new audio when it has none or its text, voice, or settings changed. If a block is changed back to an earlier version, `studio_generate` reuses that audio for free. No tool deletes projects or chapters; delete them in the editor. + +| Tool | What it does | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `studio_help` | Shows the Story Studio manual for agents, by topic. | +| `studio_list_projects` | Lists your workspace's projects. | +| `studio_project` | Shows one project's chapters, how many blocks in each still need audio, the voices in use, and the status of an import or export. | +| `studio_create_project` | Creates a project from chapters and blocks the agent writes. | +| `studio_import_document` | Imports a document's text (a novel, script, or article) into a new project, splitting it into chapters, casting voices, and adding audio tags; the project is read-only until the import finishes. | +| `studio_update_project` | Changes the project's name, description, audio settings, room ambience, chapter order, and the default voice and model for new blocks. | +| `studio_read_blocks` | Reads a chapter or specific blocks, optionally with links to their audio, or shows a chapter as a script. | +| `studio_find` | Searches text across a project. | +| `studio_write_blocks` | Edits, inserts, moves, or deletes blocks. | +| `studio_create_chapter` | Adds a chapter. | +| `studio_update_chapter` | Renames a chapter. | +| `studio_recast_speaker` | Switches every block that uses one voice to another voice. | +| `studio_enhance_text` | Returns one block's text polished for speech, with numbers, dates, and symbols spelled out and audio tags added. | +| `studio_generate` | Generates audio for blocks, a chapter, or the whole project, only for the blocks that need it unless asked to redo one. | +| `studio_list_takes` | Lists a block's earlier audio versions (takes). | +| `studio_adopt_take` | Switches a block back to an earlier take that still matches it. Free. | +| `studio_export` | Exports the project or chapters as MP3 or WAV, optionally with subtitles. The agent quotes the credits for blocks that still need audio and starts the export after you confirm. | + +## Asset library + +Your workspace's [Asset Library](https://fish.audio/app/assets) holds saved images, audio, and video in folders, plus avatars, which pair a portrait image with an optional default voice. These tools never delete anything; delete files in the web app. + +| Tool | What it does | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `list_assets` | Lists the files in a folder, or searches across the workspace. | +| `get_asset` | Shows one file's permanent link and a temporary download link. | +| `save_asset` | Saves a file uploaded with `create_media_upload`, or a finished image, video, or `text_to_speech` result, to the library. | +| `update_asset` | Renames or moves a file. | +| `list_asset_collections` | Lists folders and avatars. | +| `get_asset_collection` | Shows one folder or avatar, including an avatar's portrait and default voice. | +| `create_asset_folder` | Creates a folder. | +| `create_avatar` | Creates an avatar; it makes no video itself, but the agent can use the avatar's portrait to make a talking-avatar video. | +| `update_asset_collection` | Renames or moves a folder, or changes an avatar's name, portrait, or default voice. | + +## Account and feedback + +| Tool | What it does | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `get_credit_balance` | Shows the team this connection uses, its plan and remaining credits, and how much text one speech call can take. | +| `list_my_workspaces` | Lists your workspaces in the team you connected. | +| `submit_feedback` | Sends a short note about the tools to Fish Audio; agents are asked to send one when something is confusing or broken, so yours may do it without asking you. |