chatterbox server

[Danik-droid]

Hello, I have a question about how to run my own chatterbox (tts) as a server.

I see several versions of chatterbox. Which one should I use and how? What should the server configuration file look like?

I managed to launch Pocket-tts and it works fine. However, I couldn't figure out how to configure chatterbox from the documentation.

[0xShug0]

@Danik-droid You can start with the q8_0 gguf (less VRAM, faster) for a quick start.

Note that the task for chatterbox is "clon". An example config for chatterbox is:

{
  "host": "127.0.0.1",
  "port": 8080,
  "backend": "cuda",
  "device": 0,
  "threads": 8,
  "lazy_load": true,
  "models": [
    {
      "id": "chatterbox",
      "family": "chatterbox",
      "path": "/path/to/chatterbox-q8_0.gguf",
      "task": "clon",
      "mode": "offline",
      "session_options": {
        "chatterbox.conditionals_cache_slots": "1",
        "chatterbox.mem_saver": "false",
        "chatterbox.multilingual_t3": "v2"
      },
      "default_request_options": {
        "guidance_scale": 0.5,
        "temperature": 0.8,
        "top_p": 0.8,
        "repetition_penalty": 1.2,
        "text_chunk_size": 128
      },
      "voice_presets": {
        "default": {
          "voice_ref": "/path/to/preset_voice.wav"
        }
      },
      "default_voice_preset": "default"
    }
  ]
}

[Danik-droid]

Thanks for the reply. I've tried various gguf but I still get this error.
packed GGUF namespace does not exist: voice_encoder

[0xShug0]

Thanks for the reply. I've tried various gguf but I still get this error. packed GGUF namespace does not exist: voice_encoder

That's a missing spec error. Could you tell me you setup?

[Danik-droid]

I edited your suggested server configuration "cuda->rocm" plus the correct path to gguf (I downloaded different ones).

./build_hip/bin/audiocpp_server --config server2.json --backend hip --metrics

Maybe I should use a specific version? A link would be helpful.

[0xShug0]

Could you try the prebuilts here https://github.com/IIIIIllllIIIIIlllll/audio.cpp/releases

[0xShug0]

@Danik-droid Could you confirm you download the ggufs from the offical repo? The hash for q8 should be d586dd1aa59613cab8046176fb7ca5ba191c02a9b10ffa5b0d892ed22b470656 and for f16 b872f908605f5ccbdd8eaa9e9c92a521c602b70cbec635d46e86eb70a3f8e713

[Danik-droid]

Which one is official? https://huggingface.co/ResembleAI/chatterbox this is probably the official website, but there is no gguf here, I see that you have to download from others... the question is which one because they are not the same.

[0xShug0]

Sorry. I mean official audio.cpp HF repo here -- https://huggingface.co/audio-cpp/audio.cpp-gguf. Other GGUFs won't work with audio.cpp because for some models we have special optimizations that only audio.cpp can handle.

[0xShug0]

Another minor issue: --metrics is not a server option, please use --log

[Danik-droid]

Yep, now it works without a problem. Thank you, I didn't know there was an official repo.