Best place to host remotely or possible runpod serverless worker

[TheMindExpansionNetwork]

what do you recommend would be a good way to make this like an end point where they can connect to it as needed with other AI's

I'm not sure if it's best to just push it over to a docker image and run pond serverless or if you all know something a little easier to do this

Trying to build a little project to provide open source to trying to be as budget friendly as I can. Any advice is much appreciated thank you. I was planning on using live kit if it makes sense and possibly a. Eliza framework

[anujnayyar1]

@remsky firstly thank you for this awesome repo.

A Runpod Serverless compatible container would be extremely helpful!

[remsky]

@anujnayyar1 @TheMindExpansionNetwork

That sounds like a great idea, glad it's been helpful. I'll take a look, just finishing up some restructuring to simplify container setup which should make it more deployable

[TheMindExpansionNetwork]

yeah, it's super awesome. I'll tell you. I guess the stack that I'm looking to push with.

Serverless-runpod>kokoro>possible rvc but I like your voices > live kit > eliza>to machine or like avatar have a little Meta corner idea for digital entity that lives

I been looking something stable sounds decent because plans start deploying this stuff give people from homeless to learn free education

Throw partys and raves to fund it all

Super impressed quality amazing how you did it to super transparent

What if we did this just idea love to help

We can have AI even got the pro ai to generate a basic synthetic dataset

Of just midi notes

Then we take that bad boy with all native instruments or other stuff purchased

Then in theory we be legal train ai on it release apache or what ever

Idk how hard be but its and idea

Think we need more open source quality music models

[quetzalsly]

Hey sorry to ask this question here but, how does this version of kokoro differ from replicate/jaaari/kokoro-82m?

I really love the simplicity of using replicate in my workflow but thinking of switching to this one, but also not sure where to host it, how about huggingface dedicated endpoints?

[remsky]

Not sure to be honest, they have it always live and hosted there for cloud use, but I don't know know what source code they're using, just the same base model. This repo is geared more towards local use and development, source files so it can be modified if needed and built on over time.

For an equivalent endpoint on local hardware, or any cloud endpoints (haven't tested yet, so if you do manage to, feel free to submit a PR with instructions/mods), you'd probably want to run these to just get the voice server up fast.

docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:v0.1.0post1 # CPU or
docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:v0.1.0post1 # Nvidia GPU

If HF doesn't already have a drop in hosting for the voice images, I can take a look around to try and include a guide along with RunPod/etc instructions, just gotta finish up getting the quantized versions and model selection update worked into the code first

[dreamerseen]

Hi @remsky is it expected that everytime I hit the fastapi cpu version it doesn't return an audio file? And how could I return it as an audio file just like the GPU version?

[TheMindExpansionNetwork]

Has anyone been able to figure out anything I found out works awesome with livekit swap out url but need to host bad boy I got it with the docker image and the thing they had in new endpoint but I did not know how to access it

Think need to do something with this

https://github.com/runpod-workers/worker-template

If anyone has input be dope not many serverless endpoints especially so fast

[calicoagent]

Late to this thread, but it's still the top hit for "where do I host Kokoro-FastAPI" so figured I'd add an option.

I've been running the CPU image (ghcr.io/remsky/kokoro-fastapi-cpu) on Railway for a few months and it's the simplest path if you just want a stable /v1/audio/speech endpoint other services (LiveKit, Eliza, whatever) can hit. No GPU needed for the 82M model — CPU inference is plenty fast for TTS-sized requests — and you get a public HTTPS URL without touching RunPod's serverless config or standing up a HF Space.

Full disclosure: I maintain a one-click Railway template for this and get a kickback if you deploy through it: https://railway.com/deploy/kokoro-tts-api?referralCode=Z1xivh&utm_medium=integration&utm_source=template&utm_campaign=generic

If you'd rather skip the referral, the exact same image runs anywhere Docker does:

docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest

One tradeoff worth knowing: Railway (like most PaaS) bills by usage, so if this is going to sit mostly idle waiting on your pipeline, keep an eye on always-on cost vs. something like RunPod serverless that scales to zero. For a steady low-volume TTS backend it's been cheap for me, but it's not free-tier-forever cheap.

[calicoagent]

Small update to my comment above — we ended up also shipping a Qwen3-TTS template, and it's worth flagging here since this thread is specifically about hosting Kokoro.

We run both ourselves. Voice quality: Qwen3-TTS is noticeably better, plus it's got real expressiveness/control that Kokoro's 82M model doesn't — inline emotion markup ([joy], [sad], etc.), instruct-style steering, multi-speaker presets, all backed by a much bigger 1.7B model. Kokoro still wins on speed and footprint: it's faster than realtime on shared vCPU and needs ~1GB RAM, where Qwen3-TTS is CPU-bound at roughly RTF 2 and wants 4-6GB. So: take Kokoro if latency or RAM is your binding constraint, take Qwen3-TTS if quality/expressiveness matters more than raw speed.

Same disclosure as my comment above: I maintain this one too and get a kickback if you deploy through it: https://railway.com/deploy/qwen3-tts?referralCode=Z1xivh&utm_medium=integration&utm_source=template&utm_campaign=generic

Kokoro recommendation above still stands for anyone whose priority is speed/cheap footprint over expressive range.

[kivtxs]

Hi @calicoagent — your August comments describe running both Kokoro and Qwen speech deployments. Do you currently serve short speech-generation requests for other people's applications, or are those instances only for your own workloads?

I'm building Capability Exchange, an operator-assisted private trial where owners authorize bounded capability calls under spending caps. Participation would use non-redeemable TEST credit, not cash, and we have no confirmed recurring buyer for speech generation. I'm trying to establish whether an existing operator would find a shared-call arrangement useful before proposing any trial. A description is enough; please don't share endpoints, credentials, private text or audio.