Overview
PocketTTS (~155M params) is an autoregressive TTS backend that generates audio frame-by-frame. No espeak dependency — uses SentencePiece tokenization directly. Audio starts streaming ~80ms after prefill. Model: FluidInference/pocket-tts-coremlQuick Start
Architecture
Key State
KV Cache
- 6 cache tensors
[2, 1, 512, 16, 64]+ 6 position counters - Reset per chunk
Mimi State
- 23 tensors for convolution history, attention caches, overlap-add buffers
- Continuous across chunks — keeps audio seamless
Text Chunking
Long text splits at 50 tokens or fewer:- Sentence boundaries (
.!?) - Clause boundaries (
,;:) - Word boundaries (fallback)
Pipeline
Pronunciation Control
What can be added — anything that operates on text before the SentencePiece tokenizer: number/date/currency expansion, text substitution, abbreviation expansion.
What cannot be added without retraining — anything that requires phoneme-level control. The model decides pronunciation from text tokens alone. See Kokoro if you need pronunciation control.
CoreML Details
- All 4 models loaded with
.cpuAndGPU(ANE float16 causes artifacts in Mimi state) - Compiled from
.mlpackage→.mlmodelcon first load, cached on disk - Thread-safe via actor pattern
Benchmarks
Benchmarks in progress. Methodology follows Kyutai’s evaluation and their tts_longeval toolkit.Upstream (Kyutai, CPU)
LibriSpeech test-clean, WER via Whisper large-v3:
ELO from human pairwise evaluation (50 raters, 50 samples). Tested on Apple M3 and Intel Core Ultra 7.
FluidAudio CoreML (planned)
We will benchmark the CoreML port against the upstream PyTorch CPU baseline using the same methodology:
Additional datasets from tts_longeval:
- NTREX — monologue sentences from news translation corpus
- Synthetic Dialogs — daily life, technical, and number-heavy scripts
- SEED English — adapted from ByteDance’s SEED TTS Eval