Building Dream TTS Around Durable Local Speech Jobs

Making a laptop speak a sentence is a useful demo. It says very little about whether a local speech application can handle a long document.

Dream TTS is a local speech studio for turning text, documents, and web articles into narrated audio. The processing stays on the user’s machine. That local boundary is valuable for privacy, but it also means the application has to own work that a hosted service would otherwise hide: model storage, document cleanup, long-running jobs, retries, cache management, and final audio assembly.

That changed the architectural question. I was no longer asking only which speech engine sounded best. I needed to decide where each kind of work belonged and what state had to survive when synthesis stopped halfway through.

Start with the job, not the model

The product needed to do more than pass text to an engine. It needed to:

• import text from documents and web pages;
• clean that text before synthesis;
• generate audio without freezing the interface;
• pause, retry, and resume long jobs;
• retain completed work in a useful library;
• handle bundled, downloaded, and user-provided models; and
• avoid sending private source text to a remote service.

Those requirements led to a deliberate split.

Flutter owns the product surface: imports, controls, progress, playback, the library, and the small state transitions a user sees.

Rust owns the synthesis work: file handling, text cleanup, sentence chunking, engine execution, durable job state, cache management, and audio export.

The distinction is about failure as much as performance. A screen can be rebuilt from current state. A partially completed two-hour synthesis job should not have to start again merely because the screen was closed or the process was interrupted.

Keep the synthesis pipeline below the UI

A prototype can attach synthesis directly to a button handler. That arrangement becomes difficult to reason about once a job includes hundreds of chunks, several output files, cancellation, and recovery.

Dream TTS keeps the long-running pipeline below the Flutter/Rust bridge. The UI requests an operation and observes its state; it does not coordinate individual synthesis chunks or decide where their files live.

The practical boundary is:

The UI owns interaction.
The runtime owns synthesis and recovery.

Work that affects job recovery, generated artifacts, or engine behavior belongs in Rust. Work that affects navigation, controls, and presentation belongs in Flutter. This is not a universal Flutter rule. It is the boundary that fits this application’s failure modes.

Make chunks the unit of recovery

Chunking is often described as a way to fit work into memory or satisfy an engine’s input limit. In Dream TTS, a chunk also represents progress.

The runtime can record which chunks are pending, complete, or need another attempt. Completed audio can remain in the cache instead of being regenerated. Final assembly happens only after the required pieces are present.

That makes the cache part of the job model rather than an incidental temporary directory. It supports pause and resume, limits repeated work after a failure, and gives the runtime smaller artifacts to validate and assemble. It also imposes a responsibility: cached state and the manifest describing it must agree. Recovery is only dependable if stale or missing artifacts are detected rather than silently accepted.

For large output, assembly should not require loading one enormous uncompressed audio file into memory. The runtime can work from the chunk artifacts and choose an assembly strategy appropriate to the target format.

Treat import cleanup as speech work

A PDF often preserves page layout more faithfully than reading order. Extracted text may contain repeated headers, page numbers, broken lines, or stray characters. Web pages add navigation, cookie notices, and other text that was never meant to be narrated.

If those artifacts reach the engine, the engine will usually read them. The failure sounds like poor speech synthesis even though it began during import.

Dream TTS therefore treats source ingestion and normalization as part of the speech pipeline. The useful sequence is not simply text -> model -> audio; it is closer to:

source -> extracted text -> normalized text -> chunks -> audio chunks -> final output

Keeping those stages explicit makes failures easier to locate. It also leaves room for a user to inspect or correct the transcript before spending time on a long generation job.

Check what can be checked at chunk time

An engine returning an audio file proves that a file was produced. It does not prove that the file contains usable speech.

Dream TTS performs chunk-level checks before final assembly. A clearly silent, empty, or broken result can be rejected and retried while it is still small. This prevents a known bad artifact from being buried inside a much longer output.

These checks have a limited claim. They can catch mechanical failures; they cannot decide whether a pronunciation sounds natural or whether a voice suits the material. Those questions still need listening, fixtures, and product judgment. Keeping the distinction matters because an automated gate should not be presented as a complete measure of speech quality.

Use a narrow bridge

The bridge exposes operations such as:

configure the runtime
import source material
start, pause, or resume a job
read job status and history
retrieve the transcript
export completed audio

It passes small requests, identifiers, responses, and lifecycle events. Flutter does not need to know the current ONNX model path, the cache layout, or the engine’s retry details.

This boundary also gives each side a useful test surface. Runtime behavior can be tested without constructing widgets, while the interface can be changed without moving synthesis policy into view code. When a failure crosses the boundary, the operation and job identifier provide a narrower place to start investigating than a shared object graph spread across both languages.

Model and platform details still leak somewhere

Abstraction does not remove platform differences; it gives them a place to live.

Desktop and mobile applications have different sandboxes, data directories, audio sessions, packaging rules, and signing requirements. Dream TTS handles those details below the product interface where possible, while returning errors the interface can explain. Pretending the differences do not exist would only turn them into late release failures.

Models also have distinct lifecycles. A bundled model ships with the application. A downloaded model needs a download and validation path. A user-provided model needs compatibility checks and a clear error when it cannot be loaded. Treating all three as an arbitrary file path would make upgrades and support harder.

Privacy has a similar operational consequence. Keeping synthesis local is not enough if raw document text is copied into diagnostic logs. The runtime can record job identifiers, stages, durations, and error types without recording the text being narrated.

The release sequence followed the risks

Dream TTS developed in stages:

1. Make one local voice speak.
2. Add import, configuration, preview, and export through a usable interface.
3. Give long jobs durable state and recovery.
4. Package the runtime and models for the target devices.
5. Reject known-bad chunks before assembling and releasing output.

This order kept model quality in context. A more natural voice is useful, but it does not compensate for losing a long job, producing silent chunks, or failing when a model file is missing. Conversely, durable jobs do not make a mediocre voice good. The product needs both a capable engine and reliable machinery around it.

The most reusable decision in Dream TTS was not the choice of a particular engine. It was to put a small interface in front of a runtime that owns source cleanup, durable progress, model lifecycle, and audio artifacts. That structure cannot prevent every engine or platform failure. It can make those failures visible, bounded, and recoverable without turning the UI into the synthesis system.