feat: add expressive offline Russian book TTS
This commit is contained in:
@@ -33,3 +33,53 @@ before producing WAV files.
|
||||
`train_piper_windows.ps1` provides the equivalent native Windows path and a `-SmokeTest` mode that executes one
|
||||
training and validation batch on CUDA before a long run. See `GPU_TRAINING.md` for host prerequisites and the
|
||||
acceptance gate.
|
||||
|
||||
## Vosk BookTTS compression experiment
|
||||
|
||||
`vosk_booktts_experiment.py` reproduces the Android Vosk frontend, prepares real acoustic-model feeds from
|
||||
`booktts_calibration_ru.txt`, performs calibrated QDQ int8 quantization of convolution layers, and renders
|
||||
deterministic baseline/candidate WAV pairs. Keep generated `.npz`, candidate ONNX, and WAV files outside tracked
|
||||
source directories, for example under `.codex-temp`. Do not replace the application asset until the candidate
|
||||
loads with ONNX Runtime, runs on the target phone, and passes listening comparison.
|
||||
Use `prepare --bert-model` plus `compare-feeds` to isolate a BERT replacement while keeping the acoustic model
|
||||
identical in both WAV branches. `compare-feeds --resume` preserves existing WAVs and atomically updates its
|
||||
report after every pair; `run_booktts_ab_windows.ps1` is the persistent Windows entry point.
|
||||
|
||||
`booktts_student.py` builds token-level teacher targets from the current int8 BERT and trains a four-layer,
|
||||
256-dimensional Transformer student with a 768-dimensional drop-in output. Its small calibration corpus is only
|
||||
for validating the training/export pipeline; a candidate for the application requires a much larger licensed
|
||||
literary corpus, held-out evaluation, ONNX quantization, and listening tests. The optional
|
||||
`--max-training-batches` and `--max-validation-batches` limits are for CUDA memory smoke tests only.
|
||||
Each completed epoch atomically replaces `checkpoint.latest.pt`; pass `--resume` with the same output directory
|
||||
to continue an interrupted run.
|
||||
|
||||
`run_booktts_student_windows.ps1` is the native Windows CUDA entry point for a full or resumed student run.
|
||||
The exported student includes the dataset's original-to-compact token map so its `input_ids` remain compatible
|
||||
with the existing Vosk vocabulary. `patch_booktts_student_token_map.py` adds the same lookup to an older export
|
||||
without retraining.
|
||||
|
||||
`compare_booktts_bert.py` measures teacher/student and FP32/INT8 embedding error, inference time, and unseen-token
|
||||
coverage on a held-out text file.
|
||||
|
||||
`build_booktts_prosody_corpus.py` creates a deterministic balance of questions, exclamations, quotations,
|
||||
ellipsis, and neutral prose. Build its teacher dataset with `--reuse-token-map`, then fine-tune from the prior
|
||||
`student.pt` with `--initial-model`; the held-out listening phrases must be passed through `--exclude`.
|
||||
`run_booktts_prosody_dataset_windows.ps1` builds this reused-vocabulary teacher dataset persistently on Windows.
|
||||
After auditing that dataset, `run_booktts_prosody_finetune_windows.ps1` starts the low-learning-rate CUDA run.
|
||||
|
||||
`extract_wikisource_corpus.py` streams an official Russian Wikisource XML/BZip2 dump and writes a deduplicated,
|
||||
filtered sentence corpus plus provenance metadata. The source URL and dump size must be retained. Wikisource is
|
||||
not a blanket rights clearance for every included work, so review the resulting corpus and applicable source
|
||||
licenses before distributing a trained model.
|
||||
|
||||
`run_booktts_corpus_windows.ps1` completes the resumable BITS download on the CUDA host, verifies the official
|
||||
dump size and SHA-1, extracts the configured number of sentences, and builds the streamed float16 teacher
|
||||
dataset. It intentionally stops before training so vocabulary coverage and a held-out evaluation plan can be
|
||||
reviewed first.
|
||||
|
||||
`audit_booktts_text_corpus.py` records the corpus SHA-256, sentence-length and word-count distributions,
|
||||
residual wiki-markup counters, and deterministic review samples. Run it before training; its report is a
|
||||
technical quality check and does not replace a rights review of the source works.
|
||||
|
||||
`audit_booktts_dataset.py` opens every generated teacher shard and verifies sample counts, tensor shapes,
|
||||
token ranges, binary masks, and finite teacher values. It also prints a reproducible SHA-256 over shard bytes.
|
||||
|
||||
Reference in New Issue
Block a user