feat: add expressive offline Russian book TTS

This commit is contained in:
Курнат Андрей
2026-07-21 21:28:48 +03:00
parent 28312e3fbc
commit cf82ab61eb
47 changed files with 127843 additions and 124 deletions
+50
View File
@@ -33,3 +33,53 @@ before producing WAV files.
`train_piper_windows.ps1` provides the equivalent native Windows path and a `-SmokeTest` mode that executes one
training and validation batch on CUDA before a long run. See `GPU_TRAINING.md` for host prerequisites and the
acceptance gate.
## Vosk BookTTS compression experiment
`vosk_booktts_experiment.py` reproduces the Android Vosk frontend, prepares real acoustic-model feeds from
`booktts_calibration_ru.txt`, performs calibrated QDQ int8 quantization of convolution layers, and renders
deterministic baseline/candidate WAV pairs. Keep generated `.npz`, candidate ONNX, and WAV files outside tracked
source directories, for example under `.codex-temp`. Do not replace the application asset until the candidate
loads with ONNX Runtime, runs on the target phone, and passes listening comparison.
Use `prepare --bert-model` plus `compare-feeds` to isolate a BERT replacement while keeping the acoustic model
identical in both WAV branches. `compare-feeds --resume` preserves existing WAVs and atomically updates its
report after every pair; `run_booktts_ab_windows.ps1` is the persistent Windows entry point.
`booktts_student.py` builds token-level teacher targets from the current int8 BERT and trains a four-layer,
256-dimensional Transformer student with a 768-dimensional drop-in output. Its small calibration corpus is only
for validating the training/export pipeline; a candidate for the application requires a much larger licensed
literary corpus, held-out evaluation, ONNX quantization, and listening tests. The optional
`--max-training-batches` and `--max-validation-batches` limits are for CUDA memory smoke tests only.
Each completed epoch atomically replaces `checkpoint.latest.pt`; pass `--resume` with the same output directory
to continue an interrupted run.
`run_booktts_student_windows.ps1` is the native Windows CUDA entry point for a full or resumed student run.
The exported student includes the dataset's original-to-compact token map so its `input_ids` remain compatible
with the existing Vosk vocabulary. `patch_booktts_student_token_map.py` adds the same lookup to an older export
without retraining.
`compare_booktts_bert.py` measures teacher/student and FP32/INT8 embedding error, inference time, and unseen-token
coverage on a held-out text file.
`build_booktts_prosody_corpus.py` creates a deterministic balance of questions, exclamations, quotations,
ellipsis, and neutral prose. Build its teacher dataset with `--reuse-token-map`, then fine-tune from the prior
`student.pt` with `--initial-model`; the held-out listening phrases must be passed through `--exclude`.
`run_booktts_prosody_dataset_windows.ps1` builds this reused-vocabulary teacher dataset persistently on Windows.
After auditing that dataset, `run_booktts_prosody_finetune_windows.ps1` starts the low-learning-rate CUDA run.
`extract_wikisource_corpus.py` streams an official Russian Wikisource XML/BZip2 dump and writes a deduplicated,
filtered sentence corpus plus provenance metadata. The source URL and dump size must be retained. Wikisource is
not a blanket rights clearance for every included work, so review the resulting corpus and applicable source
licenses before distributing a trained model.
`run_booktts_corpus_windows.ps1` completes the resumable BITS download on the CUDA host, verifies the official
dump size and SHA-1, extracts the configured number of sentences, and builds the streamed float16 teacher
dataset. It intentionally stops before training so vocabulary coverage and a held-out evaluation plan can be
reviewed first.
`audit_booktts_text_corpus.py` records the corpus SHA-256, sentence-length and word-count distributions,
residual wiki-markup counters, and deterministic review samples. Run it before training; its report is a
technical quality check and does not replace a rights review of the source works.
`audit_booktts_dataset.py` opens every generated teacher shard and verifies sample counts, tensor shapes,
token ranges, binary masks, and finite teacher values. It also prints a reproducible SHA-256 over shard bytes.