86 lines
5.6 KiB
Markdown
86 lines
5.6 KiB
Markdown
# Aletheia Russian TTS corpus tools
|
|
|
|
`audit_corpus.py` verifies that an audiobook and EPUB contain the same text before any training data is produced.
|
|
It extracts the EPUB spine in reading order and compares local Whisper transcripts from the beginning, middle,
|
|
and end of the audiobook with the normalized book text.
|
|
|
|
Generated audio, transcripts, downloaded ASR models, and future training checkpoints belong under
|
|
`.codex-temp/tts-*`; they are working artifacts and must not be committed.
|
|
|
|
Example on Windows PowerShell:
|
|
|
|
```powershell
|
|
$env:PYTHONPATH = 'C:\Repos\Aletheia\.codex-temp\tts-python'
|
|
python tools\tts\audit_corpus.py `
|
|
--source-dir 'C:\path\to\audiobook' `
|
|
--output-dir '.codex-temp\tts-audit' `
|
|
--model base
|
|
```
|
|
|
|
The audit is not a copyright or voice-consent check. Training and distributing a voice model requires
|
|
separate confirmation that the recordings and narrator's voice may be used for that purpose.
|
|
|
|
After the audit passes, `transcribe_corpus.py` creates one resumable JSON sidecar per MP3 with segment and
|
|
word timestamps. Existing sidecars are skipped unless `--force` is supplied. These transcripts are alignment
|
|
anchors only; the final Piper labels must come from the exact EPUB text.
|
|
|
|
`build_piper_dataset.py` globally aligns all ASR tokens to the EPUB in monotonic reading order, rejects weak
|
|
matches, merges adjacent short segments, and writes 22.05 kHz mono PCM WAV files plus Piper's
|
|
`filename.wav|Exact book text` metadata. Use `--dry-run` first and inspect the reported exact-token match ratio
|
|
before producing WAV files.
|
|
|
|
`train_piper.sh` is the CUDA/Linux entry point for scratch training and ONNX export.
|
|
`train_piper_windows.ps1` provides the equivalent native Windows path and a `-SmokeTest` mode that executes one
|
|
training and validation batch on CUDA before a long run. See `GPU_TRAINING.md` for host prerequisites and the
|
|
acceptance gate.
|
|
|
|
## Vosk BookTTS compression experiment
|
|
|
|
`vosk_booktts_experiment.py` reproduces the Android Vosk frontend, prepares real acoustic-model feeds from
|
|
`booktts_calibration_ru.txt`, performs calibrated QDQ int8 quantization of convolution layers, and renders
|
|
deterministic baseline/candidate WAV pairs. Keep generated `.npz`, candidate ONNX, and WAV files outside tracked
|
|
source directories, for example under `.codex-temp`. Do not replace the application asset until the candidate
|
|
loads with ONNX Runtime, runs on the target phone, and passes listening comparison.
|
|
Use `prepare --bert-model` plus `compare-feeds` to isolate a BERT replacement while keeping the acoustic model
|
|
identical in both WAV branches. `compare-feeds --resume` preserves existing WAVs and atomically updates its
|
|
report after every pair; `run_booktts_ab_windows.ps1` is the persistent Windows entry point.
|
|
|
|
`booktts_student.py` builds token-level teacher targets from the current int8 BERT and trains a four-layer,
|
|
256-dimensional Transformer student with a 768-dimensional drop-in output. Its small calibration corpus is only
|
|
for validating the training/export pipeline; a candidate for the application requires a much larger licensed
|
|
literary corpus, held-out evaluation, ONNX quantization, and listening tests. The optional
|
|
`--max-training-batches` and `--max-validation-batches` limits are for CUDA memory smoke tests only.
|
|
Each completed epoch atomically replaces `checkpoint.latest.pt`; pass `--resume` with the same output directory
|
|
to continue an interrupted run.
|
|
|
|
`run_booktts_student_windows.ps1` is the native Windows CUDA entry point for a full or resumed student run.
|
|
The exported student includes the dataset's original-to-compact token map so its `input_ids` remain compatible
|
|
with the existing Vosk vocabulary. `patch_booktts_student_token_map.py` adds the same lookup to an older export
|
|
without retraining.
|
|
|
|
`compare_booktts_bert.py` measures teacher/student and FP32/INT8 embedding error, inference time, and unseen-token
|
|
coverage on a held-out text file.
|
|
|
|
`build_booktts_prosody_corpus.py` creates a deterministic balance of questions, exclamations, quotations,
|
|
ellipsis, and neutral prose. Build its teacher dataset with `--reuse-token-map`, then fine-tune from the prior
|
|
`student.pt` with `--initial-model`; the held-out listening phrases must be passed through `--exclude`.
|
|
`run_booktts_prosody_dataset_windows.ps1` builds this reused-vocabulary teacher dataset persistently on Windows.
|
|
After auditing that dataset, `run_booktts_prosody_finetune_windows.ps1` starts the low-learning-rate CUDA run.
|
|
|
|
`extract_wikisource_corpus.py` streams an official Russian Wikisource XML/BZip2 dump and writes a deduplicated,
|
|
filtered sentence corpus plus provenance metadata. The source URL and dump size must be retained. Wikisource is
|
|
not a blanket rights clearance for every included work, so review the resulting corpus and applicable source
|
|
licenses before distributing a trained model.
|
|
|
|
`run_booktts_corpus_windows.ps1` completes the resumable BITS download on the CUDA host, verifies the official
|
|
dump size and SHA-1, extracts the configured number of sentences, and builds the streamed float16 teacher
|
|
dataset. It intentionally stops before training so vocabulary coverage and a held-out evaluation plan can be
|
|
reviewed first.
|
|
|
|
`audit_booktts_text_corpus.py` records the corpus SHA-256, sentence-length and word-count distributions,
|
|
residual wiki-markup counters, and deterministic review samples. Run it before training; its report is a
|
|
technical quality check and does not replace a rights review of the source works.
|
|
|
|
`audit_booktts_dataset.py` opens every generated teacher shard and verifies sample counts, tensor shapes,
|
|
token ranges, binary masks, and finite teacher values. It also prints a reproducible SHA-256 over shard bytes.
|