Github · GitHub Repository Radar
CorentinJ Real-Time-Voice-Cloning
Clone a voice in 5 seconds to generate arbitrary speech in real-time
Stars
60,122
Forks
9,391
Watchers: 60,122
Language
License: Other
Repository Radar Score
50 / 100
Growth
- 7d
- +0
- 30d
- +0
- %
- 0.0%
Not enough metric snapshots yet to chart growth for this repository.
Score breakdown
- popularity 89
- growth 0
- activity 35
- freshness 100
- community 66
Need help integrating this stack?
Our team builds with modern open-source stacks. Tell us what you are shipping.
Get a quote →This repository is an implementation of Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (SV2TTS) with a vocoder that works in real-time. This was my master's thesis.
SV2TTS is a deep learning framework in three stages. In the first stage, one creates a digital representation of a voice from a few seconds of audio. In the second and third stages, this representation is used as reference to generate speech given arbitrary text.
Video demonstration (click the picture):
| URL | Designation | Title | Implementation source |
|---|---|---|---|
| 1806.04558 | SV2TTS | Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis | This repo |
| 1802.08435 | WaveRNN (vocoder) | Efficient Neural Audio Synthesis | fatchord/WaveRNN |
| 1703.10135 | Tacotron (synthesizer) | Tacotron: Towards End-to-End Speech Synthesis | fatchord/WaveRNN |
| 1710.10467 | GE2E (encoder) | Generalized End-To-End Loss for Speaker Verification | This repo |
Like everything else in Deep Learning, this repo has quickly gotten old. Many SaaS apps (often paying) will give you a better audio quality than this repository will. If you wish for an open-source solution with a high voice quality:
- Check out paperswithcode for other repositories and recent research in the field of speech synthesis.
- Check out Chatterbox for a similar project up to date with the 2025 SOTA in voice cloning
Both Windows and Linux are supported.
- Install ffmpeg. This is necessary for reading audio files. Check if it's installed by running in a command line
ffmpeg
- Install uv for python package management
# On Windows:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# On Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Alternatively, on any platform if you have pip installed you can do
pip install -U uv
- Run one of the following commands
# Run the toolbox if you have an NVIDIA GPU
uv run --extra cuda demo_toolbox.py
# Use this if you don't
uv run --extra cpu demo_toolbox.py
# Run in command line if you don't want the GUI
uv run --extra cuda demo_cli.py
uv run --extra cpu demo_cli.py
Uv will automatically create a .venv directory for you with an appropriate python environment. Open an issue if this fails for you
Pretrained models are now downloaded automatically. If this doesn't work for you, you can manually download them from Hugging Face.
For playing with the toolbox alone, I only recommend downloading LibriSpeech/train-clean-100. Extract the contents as <datasets_root>/LibriSpeech/train-clean-100 where <datasets_root> is a directory of your choosing. Other datasets are supported in the toolbox, see here. You're free not to download any dataset, but then you will need your own data as audio files or you will have to record it with the toolbox.
Repository Radar analysis
Deterministic insights derived from public metadata and our observations — not personal testing or reviews.
Why this repository is interesting
- High absolute popularity (60,122 stars) signals broad adoption.
Who should use it
- Developers working primarily with Python
Potential use cases
- Reference or evaluate Python open-source approaches in this domain
Strengths
- README present in our index
- Declared license: Other
- Substantial fork count (9,391) suggests reuse and contribution interest
Limitations / considerations
- Insights are derived from public metadata and our observations — not a substitute for code review
What to watch
- Re-check last push, issues, and releases on GitHub before production adoption
Strong signals: Strong community interest
Source: GitHub (public metadata) + Repository Radar analysis. We do not claim ownership of third-party repositories.