Real-time speaker
extraction
Real-time target speaker extraction is coming soon, with early access through the waitlist. Enroll a voice in seconds. As audio streams, the engine follows that one voice and returns it alone, with every other voice and the room removed. The batch version is free to try today.
Give it one voice to follow
Diarization, separation and extraction, side by side, for how the three differ.
A few seconds of one person's voice. From then on the engine follows that voice through whatever else is in the audio and returns it alone.
Diarization tells you who spoke when. Separation splits everyone onto unlabelled tracks. Extraction answers a narrower question: what is this person saying?
Real-time means the stream, not the file. Whatever sits downstream, a transcriber, a voice agent, the other end of a call, hears one speaker.
What comes back on the stream
const session = crele.stream({ voice: enrolled })
session.on("audio", chunk => {
sink.play(chunk) // your speaker, isolated
})
session.on("speaker", state => {
state.speaking // true
state.confidence // "high"
state.turn // "yours"
})- Isolated audio
- Your speaker alone. Other voices, the television, the street: gone.
- Speaking or not
- Whether the enrolled voice is talking right now, so silence is silence.
- Confidence
- How sure the engine is that what it hears is your speaker.
- Whose turn
- A turn signal that belongs to a person, not to whoever is loudest.
Where it goes
- Voice agents
- Stop answering the television. Take the turn from the caller, not the room.
- Calls and contact centers
- Two people on one line, or one person in a loud place, and one voice out.
- Meetings
- Follow the person you enrolled through crosstalk and side conversations.
- Devices
- Wearables and far-field microphones, where the demo mic never goes.
Cleaned up on the way through
Extraction hands back the voice as the microphone caught it, room and all. Live speech enhancement is the next thing we are exploring: the same stream, with the voice regenerated clean as it arrives.
The batch version exists today on the enhance tool. Live is harder, because it has to keep up with the person talking and stay honest while it does. If a live microphone is your case, say so on the waitlist. That is what decides how soon we build it.
What you can use today
The batch model is the same engine on a recorded file, and it is free.
Isolate a Speaker takes a recording and a few seconds of the voice you want, and returns that voice alone. No account, nothing to install. If you want the same thing on a live stream, join the waitlist below.
Questions
- Is the real-time engine available now?
- Not yet. It is coming soon, with early access through the waitlist. The batch version of the same model is free to use today on the speaker isolation tool, on recordings rather than live audio.
- What is target speaker extraction?
- Pulling one known voice out of audio that has other voices and noise in it. You give the system a short sample of the voice you want, and it returns that voice alone. Everything else, other speakers included, is treated as interference. Real-time extraction does it on a live stream, as the audio arrives.
- How is this different from noise suppression?
- Noise suppression keeps every voice and removes the rest. Extraction keeps one voice and removes everything else, including the other voices. On a call with a second person in the room, or a voice agent within earshot of a television, that is the difference between a tool that helps and one that does not.
- How is it different from diarization?
- Diarization labels who spoke when, after the fact, with anonymous labels. Extraction is given the identity up front and returns that person's audio, live. If what you need downstream is one clean voice rather than a labelled transcript, extraction is the one.
- Does it need a voice sample?
- Yes. A few seconds of the speaker talking on their own, recorded once. That enrollment is what the engine follows. It is also what makes the output belong to a known person rather than to whichever voice is loudest.
- What happens when my speaker is not talking?
- Silence, and a signal saying so. Knowing the difference between a quiet speaker and an absent one is part of the job, and it is why the engine returns attribution alongside the audio.
- Will it enhance the voice too?
- We are exploring it. Extraction hands back the voice as the microphone caught it, room and all. Live enhancement would regenerate that voice clean on the same stream. The batch version exists today on the enhance tool. If a live microphone is your case, say so on the waitlist.
- Can I call it from my own code?
- That is the developer API, which is also coming soon. Its page says what it will do and in what order, and its waitlist is the place to tell us what you are building.
Get early access
The real-time engine is coming soon. Join the waitlist and tell us what you'd run it on, and we'll write when there's something real to try.