What is SI-SDR? The formula, the code, and what it misses

What is SI-SDR?

SI-SDR splits an output into a scaled copy of the clean reference and everything else, then reports the energy ratio of the two in dB. Higher is better, and 0 dB means both parts carry the same energy.

s = reference - mean(reference)o = output - mean(output)a = (o · s) / (s · s)SI-SDR = 10 · log10( |a·s|² / |o - a·s|² )

• The scale a makes it blind to volume. An output turned up three times scores the same.
• Everything else counts as distortion. Noise, another voice, room echo, a missing frequency band, a small time shift: SI-SDR cannot tell them apart.
• It needs the clean reference. You can only compute it on test sets built from separate stems, which is why it lives in research, not in production monitoring.

Le Roux and colleagues proposed it in a 2019 paper to replace the older SDR in speech separation and speaker extraction results.

How do you compute SI-SDR in Python?

In NumPy, for two signals of the same length and sample rate:

import numpy as np def si_sdr(output, reference):    s = reference - reference.mean()    o = output - output.mean()    target = (o @ s) / (s @ s) * s    residual = o - target    return 10 * np.log10((target @ target) / (residual @ residual))

With a silent reference, s @ s is zero and SI-SDR is undefined. Implementations that add a small epsilon to avoid the division, as torchmetrics does, return a number that depends only on how loud the output is.

torchmetrics ships the same metric, but leaves the mean in unless you ask:

from torchmetrics.audio import ScaleInvariantSignalDistortionRatio si_sdr = ScaleInvariantSignalDistortionRatio(zero_mean=True)score = si_sdr(output, reference)

What is SI-SDRi?

SI-SDRi is the output’s SI-SDR minus the input recording’s, so it reports what the system changed rather than how easy the recording was. For scale, TF-GridNet reports 23.5 dB on WSJ0-2mix, the standard clean two-speaker test set. Raw SI-SDR flatters easy inputs, but SI-SDRi does not fix the speaker problem:

• Untouched recordings score well. In RightVoice, 16 already score 10 dB or more; the best scores 19.5 dB with 18 of the other person’s words still in it.
• It can rank outputs backwards. For the two outputs below that both score -0.097 dB, SI-SDRi gives +6.5 dB to the one with the other person in it and -2.0 dB to the one without.

SI-SDR vs SDR vs SI-SNR

SI-SDR and SI-SNR are one measure under two names; SDR, from the BSS Eval toolkit, is the older score SI-SDR was proposed to replace.

What does SI-SDR miss in a conversation?

SI-SDR cannot tell whether the leftover energy is the other person or a harmless difference, so on RightVoice’s two-person recordings it scores outputs with and without the other voice alike. These are outputs from USEF-TFGridNet, a published research model that takes a voice sample:

• Same score, different people. Items rv1-00700 and rv1-02692 both score -0.097 dB. The first has 55 of its speaker’s 57 words right and no other voice. The second brings back 38 words from the other person and a bystander.
• The ranges overlap. 35 of its 147 wrong-voice outputs score inside the middle half of its right-person outputs, -2.9 to 4.4 dB.
• It has no score for silence. Where the speaker never talks, the reference is silent and SI-SDR is undefined. USEF-TFGridNet returned speech on 120 of those 121 items.
• Harmless differences cost decibels. 244 of its outputs score below 0 dB while losing at most one word in ten and carrying no other voice.

A right-person output here loses at most one word in ten and carries no other voice; a wrong-voice output carries at least as many words from other voices as from its speaker. Every per-item score is public, so you can check the pair:

pip install -U huggingface_hubhf download crele/rightvoice --repo-type dataset --include "scores/*" --local-dir rightvoice

import json for line in open("rightvoice/scores/usef_tfgridnet.jsonl"):    r = json.loads(line)    if r["id"] in ("rv1-00700", "rv1-02692"):        print(r["id"], round(r["si_sdr"], 3),              round(r["si_sdr_improvement"], 1),              r["wrong_voice_words"]["count"])

It prints each item’s SI-SDR, SI-SDRi and wrong-voice words: rv1-00700 -0.097 -2.0 0 and rv1-02692 -0.097 6.5 38. Both input recordings are in the dataset under items/.

What should you report alongside SI-SDR?

Report what a listener notices: the speaker’s words lost, other people’s words that come through, and speech while the speaker is silent. Crele’s RightVoice benchmark scores all three from the output audio alone, next to SI-SDR:

• Target words lost. Word error rate against the speaker’s own text.
• Wrong-voice words. Other voices’ words in the output, per 100 target words and per minute.
• Talk over silence. Words a minute while the speaker is quiet, and on 121 items where they never speak.
• Activity F1, voice match and quality. When the output has speech, whose voice it sounds like, and a predicted opinion score.

To score your own system, start on the 192-clip quick subset:

git clone https://github.com/Crele-AI/rightvoice && cd rightvoiceuv sync --extra scoreuv run hf download crele/rightvoice --repo-type dataset --local-dir data/build/v1# run your system: items/<id>/mixture.wav + sample.wav -> outputs/<id>.wavuv run rightvoice score --build data/build/v1 --system my_system --outputs outputs --quickuv run rightvoice report --build data/build/v1 --quick

Results for the reference systems are on the RightVoice page.

Questions

What is a good SI-SDR?

It depends on the recording, so compare SI-SDRi rather than raw SI-SDR. TF-GridNet reports a 23.5 dB SI-SDRi on WSJ0-2mix, a clean two-speaker test set, and harder recordings score lower.

Can SI-SDR be negative?

Yes. It is negative whenever the part of the output that does not match the reference carries more energy than the part that does, as it often is for an unprocessed recording of two people talking.

Is SI-SNR the same as SI-SDR?

In practice, yes: both compare the scaled reference inside the output with everything else, and the two names come from two papers. Check whether an implementation removes the mean first; torchmetrics' SI-SNR does and its SI-SDR does not by default.

Why is SI-SDR used as a training loss?

Negative SI-SDR is differentiable, ignores output level and needs only the clean reference, so many separation models train on it, with permutation-invariant training (PIT) choosing which output to score against which speaker. It teaches a model to match the waveform, which is not the same as keeping only the right person.

Why does RightVoice keep SI-SDR out of its headline?

Crele's RightVoice benchmark writes SI-SDR and SI-SDRi to every item's scores, but its headline counts words, because on two-person recordings SI-SDR scores outputs with and without the other person alike.