pyannote / separation-ami-1.0

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: July 16 2024

Introduction of separation-ami-1.0

Model Details of separation-ami-1.0

Using this open-source model in production?
Consider switching to pyannoteAI for better and faster options.

🎹 ToTaToNet / joint speaker diarization and speech separation

This model ingests 5 seconds of mono audio sampled at 16 kHz and outputs speaker diarization AND speech separation for up to 3 speakers.

Example

It has been trained by Joonas Kalda with pyannote.audio 3.3.0 using the AMI dataset (single distant microphone, SDM). These paper and companion repository describe the approach in more details.

Requirements
  1. Install pyannote.audio 3.3.0 with pip install pyannote.audio[separation]==3.3.0
  2. Accept pyannote/separation-ami-1.0 user conditions
  3. Create access token at hf.co/settings/tokens .
from pyannote.audio import Model
model = Model.from_pretrained(
    "pyannote/separation-ami-1.0",
    use_auth_token="HUGGINGFACE_ACCESS_TOKEN_GOES_HERE")
Usage
# model ingests 5s of mono audio sampled at 16kHz...
duration = 5.0
num_channels = 1
sample_rate = 16000

waveforms = torch.randn(batch_size, num_channels, duration * sample_rate)
waveforms.shape
# (batch_size, num_channels = 1, num_samples = 80000)

# ... and outputs both speaker diarization and separation
with torch.inference_mode():
    diarization, sources = model(waveform)

diarization.shape
# (batch_size, num_frames = 624, max_num_speakers = 3)
# with values between 0 (speaker inactive) and 1 (speaker active)

sources.shape
# (batch_size, num_samples = 80000, max_num_speakers = 3)
Limitations

This model cannot be used to perform speaker diarization and speech separation of full recordings on its own (it only processes 5s chunks): see pyannote/speech-separation-ami-1.0 pipeline that uses an additional speaker embedding model to do that.

Citations
@inproceedings{Kalda24,
  author={Joonas Kalda and Clément Pagés and Ricard Marxer and Tanel Alumäe and Hervé Bredin},
  title={{PixIT: Joint Training of Speaker Diarization and Speech Separation from Real-world Multi-speaker Recordings}},
  year=2024,
  booktitle={Proc. Odyssey 2024},
}
@inproceedings{Bredin23,
  author={Hervé Bredin},
  title={{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}

Runs of pyannote separation-ami-1.0 on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About separation-ami-1.0 huggingface.co Model

More separation-ami-1.0 license Visit here:

https://choosealicense.com/licenses/mit

separation-ami-1.0 huggingface.co

separation-ami-1.0 huggingface.co is an AI model on huggingface.co that provides separation-ami-1.0's model effect (), which can be used instantly with this pyannote separation-ami-1.0 model. huggingface.co supports a free trial of the separation-ami-1.0 model, and also provides paid use of the separation-ami-1.0. Support call separation-ami-1.0 model through api, including Node.js, Python, http.

separation-ami-1.0 huggingface.co Url

https://huggingface.co/pyannote/separation-ami-1.0

pyannote separation-ami-1.0 online free

separation-ami-1.0 huggingface.co is an online trial and call api platform, which integrates separation-ami-1.0's modeling effects, including api services, and provides a free online trial of separation-ami-1.0, you can try separation-ami-1.0 online for free by clicking the link below.

pyannote separation-ami-1.0 online free url in huggingface.co:

https://huggingface.co/pyannote/separation-ami-1.0

separation-ami-1.0 install

separation-ami-1.0 is an open source model from GitHub that offers a free installation service, and any user can find separation-ami-1.0 on GitHub to install. At the same time, huggingface.co provides the effect of separation-ami-1.0 install, users can directly use separation-ami-1.0 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

separation-ami-1.0 install url in huggingface.co:

https://huggingface.co/pyannote/separation-ami-1.0

Url of separation-ami-1.0

separation-ami-1.0 huggingface.co Url

Provider of separation-ami-1.0 huggingface.co

pyannote
ORGANIZATIONS

Other API from pyannote

huggingface.co

Total runs: 318.5K
Run Growth: -55.3K
Growth Rate: -17.63%
Updated: May 10 2024
huggingface.co

Total runs: 100.8K
Run Growth: 86.7K
Growth Rate: 85.75%
Updated: November 15 2022