pyannote / speaker-diarization-3.1

huggingface.co
Total runs: 12.3M
24-hour runs: 0
7-day runs: 367.3K
30-day runs: 3.4M
Model's Last Updated: Mayo 10 2024
automatic-speech-recognition

Introduction of speaker-diarization-3.1

Model Details of speaker-diarization-3.1

Using this open-source model in production?
Consider switching to pyannoteAI for better and faster options.

🎹 Speaker diarization 3.1

This pipeline is the same as pyannote/speaker-diarization-3.0 except it removes the problematic use of onnxruntime .
Both speaker segmentation and embedding now run in pure PyTorch. This should ease deployment and possibly speed up inference.
It requires pyannote.audio version 3.1 or higher.

It ingests mono audio sampled at 16kHz and outputs speaker diarization as an Annotation instance:

  • stereo or multi-channel audio files are automatically downmixed to mono by averaging the channels.
  • audio files sampled at a different rate are resampled to 16kHz automatically upon loading.
Requirements
  1. Install pyannote.audio 3.1 with pip install pyannote.audio
  2. Accept pyannote/segmentation-3.0 user conditions
  3. Accept pyannote/speaker-diarization-3.1 user conditions
  4. Create access token at hf.co/settings/tokens .
Usage
# instantiate the pipeline
from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained(
  "pyannote/speaker-diarization-3.1",
  use_auth_token="HUGGINGFACE_ACCESS_TOKEN_GOES_HERE")

# run the pipeline on an audio file
diarization = pipeline("audio.wav")

# dump the diarization output to disk using RTTM format
with open("audio.rttm", "w") as rttm:
    diarization.write_rttm(rttm)
Processing on GPU

pyannote.audio pipelines run on CPU by default. You can send them to GPU with the following lines:

import torch
pipeline.to(torch.device("cuda"))
Processing from memory

Pre-loading audio files in memory may result in faster processing:

waveform, sample_rate = torchaudio.load("audio.wav")
diarization = pipeline({"waveform": waveform, "sample_rate": sample_rate})
Monitoring progress

Hooks are available to monitor the progress of the pipeline:

from pyannote.audio.pipelines.utils.hook import ProgressHook
with ProgressHook() as hook:
    diarization = pipeline("audio.wav", hook=hook)
Controlling the number of speakers

In case the number of speakers is known in advance, one can use the num_speakers option:

diarization = pipeline("audio.wav", num_speakers=2)

One can also provide lower and/or upper bounds on the number of speakers using min_speakers and max_speakers options:

diarization = pipeline("audio.wav", min_speakers=2, max_speakers=5)
Benchmark

This pipeline has been benchmarked on a large collection of datasets.

Processing is fully automatic:

  • no manual voice activity detection (as is sometimes the case in the literature)
  • no manual number of speakers (though it is possible to provide it to the pipeline)
  • no fine-tuning of the internal models nor tuning of the pipeline hyper-parameters to each dataset

... with the least forgiving diarization error rate (DER) setup (named "Full" in this paper ):

  • no forgiveness collar
  • evaluation of overlapped speech
Citations
@inproceedings{Plaquet23,
  author={Alexis Plaquet and Hervé Bredin},
  title={{Powerset multi-class cross entropy loss for neural speaker diarization}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}
@inproceedings{Bredin23,
  author={Hervé Bredin},
  title={{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}

Runs of pyannote speaker-diarization-3.1 on huggingface.co

12.3M
Total runs
0
24-hour runs
341.0K
3-day runs
367.3K
7-day runs
3.4M
30-day runs

More Information About speaker-diarization-3.1 huggingface.co Model

More speaker-diarization-3.1 license Visit here:

https://choosealicense.com/licenses/mit

speaker-diarization-3.1 huggingface.co

speaker-diarization-3.1 huggingface.co is an AI model on huggingface.co that provides speaker-diarization-3.1's model effect (), which can be used instantly with this pyannote speaker-diarization-3.1 model. huggingface.co supports a free trial of the speaker-diarization-3.1 model, and also provides paid use of the speaker-diarization-3.1. Support call speaker-diarization-3.1 model through api, including Node.js, Python, http.

speaker-diarization-3.1 huggingface.co Url

https://huggingface.co/pyannote/speaker-diarization-3.1

pyannote speaker-diarization-3.1 online free

speaker-diarization-3.1 huggingface.co is an online trial and call api platform, which integrates speaker-diarization-3.1's modeling effects, including api services, and provides a free online trial of speaker-diarization-3.1, you can try speaker-diarization-3.1 online for free by clicking the link below.

pyannote speaker-diarization-3.1 online free url in huggingface.co:

https://huggingface.co/pyannote/speaker-diarization-3.1

speaker-diarization-3.1 install

speaker-diarization-3.1 is an open source model from GitHub that offers a free installation service, and any user can find speaker-diarization-3.1 on GitHub to install. At the same time, huggingface.co provides the effect of speaker-diarization-3.1 install, users can directly use speaker-diarization-3.1 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

speaker-diarization-3.1 install url in huggingface.co:

https://huggingface.co/pyannote/speaker-diarization-3.1

Url of speaker-diarization-3.1

speaker-diarization-3.1 huggingface.co Url

Provider of speaker-diarization-3.1 huggingface.co

pyannote
ORGANIZATIONS

Other API from pyannote

huggingface.co

Total runs: 318.5K
Run Growth: -55.3K
Growth Rate: -17.63%
Updated: Mayo 10 2024
huggingface.co

Total runs: 100.8K
Run Growth: 86.7K
Growth Rate: 85.75%
Updated: Noviembre 15 2022