[Python] Ubuntu microphone python stream and pyannote/speaker-diarization-3.1

I'm trying to stitch together some samples and codepilot slop to put live microphone audio through pyannote/speaker-diarization-3.1. I'm getting no diarization-related output, so I added a crude volume meter, which works only if I amplify by 1000 because the root-mean-squared is coming out near .0001 which I cannot explain.

This is on Ubuntu 24 with PipeWire shimmed with pipewire-pulse, a custom build of ffmpeg v6.1 with --enable-libpulse and a bunch of other hoops.

import torch
import torchaudio
from torchaudio.io import StreamReader
from pyannote.audio import Pipeline

pipeline = Pipeline.from_pretrained("pyannote/speaker-diarization-3.1", use_auth_token="[...]")

sample_rate = 16000 # pyannote.audio typically expects 16kHz
frames_per_chunk = int(sample_rate * .1)
streamer = StreamReader(src="default", format="pulse")
streamer.add_basic_audio_stream(frames_per_chunk=frames_per_chunk, sample_rate=sample_rate)

print("Starting microphone stream...")

try:
for chunk in streamer.stream():
# chunk is a tuple: (waveform_tensor, )
waveform = chunk[0]

# Ensure waveform is mono and correct data type if necessary
if waveform.ndim > 1 and waveform.shape[0] > 1: # If stereo, convert to mono
waveform = torch.mean(waveform, dim=0, keepdim=True)
waveform = waveform.to(torch.float32)

volume = torch.sqrt(torch.mean(waveform ** 2)).item()
print("#" * int(volume * 1000 * 50)) # necessary to see anything

# Process with pyannote.audio pipeline
diarization = pipeline({"waveform": waveform, "sample_rate": sample_rate})

# Process diarization output (e.g., print speaker turns)
for turn, _, speaker in diarization.itertracks(yield_label=True):
print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker_{speaker}")

except KeyboardInterrupt:
print("Stopping microphone stream.")


My questions are:

1. Is it perfectly OK that my waveform amplitude is registering at .0001? If not, where is this supposed to be corrected?
2. Does pyannote/speaker-diarization-3.1 work iteratively like this for a real-time analysis? Does it handle buffering somehow internally? I would assume that it doesn't know how to handle streaming chunks like this and that it can only analyze the content of a single chunk at a time. If so, what strategies are there for getting good results from a continuous stream for a real-time situation or a model that can only handle short chunks which very likely fall not on boundaries between spoken words?

Continue reading...