[Python] Audio from Twilio WebSocket is distorted in browser & fails with AWS Transcribe after...

I'm working on a real-time call streaming system using Twilio, Flask, and Vosk for browser-based transcription. I also forward the audio to AWS Transcribe, but the results are empty or incorrect.

The core issue: The audio received from Twilio in μ-law format is distorted when played in the browser and fails to produce valid transcriptions via AWS Transcribe.

⚙️ Backend Summary (Python Flask): Twilio sends μ-law (8kHz mono) audio via WebSocket.

I convert the μ-law audio to PCM 16-bit using Python’s audioop:

audio = audioop.ulaw2lin(audio, 2)
audio = audioop.ratecv(audio, 2, 1, 8000, 16000, None)[0]



Then I create a WAV file:

def pcm_to_wav_bytes(pcm_data, sample_rate=16000, sample_width=2, channels=1):
buffer = io.BytesIO()
with wave.open(buffer, 'wb') as wf:
wf.setnchannels(channels)
wf.setsampwidth(sample_width)
wf.setframerate(sample_rate)
wf.writeframes(pcm_data)
return buffer.getvalue()



This base64-encoded WAV is sent via WebSocket to frontend.

Frontend Summary (JavaScript):

<audio id="audio" autoplay></audio>
<button id="start">Start Listening</button>
<div id="transcript"></div>

<script>
let audioCtx;

document.getElementById("start").onclick = () => {
audioCtx = new (window.AudioContext || window.webkitAudioContext)();
const ws = new WebSocket("wss://example.com/stream");

ws.onmessage = async (e) => {
const data = JSON.parse(e.data);
if (data.event === "media" && data.media?.payload) {
const byteArray = Uint8Array.from(atob(data.media.payload), c => c.charCodeAt(0));
try {
const buffer = await audioCtx.decodeAudioData(byteArray.buffer);
const source = audioCtx.createBufferSource();
source.buffer = buffer;
source.connect(audioCtx.destination);
source.start();
} catch (err) {
console.error("decodeAudioData error:", err);
}
}
};
};
</script>



What I've Tried: Confirmed μ-law decode logic with audioop.ulaw2lin.

Successfully created WAV with standard headers.

Ensured audio has sample rate 16000, mono, 16-bit PCM.

Base64 sent to frontend works, but audio is distorted in browser.

Same WAV audio sent to AWS Transcribe yields either:

No transcript at all

Random/wrong transcription (garbled)

❌ Issues: Distorted audio in browser

decodeAudioData sometimes throws EncodingError

AWS Transcribe gives empty or wrong results, despite proper format

Confirmed that audio does contain human speech

✅ Goal: Clean playback of Twilio call audio in browser via Web Audio API

Successful transcription via AWS Transcribe

Question: What is the correct way to convert μ-law audio from Twilio into a WAV file that can be:

Played cleanly in the browser using Web Audio API?

Sent to AWS Transcribe for accurate speech recognition? your text Is there something wrong with my conversion process or WAV headers?your text

Continue reading...