Pick the speech-to-text engine that fits your use case.

Using <Gather> for speech

Quick start

By default, <Gather> collects DTMF keypad input. To listen for spoken words, set input="speech":

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Gather input="speech" action="/handle-speech" method="POST">
        <Say>Please say your account number after the tone.</Say>
    </Gather>
</Response>

When the caller finishes speaking, VoiceML posts the transcript to your action URL. Your handler receives standard Twilio-compatible form fields:

Example handler logic (pseudo-code):

if Confidence > 0.8:
    route_to_agent(SpeechResult)
else:
    ask_to_repeat()

Input modes

For a menu that accepts both "press 1" and "say one":

<Gather input="dtmf speech" numDigits="1" action="/menu" method="POST">
    <Say>Press 1 for sales, or say sales.</Say>
</Gather>

Speech attributes

These attributes control how speech is captured and transcribed:

Spanish example with hints:

<Gather input="speech"
        language="es-ES"
        hints="Madrid, Barcelona, Valencia"
        action="/handle-speech"
        method="POST">
    <Say language="es-ES">Diga su ciudad.</Say>
</Gather>

Nested prompts

Text inside <Gather> — <Say>, <Play>, or <Pause> — plays while collection is active, just like Twilio. The caller can respond during or after the prompt.

Speech-recognition providers

VoiceML doesn't lock you into one speech engine. A free English default ships with every account so you can prototype immediately. For production, switch to the vendor that gives you the best result on the dimensions you care about — latency, accuracy on accented audio, language coverage, model selection, where the audio is processed, or just the contract you already have. Vendor flexibility is a deliberate strength of the platform.

Whichever provider you pick (or leave on the default) applies to both:

• <Gather input="speech"> and <Gather input="dtmf speech">
• <Connect><ConversationRelay> live conversations

Supported providers

Real-time vs. batch. Real-time providers (VoiceTel STT default, Deepgram, AWS, Azure) transcribe audio as the caller speaks — best for interactive menus and ConversationRelay. Batch providers (Google, OpenAI, whisper.cpp) receive the recorded clip in one request after the caller finishes — useful when you already have an OpenAI key or run your own whisper.cpp server. Both work with the same voice markup.

Picking the right engine. For most production workloads the default is a starting point, not the finish line. A real-time vendor (Deepgram or Azure) typically outperforms it on noisy audio, accented English, or specialized vocabulary; batch vendors win when you already pay them for other work or want self-hosted control. The defaults exist so you don't have to make that decision before your first test call.

How to enable a provider

Configure speech recognition from your Account page in the VoiceML portal (section Speech-to-text (STT)):

1. Open the Account page and find the Speech-to-text (STT) section.
2. Choose your STT provider from the dropdown, or leave it on none — speech recognition disabled to turn transcription off.
3. Expand that provider's section and enter its credentials (and optional model or region fields).
4. Save. New calls immediately use the new provider — no restart or redeploy needed.

API keys and secrets are write-only in the portal — leave a secret field blank to keep the value already saved, or paste a new value to replace it.

Provider setup guides

Each guide walks through portal setup with concrete examples. After saving, place a test call that uses <Gather input="speech">.

VoiceTel STT — default for English

What you need: nothing. This is the default on new accounts so English calls just work while you're prototyping. Real-time streaming, English only, no per-minute charge. Most production deployments eventually swap it for a vendor better suited to their workload — see the picking-the-right-engine note above.

Portal fields (Account → Speech-to-text → VoiceTel STT (default, English, free)):

The default runs on VoiceTel infrastructure and is restricted to English (en-*) <Gather language="…"> values. Calls in other languages need a configured BYO provider, or the call returns a speech-not-configured error.

Example voice markup:

<Gather input="speech" language="en-US" action="/handle-speech" method="POST">
    <Say>How can we help you today?</Say>
</Gather>

Deepgram

What you need: a Deepgram account and API key with speech-to-text access.

Portal fields (Account → Speech-to-text → Deepgram (streaming)):

Popular models:

Leave Model empty to use Deepgram's default for your account.

Example voice markup:

<Gather input="speech"
        language="en-US"
        speechModel="nova-2"
        hints="billing, support, cancel"
        action="/handle-speech"
        method="POST">
    <Say>How can we help you today?</Say>
</Gather>

The speechModel attribute overrides the portal model for that gather only.

AWS Transcribe

What you need: an AWS account with Transcribe enabled, an IAM access key and secret with permission to stream transcription, and the AWS region where the key was created.

Portal fields (Account → Speech-to-text → AWS Transcribe (streaming)):

Popular regions:

Set the region to match where your AWS credentials are valid. Recognition language comes from the <Gather language="…"> attribute.

Example voice markup:

<Gather input="speech" language="en-US" action="/handle-speech" method="POST">
    <Say>Please state your reason for calling.</Say>
</Gather>

Microsoft Azure Speech

What you need: an Azure Speech resource, its subscription key, and the region where the resource was created (for example eastus). The region must match your resource — a wrong region returns an authentication error.

Portal fields (Account → Speech-to-text → Azure Speech (streaming)):

Popular regions:

Set <Gather language="…"> to the locale you want recognized — for example en-US, es-ES, or fr-FR. See Azure speech language support for available locales.

Example voice markup:

<Gather input="speech" language="fr-FR" action="/handle-speech" method="POST">
    <Say language="fr-FR">Comment puis-je vous aider?</Say>
</Gather>

Google Cloud Speech-to-Text

What you need: a Google Cloud project with the Cloud Speech-to-Text API enabled, and a service-account key JSON with permission to call that API.

Portal fields (Account → Speech-to-text → Google Speech-to-Text (batch)):

Popular models (optional — enter in the portal Model field or pass via <Gather speechModel="…">):

Transcription runs after the caller finishes speaking (batch mode). For the lowest latency on live back-and-forth conversations, consider Deepgram, AWS, or Azure instead.

Example voice markup:

<Gather input="speech"
        language="en-US"
        speechModel="phone_call"
        action="/handle-speech"
        method="POST">
    <Say>Please say your full name.</Say>
</Gather>

OpenAI Whisper

What you need: an OpenAI API key. VoiceML sends recorded speech to OpenAI's transcription endpoint — you do not configure a custom URL.

Portal fields (Account → Speech-to-text → OpenAI Whisper (batch)):

Leave Model empty to use whisper-1. OpenAI Whisper is the fastest provider to try — paste an API key, select OpenAI as your STT provider, save, and test. Transcription runs after the caller finishes speaking.

Example voice markup:

<Gather input="speech" language="en-US" action="/handle-speech" method="POST">
    <Say>Tell us briefly why you're calling.</Say>
</Gather>

whisper.cpp (self-hosted)

What you need: a running whisper.cpp HTTP server that accepts audio and returns a transcript, reachable from VoiceTel over the public internet (or your configured network path).

Portal fields (Account → Speech-to-text → whisper.cpp (batch) — self-hosted server):

Popular models (depends on your server):

Leave Model empty to use your server's default.

Example voice markup:

<Gather input="speech" language="en-US" action="/handle-speech" method="POST">
    <Say>Please describe your issue in a few words.</Say>
</Gather>

ConversationRelay

<Connect><ConversationRelay> runs a live voice conversation: VoiceML transcribes the caller's speech and sends text to your WebSocket endpoint; your endpoint sends back text for VoiceML to speak.

Speech recognition uses the same STT provider you configure on the Account page. Text-to-speech uses your TTS provider (see the text-to-speech guide).

Both must be configured before ConversationRelay can run:

• STT — select a provider and save credentials under Speech-to-text.
• TTS — select a provider (or use built-in voices) under Text-to-speech.

Minimal ConversationRelay voice markup:

<Response>
    <Connect>
        <ConversationRelay url="wss://your-app.example/conversation"
                           language="en-US" />
    </Connect>
</Response>

Your WebSocket receives a setup message when the session starts, prompt messages with finalized caller transcripts, and sends text messages with replies for VoiceML to speak.

Migrating from Twilio

VoiceML uses the same <Gather> syntax as Twilio:

• input="speech", language, hints, speechTimeout, and the action callback fields (SpeechResult, Confidence, Language) match Twilio's shape.
• Existing voice markup that collects English speech works on day one against the VoiceTel STT default — no provider setup, no credentials. Pick a vendor in the portal when you're ready to tune for production.
• On Twilio, speech ran on Twilio's shared recognizer with per-minute billing baked into the platform. On VoiceML you choose: free defaults for the common case, or your own Deepgram, AWS, Azure, Google, OpenAI, or whisper.cpp account so billing and data handling follow your provider agreement — no VoiceTel platform markup on the BYO route.

For ConversationRelay, the WebSocket message types (setup, prompt, text, end) follow Twilio's ConversationRelay protocol.

Troubleshooting