AI Agents

Voice and chat AI on a Twilio-compatible API.

VoiceTel AI Agents implements the Twilio AI Assistants surface so application teams can ship voice and chat agents — with tool calls, knowledge bases, and per-session memory — without rewriting client code for a new vendor.

AI Assistants API reference Sign up

What you get

  • Assistants you configure, not train

    Each assistant points at an LLM provider and model, carries a system prompt, and attaches the tools and knowledge bases it can use. No fine-tune cycle — change the prompt, change the behaviour.

  • Tool calls into your application

    Define function-call schemas as Tools, attach them to assistants, and the runtime invokes them when the model decides to. Outputs feed back into the conversation as model context.

  • Retrieval-augmented knowledge

    Knowledge resources ingest web pages, files, or raw text. Status and per-chunk inspection endpoints surface what's in the index. Attach knowledge to specific assistants for grounded answers.

  • Sessions, messages, and feedback

    Per-session message history is read-back-able for auditing. Feedback events score assistant turns so quality can be tracked over time without bolt-on logging.

Voice agents on VoiceML

Pair an assistant with <Connect><ConversationRelay> to put it on a live call. VoiceML handles call control, media, STT, and TTS; the assistant handles turn-by-turn reasoning, tool calls, and knowledge retrieval. The Twilio compatibility surface lets existing voice-agent clients re-host with a hostname swap.

For a side-by-side endpoint map, see the AI Assistants section of the compatibility matrix.

What's covered

  • Assistants — CRUD plus POST /Assistants/{id}/Messages for single-turn or full-session inference.
  • Tools — global tool catalog and per-assistant attachments.
  • Knowledge — CRUD over knowledge resources; chunk and indexing-status inspection; per-assistant attachments.
  • Sessions — read-only session and per-turn message history.
  • Feedback — submit and list per-assistant feedback events for quality scoring.
  • Policies — read-only catalog of guardrail policies available to assistants.

AI Agents FAQ

Which LLM providers can an assistant point at?

Provider and model are configured per assistant. Available providers depend on which integrations are enabled on your account — contact support to confirm the model you want to use is wired up.

Do Twilio AI Assistants client SDKs work against VoiceTel?

Yes — the v1 paths, field names, and SID shapes match the upstream Twilio surface. Point your client at voiceml.voicetel.com, swap credentials, and the existing SDK works.

How do I put an assistant on a live phone call?

Use VoiceML's <Connect><ConversationRelay> verb in your call-control response. VoiceML handles call control, media, STT, and TTS; the assistant handles model reasoning, tool calls, and retrieval. See the VoiceML docs for the wiring.

Where do tool calls actually execute?

Tools are function-call schemas the assistant invokes. The runtime serialises the call to your endpoint or webhook — your application performs the work and returns the result, which feeds back into the model's next turn.

How are knowledge bases indexed?

Knowledge resources ingest web URLs, uploaded files, or raw text. The indexing pipeline chunks and embeds content; GET /Knowledge/{id}/Status and GET /Knowledge/{id}/Chunks let you inspect ingestion progress and what's in the index.

Can I run multiple assistants for different use cases?

Yes — assistants are per-account resources. Spin up one per use case (support triage, sales qualifier, voice IVR replacement, etc.), attach the tools and knowledge each needs, and route inbound conversations to the right one from your application code.