Outbound AI voice calling is one of the highest-ROI applications of voice AI in 2026. Sales teams, appointment reminder systems, debt collection agencies, and survey companies are deploying AI agents that make thousands of simultaneous calls, hold natural conversations, handle objections, and push structured data back to CRMs — all without human involvement. We have built 25+ production outbound AI calling systems. This is the complete engineering guide: architecture, code, compliance, and production gotchas.
System Architecture Overview
A production outbound AI calling pipeline has four layers: (1) Campaign orchestration — managing contact lists, scheduling, retry logic, and concurrency limits. (2) SIP origination — placing the actual phone call via a CPaaS provider. (3) AI voice pipeline — the real-time STT to LLM to TTS loop that drives the conversation. (4) Post-call processing — recording storage, transcription, CRM updates, and analytics.
Technology Stack
- Pipecat (Daily/Pipecat-AI) — open-source voice agent framework handling the real-time audio pipeline
- Twilio Voice or Telnyx — SIP origination, phone number management, and answering machine detection
- Deepgram Nova-3 — streaming STT with sub-200ms first-word latency and high accuracy on phone audio
- GPT-4o or Claude 3.5 Sonnet — conversation LLM; use GPT-4o Realtime API for lowest end-to-end latency
- ElevenLabs Turbo v2.5 — TTS synthesis with natural prosody and optional custom voice cloning
- Redis — campaign state, active call tracking, and DNC deduplication
- PostgreSQL — contact list, call history, outcomes, and CRM sync log
- AWS S3 — encrypted call recording storage with lifecycle policies
Campaign Orchestration Layer
The orchestration layer manages contact lists, enforces concurrency limits, checks DNC (Do Not Call) lists, schedules retries for no-answers and voicemails, and tracks daily call attempt caps per contact. This runs as an async Python service using Redis for queue management.
import asyncio
import redis.asyncio as redis
from datetime import datetime, timezone
class CampaignOrchestrator:
def __init__(self, campaign_id: str, max_concurrent: int = 50):
self.campaign_id = campaign_id
self.max_concurrent = max_concurrent
self.redis = redis.from_url("redis://localhost:6379")
self.active_calls: dict = {}
async def run(self):
while True:
slots = self.max_concurrent - len(self.active_calls)
if slots <= 0:
await asyncio.sleep(1)
continue
contacts = await self.get_next_batch(slots)
for contact in contacts:
if await self.dnc_check(contact["phone"]):
await self.mark_dnc_skip(contact["id"])
continue
asyncio.create_task(self.place_call(contact))
await asyncio.sleep(0.5)
async def dnc_check(self, phone: str) -> bool:
return await self.redis.sismember(
f"dnc:{self.campaign_id}", phone
)
async def place_call(self, contact: dict):
self.active_calls[contact["id"]] = datetime.now(timezone.utc)
try:
await initiate_twilio_call(contact, self.campaign_id)
finally:
del self.active_calls[contact["id"]]
await self.update_call_attempt(contact["id"])SIP Origination with Twilio
Twilio Programmable Voice is the most common choice for outbound AI calling due to reliability, global reach, and excellent SDK support. When a call is answered, Twilio connects the audio stream to your Pipecat WebSocket server. Answering Machine Detection (AMD) is critical — enable it on every outbound call or your AI agent will converse with voicemail.
from twilio.rest import Client
async def initiate_twilio_call(contact: dict, campaign_id: str):
client = Client(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN)
call = client.calls.create(
to=contact["phone"],
from_=TWILIO_CALLER_ID,
# TwiML URL: connects answered call to Pipecat WebSocket
url=f"https://your-server.com/twiml/{campaign_id}/{contact['id']}",
status_callback=f"https://your-server.com/call-status/{contact['id']}",
status_callback_event=["answered", "completed", "no-answer", "busy", "failed"],
# AMD: detect voicemail, adds ~2-3s to call setup
machine_detection="DetectMessageEnd",
machine_detection_timeout=30,
timeout=30, # ring timeout in seconds
record=True,
recording_status_callback=f"https://your-server.com/recording/{contact['id']}",
)
return call.sidAlways enable Answering Machine Detection. Without AMD, your AI agent will deliver its opening pitch to voicemail. Twilio AMD adds roughly 2-3 seconds to call setup but prevents wasted AI pipeline minutes. For detected voicemail, either trigger a pre-recorded message drop or hang up and schedule a callback.
Pipecat AI Pipeline
Pipecat handles the real-time audio loop: receiving 8 kHz mu-law audio from Twilio Media Streams, resampling to 16 kHz for Deepgram, streaming LLM responses to ElevenLabs, and returning synthesised audio to Twilio. The target round-trip is under 600 ms from end-of-utterance to first syllable of agent speech.
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.services.deepgram import DeepgramSTTService
from pipecat.services.openai import OpenAILLMService
from pipecat.services.elevenlabs import ElevenLabsTTSService
from pipecat.transports.network.fastapi_websocket import (
FastAPIWebsocketTransport, FastAPIWebsocketParams
)
async def create_outbound_pipeline(websocket, contact: dict, system_prompt: str):
transport = FastAPIWebsocketTransport(
websocket=websocket,
params=FastAPIWebsocketParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_enabled=True,
vad_audio_passthrough=True,
)
)
stt = DeepgramSTTService(
api_key=DEEPGRAM_API_KEY,
params=DeepgramSTTService.InputParams(
model="nova-3",
language="en-US",
punctuate=True,
endpointing=200, # ms silence before finalising utterance
interim_results=True,
)
)
llm = OpenAILLMService(
api_key=OPENAI_API_KEY,
model="gpt-4o",
params=OpenAILLMService.InputParams(
temperature=0.4,
max_tokens=150, # keep responses concise for phone dialogue
)
)
tts = ElevenLabsTTSService(
api_key=ELEVENLABS_API_KEY,
voice_id=AGENT_VOICE_ID,
params=ElevenLabsTTSService.InputParams(
model="eleven_turbo_v2_5",
stability=0.5,
similarity_boost=0.75,
)
)
messages = [{"role": "system", "content": system_prompt}]
pipeline = Pipeline([transport.input(), stt, llm, tts, transport.output()])
await PipelineRunner().run(pipeline)Conversation Design for Outbound Calls
Outbound AI conversations require a very different prompt strategy from inbound agents. The human has no context — they just received a call from a number they may not recognise. The AI must identify itself and the company within the first 3 seconds, state the purpose immediately, handle objections gracefully, and always respect opt-out requests without question.
OUTBOUND_SYSTEM_PROMPT = """
You are Alex, an AI assistant calling on behalf of Acme Corp to confirm
the appointment on {appointment_date} at {appointment_time}.
RULES:
- Start every call: "Hi, this is Alex, an AI assistant from Acme Corp,
calling to confirm your appointment on {appointment_date}."
- Keep all responses under 2 sentences. Be concise and conversational.
- If they confirm: say "Perfect, see you then!" and end the call politely.
- If they want to reschedule: collect preferred date/time and confirm it.
- If they say "remove me", "stop calling", or similar: say "Of course,
I have noted that and you will not receive any further calls from us."
Then end the call immediately.
- If asked "are you a real person?": always say you are an AI assistant.
- Do NOT discuss any topic outside this appointment confirmation.
- Maximum call duration is 3 minutes. Wrap up gracefully if approaching.
Contact name: {contact_name}
""".strip()Latency Optimisation
| Optimisation | Latency Saving | How |
|---|---|---|
| Deepgram endpointing 200ms | ~300ms | endpointing=200 in STT params |
| GPT-4o streaming + early TTS start | ~400ms TTFT | stream=True, begin TTS on first token chunk |
| ElevenLabs streaming output | ~350ms TTFT | streaming=True in TTS service params |
| Pre-warm Pipecat server pool | ~800ms on first call | Keep-alive WebSocket pool at startup |
| Barge-in (interrupt handling) | UX critical | VAD detects user speech, cancels current TTS |
| Co-locate AI servers with CPaaS | ~80ms RTT | Run Pipecat in same AWS region as your Twilio PoP |
Enable barge-in (interrupt handling) for outbound campaigns. Users will frequently start talking before the AI finishes a sentence. Without barge-in, the AI ignores them until it finishes speaking — this creates an extremely frustrating experience and dramatically increases hang-up rates. Pipecat VAD handles this natively.
CRM Integration and Post-Call Processing
Every completed call should trigger a webhook to update your CRM with the outcome, transcript, and any structured data collected during the conversation. A post-call Lambda triggered by Twilio's recording webhook handles this asynchronously.
- Outcome classification: run a GPT-4o call on the transcript to classify as confirmed, rescheduled, not_interested, voicemail, no_answer, or opted_out
- Transcript storage: Deepgram returns a full transcript via webhook after the call — store in PostgreSQL and archive to S3
- CRM sync: push outcome and extracted data fields to Salesforce or HubSpot via REST API within seconds of call completion
- Opt-outs: immediately add phone numbers to the Redis DNC set and set the CRM do-not-call flag — never retry opted-out contacts
- Analytics: track connect rate, conversion rate, average handle time, opt-out rate, and AMD accuracy per campaign
TCPA and Compliance in 2026
Outbound AI calling in the US is regulated by the Telephone Consumer Protection Act (TCPA) and updated FCC rules from 2024-2025 that specifically address AI-generated voice calls. Non-compliance can result in $500-$1,500 per-call fines — with class action exposure running into the millions for large campaigns.
- Prior written consent: for marketing calls to mobile numbers you must have explicit written consent. Document the consent source, timestamp, and IP address in your database.
- AI disclosure: FCC rules updated in 2025 require AI-generated voice calls to disclose within the first 2 seconds that the caller is an AI. Build this into every system prompt as a hard rule.
- DNC compliance: scrub contact lists against the National Do Not Call Registry before every campaign send. Update your internal DNC list immediately when any contact opts out.
- Calling hours: TCPA restricts calls to 8am-9pm in the recipient's local timezone. Use a timezone-aware scheduler that resolves timezone from the area code or ZIP code.
- Call recording consent: 38 US states require one-party or two-party consent for recording. For two-party consent states, add an audio disclosure at the start of every recorded call.
The FCC's 2025 AI Calling Order requires all AI-generated outbound calls to: state within 2 seconds that the call uses AI, provide a simple verbal opt-out mechanism, and honour opt-out requests immediately and permanently. Failure to comply creates per-call FCC fine exposure on top of TCPA class action liability.
Production Performance Benchmarks
| Metric | Our Production Numbers |
|---|---|
| Average end-to-end latency (end-of-utterance to agent speech) | 480 ms |
| Connect rate (answered vs dialled) | 38-55% (varies by industry and time of day) |
| Voicemail rate | 25-40% |
| Opt-out rate (well-designed campaigns) | 1-4% |
| AI conversation success rate | 91% (vs 84% for DTMF IVR) |
| Cost per connected AI minute | $0.08-$0.14 (TTS + STT + LLM + telephony) |
| Cost per connected human agent minute | $2.50-$4.00 |
Outbound AI voice calling delivers a 15-30x cost reduction versus human agents for structured outreach such as appointment reminders, payment notifications, lead qualification, and customer surveys. The technical barrier is real — real-time audio pipelines, latency optimisation, AMD, TCPA compliance, and CRM integration all require engineering investment. But once the infrastructure is in place it scales to thousands of simultaneous calls with negligible incremental cost. Our AI voice team has built 25+ production outbound calling systems across healthcare, real estate, finance, and e-commerce, and can architect and deploy a compliant, low-latency campaign pipeline tailored to your industry.