Marcos Medina
Leer en español
All work

01/04 Conversational AI × Telephony

AI Voice Agents

Voice agents that pick up real phone lines, understand the request, validate it and get it done.

Role
Architecture & development
Context
Integrated with real telephony
Focus
Latency, reliability, validation
  • ≈1.1–1.3 s End-to-end latency, own measurement in specific tests
  • Cerebras → Groq Primary and fallback models behind one proxy
  • P-Series Yeastar PBX integration

Interactive demo — Demo — synthetic data

  1. 01

    Ring

    The call arrives through the Yeastar P-Series PBX and is handed to the voice agent.

  2. 02

    Understand

    ElevenLabs handles voice in and out. Each turn passes through a custom orchestration proxy: Cerebras first, Groq as fallback.

  3. 03

    Validate

    Required fields and business rules are checked before anything happens. If something is missing, the agent asks instead of guessing.

  4. 04

    Act

    Validated requests become actions: messages taken, people notified, workflows set in motion.

Overview

Phone calls are still where many requests begin, and where they are most often lost: missed calls, incomplete messages, callbacks that never happen. This system puts a conversational agent on the line that can hold a natural conversation, collect exactly what is needed and trigger the right action.

It is built to live inside existing telephony rather than next to it. Calls arrive through a Yeastar P-Series PBX, and the outcome of each conversation becomes an automated action: a message delivered, a person notified, a workflow set in motion.

The challenge

Voice is unforgiving. In a text chat, two seconds of thinking is fine; on a phone call it feels like the line has dropped. Every component, from speech recognition to the language model to speech synthesis, eats into a tight latency budget.

Language models are probabilistic; the actions they trigger are not. A message without a callback number is useless, and a request that breaks a business rule should never reach automation.

Architecture

Caller Inbound call Yeastar P-Series IP PBX ElevenLabs Voice & agents Model proxy Own orchestration Cerebras Primary LLM Groq Fallback LLM Validation Fields & rules Actions Messages & workflows Caller Inbound call Yeastar P-Series IP PBX ElevenLabs Voice & agents Model proxy Own orchestration Cerebras Primary LLM Groq Fallback LLM Validation Fields & rules Actions Messages & workflows
Hover or focus a component to trace its connections.

Key decisions

  1. 01

    A proxy between the agent and the models

    Instead of wiring the agent to a single provider, every model request goes through a custom orchestration proxy. It routes to Cerebras as the primary provider for its inference speed and falls back to Groq, so a slow or failing provider does not turn into silence on the line.

  2. 02

    Validation as a first-class layer

    Required fields and business rules are checked outside the model. If something is missing or inconsistent, the agent asks again instead of guessing, and only validated data reaches automation.

  3. 03

    Telephony-native integration

    Integrating with the Yeastar P-Series PBX means the agent is reachable the way callers already reach the business, without changing how anyone makes a call.

  4. 04

    Measure the whole path

    Latency is measured end to end, from the moment the caller stops speaking to the moment they hear the reply. In specific tests of my own it came in at roughly 1.1–1.3 seconds. It is a personal measurement, not a guaranteed figure.

Stack

  • ElevenLabs
  • Cerebras
  • Groq
  • Orchestration proxy
  • Yeastar P-Series
  • Rule validation
  • Automation

Notes

Every conversation shown on this site is a synthetic demo. No real calls, recordings or customer information are published.