Finishing and deploying your ElevenLabs voice agents

Inbound receptionist and outbound qualifier. How I would scope, complete and deploy both, and how we prove they are ready for live client use.

Akhil, for Fiona Challis and John Guilfoyle
5 October 2026

ElevenLabs runs the conversation. A small service that you own handles everything that touches your data.

Each problem you saw maps to a setting, and each is proven on the phone

Problem you saw	ElevenLabs setting	What I would do
Awkward pauses	Turn eagerness, turn timeout	Normal for chat. Patient only where callers spell or give numbers.
Dead air while thinking	Soft timeout	A short "one moment" when the model runs long.
Silence during a lookup	Pre-tool speech, tool call sounds	"Let me check the diary" while the tool runs.
Slow replies overall	LLM choice, prompt length, knowledge base	Fast model, short prompt, FAQs moved to the knowledge base.
Slow integrations	Webhook tools	Direct to the Go service, target under one second.
Agent stops on "yeah", "mm"	Interruption ignore terms	Backchannels no longer cut the agent off.
Talking over the caller	Turn model, per-node settings	Tuned against recorded barge-in tests.
Background noise	Interruption and turn-taking settings	Test calls from a car, a kitchen, a busy office. Tune against them.
Distance delay	Region, phone routing	UK region and UK numbers for live clients.

Email is captured in parts, confirmed, validated live, with a safe fallback for landlines

Capture the part before the @, then the domain separately.
Clarify uncertain characters with phonetic words. "B for Bravo or D for Delta?"
Read the full address back slowly and wait for a clear yes.
Validate live: format check and a lookup that the domain accepts mail.
Still unclear? Mobile callers get an SMS link to type it. Landline callers have their callback number confirmed and the lead is flagged for a person.
Never send a calendar invite to an unconfirmed address.

A wrong invite is worse than a human callback. An unconfirmed address holds the booking for a person to verify.

Agent 1. Inbound receptionist

Answers the client's line, handles questions, captures name, email and preferred callback number, books into the diary, sends the invite, and hands over to a person when needed.

What is different from the first build

Client settings loaded from the dialled number before the greeting
Email step uses the capture flow on the previous slide
Diary slots come from the live calendar, not a script
Handover to a staff number; if nobody answers, callback details are taken
Any failed lookup ends with details captured, never a dropped call

Agent 2. Outbound qualifier

Calls leads already in your system, confirms their details, asks the qualifying questions, books a follow-up if they qualify, and writes the outcome back to the CRM.

Guardrails

Consent and calling hours checked before dialling
Lead context loaded at call start
Voicemail detected; short message or hang up
Every outcome written to the CRM

Three ways a client's existing number reaches the agent. It can differ per client.

Twilio native

Fastest setup. Outbound, SMS and voicemail detection in one account. Best for v1 and testing. UK numbers need a regulatory bundle.

SIP trunk

Keeps existing numbers and phone system. Needs carrier or PBX access. Best for clients with a PBX.

Call forwarding

No change on the client side. Works with any carrier. Caller ID passthrough and forwarding cost to check.

Handover and compliance

Human handover uses the ElevenLabs transfer-to-number tool. Outbound to leads needs consent and TPS screening. Who owns that per client is to confirm.

Calendar, CRM and Microsoft 365

Calendar and invites

Microsoft Graph on your tenant. Read free/busy, create the event, send the Outlook invite.

John owns the app registration and the scope of mailboxes it can touch.

Which calendars to confirm

Shared mailbox, per-staff diaries, or resource calendars. All three work. The choice changes how slots are offered and which Graph permissions are needed.

Google Calendar or Cal.com supported for a client not on M365.

CRM to confirm

A CRM adapter in the service, kept open until we know the CRM.

HubSpot and Salesforce have native ElevenLabs integrations. Anything else goes through the adapter or Make.

Make or direct webhooks? While the caller is waiting, every call goes direct to the service. After the call, CRM updates, summaries and alerts can run in Make, where your team can change them.

Hosted in Azure UK South, inside your estate

Security

Secret header on every tool call, held in ElevenLabs workspace secrets. HMAC check on the post-call webhook. HTTPS only, secrets in Key Vault. Least-privilege Graph permissions.

Reliability

Idempotent booking: retries never double-book. Graceful failure: the agent apologises and takes callback details. After-call jobs retried with backoff.

Observability

Every tool call logged with latency, keyed by conversation ID. Outcome stored per call. ElevenLabs evaluation criteria score every call.

Data policy to confirm

UK GDPR, data in UK South. Agreed retention for recordings and transcripts. Recording disclosure in the greeting if required.

Every test is a real phone call, and nothing goes live until the pack passes

Scenario	How we test	Pass when
Happy path booking	Inbound call, book a slot	Correct slot in the diary, invite received
Difficult emails	Hard addresses: numbers, dots, unusual domains	Email correct, or safely flagged. Never a wrong invite
Landline caller	Calls from a landline	Callback number confirmed, lead flagged when email is unclear
Mobile fallback	Force a failed email capture	SMS link arrives, typed email lands on the booking
Background noise	Car, kitchen, busy office, street	Agent stays on track, no false interruptions
Interruptions	Caller barges in, says "yeah", "okay"	Yields to real interruptions, ignores backchannels
Integration failure	Calendar or CRM switched off	Agent apologises, takes details, follow-up created
Retry, double booking	Replay the same booking request	Exactly one booking
Human handover	Ask for a person; staff line busy	Transfer works; callback taken when no answer
Voicemail, outbound	Call a number that goes to voicemail	Short message or hang up, outcome logged
Qualification, outbound	Qualified, not now, not interested	Correct outcome and next step in the CRM
Latency	Measure each turn on real calls	Responses feel natural, tool calls under target

Two weeks from access to a live client number

Start, completion and cost

Day 1

Start as soon as access is in place. Kickoff and review of the current build.

Day 7

v1: both agents working end to end on a test number.

Day 13

Pilot live on the first client number, with monitoring and handover docs.

£1,500 fixed for both agents, or £200 per day with work agreed before it starts.

Covers finishing and tuning the existing build, calendar, CRM and M365 integrations in one environment, phone testing and deployment. ElevenLabs, telephony and Azure usage are billed to you. Scope confirmed after reviewing your build; anything outside it is agreed before it starts.