
ByAhsan Nadeem@ashtech52Lead Full-Stack Engineer
AI voice agents for call centres now handle predictable inbound calls well: after-hours enquiries, bookings and order status. They still struggle with emotional or open-ended calls, and they bring legal duties. In the US, an AI voice falls under robocall law; in the EU, callers must be told they're talking to an AI.
This guide covers where voice agents help, where they don't, what a good setup looks like and what the law requires.
Where AI voice agents genuinely help call centres
The best results come from inbound, well-defined calls where the caller wants something specific:
- After-hours answering. Instead of voicemail, callers get answers to common questions and can book a slot or leave details for a callback.
- Booking and rescheduling. Appointments, deliveries and site visits, connected to your real calendar.
- Order and job status. "Where's my delivery?" or "When is the engineer coming?", answered from your systems.
- Call routing and triage. Working out why someone is calling and passing them to the right team with a summary, so they don't repeat themselves.
- Overflow at peak times. Taking simple calls when every human agent is busy, so queues shrink.
Where AI phone agents still struggle
- Emotional or high-stakes calls: complaints, bereavement, medical worries, debt. People want a human, and getting it wrong is costly.
- Open-ended problems that need judgement or exceptions to policy.
- Noisy lines, strong accents and poor connections. Speech recognition has improved a lot, but its mistakes still compound over a conversation.
- Anything without good data behind it. An agent that can't see your calendar or order system can only take messages.
The rule of thumb: automate the predictable, hand off the rest. A good voice agent knows when to transfer, and does it fast, with a summary.
What a good AI voice agent setup looks like
Speed
Conversation feels natural only when replies come back in about a second or less. Every layer adds delay: speech-to-text, the language model, your own systems, text-to-speech. Choose components and hosting with latency in mind.
Integration with your phone system (SIP)
Most businesses already have a phone system: a cloud PBX, an on-premise system such as Asterisk or FreePBX, or a contact-centre platform. The voice agent should plug into it, usually over SIP, so that:
- calls reach the agent through your existing numbers;
- the agent can transfer to a human on the same system, with context;
- recordings and call logs stay in one place.
If the phone system itself needs work first (trunks, queues, routing), that's the job of our VoIP and call-centre systems service.
Grounded answers
The agent should answer only from your approved information (policies, prices, availability) and say "let me get someone who can help" when it doesn't know. That's the same discipline as any customer-facing AI: retrieval from your own content (our guide to RAG vs fine-tuning explains the trade-off), clear limits, and testing with real questions.
Monitoring
Review transcripts weekly, track how often and why calls are transferred, and measure outcomes (bookings, issues resolved), not just call length.
AI voice agent laws: what the US, EU and UK require
This is a practical summary, not legal advice. Rules vary by country and US state, so check with a lawyer for your situation.
United States: AI voices count as "artificial" under the TCPA
In February 2024, the Federal Communications Commission (FCC) ruled that AI-generated voices fall within the meaning of "artificial" voice under the Telephone Consumer Protection Act (TCPA). In practice:
- Outbound calls to consumers using an AI voice need the called party's prior express consent, the same as prerecorded calls. For telemarketing, that means prior express written consent.
- Calls must identify who is calling and, for prerecorded or artificial-voice calls, include the required disclosures and opt-out options.
- The TCPA allows statutory damages of $500 to $1,500 per call in private lawsuits, which add up quickly across a calling list.
- State laws add their own rules, including on call recording: several states require the consent of everyone on the call.
Inbound calls, where the customer calls you, are a much lower-risk place to start. Disclosure is still good practice.
Outbound calling also needs authenticated caller ID. Under the FCC's STIR/SHAKEN framework, carriers sign calls to show the number is legitimate, and unsigned or poorly attested calls are more likely to be labelled spam or blocked.
European Union: tell people they're talking to an AI
Since 2 August 2026, Article 50 of the EU AI Act requires that people are told when they're interacting with an AI system, unless it's obvious from the context. For a phone agent, that means saying so at the start of the call; a line buried in your terms and conditions doesn't count.
This obligation was not postponed by the EU's 2026 "digital omnibus" package, which delayed other parts of the Act. GDPR also applies: call recordings and transcripts are personal data, so you need a lawful basis, a retention limit and clear information for callers.
United Kingdom
Automated marketing calls in the UK require prior consent under the Privacy and Electronic Communications Regulations (PECR). Live marketing calls must not go to numbers registered with the TPS or CTPS unless the person has consented. The ICO enforces both rules and can fine organisations that break them.
How to start with an AI voice agent safely
- Start inbound. Pick one call type (after-hours enquiries, booking, order status).
- Disclose it. "You're speaking with [Company]'s AI assistant. I can help with X, or connect you to the team."
- Connect it to real data, read-only at first.
- Make the handoff easy: "speak to a person" should always work, fast.
- Record, review and measure, with consent and a retention policy.
- Only then consider outbound, with consent records, do-not-call checks and legal review.
Frequently asked questions
As above, these answers are a summary, not legal advice.
Does an AI voice agent have to say it's an AI?
In the EU, yes: since 2 August 2026, Article 50 of the AI Act requires it unless it's obvious from the context, so say so at the start of the call. Elsewhere it's still good practice, even on inbound calls.
Can I make outbound calls with an AI voice in the US?
Only with the right consent. Since the FCC's February 2024 ruling, AI voices count as "artificial" under the TCPA, so outbound calls to consumers need prior express consent, and telemarketing needs prior express written consent. Statutory damages are $500 to $1,500 per call.
Can an AI voice agent work with our existing phone system?
Usually, yes. Most connect over SIP to a cloud PBX, an on-premise system such as Asterisk or FreePBX, or a contact-centre platform, so calls arrive on your existing numbers and the agent can transfer to a human on the same system.
The bottom line
AI voice agents can now take real work off a busy phone team. Start inbound, connect the agent properly to your phone system and data, make human handoff effortless, and take disclosure and consent seriously from day one. Our guide to AI agents for small business covers where AI fits in a small team more broadly.
Considering a voice agent for your phone lines? Book a free consultation to talk through your calls with the team behind our LLM integration and VoIP services.
Sources
- FCC, "FCC Confirms that TCPA Applies to AI Technologies that Generate Human Voices" (February 2024): https://www.fcc.gov/document/fcc-confirms-tcpa-applies-ai-technologies-generate-human-voices
- Wilson Sonsini, "FCC Rules AI-Generated Voices Are 'Artificial' Under the TCPA": https://www.wsgr.com/en/insights/fcc-rules-ai-generated-voices-are-artificial-under-the-tcpa.html
- FCC, Combating Spoofed Robocalls with Caller ID Authentication (STIR/SHAKEN): https://www.fcc.gov/call-authentication
- EU AI Act, Article 50 (transparency obligations): https://artificialintelligenceact.eu/article/50/
- ICO, guidance on electronic and telephone marketing (PECR): https://ico.org.uk/for-organisations/direct-marketing-and-privacy-and-electronic-communications/



