Skip to main content

Overview

Effective prompt engineering is the foundation of successful voice agents. Unlike text-based chatbots, voice agents require prompts optimized for spoken conversation, real-time interaction, and telephony constraints. This guide covers advanced techniques to create natural, effective, and reliable voice agent prompts.
What Makes Voice Prompts Different:
  • Optimized for spoken conversation, not text
  • Must handle real-time interruptions
  • Account for speech recognition limitations
  • Consider conversation pacing and flow
  • Designed for audio-only interaction

Fundamental Principles

1. Identity & Context

Always establish who the agent is and what it can do. Basic Identity:
Enhanced Identity:
Why This Matters: A clear identity helps the LLM maintain consistent personality and understand its boundaries. This reduces hallucinations and inappropriate responses.

2. Style Guardrails

Define how the agent communicates. Essential Style Rules:

3. Task Definition

Clearly define the agent’s goals and how to achieve them. Example Task Structure:

Advanced Techniques

Few-Shot Examples

Provide examples to guide behavior. Customer Service Examples:
Response Format Examples:

Chain of Thought

Guide the agent’s reasoning process. Problem-Solving Framework:
Variable Extraction Strategy:

Handling Edge Cases

Prepare for common failure modes. Speech Recognition Failures:
Silence Handling:
Out of Scope Requests:

Voice-Specific Optimizations

Speak-ability

Write prompts that sound natural when spoken. Written vs. Spoken:

Pacing Control

Control conversation rhythm. Chunking Information:
Confirmation Points:

Pronunciation Guidance

Help the TTS pronounce complex terms. Phonetic Hints in Prompts:

Context Management

Conversation History

Guide how the agent uses past context. Context Window Strategy:
Summarization Strategy:

Variable Context

Leverage available variables effectively. System Variable Usage:
Custom Variable Integration:

Jinja2 Template Syntax

Prompts support full Jinja2 template syntax. Variables are substituted before the prompt reaches the LLM, so you can use conditionals, filters, and expressions — not just simple substitution. Conditional blocks:
Time-based greetings:
Conditional personalization:
Filters and expressions:
Jinja2 processing happens server-side before the prompt is sent to the LLM. The LLM receives the rendered output — it never sees the template syntax itself.For the full reference of available filters, tests, operators, and control structures, see the Jinja2 Template Designer Documentation.

Testing Your Prompts

Iterative Refinement

1

Write Initial Prompt

Start with basic structure covering identity, style, and task
2

Test Common Scenarios

Run through 5-10 typical conversations
3

Identify Issues

Note where agent:
  • Misunderstands intent
  • Provides incorrect information
  • Uses wrong tone
  • Asks redundant questions
  • Fails to handle edge cases
4

Refine Specific Areas

Add examples, guardrails, or instructions for problem areas
5

Test Edge Cases

Try to break the agent with unusual inputs
6

Iterate

Repeat until quality meets standards

A/B Testing Prompts

Test Variations:

Advanced Patterns

Multi-Intent Handling

Escalation Triggers

Emotional Intelligence

Industry-Specific Examples

Healthcare

E-commerce

Financial Services

Next Steps

LLM Configuration

Optimize model settings for your prompts

Testing

Test and refine your prompts

Knowledge Base

Add knowledge to support your prompts

Flow Agents

Use prompts in complex flows