Voice Assistant Quickstart

In this tutorial, you'll learn how to configure a voice assistant with Telnyx. You won't have to write a single line of code or create an account with anyone besides Telnyx. You'll be able to talk to your assistant over the phone in under five minutes.

After this tutorial covers the basics, we will explore some optional enhancements, such as changing your voice or language model providers and empowering your assistant with built-in tools.

Try out the public demos for a real example of the finished product.

There are 2 required steps for this tutorial

1. Configure your AI Assistant
2. Configure the voice settings

First, navigate to the AI Assistants tab in the portal. You will create a new assistant to configure what context your assistant has and how it behaves.

For this tutorial, we will use a blank template.

You can use the following instructions and greeting, or use your own.

Instructions:

You are an intelligent and concise voice assistant. This is a {{telnyx_conversation_channel}} happening on {{telnyx_current_time}}. The agent is at {{telnyx_agent_target}} and the user is at {{telnyx_end_user_target}}.

Greeting:

Hi {{first_name}}, this is Nyx, your friendly Telnyx Assistant! How can I help you today?

Telnyx provides these system variables to inject details about a specific call into the instructions and greeting:

Variable	Description	Example
{{telnyx_current_time}}	The current date and time in UTC	Monday, February 24 2025 04:04:15 PM UTC
{{telnyx_conversation_channel}}	This can be phone_call , web_call, or sms_chat	phone_call
{{telnyx_agent_target}}	The phone number, SIP URI, or other identifier associated with the agent.	+13128675309
{{telnyx_end_user_target}}	The phone number, SIP URI, or other identifier associated with the end user	+13128675309
{{telnyx_sip_header_user_to_user}}	The User to User SIP header for the call, if applicable	cmlkPTM0Nzg1O3A9dQ==;encoding=base64;purpose=call
{{telnyx_sip_header_diversion}}	The Diversion SIP header for the call, if applicable	<sip:bob@example.com>;reason=user-busy
{{call_control_id}}	The call control ID for the call, if applicable	v3:u5OAKGEPT3Dx8SZSSDRWEMdNH2OripQhO

You can also define your own custom dynamic variables and set them via webhook, custom SIP headers, or an outbound API call.

In this example, you've given the assistant access to the most commonly used system variables and a custom {{first_name}} variable in the greeting.

Notice that by default, the Hangup tool is configured. This enables your assistant to end the call at an appropriate time.

In this step, you can use the default voice settings, click Create, and enable the agent for calls when prompted.

Or feel free to explore the wide range of voices in the playground.

If you have already purchased a Telnyx number with voice features, you can immediately assign it to your assistant.

You can also click Next, as completing this step is not needed for testing your assistant.

Telnyx assistants work with messaging as well. This is not covered in this tutorial.

You should now be able to interact with your voice assistant. When you are ready, you can ask the assistant to hang up to test out the Hangup tool.

If you assigned a number to your assistant, you can also have your assistant call you:

• from the portal
• via API/CLI
• in an automated workflow like Zapier using our TeXML Outbound Call action.

Note

If you are using the curl provided, make sure to set the TELNYX_API_KEY environment variable with your API Key

You can see all historical conversations in the Conversation History tab

This example shows a conversation where the Handoff tool was used. Let's dive into that next...

Besides Hangup, we offer several additional built-in tools to empower your agent to take real-world actions.

Webhook

With the webhook tool, your agent can make API requests.

You can configure headers (with integration secrets), along with path, query, and body parameters. You can also reference dynamic variables in the webhook path or in the parameter descriptions.

After you've saved a webhook tool on your assistant, you can test it out with sample data by clicking the play button icon on the tool.

Handoff

With the handoff tool, you can enable multiple assistants to support a user in a single conversation. The handoff is transparent to the user: assistants share the same context and voice, allowing for a unified experience where a variety of tasks can be handled by a team of specialists under the hood.

Transfer and SIP Refer

With the transfer and SIP Refer tools, your agent can transfer or refer a call to a list of named targets.

Send DTMF

With the Send DTMF, your agent can interact with legacy IVR systems.

You can use the Knowledge Bases tab to enable your assistant to retrieve your custom context.

First, provide a name

Then upload files or provide a URL

You can automatically run structured and unstructured analysis on every assistant conversation using the Insights tab.

You can assign an Insight Group to your assistant, which can have one or more Insights. Insights can be reused across Groups, and Groups can be reused across Assistants.

You can also configure a webhook URL to receive conversation insights after they are generated.

You can also start your assistant as part of a programmatic voice application using the Start Assistant command.

By default, every component of a Telnyx AI Assistant runs on Telnyx infrastructure. You can, however, BYO LLM or TTS using third-party providers.

If you have Conversational AI agents configured in Elevenlabs, you can import them as Telnyx AI Assistants in a single click.

If you want to use a voice from ElevenLabs in your existing Telnyx assistant, you will

1. Create an ElevenLabs API Key
2. Reference the key in your Assistant voice configuration

Note

Requests from a free plan are rejected. You will likely have to use a paid plan to set up this integration successfully.

First, check out their guide on creating an API Key

In the voice tab for your assistant, you can select ElevenLabs as the provider. A new dropdown will then appear to reference your API key. You can give the secret a friendly identifier and securely store your API key as the secret value.

To enable a multi-lingual agent, you should also set the transcription model to openai/whisper-large-v3-turbo.

Note

You will not be able to access the value of a secret after it is stored.

You can also manage all your secrets in the Integration Secrets tab in the portal. Choose a memorable identifier to refer to the API key and store it in the Secret Value field.

To use an LLM from OpenAI in your assistant, you will

1. Create an OpenAI API Key
2. Configure the language model in your AI Assistant

Note

Requests from a free plan are rejected. You will likely have to use a paid plan to set up this integration successfully.

First, check out their guide on creating an API Key

Back at the AI Assistants tab in the portal, edit your assistant.

First, change the model to an OpenAI model like openai/gpt-4o. Then follow the same API Key steps as described in the ElevenLabs section above.