Sindarin Persona Documentation [BETA]

<aside> 💡 Note: To toggle Light / Dark mode, press cmd+shift+L

</aside>

Choosing an LLM

Supported LLMs

We support any OpenAI-compatible LLM API, in addition to our hosted instances of GPT-3, GPT-4 and Nous Hermes Mixtral.

To use your own custom LLM API, or to finetune an OpenAI LLM, [contact us](mailto:[email protected]?subject=Use%20Custom%20LLM).

Finetuning LLMs

We offer finetuning custom models for Tier III and Enterprise customers.

Choosing a Voice

We use ElevenLabs as a voice provider. If you are a paid customer, you can [share any ElevenLabs voice](https://help.elevenlabs.io/hc/en-us/articles/18644643807889-How-do-I-share-a-voice-#:~:text=Sharing your voice is quite,toggle for sharing will appear.) with us to import it, including cloned voices.

If you would like to use a cheaper (but lower-quality) voice provider, please [contact us](mailto:[email protected]?subject=Use%20Cheaper%20Voice).

Custom Voices

To import a custom voice, [contact us](mailto:[email protected]?subject=Use%20Custom%20Voice) with the share link to the voice and the Persona name you would like to apply it to.

Prompt: Defining Personality and Behavior

Initial Message

The initial message is the message that the Persona will say upon conversation start.

The [***DETAILS.[detailKey]***](<https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7>) wildcard can be used in the initial message.

Rate Limit Message [optional]

The message that your Persona will say when a user hits their message limit (requires tracking user IDs; tracked automatically via phone number if using Twilio).

Saying the rate limit message does not consume messages for your account.

Prompt

A Personas’ behavior is largely defined by its prompt.

If you are unfamiliar with prompting / prompt engineering, it is important to familiarize yourself with the basics and [best practices](https://www.techtarget.com/searchenterpriseai/tip/Prompt-engineering-tips-and-best-practices#:~:text=Include context&text=To make use of this,and detail of the output.). This is an active area of research, so keeping up with the latest developments will help you get the most out of your Persona.

In general, a typical prompting pattern is:

You are [high-level Persona description, e.g. "Tifa, a hardworking, tough bartender with a heart of gold"].

Your job is to [description of job to be done, e.g. "keep the bar in order and take care of your team"].

To do that, you [description of tactics / methods to achieve goals, e.g. "use your fists when necessary"].

[Reiteration of most important details, e.g. "YOU MUST ALWAYS SPEAK IN IAMBIC PENTAMETER"].

Prompt Keywords: PERSONA_VOICE_SCHEMA, DETAILS.*, SCENARIOS_LIST, and CURRENT_DATETIME

There are 4 keywords that you can add to your Prompt to give your Persona additional capabilities. Each keyword must be written with three asterisks (*) on each side, like this:

***PERSONA_VOICE_SCHEMA***

The keywords are:

***PERSONA_VOICE_SCHEMA***: Adds your Actions Schema to the prompt, giving your Persona access to the JSON object you’ve configured.
***DETAILS.[detailKey]***: Adds any Detail you pass in on initializing the conversation.
***SCENARIOS_LIST***: Adds the top-5 most relevant Scenarios to the prompt on each conversation turn.
***CURRENT_DATETIME***: Adds the current date time as a string to the prompt, giving your Persona temporal awareness.

Actions: Making your Persona Do Stuff

In order for your Persona to “do stuff,” your Persona can take Actions via the Actions Schema.

Simply put, actions are structured JSON objects that the Persona can emit during conversation, to be handled by downstream clients — whether our hosted Twilio client or your web application, Unity game, or other client environment.

The Persona Actions Schema combines traditional JSON schema validation with Persona’s proprietary schema definitions.

Every Actions Schema has the following top-level properties:

Key	Type	Value	Description
$schema	string	“http://json-schema.org/draft-07/schema#”	[Does not change] The schema validation version.
type	string	“object”	[Does not change] The type of the schema object.
properties	object	{https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7}	Where the magic happens. You define the keys in the “properties” sub-object. See below for more information about structuring Actions.

Action Object

Each Action Object must follow the structure:

Key	Type	Value	Description
_description	string	“When the Persona should hang up the phone.”	An English-language description of the Action and the context in which the Action should be taken.
_examples	array	[Examples]	A list of Examples.
_validators [optional]	object	{ https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7 }	A dictionary of validators to apply to the specified action values.
_action_type [optional]	“https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7”	“https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7”	Either “https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7” or “https://wirehaired-caravan-e73.notion.site/Sindarin-Persona-Documentation-BETA-130182f2903742f5874cb44aaceaccb7”
_depends_on [optional]	array	Array of state value paths.	Ensure that the action can only be taken when all of the state values listed are present at the defined paths in the state.
type	string	“string”	“number”
enum [optional]

_validators

You may want to ensure that the output of an action conforms to a particular pattern or schema. In addition to generic JSON schema validation, we offer the following custom validators:

Validator	Description
email	ensures that an email address is property formatted.
date (MM/dd/yyy)	ensures that a date is formatted as (MM/dd/YYYY)
time (hh:mm aa)	ensures that a time is formatted as (hh:mm aa)
datetimeiso	ensures that a datetime is formatted as a proper Datetime ISO (yyyy-MM-ddTHH:mm)

More validators will become available as requested, so please don’t hesitate to [request a new one](mailto:[email protected]?subject=Add%20Custom%20Validators).

_action_type (“webhook” and “update_state”)

webhook:

Allows the Persona to make webhook (API) calls to external services, and react to the responses in-conversation.

For example, if the Persona has a fetch_current_weather action:

"fetch_current_weather": {
      "_action_type": "webhook",
      "_webhook_constants": {
        "url": "<https://goweather.herokuapp.com/weather/><city>",
        "method": "GET",
				"body": {}
      },
      "_description": "When Concierge should fetch the weather for the city.",
      "_examples": [
        {
          "user says": "What's the weather in Albuquerque?",
          "assistant action includes": {
            "fetch_current_weather": {
              "city": "Albuquerque"
            }
          }
        }
      ],
      "type": "object",
      "required": [
        "citys"
      ]
    }
  }

And the user says something like “What’s the current weather in Sydney?” The persona will first react to the question with something like “Let me check”, as well as emitting the action:

{
	"fetch_current_weather": {
		"city": "Sydney"
  }
}

And shortly thereafter, if the API returns a success code (200 or 201), the Persona will react with a summary of the response body from the webhook.

If the API does not return a success code, the Persona will reply with a summary of the error.

If you would like your Persona not to react to the webhook response, you can add an additional field to the _webhook_constants: "responseHandler": "fire_and_forget" .

For POST requests, you can hard-code request body values in the _webhook_constants; Details (***DETAILS.[detailKey]***) can also be injected into the request body. For instance, if you want to send an SMS through a third-party provider instead of directly via our Twilio integration, you could use:

"send_text_message": {
      "_action_type": "webhook",
      "_webhook_constants": {
        "url": "<https://text-message-service.com/send>",
        "method": "POST",
				"body": {
          "apiKey": "abc-123",
					"phoneNumber": "***DETAILS.userId***" // ***DETAILS.userId*** is set to the user's phone # for incoming calls
				}
      },
      "_description": "When Concierge should send the user a text message confirming their favorite color",
      "_examples": [
        {
          "user says": "Sure, send me a text to confirm my favorite color",
          "assistant action includes": {
						"send_text_message": {
	            "favoriteColor": "Purple"
						}
          }
        }
      ],
      "type": "object",
      "required": [
				"favoriteColor"
			]
    }
  }

The final POST body for this request, if the user says something like “Send me a text confirming my favorite color is Blue” will be:

{
	"apiKey": "abc-123" // hard-coded in the action schema
	"phoneNumber": "+1123456789" // if this is an inbound call, userId is auto-assigned as caller #
	"favoriteColor": "Blue" // decided by the LLM
}

update_state:

Assigns the emitted action body to the current state. Useful for retaining specific information over the course of the conversation.

For example, in the scenario where the current state is:

{ "user_name": "Steve" }

And the remember_user_age action definition is:

"remember_user_age": {
            "_action_type": "update_state",
            "_description": "When Charles should commit the user's age to memory.",
            "_examples": [
                {
                    "user says": "I am 42 years old",
                    "assistant action includes": {
                        "remember_user_age": {
                            "user_age": "42"
                        }
                    }
                }
            ],
            "type": "object"
        },

And the user says “By the way, I was 54 years old a week ago and just celebrated my birthday”, thus invoking the action:

{
	"remember_user_age": {
			"user_age": 55
		}
}

The final state would be:

{
	"user_name": "Steve",
	"user_age": 55
}

_depends_on

Ensures that the persona can only take the associated action if the listed path(s) exist(s) in the current state.

For example, if you’d like your Persona to only schedule a call once a time has been confirmed, you could make schedule_call depend on confirm_time like this:

"confirm_time": {
            "_action_type": "update_state",
            "_description": "When the user has confirmed the time they'd like to make the appointment.",
            "_examples": [
                {
                    "user says": "<user says what time they are available for a call>",
                    "assistant action includes": {
                        "confirm_time": {
                            "time_confirmed": "true"
                        }
                    }
                }
            ],
            "type": "object",
            "required": [
                "time_confirmed"
            ]
        },
"schedule_call": {
            "_depends_on": [
                "state.time_confirmed"
            ],
						// ...

Putting it altogether: An example

The following example illustrates how to use some of these concepts together to create an appointment-scheduling Persona name Rachel:

{
    "$schema": "<http://json-schema.org/draft-07/schema#>",
    "type": "object",
    "properties": {
        "hang_up_call": {
            "_description": "When Rachel should end the call and hang up the phone.",
            "_examples": [
                {
                    "user says": "Thank you, goodbye",
                    "assistant action includes": {
                        "hang_up_call": "true"
                    }
                }
            ]
        },
				"press_phone_keys": {
				    "_description": "When you should press a key or keys to navigate a phone tree.",
				    "_examples": [
				        {
						         "user says": "Welcome to ACME co. Please press 5 to speak to a representative.",
							       "assistant action includes": {
										    "press_phone_keys": [ 5 ]
							       }
				        }
					  ],
					  "type": "array"
				},
        "save_email_address": {
            "_action_type": "update_state",
            "_description": "When Rachel should commit the user's email address to memory. She should always confirm that she has the correct email address afterwards.",
            "_examples": [
                {
                    "user says": "<provides email address>",
                    "assistant action includes": {
                        "save_email_address": {
                            "email_address": "<email address>"
                        }
                    }
                }
            ],
            "type": "object",
            "required": [
                "email_address"
            ],
            "_validators": {
                "email_address": "email"
            }
        },
        "save_time_for_call": {
            "_action_type": "update_state",
            "_description": "When Rachel should commit the time for the follow-up call to memory. She should always confirm that she has the correct time afterwards.",
            "_examples": [
                {
                    "user says": "<user says what time they are available for a call>",
                    "assistant action includes": {
                        "save_time_for_call": {
                            "followup-datetime": "<datetime iso formatted>"
                        }
                    }
                }
            ],
            "type": "object",
            "required": [
                "followup-datetime"
            ],
            "_validators": {
                "followup-datetime": "datetimeiso"
            }
        },
        "confirm_email_address": {
            "_action_type": "update_state",
            "_depends_on": [
                "state.email_address"
            ],
            "_description": "When Rachel should confirm that she heard the user's email address correctly.",
            "_examples": [
                {
                    "user says": "Yes, that's correct",
                    "assistant action includes": {
                        "confirm_email_address": {
                            "email_address_confirmed": "true"
                        }
                    }
                }
            ],
            "type": "object",
            "required": [
                "email_address_confirmed"
            ]
        },
        "schedule_call_with_brian": {
            "_action_type": "webhook",
            "_depends_on": [
                "state.email_address",
                "state.email_address_confirmed",
                "state.followup-datetime"
            ],
            "_webhook_constants": {
                "url": "<https://hooks.zapier.com/hooks/catch/>[zapier id]/[zapier id]/",
                "method": "POST"
            },
            "_description": "When Rachel should book an appointment. You should convert what the user says into datetime ISO format (yyyy-MM-ddTHH:mm). YOU MUST ALWAYS CONFIRM WHETHER THEY RECEIVED THE CALENDAR INVITATION.",
            "_examples": [
                {
                    "user says": "<provides email address>",
                    "assistant action includes": {
                        "schedule_call_with_brian": {
                            "firstName": "<first name>",
                            "company": "<company name>",
                            "datetime": "<datetime iso formatted>",
                            "email_address": "<email address>"
                        }
                    }
                }
            ],
            "type": "object",
            "required": [
                "firstName",
                "company",
                "datetime",
                "email_address"
            ],
            "_validators": {
                "email_address": "email",
                "datetime": "datetimeiso"
            }
        }
    }
}

Scenarios

Use Scenarios to provide your Persona with specific information in specific conversational contexts.

When you have Scenarios configured, the user’s current statement — along with the Persona’s most recent statement — will be used to retrieve the 5 most relevant Scenarios, which will be injected into the prompt in real-time.

Scenarios are an implementation of RAG (Retrieval-Augmented Generation) that is optimized for ultra-low latency.

NOTE: RAG is a rapidly evolving discipline and our implementation is a first approximation. If you have

different needs than our system currently provides, please email us at [email protected].

Scenario Construction

Scenarios are composed of four sections:

Context [string]: Text that will get injected into the prompt on each conversational turn, prepended with Given the context:
Persona Says [array]: A list of strings that are examples of what the Persona might say to activate the Scenario
User Says [array]: A list of strings that are examples of what a user might say to activate the Scenario
Response Guidelines [string]: Text that will get injected into the prompt, prepended with You should respond by:

Scenarios API Example

To programmatically update scenarios via the API, you can PUT to /api/personas/:id/scenarios with a data structure of the following format:

{
	"scenarios": [
		{
			"context": "If you are discussing prices",
			"personaSays": [
				"Would you like to know about our prices?"
			],
			"userSays": [
				"What are your prices like?"
				"How much does it cost?"
			],
			"responseGuidelines": "Tell the user that our plans start at $99 per month but are negotiable"
		},
		{ ... },
		{ ... }
	]
}

While either the personaSays or userSays array can be empty, both arrays should not be empty.

Deploying your Persona

Personas are deployed via Persona Clients, which handle all incoming and outgoing audio streams, as well as expose helper methods to manage conversation lifecycle and state.

Phone (Twilio)

Our Twilio client is fully-managed, meaning that you only have to point your Twilio phone number to our TwiML webservice and submit a few details via the Persona Playground to have a fully-operational Twilio Phone Persona.

For more details, select Deploy > Phone (Twilio) in the Playground.

Our Twilio client provides some helper Actions to execute Twilio-specific behaviors:

transfer_call_to: Transfers a call to a different phone number.
hang_up_call: Hangs up the current call.
send_sms: Sends an SMS to the number currently on the call.
press_phone_keys : “Presses” phone keys (DTMF tones), e.g. for the purpose of navigating an IVR system.

Additionally, for incoming phone calls, the caller’s phone number will automatically be injectable into your prompt as ***DETAILS.userId***.