Prompting Examples

This metric evaluates whether the Main Agent initiates the conversation with a greeting.
Success is achieved if the main agent greets and makes a connection based on the testing agent's response. 
Failure occurs if the agent does not greet or fails to make a connection based on the testing agent's response.

You are tasked with determining the engagement level in a call between an AI agent and a human. You will be provided with the following information:

1. A transcript of the call. Here AI Agent is referred to as "Main Agent" and Human is referred to as "Testing Agent".

First, carefully read the following call transcript:

<call_transcript>
{{transcript}}
</call_transcript>

The call was ended because: {{call_ended_reason}}

Analyze the transcript and the call ended reason to determine if the AI agent terminated the call early. Consider the following factors:
1. Was the call ended abruptly or unexpectedly?
2. Were there unresolved issues or questions at the end of the call?
3. If the human ended the call, was it due to frustration with the AI's responses or inability to help?

Important guidelines:
- Keep your output very concise.
- Provide a maximum of 2-3 pointers (1-2 is preferable in most cases).
- Each pointer should be no more than 10 words unless absolutely necessary.

Use the following format for your response:

<detailed_analysis>
1. List key points from the transcript relevant to the task:
[List 3-5 key points, each on a new line]

2. Arguments for early termination by Human:
[Provide 1-2 concise arguments]

3. Arguments against early termination by Human:
[Provide 1-2 concise arguments]

4. Final judgment:
[Make a final decision based on the balance of evidence]
</detailed_analysis>

<evaluation>
Metric met: [Yes/No]
[If No, provide 1-2 concise pointers explaining why, with timestamps if applicable]
</evaluation>

Example output (do not use this content in your actual evaluation):

<evaluation>
Metric met: No
- Agent failed to address customer's main concern and customer ended call (2:15, 3:42)
</evaluation>

Please proceed with your analysis and evaluation.

<scenario>
- You are calling a customer support agent to return an order. 
- Begin by being slightly unsure about your exact order name.
- When asked about your name and order id, at first give incorrect order id. 
- Correct it after agent denies request. 
- Be ready to provide other information when requested. 
</scenario>

- The Main Agent asks for Name and Date of Birth 
- The Main Agent attempts to get correct date of birth again 
- After successful verification, the Main agent informs testing agent that return request is accepted.

Generate scenarios where the user is trying to fake his identity

OR 

Generate scenarios where the user is trying to cancel an appointment