Overview

AI-powered Sales Development Representative (SDR) companies face a unique challenge in crafting effective and personalized openers for lead generation. These openers are derived from analyzing company posts and generating tailored messages using large language models (LLMs). However, the iterative process of testing, evaluating, and refining prompts to achieve high-quality openers is time-consuming, subjective, and difficult to scale.

This case study illustrates how Future AGI empowers an AI SDR company to optimize its lead-generation process through scalable and intelligent evaluation.

Problem Statement

An AI SDR company leverages LLMs to generate personalized openers for lead generation. Their workflow involves:

• Crafting multiple prompts.

• Generating opener lines for each prompt.

• Manually evaluating and rating openers to identify the best-performing prompt.

Challenges include:

1. Subjectivity in Evaluation: Ratings depend on individual biases, leading to inconsistent results.

2. Scalability Issues: Evaluating hundreds of prompts and outputs manually isn’t feasible.

3. Lack of Data-Driven Insights: No systematic feedback or analytics to guide prompt refinement.

4. Versioning Complexity: Difficulty in tracking and comparing different iterations of prompts.

Solution Provided by Future AGI

Future AGI revolutionizes this process by offering an automated, scalable, and objective evaluation framework. Here’s how we help:

1. Opener Evaluation and Scoring

• The AI SDR company submits prompts and the generated openers as input to Future AGI.

• The openers can be evaluated on our platform using predefined criteria, such as:

  • Relevance: Does the opener address the specific context of the target company’s post?

  • Tone: Is the tone professional, friendly, or aligned with the SDR’s brand identity?

  • and many others.

    

• Openers are scored and tagged with these attributes to highlight their strengths and weaknesses.

• This evaluation uses the Deterministic class from our SDK with criteria focused on specific categories.

  JUDGING_CRITERIA = {
      "Engagement": "Evaluate whether the opener captures attention...",
      "Tone": "Evaluate whether the tone of the opener is respectful...",
      "Relevance": "Evaluate whether the opener aligns with the posts...",
      "Appropriateness": "Evaluate whether the selected post fits...",
      "Impact": "Evaluate how compelling the opener is in delivering its message..."
  }

from fi.evals import Deterministic
from fi.testcases import MLLMTestCase
class DeterministicTestCase(MLLMTestCase):
    opener: str
    combined_posts: str
    value_proposition: str
complete_result = {}
for criterion, description in JUDGING_CRITERIA.items():
    deterministic_eval = Deterministic(config={
        "multi_choice": False,
        "choices": ["Good", "Poor"],
        "rule_prompt": f"opener : {{{{input_key1}}}}, combined_posts : {{{{input_key2}}}}, value_proposition: {{{{input_key3}}}}. {description}",
        "input": {
            "input_key1": "opener",
            "input_key2": "combined_posts",
            "input_key3": "value_proposition"
        }
    })
    # Evaluate for each opener

2. Identification of Best Prompts

• Future AGI identifies the top-performing prompt by analyzing opener scores across multiple iterations. This eliminates the need for subjective decision-making and ensures consistent quality.

3. Improvement Suggestions

• For poorly rated openers, Future AGI provides actionable suggestions to improve the prompt. For example:

  • “Include a specific call-to-action to make the opener more engaging.”

  • “Use language that directly ties the opener to the prospect’s recent achievements.”

4. Dashboard for Model and Prompt Comparison

• An intuitive dashboard allows the AI SDR company to:

  • Compare the performance of different LLMs.

  

  • Track prompt versioning and analyze the impact of changes over time.

  • Visualize key metrics like relevance scores and engagement rates.

  • Optimize existing prompts to perform better on key metrics.

    

Original Prompt = """You have been given 3 LinkedIn posts written by the same person. You work for a company which offers the following value to their prospects:
**Value Proposition: {{value_proposition}}**
Take a deep breath, clear your mind and from the given posts first select the post most relevant to your value proposition. The entire post could be related to the value proposition or there could be a small portion in the post that might be relevant. 
After having found the most relevant post, write a **single sentence** opener for an outreach message referencing the post. Summarize the content of the post briefly to make a catchy opener. The email should start with "I recently saw your post about" and summarize the content briefly.
Posts: {{combined_posts}}""

Optimized Prompt = """You have been given 3 LinkedIn posts written by the same person. You work for a company which offers the following value to their prospects: **Value Proposition: {{value_proposition}}** Analyze the given posts and select the one most relevant to your value proposition. Consider posts that are entirely related, as well as those with small portions relevant to it. After identifying the most relevant post, craft a personalized and engaging single-sentence opener for an outreach message. The opener should start with "I recently saw your post about" and briefly summarize the key content, directly relating to the chosen post and your value proposition. Maintain a professional yet friendly tone in your opener, aligning with LinkedIn communication norms. Limit your opener to a maximum of 30 words. Posts: {{combined_posts}}"""

5. Continuous Feedback Loop

• As the AI SDR company gathers real-world performance data (e.g., response rates), it can feed this back into Future AGI’s evaluation system to further refine the scoring models.

Key Results

By integrating Future AGI’s evaluation platform, the AI SDR company achieved the following:

1. Enhanced Efficiency: Reduced manual effort in evaluating openers by 80%, allowing teams to focus on strategic tasks.

2. Improved Opener Quality: Increased response rates by 25% through data-driven prompt optimization.

3. Scalability: Evaluated 10x more prompts within the same timeframe, enabling faster iterations and experimentation.

4. Actionable Insights: Gained clear, objective metrics to guide future prompt designs and improve alignment with client goals.

5. Streamlined Workflow: Leveraged prompt versioning and model comparison to optimize lead generation strategies.

Conclusion

Future AGI’s evaluation platform empowers AI SDR companies to scale their lead-generation efforts while maintaining high standards of quality and personalization. By automating the evaluation process and providing actionable insights, we help our clients unlock the full potential of their LLMs, drive better engagement, and achieve measurable business outcomes.

About Future AGI

Future AGI specializes in AI evaluation solutions that enable organizations to optimize their AI systems and workflows at scale. Our cutting-edge platform combines robust evaluation metrics, insightful analytics, and seamless integrations to empower businesses across industries.