Personal Score Deep Dive: Measuring Response Confidence

​Welcome to your weekly Personal AI newsletter! Stay informed with practical tips, feature highlights, and real examples to help you and your team build and manage AI Personas more effectively. With Personal AI, you can create an evolving AI workforce trained on proprietary knowledge, streamlining workflows and driving innovation across your business.

This week, we’re focusing on Personal Score — a confidence metric that reflects how much reference data a persona has to support its response. A higher score means the persona had more relevant data to draw from; a lower score indicates limited supporting information. While a low score doesn’t mean the response is incorrect, it signals that users may want to refine their query, add training data, or consult a more specialized persona for better results.

​The Personal Score provides transparency into the response generation process. It allows users to assess how closely the AI's output aligns with their uploaded data, which is especially important when accuracy, specificity, or compliance with internal knowledge is required.

​For example, a Personal Score of 83% means the response is significantly influenced by the user’s specific data, as evaluated against multiple internal scoring signals.

​Scoring Criteria

​Each response is scored using three key signals:

​Accuracy:

• ​Measures how closely the response matches the actual data in your memory stack
• ​Evaluates word-for-word matches and semantic similarity to your trained content
• ​Considers the confidence level of information retrieval from your specific data
• ​Higher accuracy scores mean more direct alignment with your organization's knowledge

​Relevance:

• ​Assesses how well the response addresses the specific question asked
• ​Evaluates whether key concepts from the query are properly addressed
• ​Checks if the response stays focused on the topic without tangential information
• ​Higher relevance scores indicate more precise answers to user queries

​Context:

• ​Examines how well the response incorporates situational understanding
• ​Considers metadata like timestamps, categories, and relationships between information
• ​Evaluates the AI's ability to connect related pieces of information from different sources
• ​Works best when your data includes clear labels, tags, and organizational structure

​These three metrics work together synergistically. For example, a response might be accurate (matching your data) but lack relevance (not answering the specific question), or be relevant but missing important context. The Personal Score reflects how well all three elements come together.

​How can it be used in conjunction with the response threshold setting?

​The Personal Score and Response Threshold work together as a powerful quality control system. Here's how they complement each other:

• ​Response Threshold acts as your quality gate:
  • ​You set a minimum Personal Score requirement (e.g., 61%) that responses must meet before being delivered.
• ​When a response is generated:
  • ​If the Personal Score meets or exceeds your threshold, the response is delivered
  • ​If it falls below the threshold, you'll receive a default message instead

​You can set your fallback auto-reply in the Persona settings. To enable it, toggle on 'Default Message' in your Default AI settings for DMs or Channels.

​This setup helps organizations maintain response quality by filtering out low-confidence answers and enforcing accuracy standards. For example, with a 61% threshold:

• ​A 75% Personal Score is passed and is delivered
• ​A 55% score is blocked, triggering the “insufficient information” message

​As you explore and tune Personal Score settings, think of them as levers for improving both user trust and system performance. Small adjustments can make a big impact on how confidently your personas respond — and how confidently your users rely on them.

‍