chore: sync skills/guideline-generation/references/confidence-scoring.md for v1.2.1

This commit is contained in:
2026-07-01 13:03:14 -05:00
parent 7ec591d288
commit af9ddf5f3e
@@ -14,6 +14,8 @@ The guideline section is well-supported and actionable.
- Specific, actionable instructions (not just vague principles) - Specific, actionable instructions (not just vague principles)
- No unresolved conflicts - No unresolved conflicts
**Example:** Voice attribute "Confident but not arrogant" appears in the official style guide, is demonstrated in email templates, and matches patterns in top performer calls.
### Medium Confidence ### Medium Confidence
The section is reasonable but could benefit from more data or team confirmation. The section is reasonable but could benefit from more data or team confirmation.
@@ -24,6 +26,8 @@ The section is reasonable but could benefit from more data or team confirmation.
- Actionable but some interpretation was required - Actionable but some interpretation was required
- May have one unresolved conflict - May have one unresolved conflict
**Example:** Tone for social media inferred from email templates and one Slack thread, but no official social media guidelines exist.
### Low Confidence ### Low Confidence
The section is a best-effort recommendation. Team review strongly recommended. The section is a best-effort recommendation. Team review strongly recommended.
@@ -34,6 +38,44 @@ The section is a best-effort recommendation. Team review strongly recommended.
- Unresolved conflicts between sources - Unresolved conflicts between sources
- Limited specificity - Limited specificity
**Example:** Competitive positioning derived from a single sales call where a competitor was discussed, with no supporting documentation.
## Section-Level Scoring Guide
### Voice Attributes
- **High**: Attributes appear in official brand guide AND are demonstrated in templates or calls
- **Medium**: Attributes appear in one document type only, or are inferred from multiple conversations
- **Low**: Attributes inferred from a single source or from indirect evidence
### Messaging Framework
- **High**: Value propositions documented in official materials AND used consistently in sales conversations
- **Medium**: Documented but not observed in practice, OR observed but not documented
- **Low**: Extracted from a single pitch deck or single call
### Tone Matrix
- **High**: Explicit tone guidance exists for the context AND matches observed behavior
- **Medium**: Tone inferred from 3+ examples of content in that context
- **Low**: Tone inferred from 1-2 examples, or extrapolated from similar contexts
### Terminology
- **High**: Terms explicitly listed in a style guide or glossary
- **Medium**: Terms consistently used in templates and calls (pattern-based)
- **Low**: Terms observed in a single document or inferred from brand personality
### Language Patterns (from transcripts)
- **High**: Pattern observed in 5+ calls across multiple speakers
- **Medium**: Pattern observed in 3-4 calls or from a single top performer
- **Low**: Pattern observed in 1-2 calls only
### Transcript-Primary Scenarios
When guidelines are generated primarily from conversational sources (no AUTHORITATIVE documents available):
- Voice Attributes derived from 5+ transcripts = **Medium** (not Low)
- Messaging Framework from consistent patterns across 5+ calls = **Medium**
- Language Patterns weight increases from 10% to 20% in aggregate calculation (subtract 10% from Voice Attributes)
Note this in the guideline metadata: "Guidelines generated primarily from conversational sources — team review recommended to formalize."
## Aggregate Confidence ## Aggregate Confidence
Calculate overall guideline confidence as the weighted average of section scores: Calculate overall guideline confidence as the weighted average of section scores:
@@ -48,15 +90,39 @@ Calculate overall guideline confidence as the weighted average of section scores
Convert scores: High = 1.0, Medium = 0.6, Low = 0.3 Convert scores: High = 1.0, Medium = 0.6, Low = 0.3
**Example:**
- Voice Attributes: High (1.0 x 0.30 = 0.30)
- Messaging: Medium (0.6 x 0.25 = 0.15)
- Tone: Medium (0.6 x 0.20 = 0.12)
- Terminology: High (1.0 x 0.15 = 0.15)
- Language: Low (0.3 x 0.10 = 0.03)
- **Overall: 0.75 = Medium-High confidence**
**Aggregate score thresholds:** **Aggregate score thresholds:**
- 0.851.0 = High - 0.851.0 = High
- 0.600.84 = Medium - 0.600.84 = Medium
- Below 0.60 = Low - Below 0.60 = Low
## Presentation
Present confidence alongside each section header:
```markdown
## Voice Attributes (Confidence: High)
[content]
## Tone Matrix (Confidence: Medium)
[content — note: no official social media guidelines found, tone inferred from email patterns]
```
For Medium and Low confidence sections, include a brief note explaining why confidence is limited and what would raise it.
## Relationship to Open Questions ## Relationship to Open Questions
Low confidence sections should generate corresponding open questions:
- **Low confidence + conflict** = High Priority open question - **Low confidence + conflict** = High Priority open question
- **Low confidence + gap** = Medium Priority open question - **Low confidence + gap** = Medium Priority open question
- **Medium confidence + minor inconsistency** = Low Priority open question - **Medium confidence + minor inconsistency** = Low Priority open question
Every open question includes a recommendation that, if confirmed, would raise the section's confidence score. Every open question includes a recommendation that, if confirmed, would raise the section's confidence score.