Verification & Accuracy — Foundations Lesson 4
Scenario
You ask ChatGPT for the date of an important historical event. It responds confidently: "July 4th, 1873." You're 95% sure that's wrong, but the AI sounded so certain that you double-check. Turns out, it was completely made up. The event was in 1873, but it wasn't July 4th. This becomes your operating principle: Never trust AI without verification. But how do you verify? What's worth checking? When can you relax?
---
Learning Objectives
By the end of this lesson, you will be able to:
1. Understand why AI produces false information and what it's called
2. Identify high-risk tasks that require verification
3. Apply verification techniques appropriate to each task type
4. Recognize patterns that signal unreliable output
5. Build verification into your workflow automatically
---
Think Before You Prompt
Before you use AI output professionally:
---
Hallucinations: What They Are and Why They Happen
Hallucination = When an AI confidently generates false information.
Not a mistake. Not an accident. A predictable failure mode.
Why Hallucinations Happen
Remember: LLMs predict the next word based on patterns. They don't "know" facts. They generate plausible-sounding text.
If an LLM was trained on text that frequently associated "George Washington" with "first president," it learned that pattern. But it also learned patterns about how people write, which sometimes includes:
When the LLM predicts the next word, it's following patterns, not checking facts. If a false statement fits the pattern, the LLM generates it confidently.
Critical insight: The LLM's confidence level is not correlated with accuracy. A completely made-up fact can sound as certain as a true one.
---
Risk Assessment: When to Verify
Not all outputs require the same level of verification. Here's how to assess:
HIGH RISK (Verify Thoroughly)
Legal contexts
Employment contracts, compliance, regulatory requirements. Wrong information = legal liability.
Financial contexts
Numbers, calculations, budget proposals, tax info. Wrong numbers = money lost.
Medical/Health
Dosages, symptoms, diagnoses. Wrong health info = harm.
Customer-facing communications
Anything published externally. Errors affect your reputation.
Data & Statistics
Specific numbers, dates, sources. These are frequently hallucinated.
Security/Passwords/Credentials
Never trust AI-generated passwords, API keys, or access codes. Always regenerate.
MEDIUM RISK (Verify Selectively)
Policy Documents
Company policies should be checked against official sources.
Project Plans
Timeline estimates can be optimistic. Verify feasibility.
Job Descriptions
Should be checked for tone and accuracy before posting.
Research Summaries
Verify cited sources and key claims.
LOW RISK (Verify for Quality, Not Accuracy)
First Drafts
Emails, memos, meeting notes. You're checking tone and completeness, not facts.
Brainstorming
Ideas don't need to be factually perfect; creativity is the goal.
Formatting & Structure
Checklists, templates, outlines. Verify format, not content.
Routine Communications
Internal status updates, thank-you emails. Low stakes.
---
Verification Techniques by Task Type
Technique 1: Source Verification
Best for: Factual claims, statistics, historical events, research
How to do it:
1. Identify specific claims the AI made
2. Search for those claims independently
3. Verify against official sources (websites, databases, published documents)
4. Check when the information was published (is it current?)
Example:
AI says "The US GDP was $25 trillion in 2023."
Action: Check the US Bureau of Economic Analysis website for actual 2023 GDP.
Time investment: 2-5 minutes per claim
---
Technique 2: Internal Consistency Check
Best for: Logic, workflow steps, instructions, multi-part tasks
How to do it:
1. Read the full AI output carefully
2. Look for contradictions (e.g., "Do X first" then later "Do X after Y")
3. Check if the logic flow makes sense
4. Verify that the conclusion follows from the premises
Example:
AI writes: "First, send the email to the client. Then, get approval from your manager."
Red flag: You should get approval before sending external communications.
Time investment: 2 minutes of careful reading
---
Technique 3: Expert Review
Best for: Complex tasks, high-stakes decisions, specialized knowledge
How to do it:
1. Have someone with expertise review the AI output
2. Ask them: "Does this seem right to you?"
3. They can catch errors faster than research because of domain knowledge
Example:
AI drafts a complex legal clause. You have your company lawyer review it before signing.
Time investment: Varies by expert availability
---
Technique 4: Process Check
Best for: Workflows, calculations, multi-step procedures
How to do it:
1. Follow the AI's instructions exactly
2. Check if each step makes sense and produces expected results
3. Verify the final output matches the expected outcome
4. Note any steps that felt wrong or unclear
Example:
AI gives you Excel formula steps. You follow them and verify the result against a known correct answer.
Time investment: Depends on task complexity
---
Technique 5: Spot-Check
Best for: Large documents, lists, summaries
How to do it:
1. Don't verify everything (too time-consuming)
2. Randomly verify 20-30% of the content
3. If errors appear in your spot-check, verify everything
4. If spot-check is clean, you have reasonable confidence
Example:
AI summarizes a 20-page document. You read pages 3, 8, and 15 to verify accuracy of the summary.
Time investment: 10% of the original document read time
---
Red Flags: Patterns That Signal Unreliable Output
Watch for these warning signs:
Rule of thumb: If it sounds authoritative but makes you uncomfortable, verify it.
---
Building Verification Into Your Workflow
Fast Verification (< 2 minutes)
For low-risk, routine outputs:
1. Read for tone — Does it sound right? Does it match the context?
2. Scan for clarity — Is it easy to understand?
3. Quick logic check — Does it make sense?
That's it. Ship it.
Standard Verification (5-15 minutes)
For medium-risk outputs:
1. Source check key claims — Google 2-3 important facts
2. Internal consistency — Read for contradictions
3. Expert spot-check — If available, have someone review
4. Compare to similar work — Is this consistent with your other outputs?
Then refine and ship.
Deep Verification (30+ minutes)
For high-risk outputs:
1. Verify every claim — Full source checking
2. Process walkthrough — Test the output if it's instructions
3. Expert review — Full review by someone qualified
4. Multiple sources — Verify key facts from 2+ independent sources
5. Legal/compliance review — If it's a high-stakes document
Only use after full verification.
---
Mission: Spot the Hallucinations
The Task
You'll read three AI-generated responses and identify potential hallucinations or inaccuracies.
Response 1:
`
"The founder of Amazon, Bob Smith, started the company in 1993 in
his garage in Seattle. Amazon was originally called "Online Books"
before changing to Amazon. The company's first major product was
the Amazon Kindle, released in 2003, which revolutionized how
people read digital books."
`
Response 2:
`
"To schedule a meeting in Google Calendar, follow these steps:
1. Open Gmail and click 'Compose'
2. Select 'Schedule send' from the dropdown
3. Choose a time slot in your calendar
4. Invite attendees
5. Click 'Send'"
`
Response 3:
`
"Employee retention is consistently high in companies that offer
flexible work arrangements. Studies show that companies with
flexible hours have a 35% higher retention rate. This is a proven
strategy that all companies should implement."
`
Your Task
For each response:
1. Identify claims — What specific facts are stated?
2. Spot the problems — What seems wrong or unverified?
3. Mark the risk level — High, medium, or low risk if used as-is?
4. Note verification steps — How would you verify this?
---
Deliverables
Hallucination Analysis Table:
`
| Response | Key Claims | Suspicious Claims | Risk Level | How to Verify |
|----------|-----------|------------------|-----------|---------------|
| 1 | | | | |
`
Document your findings. This trains your pattern-recognition for hallucinations.
---
AI Coach: When to Trust (And When Not To)
Here's the uncomfortable truth: AI is unreliable at things that matter.
The pattern: Trust AI with outputs where human judgment is most important. Verify AI with outputs where facts are critical.
Key insight: The professionals who work best with AI aren't the ones who trust it blindly. They're the ones who know exactly when to verify and have systems to catch mistakes before they ship.
---
Reflection: What's Your Verification Threshold?
Every person has a different comfort level with risk. Some professions (law, finance, medicine) require extensive verification. Others (marketing, creative work) allow more room for error.
Reflect: For your role, what's your verification requirement?
Write 3-4 sentences about your personal verification threshold.
---
Portfolio Check
Portfolio principle: Every artifact in your portfolio should have verification documented.
This doesn't mean including a footnote saying "I checked this." It means:
When you submit portfolio work (starting in Lesson 5), you'll note: "I verified X by checking [method]."
---
Bonus Challenge
Hallucination hunt: Ask ChatGPT or Claude a question about something very recent (last 2 weeks). Ask it to include specific numbers and dates. Then research what it said. How many hallucinations did you spot? This trains your skepticism.
---
Key Takeaways
Next: Lesson 5 explores Hallucinations & Limitations in depth — understanding the patterns that cause AI to fail.