Catch the Wrong AI Answer Before You Trust It

Before you trust an AI answer, run a 30-second check that catches made-up facts, bad math, and confident mistakes.

AI answers sound sure of themselves. That confidence is the trap.

A polished paragraph feels true. A clean table looks checked. But the tone of an answer tells you nothing about whether it's correct.

By now you've learned to write clear prompts and shape the output. The last habit is the one that protects your reputation: a quick check before you trust what you got.

This part gives you a 30-second self-check. It's calm, it's repeatable, and it catches most of the errors that slip through.

Why confident answers fool us

An LLM predicts likely text. It doesn't know facts the way a database does. So it can produce a smooth, believable answer that is partly or fully wrong.

When a model states something false but sounds certain, we call that a hallucination. It's not lying. It's filling a gap with the most plausible-looking words.

Here's the catch. The same fluent style shows up whether the answer is right or wrong. You can't tell them apart by reading alone.

That's why guessing "this looks right" fails so often. You need a habit that doesn't depend on how the answer feels.

The 30-second self-check

You don't need to fact-check every word. You need three fast passes that catch the errors that actually hurt.

Run these in order. The whole thing takes about half a minute.

That's it. Three passes: claims, math, sources.

The goal isn't to distrust everything. It's to catch the two or three things that would embarrass you if they were wrong.

Pass 1: Verify the facts

Read the answer once and underline every factual claim. Not opinions. Facts.

A factual claim is anything that could be true or false in the real world. "The deadline is March 3." "This law passed in 2021." "Their CEO is named Dana."

For each underlined claim, ask one question: How would I know if this were wrong?

If you can't answer that, you've found something to check.

You can also ask the AI to do the underlining for you. This prompt pulls the riskiest claims to the surface.

Here is your previous answer. List every factual claim
you made — names, dates, numbers, and statistics — as a
short bullet list. For each one, rate your confidence as
high, medium, or low, and flag anything you are not sure
about.

Low-confidence flags are your shortlist. Those are the claims to verify first.

Pass 2: Sanity-check the numbers

Numbers are where AI errors do the most damage. A wrong word is awkward. A wrong number in a budget or a report is a real problem.

Models are good at sounding mathematical and bad at being reliable with math. They can add wrong, miscount, or invent a statistic that fits the sentence.

So give every number a second look.

• Do the parts add up to the total?
• Is the percentage between 0 and 100, and does its size feel right?
• Does any figure seem suspiciously round or weirdly precise?
• If two numbers should match, do they?

When math matters, ask the AI to show its work. Reasoning step by step makes mistakes easier to spot, a technique known as chain-of-thought.

Recalculate the totals in your answer. Show each step of
the arithmetic on its own line so I can follow it. Then
state the final figure and confirm it matches the table
above.

If the steps don't lead to the stated total, you've caught the error before it reached anyone else.

Pass 3: Ask for sources

When a claim matters, ask where it came from. Then check the source yourself.

A simple follow-up does a lot of the work here. (Part 6 covered the power of the follow-up — this is one of its best uses.)

For each factual claim in your last answer, tell me the
source. If you are not certain a source exists, say so
plainly instead of guessing. Do not invent citations.

Read the reply carefully. Watch for two warning signs.

First, vague sourcing. "Experts agree" and "studies show" are not sources. They're filler.

Second, made-up references. Models can invent real-sounding titles, authors, and links that don't exist. If a citation looks specific, open it and confirm it's real.

The honest move is to verify anything load-bearing in a source you already trust. The AI points you toward the answer. It doesn't get the final word.

Build it into one prompt

Once the three passes feel natural, you can fold them into the prompt itself. Ask for the check up front, and the AI does some of the work for you.

Answer my question below. Then, before you finish, add a
short "Confidence & Sources" section that does three things:
1. Lists any claim you are unsure about.
2. Re-checks any math and shows the steps.
3. Names a source for each fact, or says "no reliable source"
   instead of guessing.

Question: [your question here]

This won't make the AI perfect. It still might miss its own mistakes. But it surfaces the weak spots so your 30-second pass goes faster.

You can also paste a finished answer into a free prompt scorer or prompt optimizer to tighten the instructions that produced it. A clearer prompt means fewer errors to catch in the first place.

When to check hardest

Not every answer carries the same risk. Spend your attention where being wrong costs the most.

The pattern is simple. The further an answer travels and the more a number depends on it, the more it earns the full pass.

And here's the reassuring part. You don't have to become a fact-checking machine. You just have to stop trusting fluency, and run three fast passes when it counts.

Try this today

Take an AI answer you've used or are about to use. A real one, with at least one fact or number in it.

Run the 30-second check:

Notice what you find. Most of the time, the answer holds up. Sometimes one number is off, or one "source" doesn't exist. Either way, you caught it before it cost you anything.

Do this for a week and it stops feeling like extra work. It becomes the quiet second nature of someone who trusts AI without being fooled by it.

Keep going

Next → Part 8: Turn Your Best Prompt Into a Template You Never Rewrite

Or see the full Prompting Pro in 21 Days series.