A free playbook by The Aigent Lab

AI Systems & Workflow Literacy

AI Systems & Workflow Literacy · Playbook 07

Three ways AI lies. Three prompts that stop it.

AI lies in three distinct ways, and most people only catch one of them. A hallucinated number, a sycophantic verdict, and a biased default can each cost a client real money. Three prompts, one workflow, run before you trust any AI answer about a deal.


Why this matters when you're advising someone else's money.

Every workflow in this library depends on the model telling you the truth, and it doesn't always. A hallucinated yield can talk you into recommending a building that doesn't return what was claimed. A sycophantic answer can validate a deal a client was already emotionally attached to. A biased answer can quietly push everyone toward the same three neighbourhoods every other agent is pushing.
The model is not always wrong. But it is wrong in patterns, and patterns can be tested.

Three failure modes. Three fixes.

01

Catch hallucination

Ask the model to score its own confidence 1 to 10 and tag every claim as high, medium, or low confidence, with what would be needed to push it higher.

02

Catch sycophancy

Ask it to argue against its own answer: the three strongest reasons it could be wrong, the hidden assumptions, and a rewrite from the perspective of a sceptical committee.

03

Catch bias

Ask it to reanswer through three different lenses: a contrarian view, a pure cashflow view, and a short-hold exit view. Then ask which lens your original question was closest to.

04

Run all three in order

A bad answer is often two or three failure modes firing together. Running the sequence catches the stack, not just the loudest one.

The confidence check
Score your confidence in this answer from 1 to 10. Tag every claim as high, medium, or low confidence. Then tell me what you would need to push it higher.

What it still gets wrong.

i.

It doesn't verify sources

The confidence check tags claims by uncertainty, not truth. Still check the highest-stakes numbers against a real source.

ii.

It doesn't replace judgement

The committee prompt surfaces reasons not to do the deal. You still decide whether those risks are deal-breakers.

iii.

It can over-correct

Push hard enough and the model starts hedging on claims that were fine. Use it on irreversible decisions, skip it while brainstorming.

iv.

It's not a substitute for ground truth

None of this replaces walking the property or calling the listing agent. It catches AI-shaped lies, not ground-shaped ones.


Next Playbook

Build a grounded research stack for every submarket you cover

Read Next ↗