A new model release. What actually changes.

Every major model update gets covered as a headline. The useful question for a working agent isn't whether it's smarter in the abstract, it's which of your existing workflows just got more reliable, and which new one is now worth building.

Ignore the benchmark. Ask what changed for your workflow.

Model release posts are written for developers and researchers, not for someone underwriting a listing between showings. The translation worth doing every time a major release lands is narrow: does this change the accuracy of my underwriting prompts, does it change what fits in a single context window, and does it unlock a workflow that wasn't reliable enough to trust before.

Three questions. Every release.

01

Re-run your best existing prompt

Take the prompt you rely on most (an underwrite, a memo, a client email) and run it again on the new model. Compare the output line by line, not by feel.

02

Check the context window

A larger working memory means longer document packs and messier multi-file uploads become usable without the drift covered on Day 22.

03

Ask what's now reliable enough to automate

A capability that was 80 percent reliable last quarter and is 95 percent reliable now is the signal to move a workflow from manual review to a scheduled loop.

What it still gets wrong.

i.

Benchmarks rarely map to property work

A model's score on a coding or math benchmark says little about how well it reads a contract or a listing sheet.

ii.

Don't rebuild everything on release day

Re-test the one or two workflows that matter most before assuming everything downstream needs a rewrite.

iii.

Pricing and limits change too

A capability jump is only useful if the plan you're on actually includes it. Check before promising a client a new workflow.

Next Playbook