An AI output lands on your desk. The structure’s tight, the language confident, and it took under a minute to produce. Relief is the first reaction. Suspicion should probably be the second, but it rarely is, because the thing reads so cleanly that doubting it feels almost rude.
Here’s the catch: the tool might be completely right. The problem is you don’t actually know yet, and a polished answer is very good at making you feel like you do.
That was more or less the thread running through the panel at CPS’s Future Skills Forum on 8 October. Anton de Souza, a chartered accountant and former CEO, CFO and COO, put it plainly: something that looks ready and arrives fast gets treated as ready. Length doesn’t help either. A long, detailed answer reads as thorough, even when nobody has actually checked whether the detail holds up.
As De Souza warned during the webinar, “The competence in due care still sits with me.”
None of this is an argument against using AI, and nobody on the panel tried to make that case. The point is narrower: the easier it gets to produce something that looks finished, the harder you have to work not to mistake that for the thing actually being finished.
Why this hits differently in financial services
In most industries, a confident but wrong first draft costs you an afternoon of fixes. In banking and insurance it’s rarely that cheap. A credit recommendation, a claims decision or a piece of client advice carries regulatory and reputational weight that doesn’t disappear because a machine wrote the first version.
De Souza spoke to this from his own chair. His duty of care as a chartered accountant hasn’t changed because the tools around him have. A model can produce the analysis, but it can’t carry the accountability. That still sits with whoever puts their name on the recommendation, and skimming something that reads smoothly isn’t how that duty gets discharged.
His warning was direct: “The cost of intelligence is your career and your reputation.”
So the skill the panel kept circling back to was never really about AI at all. It’s about the gap between what checking looks like and what approving looks like, because from the outside those two can be almost impossible to tell apart.
What checking actually looks like
A handful of concrete habits came out of the conversation, worth taking literally rather than filing away as a vague reminder to be careful.
Question the answer, not just the prompt. It’s tempting to judge an AI output by whether it addresses what you asked. The harder question is what it left out, and whether you’d have noticed.
Put the work in front of a different seat. Vincent Maher, who leads Broadbrand, described a simple habit: asking AI itself to list the top five things a CFO would flag as wrong with a piece of analysis. It sounds obvious, but it reliably surfaces the number that doesn’t add up, or the assumption that only holds if you ignore something inconvenient.
Maher reinforced the principle during the discussion: “Whether you or your AI are busy doing it, own the outcome and the outputs of it.”
Know your question before you start. A vague request tends to produce volume, not an answer, and a 75 page document nobody has time to read just shifts the work onto whoever reads it next. Most executives want a short answer backed by evidence someone has genuinely reviewed, not a long one that merely looks thorough.
Treat the output like it came from a new hire. You wouldn’t put your name on a first draft from someone three weeks into the job without reading it properly. AI deserves the same instinct, however fluent the sentences sound.
Ask what you actually learned. This is de Souza’s own test before publishing anything built with AI help: did I learn something in the process, or did I just hand over what the model gave me. If it’s the second one, honestly, it isn’t ready yet.
The harder case: when you can’t check it yourself
These habits get harder when the AI output sits outside your own expertise, and that’s common, not rare. A relationship manager reading an AI-generated risk summary can’t always verify every technical claim alone. De Souza’s answer wasn’t to pretend otherwise. It was to be honest about where your own competence runs out, and route anything past that point to someone who can actually judge it, rather than nodding along because admitting the gap feels uncomfortable.
It’s close to a point CPS has made before: digital fluency is not the same as digital competence. Knowing how to get a good-looking answer out of a tool isn’t the same as knowing when that answer deserves to be trusted.
Worth adding, so this doesn’t read as entirely bleak: De Souza also uses AI to check the tone of his own writing before he sends it, a second opinion on how something will land rather than a verdict on the facts. The tool isn’t always the thing under suspicion. Sometimes it’s just a useful extra pair of eyes on your own work.
Where scepticism becomes its own problem
It would be easy to walk away thinking the safe answer is to distrust everything AI produces. That’s its own trap. Dr Indira Bhagaloo, hosting the panel, was clear that healthy scepticism isn’t reflexive doubt. Send every piece of work back for another round of critique and you don’t get quality, you get chaos, with nothing ever quite finished and nobody quite satisfied. The real skill is judgement about what genuinely deserves a second look, harder and more useful to build than blanket suspicion.
As Bhagaloo explained in her closing remarks, “Competence means that you know when, why, and with what judgment you actually need to use AI.”
That judgement doesn’t show up on its own, and no policy memo builds it for you either. At CPS, ethics and compliance has always sat as a full twenty credit module inside the Higher Certificate in Banking, taught early in a career rather than bolted on later. The logic holds regardless of the tools involved: judgement has to be built deliberately, because whatever produces the first draft keeps getting more convincing.
The question worth sitting with
None of this really comes down to whether AI is reliable enough to trust. It comes down to whether the habit of checking survives contact with something that no longer looks like it needs checking. A fast, fluent answer doesn’t announce its own blind spots. Finding them is still the job.
So next time an AI output lands on your desk, ask plainly: are we checking this, or just approving it. The two feel alike from the inside. They aren’t the same, and the gap between them is exactly where professional judgement still earns its keep.
We explored this with a panel of industry leaders at the Future Skills Forum.
