Principles

How I decide where AI belongs in a workflow, how far to trust it, and how to tell whether it’s working.

Six principles, in the order they come up on a project

UnderstandLearn the work from the people who do it.
ChooseUse AI only where it earns its place.
DesignLet AI handle language. Let rules handle policy.
OverseeKeep a person on the decisions that carry risk.
TestName the mistake you can’t afford, then test for it.
MeasureFinished isn’t the same as helpful.

Understand

Learn the work from the people who do it.

The details that make or break an AI system usually sit with the experts: where time goes, where mistakes happen, which cases are unusual. I bring them in at the start and keep them involved through testing.

The question I ask

Who knows where this process breaks, and have we asked them?

In practiceBuilt with three practicing physicians, who provided roughly 3,778 image annotations.AI-Assisted Pharyngitis Triage

Choose

Use AI only where it earns its place.

Some steps need a language model. Many are handled better by a simpler model, a form, or a clear rule, which cost less and behave more predictably. I check which one the problem calls for before building.

The question I ask

If AI weren’t an option, how would we solve this?

In practiceAn applied ML project asking whether flight-fare assessment needs an LLM at all.Is an LLM even needed?

Design

Let AI handle language. Let rules handle policy.

AI is good at conversation, messy input, and summarizing. Policy has to come out the same way every time and be easy to audit. I use AI to gather and structure information, and fixed rules to apply the policy.

The question I ask

Which decisions here must come out the same way every time?

In practiceClaude gathers the deal details in Slack. Fixed Python rules decide who has to approve.Slack AI Automation

Oversee

Keep a person on the decisions that carry risk.

When a mistake is costly, AI should prepare the decision and a person should make it. When the system is unsure, it should say so and pass the case to someone qualified.

The question I ask

If this goes wrong, who is accountable, and will they see it first?

In practiceWhen the model isn’t confident, the case goes to a clinician for review.AI-Assisted Pharyngitis Triage

Test

Name the mistake you can’t afford, then test for it.

Mistakes don’t all cost the same. An extra approver is an inconvenience; a missing one is a policy breach. I define the costly failure up front and test for it directly, including people trying to game the system.

The question I ask

What is the one mistake this system must never make?

In practiceAcross 30 simulated sales conversations, it never left out a required approver and passed all 11 adversarial cases.Slack AI Automation

Measure

Finished isn’t the same as helpful.

An AI can complete a task and still leave the person unsatisfied. I measure the outcome for the person using it, not only whether the task got done.

The question I ask

How will we know the people using this are better off?

In practiceAcross 200,000 Bing Copilot interactions, AI could appear to finish a task while the user was still dissatisfied.Where Generative AI Fits at Work

Each principle comes from a real project.

See my work Get in touch