Clinical AI architecture · 20 August 2026 · Pathways

Clinical safety rules should not be model output

Safety-critical clinical screening should be implemented as deterministic, versioned rules rather than model output, because clinical governance requires reproducibility rather than accuracy alone. A rule that fires whenever its conditions are met can be tested, audited and re-run months later to produce the same answer — a property a probabilistic system cannot offer.

There is a reasonable-sounding argument that if a language model screens perioperative risk more accurately than a rule set, it should screen perioperative risk. The argument is wrong, and the reason it is wrong is not about accuracy at all.

Governance asks a different question

A perioperative committee reviewing a case does not primarily ask whether the system was right. It asks what the system was told, what it concluded, and whether that conclusion follows from the input. Six months later, in a morbidity and mortality review, it asks the same questions again — and expects the same answers.

That is a reproducibility requirement, and it is stricter than an accuracy requirement. A system that is right 97% of the time but cannot tell you why it was right in this case, and might answer differently if asked again, fails review even when it is right.

What deterministic buys you

A deterministic rule has three properties that matter here.

It is testable. “Flag an identified SGLT2 inhibitor with no documented withholding plan for the surgery date” is a condition you can write test cases against. You can enumerate what should fire and what should not, and you can run that suite on every change.

It is versioned. Every output records the rule version that produced it. When a case is re-examined in September, you can re-run it against the rule set as it stood in March rather than against the current one, which is the difference between an audit and an argument.

It cannot be talked out of it. This one sounds glib and is the most important. A rule fires whenever its conditions are met. It does not have a bad day, it is not swayed by the surrounding narrative, and it does not become less confident because the rest of the chart looks reassuring. For the specific set of findings that exist to protect patients, that inflexibility is the feature.

Where models belong

None of this is an argument against using models. It is an argument about which part of the pipeline they occupy.

Models are very good at the messy front end: conducting a conversational interview that follows up appropriately, reading a photograph of a medication box including handwriting, classifying an uploaded document, turning free narrative into structured findings. These are tasks where the input is unbounded and brittle rules genuinely fail.

The architectural line is drawn after that. Extraction and interpretation produce structured findings; those findings are verified against their source; and only then does a deterministic engine screen them. The model’s output is an input to the rules, never a substitute for them.

Two consequences worth stating

Confidence is shown, not hidden. If an extracted value is 86% confident, the profile says so, and the missing confirmation is listed explicitly. A system that presents uncertain extractions with the same visual weight as verified ones is making a clinical judgement it has no standing to make.

The system states what it does not know. Missing investigations and unconfirmed timings appear on every profile. This is the part most easily skipped, and it is what separates a screening tool from a reassurance tool — a clean profile that has not been told about the echocardiogram is not the same thing as a clean profile.

The design rule

Use models where the input is unstructured and the cost of an error is a human reviewing something unnecessarily. Use deterministic rules where the input is structured and the cost of an error is a patient harmed. Verify the boundary between them, and record which version of everything produced the answer.

That is not a compromise between safety and capability. It is what makes the capable part deployable in a setting that has to answer for its decisions.

FAQ

Related questions

Where is Pathways patient data stored?

All Pathways patient data is stored in AWS Sydney (ap-southeast-2). Patient information is not transferred outside Australia in storage, processing or backup. Data is encrypted in transit using TLS 1.2 or above and at rest with keys managed in AWS KMS.

Data sovereignty is a procurement gate for most Australian health services, so it is worth being precise about the boundary. The Australian region covers the full lifecycle: primary storage, processing during an assessment, and backups. There is no cross-region replication of patient information.

Does Pathways send patient data to AI model providers?

No. No personally identifying information is sent to model providers. Extraction and reasoning operate on de-identified clinical content, and patient information is never used to train models. The safety-critical screening rules are deterministic code rather than model output, so they do not involve a model provider at all.

This is two separate guarantees, and they are worth separating because they fail in different ways.

The first is about identity: the content that reaches a model provider is de-identified, so a model provider never receives a named patient record. The second is about training: patient information is not used to train models, by anyone, at any point.

Underneath both sits a design decision that removes the question for the parts that matter most. The rules that protect patients — the SGLT2 withholding check, the glycaemic control threshold, the OSA screen — are deterministic, versioned code. They are not a model output that happens to be reliable; they fire every time their conditions are met and cannot be argued out of it.

What is Socrates Pre-Op?

Socrates Pre-Op is the Pathways voice agent that takes a structured anaesthetic history from a patient by phone. It is delivered as a secure, time-limited SMS link — no app, no login, no paper questionnaire — and covers 12 clinical domains conversationally, producing structured, severity-tagged findings that feed the risk engine directly.

It is a purpose-built anaesthetic variant rather than a generic intake bot. The distinction matters in practice: a general-purpose intake tool collects answers to the questions it was given, whereas an anaesthetic interview needs to follow up on a positive family history of anaesthetic problems, probe functional capacity when a patient says they are “fine”, and recognise when a described medication is worth photographing.

Completion is tracked, and a finished interview automatically triggers a fresh risk analysis.

How this compares with the two methods it usually replaces — paper questionnaires and telephone pre-assessment — including where each of those still wins.

Do patients know they are speaking to an AI?

Yes. The interview opens by identifying Pathways as an automated assistant working on behalf of your practice or hospital, and tells the patient that a clinician will review everything discussed. Patients can ask to speak to a person at any point rather than continue with the assessment.

Disclosure is not a compliance box here, it is what makes the answers usable. A patient who thinks they are filling in a form gives form-shaped answers; a patient who understands they are being interviewed, and that a clinician will read it, tends to volunteer the thing that matters — the stent nobody recorded, the sleep study that was never followed up.

Telling them a clinician reviews the result is also simply accurate. Pathways issues no orders and clears no patients; the summary is a draft until someone qualified signs it.

Does Pathways need to integrate with our PMS or EMR?

No. Pathways runs standalone from day one. The SMS-driven workflow needs no patient management or EMR integration to start, so a service can screen its next elective list without an IT project, an interface engine, or a change window. Integration is available later but is not a precondition.

This is usually the difference between a deployment measured in days and one measured in quarters. Most of the value — screening every booked patient early enough to act — does not depend on a bidirectional interface, so requiring one up front only delays the point at which the first cancellation is prevented.

When you do want it, integration is HL7 or FHIR against your patient administration and EMR systems: automated list ingestion from theatre scheduling in one direction, and write-back of the signed pre-operative summary to the patient record in the other. That is an Enterprise capability, available on Business, and it is scoped after the standalone deployment is running rather than before it.

Is Pathways a medical device, and who is responsible for clinical decisions?

Pathways is clinical decision support intended for use by qualified clinicians within Australian clinical governance frameworks. It is clinician-in-the-loop by design — it issues no automated orders, and every recommendation is reviewable and auditable. The treating clinician remains solely responsible for all clinical decisions.

The positioning is deliberate and is reflected in the architecture rather than only in the terms of service. Pathways produces risk flags with severity tiers, the evidence that triggered them, a guideline reference and a recommended action. It does not place orders, it does not clear patients, and it does not route anyone to theatre on its own authority.

How does Pathways generate its risk flags?

Pathways screens each case with deterministic, versioned rules referenced to perioperative guidelines. Every flag carries a severity tier, the clinical rationale, the triggering evidence, the guideline reference and a recommended action, plus the rule version that produced it. The same inputs always produce the same flags, months later.

Screening covers the domains that drive perioperative outcomes: unstable cardiac symptoms, marked hypertension, diabetes control and SGLT2 management, obstructive sleep apnoea, frailty, malnutrition, functional capacity, smoking and discharge support.

Pathways also states what it does not know. Missing investigations and unconfirmed timings are listed explicitly on every profile, and extraction confidence is shown rather than hidden — a profile that is 86% confident about a patient-reported HbA1c says so.

Can we audit why a patient was cleared or flagged?

Yes. Pathways keeps org-wide and per-patient audit logs with full workflow tracing on every risk analysis — the inputs used, the rule version that fired, the output produced, and the human who reviewed it, all timestamped. Logs are exportable for governance committees, M&M review and accreditation.

Because the safety-critical rules are deterministic and versioned, an audit is reproducible rather than merely archived: re-running a case against the rule version recorded at the time produces the same flags. That is the property that lets a decision made in March be examined in September without argument about whether the system “would have said something different”.

See all questions

See your list the way Pathways sees it.

Book a 30-minute demo with a real pre-admission workflow.