Dev48
Language
  • About
  • Services
  • Industries
  • Technologies
  • Articles
  • Contacts
Book a call
    Home/Articles/When probability meets accountability
Dev48
info@dev48.org+375 29 320-40-96WhatsApp@isky_by

© 2026 · All rights reserved.

When probability meets accountability

Source: Atos•November 19, 2026 at 04:18 PM

When probability meets accountability

Consider two people applying for the same service. Their relevant circumstances are identical, the same policy applies, and they submit the same information to the same system.  The first application is approved. The second is rejected. Our AI system can explain both decisions. I would…

Consider two people applying for the same service. Their relevant circumstances are identical, the same policy applies, and they submit the same information to the same system.

The first application is approved. The second is rejected. Our AI system can explain both decisions.

I would still say we have a problem.

This thought experiment gets to something fundamental about using generative AI for consequential decisions. We spend a great deal of time asking whether AI can explain an answer. But we also need to ask whether the answer should have been the same in the first place.

Probability is useful

Large language models generate responses probabilistically, giving them capabilities conventional software lacks. If I ask AI to interpret an ambiguous message, examine a complicated document or work through an unfamiliar situation, you may want that flexibility. The problem doesn't have a single path that can be fully specified beforehand.

Regulated business processes contain plenty of this kind of uncertainty. Applications arrive with free text. Customers phrase things differently. Relevant information hides in supporting documents and cases don't always fit a convenient template. AI can help make sense of that material.

Eventually, though, the process may reach a different kind of question. Perhaps legislation defines eligibility. An insurance policy specifies whether a particular condition is covered. A compliance rule determines what happens when established criteria are met.

The system has moved from interpreting the case to applying a decision that the organization has already defined. How much probability do we want there? I assume none.

An explanation may not be enough

Explainability is important, especialy when automated systems influence consequential decisions. A customer may want to understand a result. An employee may need to review it, while an auditor or regulator may need evidence of how the decision was reached and which laws and regulations it was based on.

Our example of the two applicants exposes a weakness in treating explanation as the entire problem. If a system can produce two different answers from the same relevant facts and then explain both answers convincingly, providing the explanation has not resolved the inconsistency.

For certain decisions, you require something more specific:

  • Which exact information did the system use?
  • Which rules, based on certain laws and regulations, are applied to the decision?
  • Which version of the decision and of its rules were executed at the time?
  • How did these inputs produce this particular output?

And if I submit the identical validated facts to the same decision again, can you reproduce the exact identical decision outcome? That question has direct consequences for the architecture.

Find where uncertainty ends

Imagine an application consisting of a form, correspondence and several supporting documents. An AI system could examine the material and identify case-relevant information. It could classify documents, extract candidate facts and flag something that appears to be missing.

Those are useful roles precisely because the inputs are messy. Once the system establishes and validates the relevant facts, a deterministic service can apply the approved eligibility rules. The system can record the facts provided to that deterministic service, the rule versions and the result returned.

This separation does not remove risk. If AI extracts the wrong facts from a document, software can faithfully calculate the wrong answer from those facts. We still need appropriate validation, escalation, and human oversight.

But it does give us a clearer view of where uncertainty exists and where consequential decisions are made. When something goes wrong, we have a better chance of understanding what happened.

Regulation sharpens the question

These issues become especially visible in regulated environments, where organizations may need to show auditors how systems are controlled and how consequential decisions are made. The exact requirements vary by sector, jurisdiction, and use case, so architecture teams need the relevant legal, compliance and risk expertise when evaluating or designing a real application.

I would ask the reproducibility question even when regulation didn't require it. A customer can challenge a decision months after it was made. A rule may have changed in the meantime, and the organization may need to reconstruct what happened using the information and logic that applied on that particular day. "I asked the model again and it gave me this" is unlikely to be a satisfying audit trail.

A simple test

When examining a decision inside an AI-enabled process, I like one question because it forces us to think about the nature of the work: If nothing relevant has changed, are we comfortable with the answer varying?

There are situations where the answer may genuinely be yes. A different proposed response to a service problem may be equally good. A summary can be phrased differently. More than one plan may achieve the same objective.

On the other hand, if a different answer would create a serious problem, especially when human lives are involved, we have learned something important. The decision requires explicit, deterministic logic, stronger validation, a controlled service for the agent to call and a human in the loop.

AI offers powerful ways to work with ambiguity, even in regulated operations. Still, some decisions require repeatability with identical input and 100% conformity to laws and regulations. The architecture must preserve that capability without letting ambiguity leak into decisions expected to consistently produce the same results with the same input.

Our two applicants deserved an identical decision because nothing relevant differed between them. The architecture should know that too.

We’re looking forward to our sponsorship at Adopt AI 2026, December 3-4, in Paris. Each Thursday leading up to the event, Agile & Digital Enterprise Architect Peter Kalmijn, a member of the Future Makers Research Community (FMRC) at Atos, will explain why good AI architecture starts by saying no.

← All articles

What this article says

More in Cloud & Infrastructure

All →
GitLab 19.6 release notes
GitLab

GitLab 19.6 release notes

How to find at-risk Pub/Sub messages across Google Cloud projects
Dynatrace

How to find at-risk Pub/Sub messages across Google Cloud projects

The AI Trust Gap: Why Consumers Trust Banks Over Fintechs When It Comes to AI
Fastly

The AI Trust Gap: Why Consumers Trust Banks Over Fintechs When It Comes to AI

Runtime security for Amazon EKS Auto Mode with Sysdig
Sysdig

Runtime security for Amazon EKS Auto Mode with Sysdig

Anthropic Ecosystem
HCLTech

Anthropic Ecosystem

Enhancing media supply chain operations for a global entertainment leader on AWS
HCLTech

Enhancing media supply chain operations for a global entertainment leader on AWS

More from Atos

Don’t make the agent do everything
Atos

Don’t make the agent do everything

You can afford the AI pilot. Can you afford the production system?
Atos

You can afford the AI pilot. Can you afford the production system?

Threat actor playbooks: Who is targeting the IT supply chain & how
Atos

Threat actor playbooks: Who is targeting the IT supply chain & how

Just because AI can do it, does that mean it should?
Atos

Just because AI can do it, does that mean it should?

Cyber at machine speed: Can your security program keep up?
Atos

Cyber at machine speed: Can your security program keep up?

Move from prediction to resilience: Navigating the new cyber equilibrium
Atos

Move from prediction to resilience: Navigating the new cyber equilibrium