Every few months, the tech world gives itself a new title. First it was Machine Learning. Then Deep Learning. Then Generative AI. And now the phrase showing up in every board deck and vendor pitch is Agentic AI.
The pace of change is genuinely exciting. But working across retail and healthcare ― two industries where the stakes of a wrong decision are very different but equally real-I keep coming back to a question nobody seems to be asking out loud:
Are we actually building intelligence, or are we just getting really good at prediction?
Here’s the thing-large language models don’t truly understand anything. They predict the next word. Over and over, at massive scale. When someone first told me that, it sounded like a criticism. But then I thought about what both retail and healthcare already run on-and I realized prediction has always been at the heart of it.
Retailers predict what customers will want next season. Hospitals predict which patients are at risk of readmission. Supply chains predict where the next shortage will hit. Clinical teams predict how a patient population will respond to a treatment protocol. Prediction isn’t a weakness -it’s one of the most powerful tools these industries have ever had.
The problem shows up when organizations start treating prediction as understanding. That’s when things get dangerous.
When “Looks Smart” Isn’t the Same as “Is Smart”.
I’ve spent time on the ground in both retail operations and healthcare delivery environments, and the excitement about AI is completely real and completely understandable. A model can write a clinical summary, generate a patient discharge plan, predict next week’s demand for a product category, or flag a supply chain anomaly -and the output often looks sharp enough that people start trusting it like they’d trust a senior colleague.
But that trust can outpace what the system actually knows.
In retail, a demand forecasting model might confidently recommend restocking a flagship product line. And based on historical sales data and market signals, it’s right. What it doesn’t know is that the supplier flagged a six-week lead time constraint yesterday, and the distribution center for that region is already at 94% capacity. The model saw the pattern. It missed the reality. A sharp merchandising manager caught it in review -but the uncomfortable question lingering afterward was: what if she hadn’t been looking?
In healthcare, the stakes of that same question are considerably higher. An AI-assisted triage tool might flag a patient as low-risk based on vitals and EHR history -without having access to the nurse’s most recent observation notes, or the social context that the patient lives alone and has a history of not following up. The system’s output is technically defensible. But a clinician who acts on it without that fuller picture could make a decision that harms the patient. The AI didn’t lie. It just didn’t know what it didn’t know.
“Technically correct. Operationally incomplete. That gap is the whole problem ― and it doesn’t show up in the product demo”.
The Real Failure Point Isn’t Intelligence ― It’s the Basics
Here’s what I keep seeing in both industries: the AI itself is rarely the failure. The failure is almost always in the infrastructure sitting underneath it.
Missing context. Fragmented data. Weak governance. Processes that were never designed to talk to each other.
A large retail chain I worked with had invested heavily in an AI-powered pricing engine. Genuinely impressive technology. But the product catalog data feeding it had duplication issues that nobody had fully cleaned up, and the competitive pricing signals it relied on were delayed by 48 hours. The engine was optimizing brilliantly ― for a picture of the market that was two days old. Margin leakage followed. The AI wasn’t the problem. The data pipeline was.
In healthcare, the data problem is even more acute. Patient information lives across EHR systems, lab platforms, imaging tools, pharmacy records, and handwritten nursing notes that have never been digitized. An AI clinical decision support tool working off incomplete records isn’t just less accurate ― it can create a false confidence that’s actively harmful. Clinicians trust the screen. The screen doesn’t have the full picture. Nobody flagged that to anyone.
The old rule still holds: garbage in, garbage out. AI just makes it happen faster, at larger scale, with more confidence in the output. That’s not progress. That’s amplified risk.
Where Decision Intelligence Fills the Gap
This is where I’ve become a real believer in something called Decision Intelligence ― not as a product you buy, but as a discipline you build. At its simplest, it’s the practice of designing the full decision-making environment around AI, not just the model at the center of it. Who reviews the output? What signals does the system still need? Where does a human have to sign off, and why? What does “good enough to act on” actually mean in this context?
In retail, this might look like a replenishment AI that generates purchase order recommendations ― but routes every order above a certain value through a buyer’s desk before execution, with a structured checklist that prompts them to verify supplier availability, warehouse headroom, and current promotional plans. The AI does the heavy lifting. The human does the contextual check. Together, they make better decisions than either would alone.
In healthcare, it looks like an AI that surfaces risk scores for patient deterioration ― but is explicitly designed so the attending physician sees the score alongside the factors that drove it, not just the number. The clinician isn’t replacing her judgment with the model’s. She’s using the model to sharpen her judgment. That’s a fundamentally different relationship with the technology, and it’s one that tends to produce better patient outcomes.
The best AI deployments I’ve seen in both industries share a common trait: they treat human judgment not as a bottleneck to be engineered around, but as a structural part of the system. AI handles scale and pattern recognition. Humans handle context, ethics, and the judgment calls that sit at the edge of what the model was ever trained to know. Neither works as well alone.
The New Risk: Agents That Act Without Being Watched
Agentic AI takes everything we’ve talked about and turns up the volume.
Traditional AI recommends. Agentic AI acts. It can open purchase orders, trigger patient referrals, update records, send communications, and make operational calls without a human approving each step. The efficiency gains are real. So is the exposure.
When a chatbot gives a wrong answer in a retail app, the customer is frustrated. When an agentic system in a hospital miscategorizes a patient’s urgency level and automatically schedules them for a routine follow-up instead of flagging them for immediate review ― that’s a clinical incident. When an autonomous procurement agent locks in a bulk purchase at the wrong price tier because a contract update hadn’t been fed into the system yet ― that’s a financial hit that can take months to unwind.
The risk I’d most want to flag is one that sneaks up slowly: overtrust. An agent performs well for a few months. People start relying on it. Review steps get quietly shortened. Approval thresholds drift upward. Then one day ― a supplier disruption, a formulary change, a spike in demand nobody modeled – the agent encounters something just outside its training and acts confidently on an incomplete read of the situation. Nobody catches it because the oversight mechanisms had slowly eroded. The system didn’t fail. The governance around it did. I’ve seen this play out in retail inventory replenishment and in clinical triage support. The pattern is the same in both.
This is exactly why the future of enterprise AI isn’t about building bigger models. It’s about building stronger, more honest architectures around the ones we already have.
What “Strong Architecture” Actually Means?
The things that make AI trustworthy at scale rarely make it into the sales pitch. But they’re the things that actually matter when you’re running real operations:
- Governance ― Who actually decides what the AI is and isn’t allowed to do?
- Auditability ― Can you trace exactly how a recommendation or action was generated?
- Data trust ― Is the information going into the system actually reliable and complete?
- Operational boundaries ― Does the AI actually know the constraints of your specific environment?
- Human oversight ― Where do humans stay genuinely in the loop, not just nominally?
- Accountability ― When something goes wrong ― and eventually something will ― who owns it?
In retail, getting these wrong means margin erosion, stockouts, or customer experience failures. In healthcare, the consequences can be clinical. Both industries deserve better than “we’ll sort the governance out later.”
The Question That Actually Matters
After years working across enterprise analytics, retail supply chains, and healthcare data programs, I’ve noticed that the question leaders in both industries actually ask about AI is never the one the vendors answer.
Nobody asks “Is this system truly intelligent?”
They ask something much more grounded: “Can I trust the answer enough to act on it?”
A retail buyer wants to trust the replenishment recommendation before she commits to an order. A hospital CMO wants to trust the risk stratification before it shapes care protocols. The metric they both care about isn’t accuracy on a benchmark. It’s whether the system has earned the right to influence a real decision.
That’s trust. And trust isn’t a feature you ship. It’s something you build, slowly, through consistent transparency about what the system knows and -just as importantly-what it doesn’t.
Closing Thought
Prediction is genuinely powerful. I’ve seen it change how retailers manage inventory and how hospitals manage patient flow. There’s no going back, and there’s no good reason to.
But prediction alone doesn’t create value. Trusted decisions do. And trusted decisions don’t come from smarter models alone ― they come from smarter environments built around those models. Whether you’re running a retail operation or a health system, that’s the work that actually determines whether AI delivers on its promise.
That’s the harder work. And it’s the work worth doing.
