The Black Box of AI: What the World’s Leading Expert Says We Still Don’t Know

We’ve all seen the headlines: AI is now the world’s most powerful business tool. But what would you do if it starts making things up and you don’t even notice?

That’s exactly what’s happening right now inside boardrooms, workflows, and enterprise systems. AI is rewriting job descriptions, changing how we make decisions, and becoming the new engine of business productivity.  Behind the hype lies a more complicated truth: the same systems driving productivity gains can also hallucinate facts, fabricate quotes, and break in ways we don’t fully understand.

That contradiction, AI’s immense potential versus its unpredictable behavior, is exactly what Babak Hodjat, CTO of AI at Cognizant, tackles in our latest episode of Future Ready Leadership. As one of the early minds behind Siri and a pioneer in multi-agent systems, Babak breaks down how he builds AI, leads teams that deploy it, and helps Cognizant scale it across 330,000+ employees.

In this episode, Babak pulls back the curtain on the promise and paradox of AI in the enterprise, and why Cognizant is betting on agentic AI, systems that don’t just answer questions but take actions on behalf of humans.

Read: How to Embed AI in Your Organisational DNA

What AI Is Really Doing in Your Business (and Why That Matters)

State the obvious, AI is like a smarter version of a search engine. We see it as a digital brain, an entity that reads, understands, and responds like a hyper-intelligent assistant. But Babak makes it clear, that’s not what’s happening under the hood.

The large language models (LLMs) behind tools like ChatGPT and Claude don’t “know” anything in the human sense. Today’s most powerful systems are built on a deceptively simple principle: predict the next word in a sequence. They're not reasoning or reading. That process, powered by deep learning and neural networks, creates outputs that look intelligent. Sometimes they are. Other times, they’re total fabrications.

This is what’s known as AI hallucination, a feature, not a bug, of generative AI systems. These models are excellent at producing high-probability text, but they’re not fact-checking or even “thinking” in the human sense. As Babak explains, it’s more accurate to view AI as a mathematical transformation, not an algorithm in the traditional sense. It doesn’t follow rules, it maps input to output using a sea of probabilities.

And this brings us to the problem of trust.

If an AI agent can generate something plausible that isn’t real such as a fake quote, a nonexistent book, or a fabricated statistic, how do you build critical business systems on top of it? For Cognizant, the answer is not to shut the door on AI, but to reframe how it’s used and tightly control where it takes action.

If AI had a seat at the leadership table, this is the conversation it would start:

Inside the Rise of Agentic AI at Cognizant

Rather than relying on AI for answers alone, Cognizant is embracing what Babak calls agentic AI—systems that combine powerful LLMs with actionable code to get things done. These are not just assistants. They are digital coworkers, equipped to interact with systems, perform tasks, and collaborate with other agents in what Babak calls a multi-agentic platform.

Already, Cognizant is using these AI agents to:

• Manage RFP workflows—deciding whether to respond, surfacing past proposals, and drafting responses
• Support intranet systems for over 300,000 employees—surfacing answers across apps and tools without requiring users to know which system to ask
• Power inter-agent communication—letting different AI agents coordinate across departments like sales, legal, HR, and IT

But the real innovation isn’t just in the tech. It’s in the design philosophy. These agents don’t run wild. Cognizant is not doing it blindly. They’re carefully designing for control and resilience. These agents operate within very clear parameters, are monitored by humans, and have fallback systems in place if something doesn’t go as planned.

That’s where the idea of human-in-the-loop becomes essential, not just for ethics, because while these systems are powerful, they’re not infallible. Babak’s team has even open-sourced their multi-agentic platform, inviting developers to build and run their own AI systems that mirror how humans work together in teams. Agents coordinating with other agents to get work done. It’s not just about replacing humans; it’s about extending how we operate.

Read: Culture Drift: When AI Moves Faster Than Trust

How to Think About AI in Your Organisation

Building an AI-first organisation isn’t just about automation or scaling productivity. It’s about rethinking workflows from the ground up: asking where an AI agent can take over, where a human is still essential, and how those interactions should be structured.

AI is already reshaping Cognizant’s internal structure. With over 330,000 employees, they’ve embedded AI agents into everything from HR to intranet navigation to legal workflows. And as Babak points out, we’re headed toward a world where leaders may manage five direct reports…and fifty agents.

If you’re thinking about integrating AI into your enterprise, consider this:

• Don’t think of AI as a magic brain. Understand it’s a probability machine, not a truth engine.
• Focus on structured use cases. RFP workflows, internal knowledge search, and process automation are all great starting points.
• Design with failure in mind. Assume the system will hallucinate or get things wrong, and build safeguards like human review or controlled autonomy.
• Agentify with intention. It’s tempting to offload tasks to AI, but the responsibility and liability still rests with people.
• Build for resilience, not perfection. That includes having backups, oversight, and clear triggers for when a human steps in.

Why the Future Will Be Built by Humans and Agents Together

We’ve reached the point where AI is becoming something we work alongside, not just something we interact with. And that changes everything. As Babak points out, AI won’t replace humans. But it will reshape how we work. The organisations that win won’t be the ones who blindly trust AI, or who fear it entirely, but the ones that learn how to collaborate with it.

Agentic AI may be the future, but it’s still only as good as the people who design, manage, and guide it.

This article is republished courtesy of Thefutureorganization.com.