Why large companies need an ontology before deploying AI agents
Large companies are different from medium-sized and small companies, and not simply because of their size.
In a small company, senior executives are deeply involved in operations. They know the customers, approve exceptions, solve delivery problems and often understand why each process works the way it does.
As the company grows, executives become less operational and more tactical. Eventually, the organization becomes large enough that its leaders must spend most of their time thinking about the future rather than managing the present.
This transition creates a problem that receives surprisingly little attention: where does the company’s knowledge go?
As companies grow, “We do it this way because we have always done it this way” becomes increasingly common. But why did the company start doing it that way? Does anyone remember? More importantly, does the original context still justify the decision?
The older the company becomes, the more of its knowledge disappears from view.
Some knowledge leaves with former employees. Some remains buried in documents that nobody reads or even knows how to find. Some survives in emails and WhatsApp groups that exclude the people who now need it. And a significant part of it becomes embedded in software code that business leaders cannot read or interpret.
A surprising amount of corporate knowledge is locked inside the customizations and operating rules of ERP, CRM, TMS, WMS and other enterprise systems. Those systems contain years of decisions about pricing, credit, inventory, procurement, logistics, customer service and exceptions.
The users see the screens. They rarely see the reasoning behind them.
This explains why large companies hire consulting firms to map their processes and tell them how their own organizations work. There is something almost absurd about a company paying an outsider to explain the company to itself, but this happens all the time.
Organizational silos make the problem worse. Departments isolate themselves from one another to avoid conflict and preserve what is described as a “healthy” working environment. The result may feel peaceful, but it also allows each department to develop its own vocabulary, rules and interpretation of the business.
For years, companies managed to live with this fragmentation. The transition to AI agents is exposing its cost.
The knowledge that companies have forgotten, hidden or embedded in systems is precisely the knowledge that AI needs. Its absence helps explain why so many AI implementations fail to produce reliable results.
Consider a hospital that decides to use an AI assistant.
An ambulance team sends a message to the hospital’s WhatsApp account asking whether a bed is available. The AI checks the system and answers yes. Based on that response, the team brings the patient to the hospital.
When the ambulance arrives, however, the hospital cannot receive the patient. There is no bed that can actually be used.
Did the AI make a mistake?
From the perspective of the algorithm, perhaps not. The system showed an unoccupied bed, so the AI concluded that a bed was available. The failure occurred in the meaning of the word “available.”
Without an ontological layer, “available” remains open to interpretation. A person familiar with the hospital understands that an empty bed is not necessarily an available bed. The AI does not automatically possess that institutional knowledge.
In that hospital, the correct business definition might be:
Available bed = physically unoccupied bed + completed sanitation + adequate nursing and medical coverage for that shift.
Once those conditions are included, the answer changes completely.
The ambulance team asks exactly the same question. The AI consults exactly the same hospital. But instead of recommending that the patient be brought in, it says that no bed is currently available.
The difference is not a more expensive AI model. It is not greater computing power. It is not a better-looking interface.
The difference is that the second system understands what “available” means within the operational reality of that hospital.
An ontology gives AI access to the company’s concepts, relationships, constraints and business rules. It establishes that “customer,” “delivery,” “inventory,” “risk,” “approved,” “profitable” or “available” may have precise meanings that are unique to that organization.
This is why:
The most important step toward becoming an AI-powered company is not buying AI. It is building the company’s ontological layer.
Small, medium-sized and large companies all need one, but the urgency changes with scale.
In a small company, much of the ontology still exists inside the heads of the founders and senior executives. When a term or rule is unclear, someone can walk into a room and ask the person who created it.
That becomes impossible in a large organization. Knowledge is distributed across departments, countries, systems, documents and former employees. No individual understands the entire business. Even when the knowledge still exists somewhere, the organization may no longer know where to look.
Large companies therefore face a strategic choice. They can continue deploying AI agents over fragmented systems and hope that the agents correctly interpret decades of undocumented decisions. Or they can first reconstruct the knowledge that makes the company what it is.
Before a company can teach AI how to act, it must first define its own institutional DNA. Does your organization already have the ontological layer AI needs?
Daniel R. Schnaider is Vice President of Enterprise Solutions at Luby Software, where he leads the company’s Agentic AI practice. He is co-author of Infinite Manpower: The Transition to Agentic AI and the Future of Work, Business, and the Nation.
https://linkedin.com/in/danielschnaider
https://x.com/DaniRSchnaider