Over the past two years, Generative AI has dramatically changed how people interact with information. From chatbots answering customer questions to copilots assisting analysts, AI systems are increasingly capable of retrieving relevant information and presenting it in conversational form.
A common claim across many of these systems is that they operate "with context." By retrieving documents, reports, or database records and inserting them into prompts, modern AI tools attempt to ground their responses in relevant information.
But when organizations begin applying these techniques to analytics and business decision-making, a critical limitation quickly emerges:
Having context is not the same as understanding context.
This distinction lies at the heart of a growing shift in enterprise AI architectures, from systems that merely retrieve contextual information to systems that rely on a contextual semantic layer that encodes the meaning of business data itself.
Understanding the difference between these two approaches is essential for anyone building or deploying AI-powered analytics.
What "AI with Context" Actually Means
Most modern AI assistants operate using a technique known as Retrieval-Augmented Generation (RAG). In this architecture, an AI model retrieves relevant information, typically from documents or databases, and includes that information in the prompt sent to the large language model (LLM).
The process generally looks like this:
User question
|
Intent parsing
|
Retrieve contextual documents
|
Identify relevant tables
|
Generate SQL
|
Execute database query
|
Combine results + text context
|
LLM generates answer
For example, if a user asks:
"Why did revenue decline last quarter?"
The system might retrieve:
- Quarterly reports
- Financial commentary
- Internal documents describing revenue trends
- Query results
These pieces of information become the context supplied to the model before generating an answer. The AI needs to retrieve and summarize information and infer meaning at runtime in order to deliver a response.
But trusted enterprise analytics requires something very different.
Why Context Alone Is Not Enough for Analytics
The nature of business analytical questions rarely involves retrieving a single document. Instead, they require understanding how information about your business and data elements relate to one another.
Consider a question like:
"Why did profit decline in Europe even though revenue increased?"
To answer this correctly, an AI system must understand several things:
- Profit is derived from revenue and costs
- Costs may have increased faster than revenue
- Europe is part of a geographic hierarchy
- Multiple cost categories influence profit
While these relationships and meanings exist in the business itself, they are not typically captured in documents or explicitly in modern databases.
Thus, when AI systems rely only on document retrieval, they attempt to infer these relationships indirectly. As a result, the model may produce answers that sound plausible but are inconsistent or incorrect.
This is one of the primary reasons AI systems can sometimes produce confident but unreliable analytics explanations.
Introducing the Contextual Semantic Layer
A contextual semantic layer approaches the problem differently. Rather than retrieving fragments of information, it provides the AI with a structured representation of business meaning.
Instead of documents, the system interacts with models that define:
- Business entities
- Data model and ontology
- Metrics and calculations
- Relationships between metrics
- Hierarchies such as region or product
- Domain-specific business logic
In effect, the semantic layer acts as a map of how the business works.
For example, a contextual semantic layer might represent relationships like this:
Revenue
+-- Region
+-- Customer Segment
+-- Product
Profit
+-- Revenue
+-- Cost
This structure gives AI explicit knowledge of how business concepts relate to one another.
Instead of guessing how profit might relate to revenue, the model can rely on definitions already encoded in the semantic layer.
From Information Retrieval to Business Understanding
This shift from document retrieval to structured context fundamentally changes how AI systems reason about data.
With traditional contextual retrieval:
- AI searches for relevant information
- The model interprets that information on the fly
With a contextual semantic layer:
- AI retrieves business concepts and relationships
- The model reasons using predefined definitions
The difference is subtle but powerful.
One approach provides information. The other guarantees understanding.
The Role of Embeddings
Embeddings play a key role in both architectures, but they are used very differently.
In many AI systems today, embeddings are created from documents or paragraphs of text. These embeddings enable the system to find relevant passages when a user asks a question.
For example:
- Financial report paragraph -> embedding
- Quarterly analysis -> embedding
- Strategy document -> embedding
This helps the system retrieve text related to a query.
In contrast, a contextual semantic layer generates embeddings for database entities and relationships as well as business entities and relationships.
Examples might include embeddings for:
- Metric definitions
- Entity relationships
- Database tables, joins, synonyms, descriptions
- Domain concepts
- Business rules
For instance:
- Net Revenue -> embedding
- Customer Segment -> embedding
- Profit Margin -> embedding
- Cost of Goods Sold -> embedding
Because these embeddings represent structured database and business meaning, the AI can retrieve and reason over conceptual relationships, not just documents.
This significantly improves accuracy for analytical questions.
Why This Matters for AI-Powered Analytics
Many organizations are beginning to deploy AI copilots on top of their data platforms. These systems work well for exploratory queries or simple summaries.
However, as users begin asking deeper business questions, limitations emerge.
Without a contextual semantic layer, AI systems often struggle with:
- Inconsistent metric definitions
- Fragmented business logic
- Ambiguous terminology
- Missing relationships between data elements
For example, different teams may define "revenue" in slightly different ways. An AI system relying solely on document retrieval and runtime inference may not recognize these differences.
A contextual semantic layer solves this problem by standardizing data models, business definitions and relationships before AI interacts with the data.
The AI no longer needs to infer meaning. It simply accesses it.
A Simple Analogy
An easy way to understand the difference is through an analogy.
Imagine asking two assistants the same question about your company's performance.
The first assistant is given access to a large folder of documents. They read through the material and attempt to infer what it all means.
The second assistant is given a detailed map explaining how the company operates: how revenue flows through the business, how costs are structured, and how performance metrics are calculated.
Both assistants have information.
But only one has structured understanding.
That difference is exactly what separates AI with context from AI with a contextual semantic layer.
The AI Proliferation Crisis
What makes the challenge of context for AI more alarming for enterprises is that "AI" is no longer a singular thing in the decision architecture of these businesses. Every tool and platform now has some form of AI. And with each AI addition you have an AI supported by a different LLM, using context a little differently from the next.
The end result is not just metrics confusion or business logic drift. You multiply that inconsistency by the potential number of AIs that users can interact with across the organization.
The Future of AI Analytics
As generative AI becomes more embedded in enterprise workflows, organizations will increasingly recognize that retrieval alone is not sufficient for trustworthy analytics.
To support reliable decision-making, AI systems must be able to interpret data in the same way the business does.
This requires a layer that encodes:
- Business definitions
- Metric relationships
- Domain knowledge
- Data models and hierarchies
In other words, it requires a contextual semantic layer designed specifically for AI interaction.
Because ultimately, the goal of AI analytics is not simply to retrieve information. It is to allow machines to understand the language of the business itself.
About Codd AI
To make AI work in a trusted manner at enterprise scale, you will need to architect your information and decision architecture differently. There are a few core concepts you have to design for:
-
Unified contextual semantic layer. The semantic layer must combine the technical database metadata with your business logic and rules. Simply understanding the database, tables, joins and some business-friendly terms is not a Contextual Semantic Layer.
-
Automated creation of the Contextual Semantic Layer. The modern data estate is a data swamp with no explicit data models or ontologies. Manually trying to generate the data models with the correct joins and dependencies just will not provide sufficient agility and speed to insights. Hence, AI needs to be extensively deployed to auto-design and detect ontologies, data models, and business metrics.
-
Common Contextual Semantic Execution Engine. Your number of data processing and data visualization technologies will not reduce. In fact they will continue to explode as new tech comes to the fore. What is required though is a standardized and common contextual semantic framework and execution engine underpinning all of these access paths to ensure consistency and accuracy. Think of it as your single version of AI truth.
Codd AI has been designed with this vision in mind. Our platform was natively built from the ground up for a world of GenAI, both to automate the semantic layer creation as well as powering the end-user conversational analytics itself. If you are interested in learning more about Codd AI, visit us at www.codd.ai or schedule your personal overview conversation.