Most of us have had some version of this conversation.
You ask your spouse, partner, colleague, or boss what they think you should do about something. They give you an answer. A couple of weeks later, the subject comes up again and somehow the answer is different.
Naturally, you say:
"Wait. That's not what you said last time."
I've certainly been on both sides of that conversation.
It's human. We reconsider things. We weigh information differently. Maybe the circumstances changed. Maybe we learned something new. Or maybe we simply don't remember exactly how we arrived at the original answer.
AI has a version of the same behavior.
And in many applications, that's actually one of its strengths. We don't necessarily want AI to mechanically follow exactly the same thought process every time. Its ability to reason about a problem, explore alternatives, and adapt its approach is part of what makes it so powerful.
But analytics is different.
For decades, one of the reassuring things about dashboards and reports has been their predictability. Open your monthly sales dashboard today or next Tuesday and the methodology doesn't spontaneously reinvent itself. The calculations, filters, groupings, and presentation remain consistent.
As analytics becomes increasingly AI-driven, we are trading some of that determinism for something enormously valuable: reasoning.
And that creates an interesting new challenge.
How do we get the flexibility and intelligence of AI reasoning without giving up the repeatability and consistency businesses expect from analytics?
We've Been Solving the First Consistency Problem: Data
Consider a relatively simple question:
"What were Western Region sales last quarter?"
For AI to answer this consistently, it needs to understand what those words actually mean in the context of your business.
- What constitutes "sales"?
- Which customers, states, or territories belong to the "Western Region"?
- Does "last quarter" mean the previous calendar quarter or your company's fiscal quarter?
- Are we talking about booked revenue, recognized revenue, gross sales, or net sales?
- Which transactions should be excluded?
These are semantic questions, and they are exactly why semantic layers, business ontologies, metrics layers, knowledge graphs, and other forms of governed business context are becoming increasingly important for enterprise AI.
If the definitions of Western Region, Sales, and Fiscal Quarter are governed, AI doesn't have to reinterpret them every time someone asks the question.
The underlying data and business meaning can remain consistent.
That gives us semantic consistency.
But that only solves part of the problem.
Things Get More Interesting When We Ask AI to Analyze
Now consider a different request:
"Analyze gross margin versus actual sales for the last three quarters. Show me the three worst-performing and three best-performing product categories. Then identify marketing promotions that have been effective in the top-performing regions and recommend promotions that could improve performance in the underperforming categories."
That's a very different type of question.
In fact, calling it a "question" is probably underselling it.
It's an analytical assignment.
To answer it, an AI system may need to retrieve sales and margin data, determine how performance should be evaluated, calculate trends, rank product categories, compare regional performance, analyze historical promotions, identify patterns, determine whether successful promotions might apply elsewhere, formulate recommendations, and then present everything in a useful format.
And it may discover things along the way that cause it to investigate further.
That is no longer simply natural language to SQL translation.
The AI is planning and executing an analysis.
And that distinction becomes increasingly important as organizations move from simple conversational analytics toward agentic analytics.
The Hidden Consistency Problem
Suppose the AI does a fantastic job.
You get exactly the analysis you wanted. It identifies three poorly performing categories, finds an interesting relationship between promotional activity and regional performance, and recommends several actions.
Great.
Now run the exact same request again next month.
Will the AI follow exactly the same analytical methodology?
Not necessarily.
It might take a slightly different approach.
Perhaps the first analysis emphasized gross margin percentage, while the second gives more weight to absolute margin dollars.
Maybe the first compares product categories against the company average, while the second focuses on quarter-over-quarter deterioration.
One analysis might investigate geographic differences first. Another might start with product performance and then look for promotional correlations.
Neither approach is necessarily wrong.
That's the important part.
The underlying data could be perfectly governed. The definition of Gross Margin could be identical. Product Categories could be governed. Regions could be defined consistently.
Yet the reasoning path used to analyze the business problem could still change.
This suggests that trusted AI analytics actually has at least three different consistency problems.
- Data consistency: Did AI access the correct data?
- Semantic consistency: Did concepts such as Revenue, Customer, Gross Margin, Region, and Quarter mean the same thing?
- Reasoning consistency: Did AI follow a trusted and repeatable analytical methodology?
We've spent a great deal of time in analytics thinking about the first two.
Agentic AI is forcing us to start thinking seriously about the third. It is a close cousin of the problem we described in The Hidden Risk of Analytical Drift, except that here the drift happens in the method rather than in the meaning.
But Making AI Completely Deterministic Isn't the Answer
At first glance, the solution might seem obvious: simply force AI to follow exactly the same steps every time.
But that largely defeats the point.
AI's ability to dynamically reason through a problem is incredibly valuable.
You want it to explore.
You want it to recognize that an unexpected drop in gross margin warrants another analytical step.
You want it to investigate relationships you didn't explicitly program.
You want it to adapt when it discovers something interesting.
The goal shouldn't be to eliminate dynamic reasoning.
Instead, I think we need to make an important distinction:
Exploration should be dynamic. Operationalized analysis should be repeatable.
When someone asks AI a new question, let it reason. Let it determine how to approach the problem. Let it explore.
But once the organization looks at the result and says, "Yes, this is how we want to analyze this business problem," something should change.
At that point, the reasoning itself becomes valuable. It becomes an organizational asset.
From a Great AI Analysis to an Analytical Playbook
Imagine that the AI completes our product margin analysis and the business user likes the approach.
Instead of simply saving the final answer, what if we could save the analytical process that produced it?
Think of it as an analytical Playbook.
The Playbook could preserve the analytical steps, metrics, filters, queries, comparisons, ranking methodology, thresholds, recommendation criteria, and even how the final analysis should be presented.
And importantly, the Playbook shouldn't simply be a locked recording of what AI happened to do.
It should be editable.
Maybe Finance decides that an "underperforming" category should mean more than 10% below target rather than simply the bottom three categories.
Marketing might specify that a promotion should only be recommended if it has demonstrated success in at least two comparable regions.
An analyst might decide that gross margin percentage should carry more weight than absolute sales growth.
The presentation could also change. Perhaps the business wants a summary table first, followed by category-level recommendations and supporting charts.
Now something important has happened.
The AI-generated reasoning has been transformed into a reviewable, editable, reusable analytical artifact.
Instead of asking AI to reinvent the analytical methodology every time, the organization has captured a methodology it trusts.
Playbooks Could Become the AI-Era Equivalent of Dashboards or Standard Reports
There is an interesting parallel here with traditional business intelligence.
A dashboard gives us repeatability in what we see.
Behind the dashboard are predefined metrics, calculations, filters, relationships, groupings, and visualizations.
Every time we open it, we expect those rules to remain consistent.
An analytical Playbook extends that idea into AI reasoning.
A Playbook effectively says:
When we investigate this business problem, use these metrics, perform these analytical steps, make these comparisons, apply these rules, and produce these outputs.
The difference is that the result doesn't have to be a static dashboard.
The Playbook still uses AI.
It can generate explanations.
It can interpret what changed.
It can produce recommendations based on current conditions.
It can adapt the narrative to what it discovers.
The analytical methodology is repeatable without requiring every word of the answer to be identical.
That is a subtle but important distinction.
Repeatable reasoning does not mean identical answers.
If the underlying business conditions change, the answer should change.
What we want to make consistent is the methodology used to arrive at the answer.
From Repeatable Analysis to Repeatable Action
Once an analytical process becomes reusable, things get even more interesting.
We no longer need someone sitting in front of a chatbot typing the original question every month.
Our product-performance Playbook could simply run automatically on a scheduled interval.
At the end of every month, it could analyze product category performance, identify underperforming categories, diagnose likely drivers, examine successful promotions in comparable regions, and generate recommended corrective actions.
But why stop there?
The Playbook could become part of a broader agentic workflow.
Imagine:
Monthly performance review, then identify underperforming categories, then run the approved analytical Playbook, then generate category-specific recommendations, then distribute findings to the appropriate category leaders.
Perhaps the workflow goes further.
If a category drops below a predefined threshold for two consecutive months, the system could automatically trigger a deeper analysis.
If a recommended promotion requires approval, it could route the recommendation to the appropriate marketing leader.
Now we are moving from conversational analytics to something much more powerful:
Question → AI Reasoning → Validated Analysis → Playbook → Repeatable Execution → Action
This is where the intersection of analytics and agentic AI starts to become particularly interesting.
Consistency Shouldn't Mean Freezing the Analysis Forever
Of course, there is another side to this.
Business methodologies change.
Markets change. Products change. Metrics evolve. New data becomes available. And AI itself may discover better analytical approaches.
So a Playbook shouldn't become something carved in stone.
Instead, it should evolve deliberately.
Suppose an AI system identifies a new relationship that significantly improves the product-performance analysis. It could propose an additional analytical step.
An analyst reviews it.
The business agrees that the methodology is better.
The Playbook is updated.
You now have Version 2.
That creates a very different governance model from allowing AI to independently change its reasoning every time it executes.
The goal becomes governed evolution rather than uncontrolled variation.
That is actually very similar to how organizations already manage financial models, risk models, reports, business processes, and other important analytical assets.
AI reasoning shouldn't necessarily be different.
Analytical Reasoning May Become Enterprise Intellectual Property
There is an even bigger idea here.
Companies spend enormous amounts of time developing expertise around how they analyze their businesses.
- How does a retailer determine why stores are underperforming?
- How does a manufacturer diagnose production problems?
- How does a SaaS company investigate rising customer churn?
- How does Finance analyze margin deterioration?
- How does Marketing determine why a campaign worked in one market but failed in another?
The answers to those questions represent real organizational knowledge.
Today, that knowledge is scattered everywhere.
Some of it lives in dashboards and SQL.
Some lives in spreadsheets.
Some is embedded in operating procedures or consulting methodologies.
And a great deal of it lives in the heads of experienced employees.
As AI increasingly participates in analytical work, it can help codify some of that expertise.
But the valuable asset isn't simply the answer AI generated last Tuesday.
The valuable asset is the analytical methodology behind the answer.
Capturing successful reasoning turns something temporary, an AI interaction, into reusable organizational knowledge.
That may ultimately prove to be one of the most valuable outcomes of enterprise agentic AI.
Trusted AI Needs Governed Context, and Governed Reasoning
This is also why I believe the conversation around semantic layers needs to continue evolving.
A governed contextual semantic layer is critical. It can make sure that when AI asks for Gross Margin, Customer Revenue, Western Region, or Active Customer, it gets the right data based on the right business definitions.
At Codd AI, that is a major part of what we are building.
But as AI moves from answering questions to performing sophisticated multi-step analysis, trusted context alone isn't enough.
We also need a way to capture and reuse analytical reasoning.
That is the idea behind the concept of Playbooks in platforms like Codd AI: allow AI to dynamically reason through a complex analytical problem, then let users capture, inspect, refine, and reuse the resulting analytical process.
Those Playbooks can subsequently be run by users, scheduled, or incorporated into broader agentic workflows.
It creates a natural progression:
Governed Context → AI Reasoning → Governed Playbook → Agentic Execution
The contextual semantic layer helps make the inputs and business meaning repeatable.
Playbooks help make the analytical reasoning repeatable.
Agentic workflows can then make the resulting actions repeatable.
Maybe AI Should Be Allowed to Change Its Mind, Until We Decide It Shouldn't
Changing your mind isn't necessarily a bad thing.
Sometimes new information appears. Sometimes circumstances change. Sometimes someone simply discovers a better approach.
We want AI to have that flexibility when we're exploring a new problem.
But imagine walking into your monthly executive meeting and discovering that the AI changed the methodology behind your product-performance analysis because it decided to approach things differently this month.
That's considerably less charming.
The future of AI analytics therefore probably isn't a choice between the rigid predictability of traditional dashboards and the dynamic but potentially inconsistent reasoning of AI.
We need both.
Let AI explore dynamically.
Let it reason.
Let it discover new ways of analyzing the business.
But when it discovers an analytical process worth repeating, capture it, refine it, govern it, and turn it into a reusable Playbook.
Because trusted analytics has always depended on consistency.
In the AI era, that consistency can no longer stop with the data.
We need to start governing how AI reasons, too.
About Codd AI
At Codd AI, we believe enterprise AI should reason from a shared, trusted understanding of the business, not from isolated prompts or disconnected data sources. Our AI-powered contextual semantic layer combines technical metadata, business knowledge, governance, and human certification into a reusable enterprise foundation for conversational analytics, AI copilots, and agentic workflows.
By helping organizations capture and govern business context once, Codd AI enables every AI and BI application to deliver more consistent, explainable, and trusted insights, today and as the AI landscape continues to evolve. To learn more, visit www.codd.ai or schedule a conversation with one of our co-founders.