AI Was Supposed to Improve Margins. Why Is It Showing Up in Cost of Revenue?

Somewhere in a boardroom, an executive announces that AI is going to improve margins.

Everyone nods.

The product team has a demo. Sales has a slide about differentiation. Marketing has already written the phrase “transformational efficiency” in a font large enough to require its own budget.

Finance has a question.

“How much does it cost every time a customer clicks the shiny new button?”

Silence.

Then someone says the cost is “basically negligible.”

That’s usually when I reach for a calculator.

I like AI. I like automation. I especially like the possibility of removing work that adds no value except proving that we know how to suffer in Excel. But I am increasingly interested in a less glamorous question: what happens to the economics of the business when AI moves from an internal experiment into the product customers actually use?

Because the minute we start paying for compute, inference, model calls, retrieval, monitoring, human review and all the other things that make the feature work, the margin story becomes more complicated.

AI can absolutely improve productivity. It can also create a new variable cost. Both can be true at the same time, which is unfortunate for anyone hoping the business case would fit on one enthusiastic slide.

The Demo Was Free. The Customer Isn’t.

An AI demo is a wonderful thing. Someone asks a question, the model produces an answer, and the room reacts as though we’ve discovered electricity for the second time.

The demo is also an unusually friendly environment for a financial forecast. There are few users. Nobody is trying to break the system. The sample questions are carefully selected. The usage bill is small enough to hide behind the coffee budget.

Then the feature launches.

Customers ask longer questions. They upload larger documents. They repeat prompts when they dislike the first answer. A few customers discover that the system will happily process their entire digital filing cabinet if given the opportunity.

Product usage is growing. The dashboard is green. The sales team is thrilled.

Meanwhile, the infrastructure bill has developed a personality.

I’d want to know the cost per successful customer outcome, not just the cost per model call. A single useful answer might require multiple model calls, retrieval, validation, retries and support. If the feature sometimes fails, the unsuccessful attempts cost money too.

That isn’t an argument against the product. It’s an argument for knowing what we built before we declare victory.

Gross Margin Doesn’t Care How Impressive the Technology Is

Gross margin has a stubborn habit of ignoring our excitement.

If an AI capability is part of delivering a paid product or service, some associated costs may belong in cost of revenue, depending on the company’s accounting policies, contractual arrangements and the nature of the service. I wouldn’t assume every AI invoice is automatically a cost-of-revenue item. An internal finance assistant and a customer-facing AI product are different economic animals.

But wherever the costs are classified, they still need to be forecast. And if usage-related costs are required to deliver the product, gross-margin analysis needs to reflect that reality.

Imagine a software company with $20 million in annual revenue and $4 million in cost of revenue. That’s an 80% gross margin.

Now it launches a popular AI feature. Revenue increases by $2 million, but AI-related delivery costs add $1.4 million and other delivery costs stay unchanged. Revenue becomes $22 million. Cost of revenue becomes $5.4 million. Gross profit rises from $16 million to $16.6 million.

Wonderful. Gross profit is up.

But gross margin falls from 80% to approximately 75.5%.

I can already hear the meeting. “But the feature is growing!” Yes. “And customers love it!” Excellent. “Then why is margin down?” Because customers are using the thing we encouraged them to use.

Growth and margin are not enemies. They just need to be introduced properly before the forecast meeting.

The $1 Million Feature That Eats Its Own Lunch

Let’s make the economics more concrete with a hypothetical AI feature.

Suppose 10,000 customers pay an additional $100 a year for it. That’s $1 million in incremental annual revenue.

Each customer averages 200 AI interactions per month, or 2.4 million interactions a year across the customer base. Assume the average direct model and orchestration cost is $0.18 per interaction.

That produces $432,000 in annual usage-related cost. Add $120,000 for additional infrastructure, monitoring and support directly associated with the service. The feature generates $448,000 in incremental gross profit, or a 44.8% incremental gross margin, before considering other potential delivery costs.

That might be a good investment. It might not. The answer depends on the company’s pricing, retention economics, strategic objectives and margin expectations.

Now imagine engagement doubles, but the price stays fixed. The $432,000 interaction cost becomes $864,000. Add the same $120,000 of other delivery costs and incremental gross profit falls to $16,000.

Customers are twice as engaged. The product manager is celebrating. The finance team is quietly checking whether the company has accidentally invented a very popular nonprofit.

The lesson isn’t to discourage usage. It’s to build a pricing and cost structure that can survive success.

Averages Are Where Bad Unit Economics Go to Hide

One average cost per AI interaction is a useful starting point. It is not a business model.

Some customers ask short questions. Others submit long documents and request detailed analysis. Some workflows need a cheap model. Others require a more expensive model, additional retrieval or human review. Some customers barely use the feature after onboarding. Others use it as though their personal mission is to stress-test the vendor’s cloud contract.

If we price everyone the same and model only the average, the heavy users may consume a disproportionate share of the gross profit.

I’d ask for a distribution, not just a mean: cost per customer, per workflow, per successful outcome, and by usage cohort. What does the median customer cost? What about the top 10%? What about the customer whose usage made the engineering team send a Slack message consisting entirely of question marks?

I’d also separate fixed and variable costs. Some platform commitments won’t move with usage in the short term. Others will. Discounts, minimum commitments and volume tiers can make the curve nonlinear.

That’s why I’d rather see a contribution margin by product and customer segment than one cheerful average cost pasted into every row of the plan.

The Pricing Question Nobody Wants to Ask at Launch

Product teams understandably want adoption. Sales understandably wants fewer objections. Customers understandably want useful features without a pricing document that requires legal counsel.

All reasonable preferences.

But if the feature’s cost scales with usage and the revenue doesn’t, somebody has to ask how we get paid.

Maybe the answer is a higher subscription price. Maybe it’s usage tiers, included allowances, overage charges or a premium package. Maybe the feature improves retention enough that the economics work even without direct monetization.

I wouldn’t prescribe one model for every product. I would insist that we model the trade-offs.

A usage cap can protect margin but frustrate customers. Unlimited usage can simplify selling but expose the business to unpredictable costs. A higher price may recover costs but reduce conversion. Bundling can increase perceived value while hiding the economics of individual features.

Finance’s role is not to arrive with a giant red stamp that says NO. It’s to show the product and commercial teams what each choice means for revenue, usage, retention, margin and cash.

And perhaps gently suggest that “unlimited” is a pricing strategy, not a mathematical law.

Productivity Gains Aren’t Automatically Cost Savings

Now let’s move inside the company, where AI is supposed to save time.

Suppose 100 employees each save four hours a week using AI. At 48 working weeks, that’s 19,200 hours of capacity a year.

Impressive.

If we multiply those hours by a fully loaded labor rate of $60, we get a theoretical value of $1.152 million. I understand why that number looks lovely on an ROI slide.

But did payroll decline by $1.152 million? Probably not.

This is the same distinction I keep making when building a driver-based budget instead of rolling forward last year’s numbers: assumptions need to describe what changes operationally. If the employees remain employed at the same compensation, the organization has created capacity. That may be tremendously valuable. It could improve service, reduce backlogs, support growth, or prevent future hiring. But those outcomes need their own assumptions.

I’d separate realized cash savings, avoided future costs, increased capacity and improved quality. Then I’d ask which benefits actually show up in the financial statements, which show up in operational metrics and when.

The hours-saved calculation isn’t wrong. Calling every saved hour a cash saving is where the math puts on a little costume and hopes nobody notices.

There’s also a cost to adopting the tool: licenses, integration, training, controls, oversight and the work of changing processes. Those don’t disappear because the vendor’s demonstration included a smiling robot.

The Model Can Get Cheaper While the Bill Gets Bigger

Here’s an AI cost dynamic I think FP&A teams need to understand: lower unit costs do not guarantee lower total spending.

Suppose model optimization reduces the cost per interaction from $0.20 to $0.12, a 40% reduction. Excellent engineering work.

Now suppose usage grows from one million to three million interactions. The model bill goes from $200,000 to $360,000.

The unit economics improved. The total expense rose 80%.

Both statements are true.

This is not necessarily bad news. If revenue, retention or customer value increased enough, the higher bill may be entirely justified. But I’d want a bridge that separates the rate effect from the volume effect.

At the original volume, the new rate would save $80,000. The additional two million interactions at $0.12 add $240,000. Net expense increases $160,000.

That’s a much more useful conversation than “AI costs increased despite optimization,” which makes Engineering sound incompetent when it may have done exactly what we asked.

Finance should be able to explain why the cost moved, not just announce that it moved.

Forecasting AI Costs Without Pretending We Know the Future

I wouldn’t build an AI forecast around one heroic assumption called “adoption.” That is a lovely way to make uncertainty look organized.

I’d break the forecast into drivers. How many customers or employees can use the feature? What percentage will activate it? How frequently will active users engage? How many calls or processing steps are required per useful outcome? What’s the cost per unit? How much retry, retrieval, monitoring and support overhead do we expect?

Then I’d separate the factors we control from the ones we don’t. Product design, routing and pricing are partly controllable. Customer behavior, vendor prices and model requirements can change faster than the annual budget cycle.

I’d use scenarios that describe actual behavior, not just a neat plus-or-minus 10% around the spreadsheet.

In one case, adoption is modest and usage is light. In another, adoption is strong and customers use the feature intensively. In a third, a new model or design change improves quality but raises the cost per outcome.

Each scenario should show revenue, cost of revenue, gross profit, margin, cash implications and the decision we’d make if it occurred.

And yes, I’d reforecast. Frequently. A twelve-month-old usage assumption for a fast-changing product is less a forecast and more a historical artifact with a nice filename.

What I’d Put on the AI Margin Dashboard

I’d keep the management dashboard brutally practical.

First, revenue and gross profit attributable to the AI-enabled product or feature, with the limitations of attribution stated clearly. Second, total AI delivery cost and the cost per successful outcome. Third, usage volume, frequency and distribution by customer segment. Fourth, the share of usage that falls into expensive workflows. Fifth, model efficiency and quality together, because making the answer cheaper by making it useless is not a victory.

I’d also show a margin bridge: volume, pricing, model cost, workflow mix, infrastructure, support and any material fixed commitments.

Then I’d add the business measures. Retention. Conversion. Customer satisfaction. Support burden. Adoption. Revenue per account. The point is to see whether higher AI expense is purchasing something valuable.

One metric I’d be careful with is cost per token or per call. It matters operationally, but it’s not the whole customer experience. A cheaper call that needs three retries may cost more than one expensive call that works.

I want a dashboard that helps us make decisions. Not a museum of technical metrics that Finance is expected to admire from a respectful distance.

The Conversation I’d Have Before Approving Another AI Initiative

I wouldn’t start with “What’s the ROI?” because that question can produce a remarkably creative answer when the project already has executive sponsorship.

I’d start with: What problem are we solving? Who benefits? What behavior changes? How does that change revenue, cost, capacity, quality or risk? What must be true for the economics to work?

Then I’d ask about scale. What does this cost with 100 users? Ten thousand? A million? What happens if usage is twice our estimate? What happens if customers love the feature but refuse to pay extra?

I’d want the product owner to explain the operating assumptions and Finance to translate those assumptions into economics. Engineering should own the technical drivers it understands. Sales should challenge the pricing assumptions. Nobody should get to hand Finance a number and disappear until the budget is approved.

And I would want a stopping rule. At what point do we change the model, adjust pricing, redesign the workflow or reconsider the investment?

That’s not negativity. That’s management.

An initiative without a measurable outcome, a cost model and a decision threshold isn’t a strategy. It’s a hope with a purchase order.

Accounting Classification Matters, but Economics Come First

I also don’t want the discussion to get trapped in the general ledger.

Whether a particular AI cost is classified in cost of revenue, research and development, sales and marketing or general and administrative expense depends on what the cost supports and the applicable accounting framework and company policy. Finance should work with controllership to classify it consistently and correctly.

But a classification decision does not change the underlying cash economics.

If a customer-facing AI service costs $500,000 more to deliver, the company needs to understand that cost even if two departments disagree about the account code. The income-statement presentation matters for reporting and margin comparisons. The operating decision still requires a clear view of incremental cost and value.

I’d maintain both perspectives: the accounting view for accurate reporting and the unit-economics view for better product decisions.

Otherwise we can spend three meetings arguing about where the expense belongs while nobody asks whether the feature makes money. I have attended enough meetings to know that’s a very real possibility.

AI Can Improve Margins. It Just Has to Earn Them.

I am not bearish on AI. I’m bearish on business cases that treat the benefits as guaranteed and the costs as somebody else’s problem.

AI can lower service costs, reduce manual work, improve customer outcomes and create products that weren’t economical before. Those are real possibilities worth testing.

But the margin improvement has to survive contact with actual usage.

I’d want a clear distinction between internal productivity and customer-facing delivery costs. I’d want pricing that reflects how the product is consumed. I’d want a driver-based forecast, a realistic adoption range and a gross-margin bridge that explains what changed.

And I’d want management to celebrate the right thing. Not just that people used the feature. Not just that the model got cheaper. Not just that the demo made everyone gasp.

Celebrate when customers get value and the company understands what it costs to provide it.

Until then, the AI feature may be revolutionary.

The invoice is still very traditional.