Picture this: your marketing manager has been using ChatGPT to draft client proposals for months, pasting contract details into the prompt window each time. Your finance analyst uses an AI tool to pull together board reports, and nobody's quite sure which version of the numbers is authoritative. Your customer service lead installed a browser-based AI assistant last quarter without telling IT. And your CEO just asked the leadership team to report back on your AI strategy.
That scenario isn't hypothetical. It's what "68% of small businesses use AI regularly" actually looks like from the inside.
The number sounds impressive. It's been cited in Goldman Sachs research, the U.S. Chamber of Commerce's 2025 Empowering Small Business report, and a 2026 analysis from Digital Applied that digs behind the headline. But the same research surfaces the number that actually matters: 77% of those businesses have no formal AI policy. No written guidelines on what data employees can share with AI tools. No approved tool list. No review process for AI-generated outputs. No designated owner for AI-related decisions.
For mid-market executives, that gap between adoption and governance is a liability that's currently misclassified as progress.
What "No Policy" Actually Looks Like in Practice
It's tempting to read "77% have no formal AI policy" and think: we're fine, everyone's in the same boat. But being in the majority doesn't reduce your exposure. It just means the consequences haven't arrived yet.
The failure modes of ungoverned AI use fall into four categories, and each one is currently active in most mid-market companies.
Inconsistent outputs. When every employee uses different tools, different prompts, and different levels of scrutiny, the quality of AI-assisted work varies wildly. One team's AI-generated report went through a rigorous review process. Another team's is essentially first-draft-as-final. From a leadership perspective, you can't tell which is which. Neither can your clients.
Untracked spend. The average small business spends around $2,400 per year on AI tool subscriptions, according to Digital Applied's 2026 analysis. That figure is almost certainly an undercount. It doesn't capture the shadow subscriptions: the tools employees are expensing, the free tiers they're using personal accounts for, or the API costs that appear quietly in someone's department budget. Without an inventory, you don't know what you're spending or what you're getting for it.
Data exposure. IBM's 2025 Cost of a Data Breach Report found that one in five organizations reported a breach due to shadow AI — AI tools employees used without organizational knowledge or approval. Breaches involving high levels of shadow AI added an average of $670,000 to breach costs. They were also more likely to result in compromise of personally identifiable information (65% of cases) and intellectual property (40%). Detection and containment ran about a week longer than the global average, adding another $200,000 in cost on top of the premium. The U.S. average cost of a data breach has hit $10.22 million, an all-time regional record driven by steeper regulatory fines.
Companies that let shadow AI proliferate aren't taking on a theoretical risk. They're statistically likely to absorb hundreds of thousands of dollars in avoidable breach costs. And 63% of breached organizations in IBM's study either had no AI governance policy or were still developing one when the breach occurred.
High-stakes decisions made with unvetted tools. When an employee uses an AI tool to summarize a contract, flag a supplier risk, or draft a client-facing recommendation, they're often not thinking of it as a "decision." They're thinking of it as getting their work done faster. But the output influences real decisions, and if the tool hallucinated a fact or drew from outdated training data, you have no audit trail and no accountability structure. That's the highest-stakes failure mode — and the one that shows up in the incident report you weren't expecting.
The Data Is Worse Than You Think
The 68%/77% split from the Digital Applied survey tells a clear story at the SMB level, but mid-market firms aren't doing much better.
The RSM Middle Market AI Survey, conducted in early 2025 across 966 decision-makers at mid-market firms in the U.S. and Canada, found that 91% of those companies were using generative AI. Of those, only 37% described their AI approach as "well-formulated." The majority of mid-market AI users — companies with tens of millions to billions in revenue — are operating on instinct rather than strategy.
Organizations that deployed AI without measurement couldn't demonstrate value, couldn't justify investment, and couldn't scale what was working. The RSM data captures exactly that dynamic: high adoption, thin strategy, widening exposure.
Adoption is outpacing oversight at a structural level, and the gap is widening as AI tools become cheaper and easier to access.
Why This Is a Competitive Problem, Not Just a Compliance One
Most conversations about AI governance frame it as risk management: something you do to avoid getting into trouble. That framing misses the bigger cost — the performance gap between governed and ungoverned AI users.
Deloitte's 2025 AI ROI research identified what sets AI leaders apart from the rest. The firms achieving the strongest financial returns from AI — classified as "AI ROI Leaders" — shared specific characteristics. One that stands out: they built governance frameworks early, treating AI as enterprise transformation rather than a collection of productivity hacks, with governance as the infrastructure that made scaling possible.
McKinsey's 2025 State of AI survey reinforces this. Organizations that tracked well-defined KPIs for AI solutions showed the strongest correlation with bottom-line EBIT impact. Not just better AI outcomes: better business outcomes, full stop.
If most competitors are ungoverned AI users and you build governance infrastructure now, you gain a real advantage. Your AI outputs are more reliable. Your data is better protected. Your spend is tracked and optimized. And when regulation inevitably tightens — Stanford's AI Index documented 131 state-level AI laws passed in 2024 alone, more than double the 49 from 2023 — you're not scrambling to catch up.
Companies that treat governance as a compliance burden will implement it under pressure and pay twice for the privilege. The ones building it now will be setting the pace in 2027.
Where to Start: The Minimum Viable AI Policy
The word "policy" makes executives think of thick compliance manuals that live in SharePoint and nobody reads. Forget that version. A minimum viable AI policy for a mid-market company is a document your leadership team can read in 15 minutes and your employees can understand in five.
Based on practical frameworks from governance practitioners and aligned with NIST's AI Risk Management Framework, here's what a first-version policy needs:
1. An approved tool inventory. List the AI tools your company sanctions for use. This doesn't have to be exhaustive on day one — start with the tools you already know people are using. The goal is to draw a clear line between "tools we know about and accept" and "tools employees are using without our knowledge." That single step begins to surface shadow AI.
2. Data access boundaries. Define what data employees can and cannot input into AI tools. The rules here should be binary, not layered with exceptions: no customer PII into third-party AI tools without explicit data processing agreements, no confidential financial data into consumer AI products, no proprietary IP into unsanctioned tools. Clear enough that an employee can apply them without asking someone.
A lot of companies stall here trying to write a comprehensive data classification policy instead of a simple list of prohibitions. Start with prohibitions. You can build sophistication later, but the prohibitions are what prevent the breach that costs you $670,000.
3. Human review requirements. Specify which AI outputs require human review before use. Customer-facing communications, financial projections, legal document summaries, and any AI-generated content used in client deliverables should all require a sign-off step. This creates quality standards and builds an audit trail simultaneously.
4. A designated owner. At a mid-market company, this is usually the COO or IT director. They don't need AI expertise. They need accountability: maintaining the policy, fielding employee questions, running a quarterly review.
5. A vendor evaluation checklist. Before any new AI tool gets approved, run it through three questions: Where does user data go? Is there a data processing agreement? What are the retention policies? This doesn't require legal expertise. It requires a consistent process applied consistently.
Governance advisor Di Tran's framework for small and mid-sized organizations makes a useful point: a minimum viable governance model should deliver 80% of the risk mitigation value through 20% of the administrative effort. A governance document that's actually used beats a comprehensive one that lives in SharePoint. Defensible and consistently applied is the standard, not exhaustive.
Which Departments Need Guardrails First
Not all AI use carries the same risk profile. Two departments sit at the top of the priority list — and for meaningfully different reasons.
Customer Service
Customer-facing AI use is where errors become public fastest. Whether it's an AI chatbot handling support requests or a rep using AI to draft customer responses, the risks are specific: incorrect information delivered with confident AI-generated authority, customer PII processed through unsanctioned tools, and tone inconsistencies that erode brand trust.
The practical starting point is a two-layer rule: all AI-generated customer communications go through a human before they're sent, and the tools used in customer service are restricted to an approved list with documented data handling practices. Optimize after you've got that baseline working.
Finance
Finance is where AI hallucinations get expensive fast. An AI tool that generates a financial summary with plausible-sounding but incorrect numbers can influence board decisions, investor reporting, or lending relationships. The error rate of even well-regarded AI tools on numerical reasoning tasks is non-trivial, and the downstream consequences of a wrong number in the right report are significant.
Finance AI guardrails come down to three requirements: prohibit consumer AI tools for anything involving sensitive financial data, require that AI-generated financial analysis be reconciled against source data by a human before it circulates, and log which AI tools contributed to which outputs. That last point becomes critical when an auditor or board member asks how a number was derived — and they will ask.
Marketing and operations typically carry a lower risk profile. Hallucinated copy is embarrassing, not catastrophic. But even those departments benefit from the approved tool inventory and data boundary rules. Consistency prevents the slow accumulation of shadow AI tools that nobody's tracking, which is how the customer service and finance problems get seeded in the first place.
Building Measurement In From Day One
Most companies write a policy and stop there. Measurement is where the policy pays off — and where most mid-market firms skip a step.
McKinsey's research makes this concrete: the single management practice most correlated with bottom-line EBIT impact from AI is tracking well-defined KPIs for AI solutions. Not having a policy. Not having a governance committee. Measuring the right things, from the start.
For mid-market executives, this means defining what success looks like for each AI use case before deploying it, not after. Three questions to answer upfront:
What does this AI use case replace or augment? Be specific. "We're using AI for customer service responses" is too vague. "We're using AI to draft first responses to tier-1 support tickets, reducing average handle time from 12 minutes to 4" is measurable.
What's the baseline? What's the current state you're comparing against? Time, cost, error rate, customer satisfaction score — whatever fits the use case. You need a before to measure an after.
Who reviews the results and when? Assign a cadence. Quarterly is realistic for most mid-market teams. If the numbers don't support the use case, adjust or deprioritize it and reallocate budget to what's working.
This discipline shifts how the organization thinks about AI. "Tools we use" is a different posture from "investments we manage," and the financial outcomes reflect that difference. Companies that measure AI like an investment stop paying for subscriptions they can't justify. That's how AI spend gets smarter over time rather than just bigger.
The Window Is Narrow
The regulatory clock is moving faster than most executives expect. Stanford's AI Index documented a 21.3% increase in legislative AI mentions across 75 countries in 2023-2024. U.S. federal agencies introduced 59 AI-related regulations in 2024, more than double the prior year. The EU AI Act's first obligations are already in effect.
Mid-market companies that build governance frameworks now, while it's still a choice, will have a structural advantage over those forced to build them under regulatory pressure. The difference shows up in control, not just cost. A framework you build under regulatory pressure is written for the regulator. One you build now is written for your business.
The companies that will look sharpest in 2027 are making unglamorous decisions today: inventorying their AI tools, writing a five-page policy, designating an owner, defining KPIs before deploying. None of it looks impressive in a board deck. But it's the operational foundation that determines whether your AI spend turns into an advantage or a liability.
The companies that move on this in the next six months will be setting AI policy from a position of strength when federal regulation firms up. The ones that wait will be writing it in response to something they'd rather not explain.