Chapter Six

Democratization of IT and the Accountability Problem

^{Democratization of IT and The Accountability Problem}

AI democratizes creation. Business users can design workflows, generate code, build copilots, automate analysis and orchestrate agents with far less dependence on traditional IT delivery.

This can unlock enormous productivity, but it also shifts risk into parts of the organization that may not be trained to manage technology, data and control obligations.

The future role of IT is therefore not to be the sole builder of every solution. It is to provide platforms, guardrails, reusable patterns, observability, and governance that allow the business to build safely.

This is a profound identity shift: from bottleneck to infrastructure steward, from gatekeeper to capability platform, from project delivery function to enterprise control plane.

In your experience, what’s your largest AI cost category?

Model calls
Integration & Governance
Human Review
Don't know

_{For decades we governed applications. In the AI era we must govern decisions. If an autonomous workflow cannot answer who decided, why it decided, and who is accountable for the outcome, it is not ready to scale."}

Sumita Dutta, Global IT and Digital Head & CIO, Shell Downstream, Customer Operations, Shell

The AI-Ready Infrastructure Maturity Curve

Object to Govern	Minimum Control	Human Accountability
AI Agent	Identity, access, purpose, allowed tools, budget envelope, evaluation criteria, kill switch	Named agent manager and business owner
AI-Enabled Workflow	Process map, input/output classification, exception handling, audit trail, production approval	Product owner plus accountable executive for outcomes
Prompt/Instruction Set	Version control, testing, red-team review for high-risk uses, change history	Technical owner and domain approver
Data Product	Lineage, quality, sensitivity classification, retention rules, model-use permissions	Data owner and platform steward
Saas AI Feature	Usage telemetry, contract transparency, data-processing terms, chargeback logic	Application owner and procurement/risk lead

Regulation Reinforces the Model: Mapping to the EU AI Act

For enterprises operating in or serving the European market, this accountability model is not only good practice; it increasingly anticipates regulatory obligation

The EU AI Act uses a risk-tiered approach:

Prohibited uses are banned, high-risk systems are subject to obligations around risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy, robustness and cybersecurity, and lower-risk uses carry lighter transparency duties.

Implementation is phased:

Prohibited-practice and AI-literacy provisions began applying in 2025, general-purpose AI and governance provisions apply from 2025, transparency obligations apply from 2026.

High-risk obligations are subject to later application dates, including 2 December 2027 for certain stand-alone high-risk systems and 2 August 2028 for high-risk AI embedded in regulated products, following the current Commission guidance on the updated timeline.

The Act’s requirements for traceability, human oversight and record-keeping are, in substance, the same disciplines as named ownership, kill switches, audit trails and evaluation pipelines. [8]

The practical implication for executives is that an enterprise which builds the accountability model described here is also building much of the evidence base a regulator will expect.

Conversely, an AI estate that cannot identify owners, scopes, data classifications and decision logs will struggle to demonstrate compliance under the Act and comparable regimes emerging elsewhere.

Sovereignty, data residency and the ability to audit and explain AI-driven decisions therefore become design constraints to be engineered from the start, not retrofitted under audit pressure.

Governance as a Head Start, Not a Tax

Treating the accountability model as a compliance burden is the wrong frame. The same telemetry, ownership and evaluation that satisfy a regulator are what make autonomous capability safe to scale at all.

Enterprises that govern agents well will deploy them faster and more widely than those that do not, because they will be able to prove to a board, an auditor or a regulator that the risk is contained.

The 3am Question

Every AI-enabled workflow should be able to answer one question: if the agent makes a consequential error at 3 a.m., who is accountable, who is alerted, what evidence exists, what can be stopped, and how is the customer, regulator or employee protected?

This is where many AI programs remain immature. They can measure hours saved, but struggle to translate those hours into accountable value: fewer handoffs, faster cycle time, improved quality, higher revenue conversion, risk reduction, better customer experience, or strategic redeployment of scarce experts.

_{^{AI-enabled workflow will not only have to map the happy paths, but also what happens when things go wrong; how will this be communicated to the customer and internally to process and agent owners? Will there be agents to fix agents, and what if those agents fail? Design of suitable guardrails is a starting point, but some form of “exception handling” will need consideration, whether it's ‘Human in the Loop’ or another AI-enabled solution.}}

Steve Crombie, CIO Bulgaria, ING Bank

_{^{Organizations have spent decades designing governance for people. We now need an equivalent governance model for digital labor.}}

Sumita Dutta, Global IT and Digital Head & CIO, Shell Downstream, Customer Operations, Shell

A Practical Accountability Pattern

Assign a named human owner for every agent, AI-enabled workflow and high-risk automation

Instrument the workflow for audit, evaluation, exception review and incident response

Classify data inputs and outputs, including whether personal, regulated, confidential or commercially sensitive data is involved

Define the scope of action – what the agent may observe, decide, recommend, or execute

Review value and risk on a recurring cadence, then scale, redesign or retire

Set budget envelopes and usage thresholds before production deployment

Getting There from Here: A Transition Path, Not a Leap

The accountability model above describes a Stage 4 end-state. Most enterprises today sit at Stage 2 to 3, with shadow AI in use and no agent registry.

The honest question is therefore not what perfect governance looks like, but how an organization moves from where it is to where it needs to be without either freezing experimentation or losing control.

The following sequence is deliberately incremental: each step is achievable in weeks, creates value on its own, and earns the right to the next:

Discover before you control: Survey what AI tools and agents are already in use, by whom and on what data. The goal is visibility, not enforcement; shadow AI is the signal of demand you will later channel, not the enemy.
Stand up a lightweight registry: Begin with a simple list of known agents and AI workflows with an owner and a purpose for each. A spreadsheet that is maintained beats a sophisticated platform that no one updates.
Govern by risk tier, not uniformly: Apply the full control set – evaluation, audit logging, kill switch, budget envelope and only to high-risk and production agents first. Low-risk experimentation stays fast inside a sandbox; this is what stops governance from becoming the new bottleneck.
Add cost and evaluation telemetry to what you already run: Instrument the handful of agents that matter most before instrumenting everything. Visibility on the critical few buys more control than partial visibility on all.
Industrialize only what proves out: Promote the registry, evaluation and cost controls into platform capability once the patterns are stable, so the governance plumbing is built around real use rather than anticipated use.

This resolves the central tension in governing AI. Shadow AI emerges when official channels are too slow, so the cure cannot be more process applied uniformly.

Risk-tiered, incremental governance keeps low-risk experimentation fast while reserving heavier control for where the consequences are real, making the agent-ready operating model reachable rather than aspirational.

_{^{The winning strategy is not centralization or decentralization. It is federated accountability supported by common standards and platforms.}}

Sumita Dutta, Global IT and Digital Head & CIO, Shell Downstream, Customer Operations, Shell