Your AI Is Making Decisions: Who Is Checking Its Work?

Introducing the AI Watchdog: An independent check before the business acts

Imagine an AI system has helped your team schedule production for months. Today it recommends moving a major order to another plant. The capacity figures are current. The calculations look convincing. An automated workflow is ready to release the change.

But the analysis has overlooked a plant-specific restriction. Nothing crashes. The recommendation arrives on time and looks like every other answer the team has accepted.

Who would notice before the order moves?

That is the role proposed for an AI Watchdog (AW): a separately controlled checking system that uses an alternative AI model to assess the primary AI engine’s analysis, outputs, and observable behavior. Its purpose is to detect reasons to question reliance, whether caused by error, degraded performance, or possible compromise—not simply ask the same AI to check itself.

A lesson from the flight deck

Aviation offers a useful analogy. GPS can provide precise position and ground-track information. Gyro-based instruments provide heading or, within an inertial navigation system, additional motion and position estimates. A magnetic compass offers another heading reference. These use different physical principles, providing opportunities to notice an inconsistency.[1]

They are not three interchangeable readings: heading is where the aircraft points; track is where it travels over the ground. Wind, magnetic reference differences, instrument errors, and inertial drift matter. A compass cannot independently confirm a GPS position. The lesson is cross-checking with understood limitations, not taking a majority vote.[1]

Independence also matters. The FAA notes that self-contained inertial systems do not depend on GPS, whereas hybrid systems can inherit errors through GPS updates.[1]

For AI, another opinion is most valuable when it can reveal a failure the primary system might miss.

Know what the answer is built on

The Information Bill of Material (IBOM) records what information a decision requires and what actually supports the output, including sources, transformations, limitations, and checks. Comparing the Required and As-Used IBOMs exposes gaps before reliance.[2]

Check the analysis, not just the answer

A Watchdog should therefore be defined by its assurance objective, not simply by being a second model. Before deployment, specify which failure modes it is expected to detect, what evidence it can independently obtain, what it cannot observe, and what action follows a failed or unavailable check.

The watchdog needs this independent evidence for the validation of critical elements of AI output. It might independently recompute capacity, check a restriction, or compare the recommendation with an approved operating limit. It should form its assessment before or concurrently with assessing the validity of the primary conclusion where practical. Independent calculations and rule checks should, for critical outputs, supplement the AI model’s primary mission analysis.

An AW adds evidence about integrity, processing reliability, output verification, independent corroboration, and continuing performance. It does not replace checks of provenance, information quality, task competence, consistency and robustness, or uncertainty.[3]

The Watchdog itself becomes part of the assurance boundary. Its model, prompts, tools, data sources, configuration, access rights, update path, logging, and release controls should be governed and testable. A poorly controlled Watchdog can create false reassurance rather than independent assurance.

Output agreement alone cannot certify that a model’s internal operation is uncompromised. Checks of model identity, configuration changes, and tool activity still matter.

Make independence real

Three design choices deserve attention:

Independence should be treated as a property to be demonstrated. Different model providers can still share training patterns, retrieval sources, software dependencies, cloud infrastructure, or common weaknesses. The relevant test is whether the Watchdog provides meaningful additional coverage against the failure modes that matter for the decision.

Different models and methods. Use an alternative model suited to the checking task, not merely another session of the primary model. Test whether it actually catches different errors.
Separate compute and control. Use an alternative compute platform with separately managed infrastructure, credentials, administration, and update paths. Select different hosting, software, or hardware where they reduce shared failure or compromise risks.
Independent evidence where needed. Keep the same decision requirements, but verify critical facts through independently obtained evidence. Two models reading one corrupted record can agree for the wrong reason.

Diversity is an established resilience technique, but shared software and infrastructure can undermine it.[4] Research also finds correlated errors across different language-model providers.[5]

Different brands are a starting point, not proof of independence

Let agreement earn trust—not grant permission

When capable, sufficiently independent watchdogs concur within defined tolerances and the required evidence is present, their findings can strengthen trust in that output. Feed these findings into the trust score, with their scope and limitations. Verified outcomes over time can support broader confidence in the system.

But concurrence is evidence, not proof. Compare material facts, calculations, constraints, and proposed actions—not identical wording. Record what was checked and what remains uncertain. No favorable average should override a missing critical input, failed security check, or required approval.

A useful message might be:

Capacity calculations independently confirmed.
Plant restriction unresolved.
Production transfer not authorized.

`People and downstream systems need that status, not just a reassuring score. Unavailable watchdog checks must remain visible—not be counted as agreement.

IBOM APPLIES TO EVERY INFORMATION SOURCE, NOT ONLY AI

Although AI has drawn attention to Information Trust, the IBOM is not an AI construct. The vast majority of enterprise systems consume information to perform their intended function: a control system reads instruments, a billing system reads meters, a logistics platform reads location data, and a trading system reads market feeds. Every one of these inputs is an IBOM component, and every one can be wrong, substituted, degraded, or manipulated.

Consider a pressure sensor feeding an industrial control loop. The controller may open a relief valve, stop a pump, or shut down a process based on that reading. Nothing about the sensor is intelligent, yet the action it drives may carry safety, environmental, and financial consequences greater than those of most AI-generated reports. For that reading to be trusted, the IBOM should establish four things:

Defined source.
The Required IBOM designates the specific device authorized to supply the reading: its location, type, measurement range, tolerance, and calibration requirements.

Verified identity.
The enterprise can confirm that the data actually originates from that designated device, and not from a substituted, cloned, spoofed, or misconfigured one.

Continuous monitoring.
The device’s health, calibration status, and behavior are monitored over time, including drift, stuck or frozen values, and readings that are physically implausible or inconsistent with redundant instruments.

Validated stream integrity.
The data stream is tested and validated from device to decision: messages are authentic and unaltered, timestamps and sequence are consistent, and gaps, replays, or injected values are detected.

These concerns are not theoretical. Technical analyses of Stuxnet describe malware that recorded normal sensor values from industrial controllers and then replayed them to operators while the equipment was being driven outside its intended operating range, so that everything on the operator’s screen appeared to be in order.6 The operators were relying on data that looked authoritative, but its source and currency had been compromised. That is precisely the gap that verified source identity and validated stream integrity are intended to expose.

The same discipline applies to laboratory instruments, meters, cameras, GPS receivers, market data feeds, and third-party APIs. In operational technology environments, it complements established industrial-security practice, such as the ISA/IEC 62443 series of standards,7 by connecting device and network controls to the decisions the data ultimately supports. Seen this way, AI is one more consumer and producer of information whose trustworthiness must be demonstrated, not the reason the discipline exists.

COMPARTMENTALIZATION PROTECTS DECISION INTEGRITY

Compartmentalization is commonly associated with protecting confidential information from unauthorized access. For Information Trust, its role is broader: it prevents lower-trust, restricted, unrelated, or improperly authorized information from contaminating a higher-consequence output.

A system should not be allowed to use every available source merely because access is technically possible. Public commentary should not silently shape a regulated determination. Customer information approved for service delivery should not automatically become training material. Preliminary analysis should not flow into a published conclusion without review. A system permitted to explore options should not automatically receive authority to execute a transaction.

An IBOM makes five boundaries visible: source boundaries define approved information domains; processing boundaries separate exploratory, production, regulated, and high-impact uses; access boundaries enforce least privilege; decision boundaries define what an output may inform, recommend, approve, or execute; and distribution boundaries control publication, sharing, retention, reuse, and downstream ingestion.

Compartmentalization is also a matter of purpose control: information should not become eligible for a new use merely because it is technically accessible. The IBOM can make the intended purpose, permitted downstream uses, and boundaries of reuse explicit, helping distinguish availability from authorization.

Compartmentalization is not intended to prevent useful information combination. It makes combination deliberate. The more consequential the decision, the more precisely the enterprise should define which information may enter, how it may be transformed, and where the resulting output may go.

AI MAKES INFORMATION TRUST MORE URGENT

AI is one system type to which IBOM discipline should be applied; it is not part of the core definition. Its importance arises from scale and opacity. AI can retrieve thousands of sources, merge information, infer relationships, generate language, and pass outputs into other workflows. These capabilities create value, but can hide where a claim originated, whether a source existed, whether rights were respected, or whether a conclusion was generated rather than established.

NIST’s Generative AI Profile treats generative-AI risk management as a lifecycle activity.3 Within that lifecycle, the IBOM can serve as an evidence layer that records the information inputs and trust conditions behind a specific AI-assisted output or action. The IBOM is not itself an AI governance standard; it supplies evidence that such standards depend on.

Several 2026 events show the consequences of allowing authoritative presentation to substitute for source-level trust.

In professional services, EY withdrew a study after researchers identified fabricated or unsupported citations. KPMG pulled an agentic-AI report after organizations disputed claims about their AI use. PwC began correcting reports containing fabricated footnotes, misattributed statements, and unverifiable sources.8,9,10 The incidents differed, but shared a pattern: information entered an authoritative publication channel without sufficient evidence that its sources and assertions deserved that authority.

In April 2026, Sullivan & Cromwell apologized to a federal judge after a court filing contained inaccurate citations and other AI-generated errors. The firm stated that its AI policies had not been followed and that a secondary review process failed to identify the inaccuracies.11 The output crossed a release boundary without required source verification and operating evidence.

A June 2026 lawsuit by MeetingTV against Koi Security and Palo Alto Networks illustrates a related downstream risk involving a disputed threat-intelligence finding.12 The lawsuit alleges that published security findings were hallucinated; those allegations have not been adjudicated, and the litigation does not establish that AI generated the disputed finding. The example is therefore best read not as a confirmed AI hallucination but as a case study in downstream propagation and verification risk. Once a security finding is published, other organizations may ingest it into their own tools, alerts, and decisions without independent verification. An IBOM for such a finding would make visible the evidence supporting it and the validation performed before it was released for downstream reliance.

Rights and attribution create another dimension. In March 2026, Encyclopaedia Britannica and Merriam-Webster sued OpenAI, alleging unauthorized use of copyrighted reference material, near-verbatim outputs, and false attribution of AI-generated content. OpenAI disputes the claims and argues that its models are trained on publicly available information under fair-use principles; the case remains unresolved.13 The enterprise lesson does not depend on the ruling. Access does not automatically establish the right to copy, transform, attribute, publish, or commercialize information. The U.S. Copyright Office has also emphasized that copyrightability of AI-assisted output depends on sufficient human authorship, while separate training and licensing questions remain.14

For enterprise use, the relevant question is therefore not only whether information can be accessed, but whether the organization is authorized to use, transform, reproduce, disclose, retain, or pass that information into another system for the intended purpose. Rights metadata belongs in the information’s trust and permitted-use record.

An effective IBOM would not eliminate every error or dispute. It would make critical evidence visible before reliance: supporting sources, authority, rights, transformations, validation, warranted confidence, release approval, and permitted downstream use.

IBOM, INFORMATION TRUST, AND AI VALUE-RISK MATURITY

A value-risk view of AI maturity defines it as the enterprise’s ability to produce measurable AI-enabled business value at a level of residual risk that is understood, governed, monitored, and within risk appetite. It does not treat the number of tools, pilots, models, or users as maturity by itself.

The IBOM provides evidence for the trust dimension of that judgment. Trust should attach to a specific output or action in a specific business context, not to a model in the abstract. A model that performs well for summarization may be unsuitable for a legal conclusion. A system that is acceptable for drafting may not be acceptable for approving a payment. A high-confidence output may still require human approval when the action is irreversible, regulated, safety-related, customer-impacting, or strategically material.

Three factors should determine permitted reliance:

Enterprise value and decision consequence:
What value is being pursued, and what happens if the information is wrong?

Warranted Information Trust:
Does the Required and As-Used IBOM provide sufficient evidence regarding provenance, integrity, rights, validation, uncertainty, and context?

Control sufficiency:
Are access, compartmentalization, human oversight, monitoring, approval, rollback, and accountability controls proportionate to the risk?

AI maturity advances only when value, trust, and control capability advance together. High value with weak Information Trust is not advanced maturity; it is accelerated exposure. Strong controls without meaningful value may be safe, but they do not demonstrate mature utilization. Mature enterprises can explain why the information was trusted, why the action was permitted, who remained accountable, and why the residual risk was acceptable.

7 DRIVING QUESTIONS FOR ENTERPRISE LEADERS

1. Which decisions and actions would create the greatest financial, operational, legal, safety, regulatory, customer, employee, or reputational consequences if the supporting information were wrong or untrustworthy?

2. Has the enterprise defined the information required for those decisions, or is it assembled each time differently by individuals, systems, vendors, or AI tools?

3. Can the enterprise demonstrate the provenance, authority, integrity, accuracy, completeness, currency, context, and usage rights of each material information component?

4. Can it distinguish between the information that was required and the information actually used, including substitutions, omissions, transformations, assumptions, and conflicting sources

5. Are access, compartmentalization, validation, human oversight, and decision authority proportionate to the consequence of reliance?

6. Where is AI retrieving, combining, interpreting, generating, recommending, or acting on information, and what trust threshold must be met before its output may influence a decision or action?

7. Can the enterprise reconstruct and defend why a material decision was made, what information supported it, how that information was validated, who authorized reliance, and why the remaining risk was accepted?

An enterprise that cannot answer these questions may possess substantial information while still lacking an effective Information Trust capability.

A starting template.
Organizations that want to begin can use a compact field checklist such as the one below. It can be kept as a simple form for a first pilot and later carried as structured metadata in workflow and data-catalog systems.

IBOM field	What it records
Output / decision ID	Unique identifier for the output, decision, or action the IBOM supports.
Purpose	Intended use, decision consequence, and the level of trust required.
Required inputs	Information components that must be present before the output may be accepted.
Approved sources	Sources authorized for each required input, and any prohibited sources.
Actual sources used	The specific sources, devices, instruments, services, or systems actually relied upon, and how their identity was verified.
Source owner	Accountable owner of each source.
Version / timestamp	Version, observation time, or extraction date of each input.
Transformations	Filtering, calculation, summarization, inference, or AI processing applied.
Rights / restrictions	License, consent, confidentiality, retention, and purpose limitations.
Validation	Checks, reconciliations, testing, and confidence assessment performed.
Exceptions	Substitutions, missing inputs, and conflicts, and how each was resolved.
Reviewer	Who examined the evidence before reliance.
Approval	Who authorized reliance, and at what trust level.
Permitted use	What the output may inform, recommend, approve, or execute, and where it may go.
Expiry / review trigger	When the IBOM must be revisited: source change, new use, or elapsed time.
Audit reference	Where the record and its supporting evidence are retained.

A PRACTICAL ENTERPRISE AGENDA

Improving Information Trust should not begin by cataloging every piece of enterprise information. It should begin where misplaced trust could create the greatest harm. The progression is priority-based and cumulative: Visible, Defined, Controlled, Assured, Integrated, and Adaptive.

Priority	Maturity objective	Principal actions
1. Establish visibility and accountability	Information dependencies are recognized.	Identify high-consequence outputs, decisions, and actions; map the people, sources, instruments, systems, vendors, and AI capabilities that contribute; assign business, information, technology, and risk owners.
2. Define required Information Trust	Decision-specific trust requirements are explicit.	Create the Required IBOM; define approved and prohibited sources, accuracy, currency, completeness, provenance, rights, uncertainty, validation, and human oversight requirements.
3. Control information composition and use	Information enters and moves through governed boundaries.	Enforce source, access, processing, compartmentalization, transformation, and distribution controls; create the As-Used IBOM for material outputs.
4. Establish assurance and decision-grade evidence	The enterprise can demonstrate why reliance is warranted.	Compare Required and As-Used IBOMs; detect gaps and substitutions; validate outputs; monitor source and control integrity; document exceptions, approvals, permitted use, and residual risk.
5. Integrate, scale, and continuously improve	Information Trust becomes a repeatable enterprise capability.	Embed IBOM requirements into workflows, information catalogs, analytical systems, publications, decision gates, third-party management, and AI platforms; reassess when sources, uses, autonomy, or consequences change.

The maturity objective is not to make every output equally trusted. It is to make the level of trust visible and to align permitted use with evidence. Information may be labeled exploratory, draft, reviewed, decision-support, decision-grade, or authorized for bounded automated action. The label should reflect actual evidence and control performance, not presentation quality or organizational reputation.

The enterprise can start with a few consequential decisions, create Required and As-Used IBOMs, test whether controls support the intended reliance, and expand based on risk and value. AI use follows the same progression, with added attention to retrieval boundaries, generated assumptions, source manipulation, hallucination, rights, sensitive inference, downstream automation, and reversibility.

Measuring progress. To make the agenda operational, organizations can track a small set of indicators:

Percentage of high-consequence outputs with a defined Required IBOM.
Percentage of critical sensors, instruments, and data feeds with verified source identity and monitored stream integrity.
Percentage of those outputs with a completed As-Used IBOM.
Number of material source substitutions detected before reliance.
Number and age of unresolved validation exceptions.
Percentage of material outputs with documented approval and permitted use.
Time required to reconstruct the evidence behind a decision.

CONCLUSION: INFORMATION TRUST MUST BE DEMONSTRABLE

Information is already one of the most important assets used by every enterprise. Its value, however, does not arise merely from possession or availability. Its decision value depends on whether the enterprise can establish that the information is sufficiently trustworthy for the purpose and consequence of its use.

The Information Bill of Material provides an enabling structure for that determination. It identifies what a defined output, decision, or action is made from; distinguishes what information was required from what was actually used; and preserves the evidence needed to judge provenance, integrity, rights, transformation, validation, access, and permitted reliance.

AI did not create the need for Information Trust. It has increased the urgency. As enterprises use AI to retrieve, combine, generate, recommend, and act, speed and apparent confidence must not substitute for warranted trust.

Put simply: if an enterprise cannot show what information materially influenced an output, where it came from, what happened to it, and who authorized reliance on it, then it cannot fully demonstrate Information Trust.

The central leadership question is therefore not simply, “Can the enterprise use AI?” It is: Can the enterprise demonstrate that the information supporting its decisions and actions deserved to be trusted?

ABOUT THE AUTHOR

Clif Triplett is a cybersecurity and enterprise-risk advisor and served as a Presidential Executive Fellow for IT and Cybersecurity and as Senior Cyber and IT Advisor at the U.S. Office of Personnel Management following the 2015 OPM breach.
He has served as a former Chief Information Officer of Baker Hughes and given the responsibilities for remote offshore drilling. He has held senior technology leadership roles at Motorola, General Motors, AlliedSignal, and Entergy Services.
He is a former President of the Greater Houston Chapter of ISACA, former President of the Houston Society of Information Management (SIM) and is a graduate of the U.S. Military Academy at West Point, and holds a master’s degree in computer information systems from Boston University.

SOURCES AND NOTES

The external examples are used for educational analysis. Allegations in unresolved litigation are identified as allegations and should not be read as findings of liability. Publication dates and case status are current through July 29, 2026.

1	National Institute of Standards and Technology. “Software Security in Supply Chains: Software Bill of Materials (SBOM).”
2	National Institute of Standards and Technology. “AI Risk Management Framework (AI RMF 1.0).” NIST AI 100-1, January 2023.
3	National Institute of Standards and Technology. “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.” NIST AI 600-1, July 2024.
4	International Organization for Standardization. “ISO/IEC 5181: Data provenance.” Under development.
5	Coalition for Content Provenance and Authenticity (C2PA). “C2PA Technical Specification,” version 2.2.
6	Falco, M. “Stuxnet Facts Report: A Technical and Strategic Analysis.” NATO Cooperative Cyber Defence Centre of Excellence, 2012.
7	International Society of Automation. “ISA/IEC 62443 Series of Standards.
8	Financial Times. “EY retracts study after researchers discover AI hallucinations.” May 15, 2026.
9	Financial Times. “KPMG report contained AI hallucinations on benefits of AI.” June 2026.
10	Financial Times. “PwC published reports on AI marred by AI hallucinations.” July 29, 2026.
11	Reuters. “Sullivan & Cromwell law firm apologizes for AI ‘hallucinations’ in court filing.” April 21, 2026.
12	Axios. “Lawsuit accuses AI security company of publishing hallucinated findings.” June 29, 2026.
13	Reuters. “Encyclopedia Britannica sues OpenAI over AI training.” March 16, 2026.
14	U.S. Copyright Office. “Copyright and Artificial Intelligence.