ճ
- Generative Artificial Intelligenceintroduces new risks, such asinaccurate outputs, data exposure, bias, and rapid change,that traditionalinternalcontrols cannot fully address.
- COSO’s Internal Control–Integrated Framework offers a proven foundation for governing generative AI.
- This articleprovidespracticalstepsorganizations can usetominimize risk when usinggenerative AI.
______________________________________________________________________________________________
Generative AI: A Solution That Brings New Challenges to Traditional Internal Controls
Generative AI israpidlymoving from experimentation toreal businessuse.Your employeesmay already beusing it to summarize contracts, support forecasting, automate workflows, draft communications, and analyze large sets of data.
That speed creates opportunity, but it also raises a serious question:Can your organization trustGenerative AI’soutputs enough to rely on them?
Generative AI can improve efficiency and insight, but it can also introduce new forms of risk, including inaccurate outputs,unreliable or factually correct information, privacy concerns, bias, and changes that happen faster than governance processes can keep up. Organizations that use AI well need more than enthusiasm. They needstructure.
That’swhere internal control comes in.
In this article,you’lllearn:
- Why internal control matters for generativeAI.
- Whatrisksorganizations need to addressfirst.
- Howframework can be adapted for AI usecases.
- What practical steps leaders can take now to buildconfidence.
WhyGenerative AINeedsStrongerInternalControls
Generative AI is different from traditional software. Most business systems are designed to produce the same result when given the same inputs. Generative AI does not work that way. It is“probabilistic,”which means it can produce variable outputs, even when the question appears similar.
That creates a different control challenge.
A generative AI tool may sound confident while producing inaccurate information. It may rely on incomplete source material. It may change behavior afteran updateto the tool. It may even be adopted by employees outside approved channels, creating “shadow AI” use that bypasses formalcontrols.
These are not small issues. If AI outputs influencebusiness decisions,financial reporting, compliance activity,or customer communication, control gaps can quickly becomeseriousproblems.
Key Takeaway:If your organization is using generative AI in any meaningful way, internal controlsshould not be treated as a future-state issue. It is acurrent-staterequirement.
TheTop 5RisksGenerative AIIntroduces
Beforeyourorganization canimplementthe right controls,youneed to understand where AI risk shows up in practice.
While every use case is different,theprimaryissuesincreasingrisk in organizations are:
1. Reliability and Accuracy
Generative AI can produce outputs that look polished and useful but are factually wrong. This is calleda hallucination. In a business setting,hallucinationscouldresult inan incorrect summary of a contract, a flawed forecast narrative, or an unsupported recommendation.
A finance team, for example, may use AI to draft management commentary for monthly reporting. If the tool introduces a false explanation for margin changes, and no one catches it, the issue canlead to an incorrect decision.
2. Data Quality and Source Integrity
AI systems are only as strong as the information they use. If source data is incomplete, inconsistent, outdated, or poorly governed, output quality will suffer. In many cases, organizations also struggle tounderstand anddocumentthesourceAI pulled information from and how it used it.
That matters for accountability, transparency,and auditability.
3. Security and Privacy
Generative AI creates new entry points for data exposure. Employees may upload sensitive information into tools thatarenot approved for regulated or confidential data. Prompt injection attacks can also manipulate how tools behave or what they reveal.
For organizationsoperatingin regulated environments, this risk is especially important. Internal controls help define what data can be used, where it can be used, and who has access.
4. Bias and Fairness
AI models can reflect bias from training data, source materials, or design choices. That can affect outputs in ways that are difficult to detect without deliberate testing and oversight.
Bias is often discussed as an ethical issue, but it is also a business issue. It can increase legal exposure, damage trust, and lead to poor operational decisions.
5. Third-party and Change risk
Many organizations rely on third-party AI vendors. That means key components of the system may change outside the organization’s direct control. A vendor update, new model version, or revised retrieval setting can shift output behavior quickly.
Without clear monitoring and change management, teams may continue relying on a tool that no longer performs as expected.
How COSOCan Help Protect Organizationsthat UseGenerative AI
The good news is organizations do not need to invent an entirely new control model.TheCommittee of Sponsoring Organizations of the Treadway Commission (COSO), a globally recognized organization dedicated to helping entities reduce fraud while improving operations and oversight, has introducedanInternal Control – Integrated Framework.
䰿’sInternal Control – Integrated Frameworkprovidesa strong foundationcomprisingfive componentsitdeemscrucial toeffective internal controls in the age of generative AI:
- Control Environment
- Risk Assessment
- Control Activities
- Information and Communication
- Monitoring Activities
When adapted thoughtfullyto your organization’s unique processes, these componentscan help yourorganizationgovern generative AI in a way that supports innovationand protects businessobjectives.
Best Practices for Implementing the COSO Components
To effectivelyimplementeach of the five COSO components, organizations must approach them with strategic planning and practical implementation.
Here’s an introduction to tackling eachcomponent, ensuring they align with modern business needs:
1. Control environment: Set the tone for responsible AI use
A strong control environment defines expectations before problems arise. With generative AI, that means leadership should make it clear that speed and experimentation do not replace accountability.
Organizations shouldestablish:
- Clear acceptable use policies for AI tools.
- Defined ownership for each AI use case or platform.
- Role-based responsibilities for development, review, and approval.
- Training tailored to technical users, business users, and reviewers.
- Consequences for misuse or poor oversight.
This is where integrity becomes visible. If an organization says responsible use matters, its policies, oversight structure, and leadership behavior should reflect that.
A practical example:Ifa legal summary tool is used to review contracts, someone should own that tool, define approved document types, oversee changes, and confirm users understand its limits.
Key takeaway: Governance starts with clarity. People need to know what is allowed, who is responsible, and when human judgment must override automation.
2. Risk assessment:Identifywhere AI can go wrong
Risk assessment for generative AI should be active, specific, and ongoing. Annual review cycles alone are not enough. Models, data sources, prompts, and vendor settings can change too quickly.
Organizations should begin by asking a simple question:Is generative AI the right tool for this task?
In some cases, traditional automation or rules-based systems may be more reliable and easier to control. If AI isappropriate, the next step is to assess risk based on the use case.
That includes evaluating:
- Theobjectiveof thetool.
- The business impact of wrong outputs.
- The likelihood of hallucinations or“drift,”when the inputdata’sdistribution shifts away from what the model initiallyencounteredduring training, leading to inaccuracies.
- Data sensitivity and privacy exposure.
- Vendor dependence.
- Fraud and manipulation scenarios.
- Regulatory or financial reporting implications.
For example, an AI tool that drafts internal brainstorming notes carries a different risk profile than one that supports reconciliations or compliance monitoring.
This is where business wisdom matters. Strong risk assessment is not about slowing everything down. It is about applying the right level of discipline to the right level of risk.
3. Control activities: Build safeguards into the process
Control activities are the practical actions that reduce risk. For generative AI, these controls should reflect how the tool is used, how much reliance is placed on it, and how quickly errors could spread.
Common control activities include:
- Human review for high-risk outputs.
- Confidence thresholds before automationisallowed.
- Approval workflows for prompts, rules, and model changes.
- Segregation of duties between those who configure and those who approve.
- Side-by-side testing before deployment.
- Source citation requirements for key outputs.
- Logging of prompts, outputs, and model versions.
- Rollback plans if performance declines.
Consider an AI-enabled reconciliation process.If the system is allowed to auto-post entries, there should bepostingthresholds, clear exception routing, and multi-layerapproval for any changes to those thresholds.If not, a small configuration issue could affect a large volume of transactions.
The goal is not toeliminateautomation. It is to ensure automationremainstrustworthy.
4. Information and communication: Make AI use traceable
Many AI-related failures become harder to manage because organizations cannot easily answer basic questions, such as:
- What data did the tool use?
- Which model version generated the output?
- Was the output reviewed?
- Were limitations communicated to users?
- Did anyone know the tool changed?
Strong information and communication practices help solvethis.
Organizations shouldmaintainrecords of prompts, inputs, outputs, model configurations, source references, and known limitations whenappropriate tothe use case. They should also communicate changes clearly to the people affected.
For example, if an AI summarization tool used by compliance teams receives a model update, reviewers should know what changed, when it changed, and whether performance was revalidated before continued use.
Internal communication matters just as much as documentation. Users, reviewers, managers, and governance teams all need the right level of information to make sound decisions.
Key takeaway:If AI supports an important process, its use should be understandable, traceable, and communicated clearly enough to support confidence and oversight.
5. Monitoring activities: Keep controls current as AI evolves
Generative AI cannot be controlled with a “set it and forget it” mindset. Even a well-designed control can weaken over timeasthe modelchanges,the data shifts, or users begin working around the process.
Monitoring should include both ongoing review and periodic assessment.
Ongoing monitoring may track:
- Accuracy rates
- Exception volumes
- Drift indicators
- Response quality
- Bias metrics
- Security incidents
- User behavior patterns
Separate evaluations may include:
- Periodic control testing
- Historical back-testing
- Independent challenge reviews
- Adversarial testing
- Validation after vendor updates
A useful example is an expense monitoring model that performs well at launch but slowly declines as employee spending patterns change. Without monitoring thresholds and formal retraining triggers, the decline may go unnoticed until reporting or compliance issuesemerge.
This is where client success comes into focus. Monitoring is not just about catching failure. It is about sustaining value over time.
CommonGen AI MistakesOrganizationsMake
As companies expand AI use, several control issues tend to appear early.Here’swhat to look out for andavoid:
- Treating AI like standard software: Traditional IT controls still matter, but they are not enough on their own. Generative AI introduces issues like probabilistic outputs, prompt-based configuration, and source reliability that require added attention.
- Allowing adoption before ownership is clear: If no one owns the use case, no one owns the risk. Every meaningful AI tool should have aknowledgeablebusiness ownerthat hasauthority andis accountable.
- Over-relying on outputs without review: Even useful tools can produce flawed results. High-impact outputs should bevalidatedby qualified reviewers, especially when they affect reporting, compliance, or external communication.
- Failing to update controls as the tool changes: A control that worked at implementation may not work six months later. Monitoring, change control, and revalidation are essential.
APracticalPathForwardfor Gen AI Governance
Organizations do not need to solve every AI governance issue at once. A more effective approach is to build control maturity in stages.
Here is a practicalseries of steps tostartwith:
- Inventory current AI use casesacross departments, including unofficial tools.
- Classify each use caseby purpose, data sensitivity, and business impact.
- Assign ownershipfor each approved tool or process.
- Assess risk using 䰿’s five componentsasthe foundation.
- Design and document controlsbased on the level of reliance and risk.
- Train users and reviewerson expectations, limitations, and escalation paths.
- Monitor performance and update controlsas use cases evolve.
This kind of structure helps organizations move from reactive concern to proactive governance.
Next Steps
Achieving effective internal control over generative AI is not about resisting change. It is about making sure change creates value without compromising trust, compliance, or decision quality.
䰿’s framework gives organizations aprinciple-based approachto govern AI use with discipline and flexibility. When companies pair that structure with integrity, a focus on client success, and sound business judgment, they are betterpositionedto use generative AI confidently and responsibly.
If your organization is evaluating howtousegenerative AI, now is the time toapply the COSO principlesto lay the foundation in the control environment. Start byidentifyingwhere AI is already influencing decisions, then assess whether your controls are keeping pace. That foundation can help you move forward with greater confidence and clarity.
Don’t let your controls lag behind your AI adoption
Evaluating where AI influences your business decisions is a critical first step—but youdon’thave to navigate the complexities alone.At ݮƵ, we believe effective governance should do more than reduce risk. It should help you move forward with confidence, integrity, and clear businesspurpose.Schedule a readiness assessmentwith our risk advisory team todayto ensure your organization moves forward responsibly and securely.
FrequentlyAskedQuestions
Q. What is “shadow AI” and why is it a risk?
A.Shadow AI refers to generative AI tools that employees adopt outside of approved channels, bypassing formal controls. It is a risk because unsanctioned use can expose sensitive data, introduce unreviewed outputs into business decisions, and leave organizations unable to track how AI is influencing operations.
Q. How often should organizations review their generative AI controls?
A.Generative AI controls should be reviewed continuously rather than on a fixed annualcycle, becausemodels, data sources, prompts, and vendor settings can change quickly. Most organizations combine ongoing monitoring—such as tracking accuracy and drift—with periodic evaluations like control testing and validation after vendor updates.
Q. Can existing internal control frameworks govern generative AI, or is a new one needed?
A.Existing frameworks like COSO’s Internal Control – Integrated Framework can effectively govern generative AI without the need to create a new model. The key is adapting how each of the five COSO components is applied to account for AI-specific factors such as probabilistic outputs, source reliability, and third-party change risk.