When AI Becomes a Quality Risk: Why Governance Alone is Not Enough

WASHINGTON, D.C.: Johnson Global Advisory is proud to support Santa Monica College through a donation to its STEM Program—investing in educational opportunities that prepare students for careers in science, technology, engineering, and mathematics. Santa Monica College’s STEM Program provides students with access to high-quality academic resources, hands-on learning experiences, and pathways to transfer to four-year institutions and enter in-demand fields. By fostering critical thinking, innovation, and technical skills, the program helps equip students with the tools they need to succeed in an evolving workforce. Katherine Moe writes, “We are deeply grateful to Johnson Global Advisory for its sponsorship of Santa Monica College’s Launch the Future campaign and its investment in the next generation of STEM leaders and innovators. This support expands access to hands-on research, industry-standard technology, scholarships, mentorship, and professional connections—ensuring financial barriers do not stand in the way of talented students pursuing careers that will shape the future of science, healthcare, technology, and innovation.” "My connection to California makes this especially meaningful, " said Jackson Johnson, JGA President. "Supporting the Santa Monica College STEM program reflects our broader commitment to education, access, and we’re proud to invest in opportunities that help shape the next generation of leaders." About Johnson Global Advisory Johnson Global partners with leadership of public accounting firms, driving change to achieve the highest level of audit quality. Led by former PCAOB and SEC staff, JGA professionals are passionate and practical in their support to firms in their audit quality journey. We accelerate the opportunities to improve quality through policies, practices, and controls throughout the firm. This innovative approach harnesses technology to transform audit quality. Our team is designed to maintain a close pulse on regulatory environments around the world and incorporate solutions which navigate those standards. JGA is committed to helping the profession in amplifying quality worldwide. Visit www.johnson-global.com to learn more about Johnson Global.

As discussed in our prior articles, What Regulators Expect to See When AI is Used and AI Governance Belongs in the Boardroom, Not the Server Room, firms increasingly recognize that AI governance belongs within the system of quality management. However, inspection experience shows that even well-designed governance frameworks do not eliminate risk. Significant failures occur not only at the policy level, but also at the engagement level, where AI outputs are relied upon as audit evidence without sufficient validation. This article focuses on that execution gap. Specifically, it examines why validation of AI is emerging as one of the most significant audit evidence risks facing public company auditors today. For public company auditors, AI validation is no longer a technical exercise. It is an audit quality issue — and increasingly, an inspection issue. In the eyes of regulators, AI does not reduce evidentiary requirements; it changes how evidence must be evaluated, corroborated, and defended . How AI Changes Audit Evidence—and Raises the Validation Stakes PCAOB auditing standards governing audit evidence have not been rewritten for AI. The fundamental requirement remains the same: auditors must obtain sufficient appropriate audit evidence to support their opinion. What has changed is the evidence pipeline: when AI is used, outputs are often indirect (generated through models rather than procedures alone), abstracted (summaries, risk flags, or scores rather than raw data), and less intuitive to evaluate using traditional audit instincts. This creates a new risk: auditors may rely on AI assisted outputs without fully validating how those outputs were produced, what they mean, or whether they are reliable. From an inspection perspective, AI introduces a simple but critical question: How does the auditor know the AI result is reliable enough to rely on as audit evidence? Inspectors are increasingly focused on whether the engagement team can demonstrate the completeness and accuracy of inputs, the reasonableness of assumptions/logic (including prompts), the consistency and explainability of outputs, and the auditor’s independent evaluation and corroboration. A common misconception is equating firm tool approval (vendor diligence, IT review, or risk assessment) with audit evidence validation. Approval is necessary, but it is not sufficient: validation must occur at the engagement level, in the context of the specific audit objectives, data, and risks. Where AI Validation Commonly Breaks Down In practice, AI validation risk often arises in predictable ways:

Johnson Global Advisory is pleased to announce that Jackson Johnson, CPA, President, has been appointed to serve on the AICPA & NASBA International Qualifications Appraisal Board (IQAB). The IQAB is responsible for evaluating international accounting qualifications and facilitating mutual recognition agreements between the United States and other countries, helping to support global mobility and consistency in professional standards. “It’s an honor to serve on the IQAB and contribute to efforts that strengthen the global accounting profession,” said Johnson. “As the profession continues to evolve, collaboration across jurisdictions is critical to maintaining high standards and enabling greater mobility for accounting professionals worldwide.”

Few technologies have generated as much excitement—and as much promise—for accounting firms as artificial intelligence (“AI”). The potential to streamline audit execution, reduce hours, and enhance firm profitability is real and already being realized. However, AI does not simply change how audits are performed; it fundamentally alters how firms must think about oversight, responsibility, and quality management. As regulators sharpen their focus on AI‑enabled audits, firm leadership must move beyond adoption and address a more complex challenge: establishing clear and scalable AI governance. This article outlines why AI governance is now a strategic imperative for accounting firm leadership. As discussed in JGA’s article What Regulators Expect to See When AI is Used , inspectors do not evaluate AI tools in isolation. They evaluate whether the engagement team obtained sufficient appropriate audit evidence, exercised professional skepticism, and applied appropriate supervision and review when AI was used. Those expectations are grounded in existing auditing standards and apply regardless of whether AI was used for risk assessment, testing, or documentation support. Against that backdrop, AI governance is not simply about approving tools or managing technology risk. It is about ensuring the firm’s system of quality management supports consistent, supervised, and well-documented use of AI that aligns with audit objectives and withstands inspection scrutiny. When firms treat AI as an IT matter, governance discussions tend to center on 1) Data security, 2) System access, 3) Vendor due diligence, and 4) Infrastructure controls. Those topics matter—but they are only the baseline. Inspectors do not evaluate whether AI systems are well engineered; they evaluate whether AI enabled audit work complies with standards, supports professional judgment, and is governed within the firm’s system of quality management. In short, AI governance is a firmwide audit quality issue, not a back office technology function. Using AI does not change the auditor’s responsibilities. Requirements still apply when AI is used for 1) Audit evidence, 2) Professional skepticism, 3) Supervision and review, 4) Engagement partner accountability and 5) Firm level quality controls. From an inspection standpoint, AI introduces new audit quality risks, including: Over reliance on automated outputs Reduced professional skepticism (automation bias) Inconsistent application across engagements Insufficient documentation of judgment Lack of transparency around how conclusions were reached These are not IT risks—they are audit quality risks. AI Touches Nearly Every Component of a QC System Under modern quality management frameworks (including PCAOB QC 1000 , AICPA SQMS No. 1, IAASB ISQM 1), AI affects nearly every component of a firm’s QC system, not just technology or data governance. 

Johnson Global Advisory ("JGA") is proud to announce that Joe Lynch, Shareholder, will be speaking on a panel at the 41st Midyear SEC Reporting & FASB Forum . Joe will deliver the PCAOB update on June 5, with attendance available both in person and virtually. This panel will summarize the activities of the PCAOB including: Recite new requirements for the lead auditor’s use of other auditors Anticipate the new standard, “The Auditor’s Use of Confirmation” Enumerate the new requirements of QC 1000, “A Firm’s System of Quality Control” Recall the guidance of the new auditing standard “General Responsibilities of the Auditor in Conducting an Audit” Understand the amendments addressing aspects of audit procedures that involve technology-assisted analysis of information in electronic form Learn about the proposal to replace existing auditing standards related to an auditor’s use of substantive analytical procedures Anticipate other Standard-Setting and Research Projects Summarize PCAOB inspection findings and enforcement activities Understand recent PCAOB publications, including: Spotlight Publications Audit Focus Publications Data Points Publications Click here to register and learn more. Johnson Global partners with leadership of public accounting firms, driving change to achieve the highest level of audit quality. Led by former PCAOB staff, JGA professionals are passionate and practical in their support to firms in their audit quality journey. We accelerate the opportunities to improve quality through policies, practices, and controls throughout the firm. This innovative approach harnesses technology to transform audit quality. Our team is designed to maintain a close pulse on regulatory environments around the world and incorporates solutions which navigates those standards. JGA is committed to helping the profession in amplifying quality worldwide. 

Johnson Global Advisory (JGA) has submitted its response to the PCAOB’s request for input on its 2026–2030 strategic priorities. Drawing on extensive experience supporting firms subject to PCAOB oversight, JGA’s comments emphasize a more modern, risk-based approach to regulation focused on audit quality, scalability, and transparency. View JGA's comments here. Johnson Global partners with leadership of public accounting firms, driving change to achieve the highest level of audit quality. Led by former PCAOB staff, JGA professionals are passionate and practical in their support to firms in their audit quality journey. We accelerate the opportunities to improve quality through policies, practices, and controls throughout the firm. This innovative approach harnesses technology to transform audit quality. Our team is designed to maintain a close pulse on regulatory environments around the world and incorporates solutions which navigates those standards. JGA is committed to helping the profession in amplifying quality worldwide.

In our work with firms, we have seen a clear shift in how monitoring and remediation are viewed under modern quality management frameworks. They are no longer treated as retrospective compliance exercises. Instead, engagement deficiencies are increasingly used as meaningful inputs into an ongoing, risk-based system designed to identify issues early, address them thoughtfully, and reduce the likelihood of recurrence. Regulatory messaging reinforces this evolution. Oversight bodies are signaling a shift in focus from isolated engagement outcomes and more on whether firms have a system of quality management that consistently detects quality risks, responds appropriately, and demonstrates that remediation is working in practice. Based on our experience, while individual engagement deficiencies remain important, the more critical question is becoming how firms analyze, respond to, and learn from those issues over time. Engagement Deficiencies Are Signals, Not Endpoints Engagement deficiencies can surface through many channels, including pre-issuance reviews, internal inspections, post-issuance reviews, peer reviews, and regulatory inspections. Regardless of source, firms benefit most when these findings are evaluated through a consistent quality management lens. In practice, we encourage firms to look beyond whether a single engagement fell short . The more meaningful consideration is whether the deficiency points to potential weaknesses in governance, methodology, training, supervision, resourcing, or monitoring activities. We often observe that when issues are quickly labeled as engagement-specific, without assessing whether they reflect broader quality risks, valuable insight is lost. Modern quality management frameworks are designed to use these signals to strengthen the system, not simply close individual findings. What Effective Monitoring and Remediation Looks Like in Practice Firms that navigate this environment effectively tend to apply a disciplined and repeatable approach when deficiencies are identified. Based on our experience supporting firms across a range of practice areas, several elements consistently make a difference: Assess whether the issue may be systemic Recurring observations across engagements, service lines, or time periods often indicate system-level risk. Similar documentation gaps, inconsistent application of methodology, or supervision challenges rarely arise in isolation. Perform meaningful root cause analysis Effective root cause analysis typically moves beyond surface explanations. Firms benefit from evaluating whether policies and procedures were designed appropriately, implemented as intended, and supported by sufficient training, time, and resources. Design remediation that directly responds to the quality risk Remediation is most effective when it is clearly linked to the underlying risk. Depending on the circumstances, this may include enhancements to methodology, targeted training, revised review requirements, or changes to engagement acceptance, staffing, or oversight processes. Validate remediation through timely monitoring Implementing corrective actions is only part of the process. In our experience, firms are most successful when they also confirm that remediation operates as intended. Follow-up monitoring performed early enough to prevent recurrence is a critical component of this step. Failure to validate remediation remains one of the most common and consequential weaknesses we observe across firms. Case Study: When Remediation Is Not Validated In one situation we encountered, a firm identified engagement deficiencies through post-issuance reviews. The issues mirrored observations that had previously been noted during peer review and were communicated as having been addressed by the group responsible for report issuance. However, responsibility for validation was not clearly assigned, and no follow-up procedures were performed to evaluate whether the revised processes were effective. Subsequent post-issuance reviews, triggered by an organizational change, revealed that similar and additional deficiencies had re-emerged. From a quality management perspective, this was not an engagement execution failure. It reflected a breakdown in monitoring and remediation. The firm had information indicating quality risk but did not adjust its monitoring activities to confirm that remediation was working. Viewed through a system lens, this represents a system-level deficiency rather than an isolated engagement issue. Quality Management Applies Across All Engagement Types Modern quality management frameworks apply across a firm’s assurance and attestation practice, including private company audits, public company audits, SOC engagements, nonprofit audits, and other services. Deficiencies identified in any practice area may signal broader weaknesses in: Governance and leadership Methodology and training Monitoring activities Remediation processes In our experience, firms struggle to maintain an effective system of quality management when certain practices are treated as exempt from system-level evaluation. Key Takeaways Engagement deficiencies are inputs into the system, not endpoints. Recurring issues often indicate systemic quality risk. Remediation should be validated, not assumed. Monitoring activities should evolve as risks emerge. Quality management applies across all engagement types. Firms that treat monitoring and remediation as a continuous feedback loop, rather than a periodic exercise, are typically better positioned to improve engagement quality and respond to evolving regulatory expectations. Looking for an independent perspective on whether engagement deficiencies have been fully addressed? Based on our experience working with firms across assurance and attestation practices, Johnson Global Advisory supports clients by performing independent reviews, validating remediation efforts, and strengthening monitoring processes. If you would like support refining policies, training, workflows, or documentation standards, or would benefit from an objective assessment ahead of regulatory, peer, or internal inspections, contact your JGA audit quality advisor to discuss your needs.

Artificial intelligence (“AI”) is no longer experimental in public company audits. From risk assessment and scoping decisions to population testing, anomaly detection, and documentation support, AI enabled tools are increasingly embedded in audit execution and workflow. As use expands, the auditor’s core obligations do not shift to the technology, they remain with the engagement team. If AI is used to inform judgments, influence the nature, timing, or extent of procedures, or summarize and interpret information, auditors must still demonstrate that they obtained sufficient appropriate audit evidence and applied professional skepticism throughout. In practice, auditors must understand what the tool is doing, confirm that inputs are complete and accurate, and evaluate whether the outputs are reliable and fit for purpose in the specific audit context. While the auditing standard devoted solely to AI have not been issued, our experience is that inspectors have been increasingly direct—through staff publications, questions from inspectors in the field, and public remarks—about what they expect to see when AI is used. The expectations are grounded in existing standards and longstanding inspection focus areas: audit evidence, supervision and review, professional skepticism, and firm quality control (now quality management). In other words, AI does not create a “new” audit; it amplifies the need to show your work. Firms that treat AI as a “shortcut”, rely on outputs that cannot be explained or reproduced, or fail to govern and document how tools were selected, configured, and monitored are inviting new risks to support their audit conclusions. Conversely, firms that can clearly articulate the purpose of the tool, how it aligns to audit objectives, how inputs and outputs were validated, and how experienced personnel supervised and challenged the results will be far better positioned during inspection. The table below summarizes what inspectors typically expect to see documented when AI is used in a public company audit. Firms can use these themes to evaluate whether their engagement documentation tells a complete story that an experienced auditor (and an inspector) can follow from objective, to procedure, to results, to conclusion. 

In a previous article, Back to Basics: Audit Documentation Failures Have Become Dangerous Low Hanging Fruit , we highlighted how audit documentation had quietly re-emerged as a source of regulatory risk after years of relative deprioritization. While PCAOB Auditing Standard 1215, Audit Documentation (AS 1215), has historically been cited less frequently than other standards, our direct experience from recent inspection activity, enforcement actions, and internal inspection results, demonstrate that documentation failures are increasingly treated as indicators of deeper execution, supervision, and quality management breakdowns. In today’s environment, audit documentation is no longer merely a record of work performed. It is the primary evidence inspectors rely on to evaluate whether an engagement was properly planned, executed, and supported at the time the auditor’s report was issued. What has been low-hanging fruit now requires firms to close these gaps and transform them into a load-bearing foundation for audit quality. From Rare Enforcement to Systemic Inspection Risk AS 1215 establishes clear requirements regarding what must be documented, when documentation must be completed, and how engagement files must be assembled and retained. As discussed in our prior article, failures to comply with these requirements were historically viewed as technical or secondary issues, often resulting in inspection comments rather than enforcement action. That distinction is no longer meaningful. Recent enforcement actions involving backdating, improper (both intentionally, and inadvertent) modification of workpapers, and failure to timely assemble a complete audit file reflect an evolving regulatory view. Documentation failures do not simply violate procedural requirements; they call into question the credibility of the audit opinion itself. More importantly, beyond enforcement, documentation deficiencies are increasingly cited as core inspection findings. Inspectors are challenging situations where engagement teams assert that work was performed but cannot demonstrate that work within the archived file. In these cases, the absence of timely, complete, and clear documentation is no longer treated as a formality. It is treated as evidence that the engagement may not have been properly executed, supervised, or supported in accordance with PCAOB standards. This represents a fundamental shift. Documentation is no longer “low-hanging fruit.” It is a systemic inspection risk that cuts across execution, supervision, and firm-level quality management. From Misconduct to Execution Failures Pervasive documentation failures that do not involve intentional misconduct but still result in non-compliance are increasingly observed. For example, reviewer signoffs occurring near the documentation completion date, rather than contemporaneously with the performance of audit procedures, raise questions about whether effective supervision occurred during the audit or was deferred to meeting archiving deadlines. Similarly, engagement teams may assert that key judgments can be explained verbally, even when those judgments are not clearly documented in the audit file. In today’s environment, the distinction between “we can explain it” and “it is clearly documented” is critical. If procedures, judgments, and conclusions are not evident in the documentation itself, inspectors increasingly conclude that the work was not performed in accordance with PCAOB standards. The issue is not whether the engagement team can explain what they did after the fact. The issue is whether the archived documentation allows an experienced auditor, with no prior connection to the engagement, to understand the procedures performed, evidence obtained, and conclusions reached at the time of the auditor’s report. When documentation fails to reach that standard, inspectors are increasingly concluding that the audit itself was not properly executed, regardless of intent. This reflects an important shift. Documentation failures are no longer viewed primarily as misconduct. They are viewed as symptoms of execution breakdowns, including delayed supervision, compressed review cycles, and audit workflows that defer documentation until the end of the engagement. As a result, AS 1215 has become a direct proxy for how audits are actually performed in practice. How the 14-Day Documentation Completion Requirement Changes the Risk Profile The execution risks are further amplified by the PCAOB’s shortened documentation completion timeline. Recent amendments to AS 1215 reduce the timeframe to assemble a complete and final audit file from 45 days to 14 days after the report release date. While this change may appear procedural, its implications are operational. Under this accelerated timeline, engagement teams no longer have a meaningful post-issuance window to resolve review notes, complete documentation, or finalize supervisory evidence. What were once viewed as “clean-up” activities are now more likely to result in timing violations and non-compliance. This shift places increased emphasis on: Contemporaneous documentation Real-time supervision Realistic workload and staffing models Audit Documentation as a Cornerstone of Audit Quality Audit documentation has long been described as low-hanging fruit in the inspection process. That characterization no longer reflects its role in today’s regulatory environment. Documentation now serves as the primary lens through which regulators assess whether an engagement was properly executed, supervised, and supported. With shortened timelines, expanded quality management expectations, and increased regulatory scrutiny, firms can no longer treat documentation as a downstream activity. It must be embedded into how engagements are planned, staffed, reviewed, and completed. In an environment where inspection conclusions are driven by what is, and what is not, in the audit file, strong documentation is not merely defensive. It is foundational to audit quality. At Johnson Global Advisory , we support firms in selecting, implementing, and optimizing these tools to meet their unique needs. For more insights, visit our blog or contact us to learn how we can help your firm AmplifyQuality®. For more information, please contact your JGA audit quality expert .