Auditing Estimates: An Update for Unprecedented Times

For anyone reading the headlines, it sometimes feels as if we are living in unprecedented times, but the reality is, Shakespeare was right: “the past is prologue.” We’ve been through wars before. We’ve experienced inflation. We’ve survived recessions. What’s perhaps unique however, is the confluence of so many uncertainties which feels like uncharted territory for many of the younger generations. 

For instance, in March 2020, while typically a lagging indicator of economic health, we saw unemployment uncharacteristically lead the way for economic deterioration with the onset of the pandemic. While the markets tanked in the short term, by Q3 2020, the stock market had fully recovered and then went on to rally through Q4 2021. Though market performance does not equate to economic strength, certainly the pandemic seemed to de-correlate the two metrics. Fast forward to Q3 2022 and headlines are struggling to know what to call the current economic situation. Is it a recession or just a correction? Despite two quarters of negative economic growth, companies across many industries are still posting profits (albeit perhaps less than anticipated) and almost every company is struggling to hire sufficient resources. Supply chains are still disrupted, given the war in Ukraine and the reverse impact of sanctions, as well as the ongoing nature of the pandemic. And finally, we’re all aware of the red-hot inflation trend, leading the Fed to post several interest rate hikes in a very short time. 

While we can all acknowledge the economic uncertainties, how do we incorporate these new realities into our audits? Specifically, how does management compensate for these uncertainties in its estimates and how do auditors test these assumptions given how new or different they are from the past economic cycles? 

In our first article on Auditing Estimates, we provided various audit considerations for teams when evaluating subjective management assumptions. We stated (and many of our readers echoed their frustrations) that “auditing a management estimate can feel like trying to make concrete out of Jell-O.” Several years later, in its most recent inspection observations, the PCAOB still finds issues with estimates, stating: 

“While we have observed improvements in auditing accounting estimates, deficiencies continue to occur, particularly in auditing the allowance for loan losses (ALL), estimates related to accounting for business combinations, investment securities, and long-lived assets.” 

The most common deficiencies stemmed from audits where engagement teams:

• Did not sufficiently evaluate the appropriateness of models used in valuations; 
• Did not sufficiently obtain audit evidence for assumptions used in valuations; 
• Did not sufficiently evaluate changes, or lack of changes, in recurring assumptions used in valuations (specifically for ALL); and 
• Did not sufficiently evaluate contradictory evidence when concluding on the reasonableness of assumptions. 

Building on our previous article, below we expand on the common deficiencies and additional considerations to incorporate into audits of estimates, especially given current economic conditions. 

Auditing Estimates Considerations 

Valuation Models 

While I have rarely seen inappropriate models used in valuations, I have often seen teams fail to sufficiently document its evaluation of the valuation models used in estimates. AS 2501.10 and 11 explicitly require the auditor to evaluate whether the method used by management is in accordance with the financial reporting framework and is appropriate for the specific account. In addition, all changes to models need to be considered. Regardless of the type of model used/applied, it must be evaluated. The more complex the model, the more there is a need for a qualified valuation specialist that can specifically evaluate the appropriateness of the model itself, whether at the macro level (i.e. use of an income approach) or at the micro level (i.e. the appropriate factors to incorporate in building a discount rate). 

Support for Assumptions 

This finding is arguably the most difficult for auditors to fulfil given the judgment involved in what defines “sufficiency” or “reasonableness.” While we can debate the definitions, the reality is that many teams are still failing to obtain solid evidence and support for assumptions embedded into valuations. I often see teams inquire with management to understand how management derived its assumptions while failing to perform further procedures to obtain actual support for the inputs. Below are some considerations for teams to incorporate into their evaluation of assumptions: 

• Availability of data / information: In the current economic environment, is there relevant historical or industry data that can support specific assumptions? 

• For instance, given supply chain disruptions, do the past two or three years of historical internal data support future projections? How long will supply chain disruptions last? What will be the impact on production and sales? What will be the impact on costs and margins? 

• For start-ups with less operating history or smaller companies with less internal information tracking/monitoring, or less controls around internally derived information, management and auditors may be forced to look to external sources of information to support specific assumptions. 

• Accuracy and completeness as well as relevance and reliability of information: Engagement teams need to evaluate the accuracy and completeness of any data used by management that is internally derived (i.e. company specific data). In addition, for all externally derived information, auditors need to evaluate the relevance and reliability of that information. Regardless the source of the data, AS 2501.14 specifically requires auditors to evaluate whether “the data is relevant to the measurement objective for the accounting estimate.” The current economic uncertainties will challenge the relevance of information given some of the current conditions have not been seen in 30 or 40 years (i.e. inflation). 

• Qualitative inputs: Management often discusses qualitative factors that impact the valuations. Somehow, these qualitative inputs need to translate into quantitative figures used in the valuation model. Management is responsible for creating and supporting the quantitative assumptions, so auditors should not hesitate to challenge management on how it derived a specific assumption. I encourage teams to keep asking: “How? Why? Tell me more.” Be curious. 

Changes in Recurring Assumptions 

Given the changes in economic conditions, management and auditors need to consider changes (or the lack thereof) in recurring assumptions. Part of this evaluation should be built into retrospective reviews over management estimates (as required under AS 2401.63-65). Retrospective reviews will help audit teams evaluate how accurate previous management estimates were. To the extent management missed the mark in prior years, I would expect that current year assumptions would change to more accurately reflect the most recent information. In addition, to the extent economic conditions change, again, assumptions should also adjust year over year. For instance, although historical inflation assumptions typically ranged from 2-3%, I would expect current year inflation assumptions to reflect the higher trends being reported in the news. 

Too often, auditors simply apply a “status quo” blanket expectation for all assumptions, but the challenge will always be: 

• If assumptions changed year over year, what supports the change in assumptions? 
• If assumptions remained static, should they have remained constant? Or should they have changed to reflect evolving macro-economic or company-specific factors? 

These same concepts apply for analytics and fluctuation analyses where teams often just use a blanket “status-quo” expectation and investigate any changes greater than $X and/or X%. Well, why is the status quo the appropriate expectation to start? These are the auditor judgments that need to be documented to evidence the team’s considerations. 

Contradictory Evidence 

Auditors often review large sums of information. Invariably, there will be data that appears contradictory to management’s assumptions/assertions. It is critical for auditors to challenge this information and resolve any discrepancies that arise from contradictory evidence. Auditors should consider the following: 

• Obtain support from management to validate its assumption and ask management to speak to why the contradictory evidence is irrelevant or unreliable and should not be factored or weighted in the valuation. 
• Perform a sensitivity analysis to demonstrate how the contradictory evidence does not materially impact the valuation. 
• If contradictory evidence could materially impact the valuation, consider different scenarios and obtain additional support that further validates management’s assumption and/or invalidates the contradictory evidence. For instance, look at historical performance with the presence of the same contradictory evidence but that would still support management’s assumption. 

The extent of additional procedures needed to resolve the contradictory evidence will depend on various factors, such as the risk assessment linked to the estimate, including the fraud risk assessment, the overall evaluation of management bias, the materiality of the valuation and the correlated contradictory evidence, etc. The key here is that auditors cannot simply ignore contradictory evidence. Teams need to document the evaluation. 

Bank-Specific Considerations 

While estimates for all companies are difficult to audit, it is perhaps even more complex for banks given the allowance for loan losses (or now the allowance for credit losses) has so much tied to economic conditions. How are banks incorporating new realities such as the interest rate volatility? Or supply chain disruptions that may impact borrowers’ abilities to service loans? What about conflicting economic conditions such as declining unemployment figures coupled with two quarters of negative economic growth? Do banks have sufficient historical data from previous time periods that mimicked the current economic conditions? Depending on the source of that information, is that information accurate and complete or relevant and reliable? 

For banks, engagement teams should specifically consider the following: 

• Since the allowance is often predicated on historic loss data, how has the engagement team evaluated the accuracy and completeness of that information? Recurring audits will often use recent historical loss data pulling from systems and reports that have been tested in previous audits. However, what if the engagement team decides to look at information from the 2008 recession or from the inflationary decades such as the 1970s and 1980s? What procedures has management and/or the engagement team performed to validate the accuracy and completeness of that information? 

• How has the engagement team evaluated the relevance of information? For instance, a two or three-year historical loss lookback would not necessarily reflect the current economic conditions such as inflation, interest rates, unemployment rates, etc. Engagement teams should consider the relevant economic factors that are built into the allowance and evaluate how closely (or not) the historical loss data reflects the current economic conditions. To the extent the data is dissimilar, then management should be adjusting assumptions, such as qualitative factors (Q-factors), to incorporate these differences. 

• For banks that may not have relevant historical loss data sets, management may be forced to look for external sources to support their assumptions (i.e. look for other banks and their loss ratios). Engagement teams need to consider the relevance and reliability of this information when evaluating the assumptions. For instance, where were the loss ratios obtained? Which industries/segments were included? How similar are the loan portfolios? 

• Are inputs to qualitative factors auditable? What support is there for changes in qualitative factors? Do changes (or lack of changes) in qualitative factors correlate with macro-economic trends (i.e. did the bank adjust for unemployment and did that adjustment mirror current unemployment trends)? How did the bank determine the percentage change given the qualitative consideration? Often teams will need to look in aggregate at the impact of all changes to qualitative factors on the overall reserve. 

• In testing controls, are engagement teams considering all relevant controls that might provide comfort over accuracy and completeness of information used to derive assumptions or data used in the valuation? How precise are management review controls around the valuation and how much comfort can engagement teams leverage from the testing of these management review controls? For example, would an entity level credit committee review be sufficiently precise to detect material misstatements in estimation and calculation of allowance, or should the auditors identify and test more precise process level controls? 

One tool we often recommend to our clients who perform bank audits is to perform an anchoring exercise, or a look-back analysis performed to locate historical periods with similar economic conditions/outlooks. This requires historical information about losses reported in a time period with similar risk characteristics (e.g. Y1 of recession). Then compare the loss reserves to actual charge-offs (of the loans existed at Y1 YE) that occurred in the periods subsequent to Y1. The difference would be a good indicator of how accurate the historical loss model was and what assumptions / inputs might need to be adjusted in estimation of relevant Q-factors to fully reserve for anticipated losses in the current year. 

Key Takeaways 

Auditing estimates is never easy. As with all things audit, the nature, timing, and extent of procedures are driven by the risk assessment. Given the confluence of numerous economic uncertainties, many of which are “new” compared to the last couple of decades, the risks surrounding subjective management judgments and assumptions used in valuations will increase the overall risk linked to an estimate, including the potential for fraud risk through management bias. As auditors plan and prepare for audits, consider the following:

• Engagement teams must always evaluate the appropriateness of valuation models used in estimates. Some models may require a qualified valuation specialist to conclude. 

• Auditors need to continue to expand on testing the reasonableness of assumptions by obtaining support from management that is complete and accurate and relevant, or from other external sources (such as industry data) that is relevant and reliable. Given so many changes to economic conditions, relevance will be an important consideration for teams to document. 

• When the status quo is disrupted and the economy is in a period of significant uncertainty, auditors should consider all changes, or lack of changes, in assumptions and inputs. This is an important part of reviewing estimates for management bias from previous periods and for truly concluding on the reasonableness of current year estimates. 

• Contradictory evidence must always be considered and sufficiently documented and resolved to conclude on the overall reasonableness of accounting estimates. 

• Q-factors should be supported by reasonable estimates which are based on accurate, and relevant and reliable information, especially in times of significant uncertainties. 

While we’ll never make concrete out of Jell-O, no matter the economy, we must continue to perform robust audit procedures and build in additional considerations to account for the economic changes and uncertainty we’re experiencing today. The hope is not to make concrete, but merely a Jell-O that holds it shape (and jiggles) despite a dynamic, changing environment. 

Farkhod Ikramov, JGA Director, has over 25 years of public accounting and audit regulation experience. Most recently, Farkhod held a ten-year tenure as a PCAOB inspector. Throughout his experience there, he inspected a variety of industries, focusing the last four years on financial services, insurance and mining. His experience positions him as a passionate and practical advisor to public accounting firms, assisting leadership in the implementation of the right controls, policies and practices throughout the organization.