This is the accessible text file for GAO report number GAO/OIG-10-4 entitled 'Performance Measure: GAO Had a Reasonable Basis for Reporting Its Largest Financial Benefits for Fiscal Year 2009' which was released on May 21, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Office of the Inspector General: United States Government Accountability Office: GAO: May 2010: Performance Measure: GAO Had a Reasonable Basis for Reporting Its Largest Financial Benefits for Fiscal Year 2009: GAO/OIG-10-4: Office of the Inspector General: United States Government Accountability Office: Memorandum: Date: May 21, 2010: To: Acting Comptroller General Gene L. Dodaro: From: [Signed by] Inspector General Frances Garcia: Subject: Performance Measure: GAO Had a Reasonable Basis for Reporting Its Largest Financial Benefits for Fiscal Year 2009: We have completed our audit of GAO's fiscal year 2009 financial benefits of $1 billion or more. GAO annually assesses its performance using a combination of performance measures to help determine how well it is meeting the needs of the Congress and maximizing its value to the public. One of these measures—financial benefits—is an estimate of the federal monetary effect of agency or congressional actions or changes in agency practices that are directly related to GAO's work. Based on our audit of 10 GAO accomplishment reports[Footnote 1] that claim financial benefits valued at $1 billion or more, we determined that GAO had a reasonable basis to claim financial benefits of $24.95 billion cited in the reports. These large dollar value accomplishment reports accounted for 58 percent of GAO's $43 billion in total financial benefits claimed for fiscal year 2009. As a result of our work, GAO took additional steps to better document these benefits and made adjustments in amounts claimed before reporting the final benefit total in its fiscal year 2009 Performance and Accountability Report. [Footnote 2] In addition, our audit identified an opportunity to strengthen GAO's process for reporting financial benefits by incorporating more technical expertise in calculating benefit amounts. In January 2010, GAO announced new procedures for expanding the role of GAO's Center for Economics to assist the mission teams in calculating financial benefits for accomplishment reports if they are reported to be $500 million or more. Because of the adjustments made to the amounts claimed for the accomplishments reports we audited, we are recommending that GAO explore additional ways to assist teams in developing accomplishment reports that meet GAO's policy requirements. GAO concurred with our recommendation. The primary objective of our audit was to determine whether the 10 accomplishment reports that originally claimed financial benefits of $1 billion or more and their supporting documentation met policy requirements and, thus, whether GAO had a reasonable basis to claim these financial benefits. In addition, we examined GAO's compliance with key internal controls[Footnote 3] for reviewing and approving these large dollar value accomplishment reports and whether these controls provided reasonable assurance for claiming financial benefits. (See attachment I for a detailed discussion of our methodology.) Background: GAO's findings and recommendations may produce measurable financial benefits for the federal government when the Congress or agencies act on them to (1) reduce annual operating costs or the costs of multiyear projects or entitlements or (2) increase revenues from asset sales or changes in tax law or user fees. The funds made available in response to GAO's work may be used to reduce government expenditures or may be reallocated to other areas. GAO uses financial benefit information, in part, to indicate a return on investment for the agency. For example, in its fiscal year 2009 Performance and Accountability Report, GAO noted that its $43 billion in financial benefits represented a return of $80 for every $1 invested in the agency. To record a financial benefit, GAO staff prepare an accomplishment report according to guidance provided in the GAO Policy Manual and in any supplemental guidance provided by the Chief Quality Officer, who manages the Quality and Continuous Improvement (QCI) office.[Footnote 4] This guidance specifies that certain conditions be met, including that the agency or congressional action for which the financial benefit is claimed must be either completed or substantially completed and must have been taken within a certain time frame of filing the accomplishment report (generally within the past 2 fiscal years). Also, it must be shown that GAO influenced the action—that is, there should be a cause-and-effect relationship between GAO's work and the action taken that resulted in the financial benefit. Another condition specifies the number of years' worth of benefits that can be claimed according to benefit type. GAO also prefers that, if possible, estimates of financial benefits be from non-GAO parties, such as the affected agency, congressional committees (such as the Joint Committee on Taxation), or the Congressional Budget Office. However, third-party estimates may not be available, thus requiring GAO to develop its own estimates, often using agency-provided data. Internal controls are an important element of GAO's policy guidance and include an extensive review and approval process that helps to ensure the reliability of the financial benefits claimed. For example, at the mission team level, each accomplishment report for financial benefits is documented to show that all policies have been met, and a senior executive (referred to as a director) confirms compliance through a supervisory review. Then, another GAO staff member not involved in the work that led to the accomplishment report independently checks the supporting statements of facts, figures, and dates for accuracy and verifies that the accomplishment report is supported by the evidence (referencing). Additionally, the managing director of the mission team conducts a supervisory review before the accomplishment report is forwarded to the Chief Quality Officer for final agency approval. In accordance with new procedures adopted for fiscal year 2009 financial benefits, the managing director signs the accomplishment report and prepares a separate memorandum addressed to the Chief Quality Officer attesting that the proposed accomplishment report meets GAO's standards. In addition, for fiscal year 2009 accomplishment reports that claim financial benefits between $500 million and $1 billion, the Chief Quality Officer required a second review (in addition to QCI's internal review). After completing a review, the second reviewer provides the Chief Quality Officer a memorandum stating that the proposed accomplishment report meets or does not meet GAO's standards. The Chief Quality Officer approves all financial accomplishment reports claiming financial benefits of $100 million or more. Also, for fiscal year 2009, the Acting Comptroller General requested that the Office of the Inspector General (OIG) audit all accomplishment reports valued at $1 billion or more after they have been approved by the Chief Quality Officer but before the agency published its final benefit amount in its fiscal year 2009 Performance and Accountability Report. Amounts for Financial Benefits We Reviewed Are Reasonable, but Opportunities Exist to Improve the Accomplishment Reporting Process: We determined that GAO had a reasonable basis to claim $24,947,909,935 in financial benefits for the 10 accomplishment reports of $1 billion or more we audited. Based on our work, mission teams made adjustments to 6 accomplishment reports that both increased (by a total of $1.19 billion) and decreased (by a total of $2.45 billion) the amounts originally claimed. Adjustments ranged from a $281,000 decrease to a $1.19 billion increase. Therefore, while GAO originally planned to claim a total of $26.21 billion for these 10 accomplishment reports, the agency ultimately claimed and reported a total of $24.95 billion for 9 accomplishment reports. As discussed below, GAO plans to claim the financial benefits for the tenth accomplishment report in fiscal year 2010. In the six cases where adjustments in benefit amounts were made, the mission teams reconsidered the amounts originally claimed in light of specific questions we raised and additional efforts taken by the teams to respond to those questions. For example, a mission team decided to defer 1 of the 10 accomplishment reports we audited—which originally claimed financial benefits of $1.15 billion—to ensure that the agency's action in restructuring the program was final and not subject to further change. GAO's policy is to claim benefits once an action is completed or substantially completed, and in this case, the agency revamped the program GAO reviewed for the second time since 2008. Additionally, the agency announced its latest decision to restructure the program after the President's budget went to the Congress and after an oversight committee had deliberated on the agency's budget. Therefore, the extent of congressional review of and concurrence with the agency was not clear. Upon reconsideration, the mission team decided to postpone completing the accomplishment report until next year, and the entire benefit amount was removed from the total amount of financial benefits claimed and reported for fiscal year 2009. In the five other cases where adjustments in benefit estimates occurred, the teams took additional steps to comply with specific agency policies relating to documentation of benefits and to avoid any potential overlap in claimed benefits. * In two of these cases, we questioned whether the mission teams had calculated benefits in accordance with GAO's policy that generally requires the first year in which the changes are implemented to be included in the time frame used to calculate the claimed amount. In one of these cases, in which GAO's work resulted in a cost reduction to a multiyear project, the mission team claimed 5 years of financial benefits beginning with fiscal year 2009. However, supporting documentation indicated that these benefits actually began to accrue during fiscal year 2008. The team adjusted its benefit estimate accordingly and, as a result, reduced its originally claimed amount by $728.7 million to $5.07 billion. In the other case, we raised a question about when the financial benefits actually started to accrue. Upon further review, the team determined the benefits began to accrue much earlier than originally claimed. Also, while GAO's policy only allows mission teams to claim as benefits the difference in the amounts between what the agency was already doing (generally referred to as the "baseline") and what actually happened after GAO's involvement, the team had not subtracted the baseline amount from each year of claimed benefit amounts. After reconsideration, the team recalculated the amount of benefits, resulting in an increase of $1.19 billion to $3.97 billion. These two cases illustrate the importance of correctly establishing the year benefits begin to accrue, as well as the baseline amount. * For a third accomplishment report, we identified two fiscal year 2008 accomplishment reports that appeared to claim portions of the benefits being claimed in this fiscal year 2009 accomplishment report. To avoid any potential for overlap, the mission team decided to reduce its claimed benefit amount by $466 million to $5.25 billion. * In the fourth case, we questioned whether the documentation for the accomplishment report adequately supported the financial benefits estimate. In response, the team obtained additional written documentation, which resulted in a $106 million decrease in benefits to $1.27 billion. * In the last case, the mission team had inadvertently rounded up the benefit amount rather than reporting the full amount. Once this correction was made, the claimed benefit was reduced by $281,000 to $2.89 billion. In all six cases, the amounts of financial benefits claimed were adjusted prior to being reported in GAO's fiscal year 2009 Performance and Accountability Report. Our audit showed that GAO's mission teams generally documented compliance with the key elements of GAO's internal control process. For example, the files contained documentation that the referencing process was completed, including supervisory reviews. They also contained documentation of the newly required managing director's memorandum attesting that GAO's standards for claiming these financial benefits were met. In addition, the accomplishment reports were reviewed and signed by GAO's Chief Quality Officer, signifying final agency approval. At the same time, our audit indicated other factors may have contributed to claimed amounts being adjusted as a result of our work. One is that GAO's policies for claiming financial benefits are complex and require judgments. Another contributing factor may be that the mission teams were not seeking out advice from GAO's in-house economists—for example, we identified only one instance where a mission team consulted with GAO's Applied Research and Methodology (ARM) team's Center for Economics prior to final agency approval. In the one case where consultation occurred before our audit, QCI staff asked the team to seek assistance from both GAO's Center for Economics and the agency. As a result, the team recalculated the amount to be claimed, reducing the benefit by $964 million. As part of our audit procedures, we directed technical questions regarding benefit calculations to GAO's Center for Economics, and the center's Director was able to assist one of the mission teams in better documenting financial benefits. In this instance, the Director's review of this accomplishment report raised questions about the team's methodology and the reasonableness of the 5-year estimate. He identified technical issues regarding the comparison of the amount of benefits that would be achieved before and after GAO's involvement, particularly as it might affect the net present value calculation. [Footnote 5] With the additional data provided by the team, the center's Director determined that the team had a sound methodological basis for its benefit estimate, and no adjustment in the amount was required. Although the amount did not change in this case, the ARM review provided additional assurance that the benefit estimate was reasonable. In a January 28, 2010, memorandum to managing directors, GAO's Chief Quality Officer introduced new procedures for approving financial accomplishment reports beginning in fiscal year 2010. (See attachment II for a copy of this memorandum.) These new procedures will require teams to consult with the agency's Center for Economics on the calculation of amounts for accomplishment reports that claim financial benefits of $500 million or more. In this memo, the Chief Quality Officer noted that these procedures would formalize a good practice that several teams currently use to help document financial benefits. For reporting financial benefits of $500 million or more, the new procedures require teams to provide the Center for Economics with documentation supporting (1) the baseline amount from which the financial benefit is calculated, (2) the estimated financial benefit, and (3) the calculation of present value. The center will review the documentation, work with the teams if needed, and sign off on the estimated financial benefit amount. Further, the managing directors, in their memo attesting that an accomplishment report meets GAO's standards, will now include a statement that the Center for Economics concurs with the financial benefit amount. We believe that implementation of these newly announced procedures is a positive step toward strengthening GAO's internal controls for accomplishment reports and could lead to better supported and more reliable estimates of benefits. Given the number and dollar value of adjustments that were made to the accomplishment reports we audited, we believe the mission teams could benefit from additional assistance in developing accomplishment reports. As previously mentioned, GAO's policies for claiming financial benefits are complex, and we believe these policies may not be fully understood by the mission teams. Therefore, GAO may want to focus more attention on assisting mission teams in implementing these policies, as well as in complying with internal controls. To provide added assurance that GAO's benefit claims are reasonable, GAO management asked the OIG to audit GAO's largest financial accomplishment reports and to provide our preliminary results on benefit amounts before publication of GAO's fiscal year 2009 Performance and Accountability Report. GAO management is committed to ensuring the reliability of its financial benefits performance measure. Because an audit cannot be part of an agency's controls, we believe GAO should examine how it can better assist teams in a way so as to make these controls more efficient and effective for producing reliable estimates of financial benefits. Conclusions: Financial benefits are one of the key measures of GAO's performance. We determined that GAO had a reasonable basis to claim financial benefits of $24.95 billion for its large dollar value accomplishment reports for fiscal year 2009. We also identified opportunities for GAO to strengthen its accomplishment report process, for example, by establishing a greater role for GAO's Center for Economics in reviewing estimates of financial benefits. To its credit, GAO has already taken action to create a review process that includes GAO's Center for Economics. Moreover, we believe GAO may be able to develop a more effective and efficient system of internal controls over large dollar value accomplishment reports by focusing more attention on the mission teams in their development of large dollar value accomplishment reports. Such assistance would increase management's confidence in the reliability of the agency's financial benefit claims. Recommendation: To provide added assurance that GAO's financial benefits performance measure reflects a reasonable estimate of benefits and to provide a more efficient and effective system of internal controls over large dollar value accomplishment reports, we are recommending that GAO's Acting Comptroller General direct the Chief Quality Officer to further analyze the issues we identified and take appropriate steps to assist mission teams in complying with agency policies to achieve greater reliability in the amount of financial benefits GAO reports. Agency Comments: The Inspector General provided GAO with a draft of this report for review and comment. In written comments, which are included in attachment DI, GAO concurred with our recommendation and indicated that it would consider ways to further assist mission teams in reporting on performance measures and described some actions already under way that the agency believes addresses our recommendation. GAO also said it would continue to look into this issue. GAO provided technical comments that we incorporated, as appropriate. Actions taken in response to our recommendation should be reported to my office within 60 days. We are sending copies of this report to the other members of GAO's Executive Committee (the Chief Administrative Officer and the Acting General Counsel), the Managing Director of Quality and Continuous Improvement, and GAO's Audit Advisory Committee. The report also is available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-5748 or garciaf@gao.gov. Key contributors to this report were Cathy Helm (Deputy Inspector General), Alan Belkin (General Counsel), Wendy Jaffe, Ben Ritt, and Keith Steck. Attachments: [End of section] Attachment I: Scope and Methodology: The primary objective of our audit was to determine whether the 10 accomplishment reports that originally claimed financial benefits of $1 billion or more and their supporting documentation met policy requirements and, thus, whether GAO had a reasonable basis to claim these financial benefits. For each accomplishment report, we compared the documentary support for the claimed financial benefits against the GAO Policy Manual requirements, focusing primarily on requirements that would affect the benefit amount. To assist in our analysis, we used a standardized worksheet linked to specific GAO Policy Manual requirements. Also, to ensure consistency in our analysis, we had one auditor conduct a primary review of the supporting documentation for each accomplishment report and a second auditor go over the results of the first auditor's work and raise any questions and concerns. After agreement was reached by the two auditors for each report, we discussed the results of our review with the appropriate mission team officials, as needed. Also, as needed, we obtained additional support from GAO's Center for Economics, to allow us to complete our review and make a determination as to the amount of financial benefits that we believed was reasonable for GAO to claim. We provided the Acting Comptroller General the results of our audit of GAO's fiscal year 2009 financial benefits to assist in his determination of the amount of benefits GAO would claim in its fiscal year 2009 Performance and Accountability Report. In addition, we examined GAO's compliance with key internal controls for reviewing and approving these large dollar value accomplishment reports and whether these controls provided reasonable assurance for claiming financial benefits. We compared the documentary support for the claimed financial benefits against GAO's internal control requirements for accomplishment reports, such as referencing and various levels of management approval. To determine if the controls ensured the reliability of financial benefit claims, we compared the results of this analysis (i.e., whether there was documentation to show the controls were present) with the results of our analysis relating to benefit amounts (i.e., whether there were adjustments made to benefit amounts). We conducted this performance audit from September 2009 to May 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Attachment II: GAO: Memorandum: Date: January 28, 2010: To: GAO Managing Directors: From: [Signed by] Chief Quality Officer - Timothy P. Bowling: Subject: Procedures for Approving Financial Accomplishment Reports: In FY 2009 we introduced and tested some changes in procedures for approving financial accomplishment reports. These changes, as outlined in my memo of August 4, 2009, were intended to strengthen and better document the underpinnings of financial benefits resulting from GAO work. Based on our experience using these procedures, we have decided to formalize a good practice that several teams currently use to help document financial benefits, i.e., consulting with ARM's Center for Economics. Effective immediately, teams should consult with the Center for Economics on the calculation of the financial benefit amounts for accomplishments of $500 million or more as described above. For financial accomplishments of $500 million or more, teams should provide the Center for Economics with the documentation supporting (1) the baseline amount from which the financial benefit is calculated, (2) the estimated financial benefit, and (3) the calculation of present value. The Center will review the documentation, work with the teams if needed, and sign off on the estimated financial benefit amount. Tom McCool is the Center for Economics' contact for this step. Consistent with our current process, once the Managing Director reviews and signs the proposed accomplishment report, the report along with the associated workpapers and the attestation memorandum are forwarded to QCI. The attestation memo should now include a statement that ARM's Center for Economics concurs with the financial benefit amount. If you have questions, please call me (x27680), Steve Backhus (x27111), or Chip Breen (x29035). [End of section[ Attachment III: Comments from GAO: GA0: Memorandum: Date: May 12, 2010: To: Inspector General —Frances Garcia: From: [Signed by] Chief Quality Officer - Timothy P. Bowling: Subject: Agency Comments on OIG's Financial Benefits Performance Measure Report: Thank you for the opportunity to review your draft report, Performance Measure: GAO Had a Reasonable Basis for Reporting Its Largest Financial Benefits for Fiscal Year 2009 (GAO/OIG-10-4), which evaluates GAO's financial benefits of $1 billion or more for FY 2009. We are pleased that you found that GAO had a reasonable basis for reporting the financial benefits in your review. We are always looking for opportunities to refine our performance measures and will consider ways to further assist mission teams in this area. In concert with your recommendation, QCI has already initiated the following actions: 1. Teams are now required to consult with the Center for Economics on the calculation of financial benefits of $500 million or more, and; 2. All proposed accomplishment reports over $100 million will receive a second review by independent reviewers. We will continue to look into this issue and keep you informed of any further developments in GAO's approach to measuring financial benefits. cc: Gene Dodaro, Acting Comptroller General: Sallyanne Harper, Chief Administrative Officer: Lynn Gibson, Acting General Counsel: Pam LaRue, Controller: Pam Frere, CASO/PAO Director: Thank you for the opportunity to review your draft report, Performance Measure: GAO Had a Reasonable Basis for Reporting Its Largest Financial Benefits for Piscal Year 2009 (GAO/OIG-10-4), which evaluates GAO's financial benefits of $1 billion or more for FY 2009. We are pleased that you found that GAO had a reasonable basis for reporting the financial benefits in your review. We are always looking for opportunities to refine our performance measures and will consider ways to further assist mission teams in this area In concert with your recommendation, QCI has already initiated the following actions: 1. Teams are now required to consult with the Center for Economics on the calculation of financial benefits of $500 million or more, and 2. All proposed accomplishment reports over $100 million will receive a second review by independent reviewers. [End of section] Footnotes: [1] Accomplishment reports are used to document the amounts claimed as financial benefits and are linked to specific recommendations or actions taken in response to GAO's work. [2] GAO, Performance and Accountability Report-—Fiscal Year 2009, [hyperlink, http://www.gao.gov/products/GAO-10-234SP] (Washington, D.C.: Nov. 13, 2009). [3] Internal control comprises the plans, policies, methods, and procedures used to meet the organization's mission, goals, and objectives. Internal control includes the processes and procedures for planning, organizing, directing, and controlling program operations and management's system for measuring, reporting, and monitoring program performance. [4] QCI assists the Comptroller General and the Chief Operating Officer in ensuring that GAO's products and services are of the highest quality. [5] Present value is the discounted value of a payment or stream of payments to be received or paid in the future, taking into consideration a specific interest or discount rate. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: