Architect of the Capitol: Incorporating All Leading Practices Could Improve Accuracy and Credibility of Projects' Cost Estimates
Highlights
What GAO Found
GAO's Cost Estimating and Assessment Guide (Cost Guide) defines 12 leading practices that are associated with four characteristics—comprehensive, well documented, accurate , and credible —that are important to developing high-quality, reliable project-cost estimates. Using the Cost Guide , GAO determined that the Architect of the Capitol's (AOC) cost-estimating guidance conforms to leading practices for developing estimates that are, in general, comprehensive and well-documented . However, AOC's guidance does not substantially conform to leading practices related to developing cost estimates that are accurate and credible . For example, pertaining to the credible characteristic, AOC's guidance does not require determining the confidence level of estimates or quantifying the extent to which a project's costs could vary due to changes in key assumptions. GAO found the strengths and weaknesses of AOC's guidance generally reflected in the cost estimates for AOC's Cannon House Office Building's (Cannon Building) renewal project ($753 million) and Capitol Dome's restoration project ($125 million).
- Cannon Building renewal—GAO found the estimate is substantially comprehensive, well documented, and accurate, but several factors that affect its credibility are lacking. For example, AOC's risk analysis does not allow for determination of which risks have the greatest influence on project costs and may overstate the effect of the risks.
- Capitol Dome restoration—GAO found the estimate is substantially comprehensive and well documented, but lacking key analysis that support accurate and credible estimates. For example, AOC did not use actual costs from completed phases to update its estimates and did not complete a risk and uncertainty analysis.
Overall, AOC's cost-estimating guidance may not enable fully reliable estimates because it incorporates some, but not all, leading practices. Without reliable cost estimates that convey their confidence levels, AOC's projects risk experiencing cost overruns or budget surpluses, missed deadlines, and performance shortfalls. Potential limitations in the reliability of AOC's estimates may make it difficult for Congress to make well-informed funding decisions and affect how AOC allocates resources across competing projects in its capital portfolio.
|
GAO's Assessment of AOC's Cost-Estimating Characteristics in Its Guidance for Two Project Cost Estimates |
|||
|
Characteristic |
AOC's guidance |
Cannon Office Building |
Capitol Dome |
|
Comprehensive |
● |
◕ |
◕ |
|
Well documented |
● |
◕ |
◕ |
|
Accurate |
◑ |
◕ |
◑ |
|
Credible |
◔ |
◑ |
◔ |
Source: GAO analysis of AOC documents and data.
●= Fully Meets ◕=Substantially Meets ◑=Partially Meets ◔=Minimally Meets ○= Does Not Meet
Note: A characteristic is fully met when the associated tasks of underlying leading practices are completely satisfied; substantially met when a large portion of the associated tasks are satisfied; partially met when about half of the associated tasks are satisfied; minimally met when a small portion of the associated tasks are satisfied; and not met when none of the associated tasks are satisfied.
Why GAO Did This Study
AOC is responsible for the maintenance, renovation, and new construction of the U.S. Capitol complex, which comprises more than four dozen facilities. Reliable cost estimates for projects are crucial to AOC's capital-planning and construction processes. The House Appropriations Committee report accompanying the fiscal year 2014 Legislative Branch Appropriations bill mandated that GAO review AOC's cost-estimating methodology.
This report addresses the extent to which AOC's policies and guidance for developing cost estimates adheres to leading practices. GAO analyzed AOC's cost-estimating guidance, interviewed AOC officials, and compared AOC's cost-estimating guidance and documentation and two projects' cost estimates to leading practices in GAO's Cost Guide . When most or all of the practices associated with each characteristic of a high-quality, reliable estimate are followed, GAO considers the characteristic to be “fully” or “substantially” met. When, in turn, all four characteristics are at least “fully” or “substantially” met, GAO considers a cost estimate to be reliable.
Recommendations
GAO recommends that AOC incorporate additional leading practices into its cost-estimating guidance and submit the confidence levels of project estimates to Congress. AOC concurred with the recommendations and provided context and clarification on its cost-estimating guidance and policies.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Architect of the Capitol | To improve the Architect of the Capitol's project-cost-estimating process, enhance the transparency of its related process, and allow for more informed decision making related to projects' costs, the Architect of the Capitol should, to the extent that the benefits exceed the costs, incorporate leading practices GAO identified as lacking for cost estimating in the AOC's cost-estimating guidance and policies. |
In March 2014, we found that the Architect of the Capitol's (AOC) project cost estimates may not be fully reliable because its policies and guidance for developing cost estimates did not incorporate all leading practices in cost estimating. GAO's Cost Estimating and Assessment Guide defines 12 leading practices that are associated with four characteristics, comprehensive, well documented, accurate, and credible, that are important to developing high-quality, reliable project cost estimates. In particular, we found that AOC's policies and guidance conformed to leading practices for developing estimates that are, in general, comprehensive and well documented. However, AOC's guidance did not substantially conform to leading practices related to developing cost estimates that are accurate and credible. For example, pertaining to the credible characteristic, AOC's guidance did not meet requirements of underlying leading practices to (1) quantify the extent to which a project's costs could vary due to changes in key assumptions or (2) determine the confidence level of estimates. We compared the Cannon Office Building's renewal and Capitol Dome's restoration cost estimates to GAO's leading practices. We found strengths and weaknesses that generally correspond to our assessment of AOC's overall policies and guidance for developing cost estimates. We initially determined that both estimates were comprehensive while lacking, to a varying extent, in their documentation, accuracy, and credibility. To improve AOC's project cost estimating process and allow for more informed decision making related to projects' costs, we recommended that the AOC incorporate leading practices we identified as lacking for cost estimating in its cost estimating policies and guidance. In response to our recommendation, in April 2014, AOC directed its staff to take immediate and long term actions to incorporate changes to its cost estimating process. As part of its actions, AOC revised its policies and guidance for cost estimating to incorporate the leading practices we identified as lacking. AOC now requires, among other things, quantitative sensitivity and risk assessments of estimates for applicable projects. Conducting these assessments will enable AOC to quantify how costs vary based on changes in key assumptions and determine the confidence level of estimates, thereby enhancing the estimates' credibility. By incorporating changes to its estimating process, AOC's project estimates will more closely be based on leading practices associated with high-quality and reliable estimates, and projects will be less at risk of experiencing cost overruns or budget surpluses, missed deadlines, and performance shortfalls. Furthermore, the improved reliability of AOC's estimates will help Congress to make better-informed funding decisions and improve AOC's ability to efficiently allocate resources across competing projects in its capital portfolio.
|
| Architect of the Capitol | To improve the Architect of the Capitol's project-cost-estimating process, enhance the transparency of its related process, and allow for more informed decision making related to projects' costs, the Architect of the Capitol should, to the extent that the benefits exceed the costs for ongoing and future projects, submit the confidence level derived from risk and uncertainty analyses along with budget documentation to appropriate congressional decision makers, so that Congress is aware of the range of likely costs and AOC's associated confidence levels. |
The Architect of the Capitol (AOC) is responsible for the maintenance, renovation, and new construction of the U.S. Capitol complex in Washington, D.C., which comprises more than four dozen facilities. Reliable cost estimates are crucial for supporting AOC's capital planning and construction processes. Without a reliable cost estimate, an AOC project is at risk of experiencing cost overruns, budget surpluses, missed deadlines, and performance shortfalls. In March 2014, GAO reported that the GAO Cost Estimating and Assessment Guide (Cost Guide) defines 12 leading practices that are associated with four characteristics--comprehensive, well documented, accurate, and credible--that are important to developing high-quality, reliable project cost estimates. Using the Cost Guide, GAO found that AOC's policies and guidance conformed to leading practices for developing estimates that are, in general, comprehensive and well documented. However, AOC's guidance did not substantially conform to leading practices related to developing cost estimates that are accurate and credible. In particular, pertaining to the credible characteristic, AOC's guidance did not require quantitative risk and uncertainty analyses be performed. The results of a quantitative risk and uncertainty analysis--the range of costs around a point estimate--can be useful to decision makers because it conveys the confidence level in achieving the most likely cost and informs them about cost, schedule, and technical risks. Therefore, GAO recommended that AOC submit the confidence level derived from risk and uncertainty analysis along with budget documentation to appropriate congressional decision makers so that Congress could be aware of the range of likely costs and AOC's associated confidence levels. In 2014, GAO confirmed that AOC revised its guidance to require quantitative risk assessments be performed on projects for which construction costs are expected to be $10 million or more and funding for the assessment is available. In 2017, AOC began providing Congress information on project cost estimate confidence levels in its budget documentation. In particular, to support its fiscal year 2018 budget request, AOC completed, or as of April 2017 was in the process of completing, a quantitative risk assessment on 6 of 10 projects that exceeded the $10 million threshold. AOC planned to provide to Congress the results of the assessments, including associated confidence levels, at meetings to discuss its budget. AOC's efforts to improve the credibility of its estimates by requiring quantitative risk assessments and reporting the associated confidence levels will help Congress to make better-informed funding decisions and improve AOC's ability to efficiently allocate resources across competing projects in its capital portfolio.
|