Information Technology: Agencies Need to Strengthen Oversight of Billions of Dollars in Operations and Maintenance Investments
Highlights
What GAO Found
Federal agency assessments of the performance of information technology (IT) investments in operations and maintenance (O&M)--commonly referred to as operational analyses (OAs)--vary significantly. Office of Management and Budget (OMB) guidance calls for agencies to develop an OA policy and perform such analyses annually to ensure steady state investments continue to meet agency needs. The guidance also includes 17 key factors (addressing areas such as cost, schedule, customer satisfaction, and innovation) that are to be assessed. The five agencies GAO reviewed varied in the extent to which they carried out these tasks.
The Departments of Homeland Security (DHS) and Health and Human Services (HHS) developed a policy which included all OMB assessment factors and performed OAs. However, they did not include all investments and key factors. In particular, DHS analyzed 16 of its 44 steady state investments, meaning 28 investments with annual budgets totaling $1 billion were not analyzed; HHS analyzed 7 of its 8 steady state investments. For OAs performed by DHS and HHS, both fully addressed approximately half of the key factors. With regard to the DHS and HHS investments that did not undergo an analysis or were not fully assessed against key factors, agency officials said this was due in part to program officials inconsistently applying OMB and agency guidance in conducting OAs and that OAs were not a priority. DHS and HHS have recently begun to take action to make OAs a priority and improve consistency. For example, DHS's chief information officer recently issued a directive requiring all steady state IT investments to conduct analyses annually and plans to assign staff in the office of the chief information officer to review them to ensure they are complete.
The Departments of Defense (DOD), the Treasury (Treasury), and Veterans Affairs (VA) did not develop a policy and did not perform analyses on their 23 major steady state investments with annual budgets totaling $2.1 billion. DOD and VA officials said that they did not have a policy or perform analyses because they measure the performance of steady state investments via development of plans and business cases submitted to OMB (called exhibit 300s) as part of the budget process. While these can be helpful in managing performance and do address aspects of the 17 key factors, they do not address 11 of the key factors. For example, the exhibit 300 does not address reviewing strategic business results and making recommendations to modify or terminate an investment. Treasury officials stated that they did not to perform OAs in 2011 and instead decided to use the time to develop a policy. However, the officials stated that they did not anticipate the policy to be completed until the end of this calendar year.
Overall, these five agencies have steady state investments with a fiscal year 2011 budget of over $3 billion which have not undergone needed analyses. While OMB requires agencies to perform OAs, its existing guidance does not provide mechanisms that ensure the OAs are completed and allow public transparency into the results of the assessments. Until agencies address these shortcomings, there is increased risk that these agencies will not know whether the multibillion dollar investments fully meet their intended objectives.
Why GAO Did This Study
Of the $79 billion federal agencies budgeted for IT in 2011, $54 billion (about 69 percent) was reported to have been spent on the operations and maintenance of existing legacy IT systems--commonly referred to as steady state investments. Given the size and magnitude of these investments, it is essential that agencies effectively manage them to ensure they continue to meet agency needs. As such, OMB directs agencies to periodically examine the performance of such investments against, among other things, established cost, schedule, and performance goals by performing annual OAs.
GAO was asked to determine the extent to which federal agencies analyze the performance of steady state investments in accordance with OMB guidance. To do so, GAO (1) selected five agencies, DOD, HHS, DHS, Treasury, and VA, which reported spending $4.6 billion annually on major steady state investments; and (2) and compared their fiscal year 2011 OAs to OMB criteria. GAO also analyzed documents and interviewed agency officials regarding any variances as well as their causes.
Recommendations
GAO is recommending that DOD, Treasury, and VA develop an OA policy and conduct annual OAs; and that DHS and HHS ensure OAs are being performed for all investments and that all factors are fully assessed. GAO is also recommending that OMB revise its guidance to incorporate mechanisms to ensure OAs are completed and provide for increased transparency. In commenting on a draft of this report, OMB and the five agencies GAO reviewed agreed with its content and recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To ensure that major steady state IT investments are being adequately analyzed, the Secretaries of Defense, Veterans Affairs, and the Treasury should direct appropriate officials to develop an OA policy, annually perform OAs on all investments, and ensure the assessments include all key factors. |
In January 2015, DOD issued Instruction 5000.02, which addressed more than half of the related key factors identified in OMB's guidance. While the department did not perform operational analyses for its major IT investments in operations and maintenance, in November 2018, GAO was able to identify and review other documents which demonstrated that DOD was performing equivalent annual assessments. These documents included business cases and portions of the agency's budget justification. These annually reported documents generally addressed 11 of the 17 key factors required by OMB to be in an operational analysis. Thus, DOD is better positioned to more effectively measure the performance of its information systems and more likely to ensure that these multibillion dollar investments fully meet their intended delivery and effectiveness objectives.
|
| Department of the Treasury |
Priority Rec.
To ensure that major steady state IT investments are being adequately analyzed, the Secretaries of Defense, Veterans Affairs, and the Treasury should direct appropriate officials to develop an OA policy, annually perform OAs on all investments, and ensure the assessments include all key factors. |
The Department of Treasury has developed an operational analysis policy and is currently performing operational analyzes on their major IT investments in steady state. In September 2017, the department provided operational analyses for its fiscal year 2015 investments, which addressed a majority of the evaluation factors.
|
| Department of Veterans Affairs | To ensure that major steady state IT investments are being adequately analyzed, the Secretaries of Defense, Veterans Affairs, and the Treasury should direct appropriate officials to develop an OA policy, annually perform OAs on all investments, and ensure the assessments include all key factors. |
In July 2018, the Department of Veterans Affairs provided its operational analyses to GAO for its major information technology investments for fiscal years 2013, 2014, and 2016. These operational analyses addressed a majority of the key factors identified in the Office of Management and Budget guidance. In September 2021, the department provided GAO with its finalized operational analysis policy, dated August 2021. The policy addressed most of the related key factors identified in OMB's guidance. By implementing this recommendation, the department is in a better position to more effectively measure the performance of its information systems, and, therefore, more likely to ensure the delivery and effectiveness that these multibillion dollar investments fully meet their intended objectives.
|
| Department of Homeland Security | The Secretaries of Homeland Security and Health and Human Services should direct their Chief Information Officers to ensure OAs are performed annually on all major steady state investments and the assessments include all key factors. |
The Department of Homeland Security has updated their operational analysis policy and is currently performing operational analyzes on their major IT investments in steady state. In September 2017, the department provided operational analyses for its fiscal year 2016 investments, which addressed a majority of the evaluation factors.
|
| Department of Health and Human Services | The Secretaries of Homeland Security and Health and Human Services should direct their Chief Information Officers to ensure OAs are performed annually on all major steady state investments and the assessments include all key factors. |
In May 2015, the Department of Health and Human Services issued an updated operational analysis policy that addressed all the evaluation factors. In January 2022, the department provided GAO with several operational analyses for fiscal year 2020 investments that addressed a majority of the key factors identified in the Office of Management and Budget guidance. By implementing this recommendation, the department is in a better position to more effectively measure the performance of its information systems, and, therefore, more likely to ensure the delivery and effectiveness that these multibillion dollar investments fully meet their intended objectives.
|
| Office of Management and Budget | To ensure that OA policies are developed and that annual analyses are conducted and to promote transparency into the results of these analyses, the Director of OMB should revise existing guidance to include directing agencies to report on the IT Dashboard the results from the OAs of their steady state investments. |
The Office of Management and Budget (OMB) issued guidance to the agencies directing them, starting in fiscal year 2015, to report operational analysis (OA) results, along with their annual budget request documentation, to OMB. In addition, OMB established a new process for how agencies are to provide OA information to OMB, which OMB officials said was effective as of March 2014. While OMB has taken these steps to improve OA reporting and dissemination, as of November 2015, it did not plan to make OAs publicly available, asserting that information contained in OAs can be sensitive in nature. We agree that sensitive information should not be made publicly available.
|