Information Technology Reform: Agencies Need to Improve Certification of Incremental Development
GAO-18-148
Published: Nov 07, 2017. Publicly Released: Nov 07, 2017.
Skip to Highlights
Highlights
What GAO Found
Agencies reported that 62 percent of major information technology (IT) software development investments were certified by the agency Chief Information Officer (CIO) for implementing adequate incremental development in fiscal year 2017, as required by the Federal IT Acquisition Reform Act (FITARA) as of August 2016. However, a number of responses for the remaining investments were incorrectly reported due to agency error. Officials from 21 of the 24 agencies in GAO's review reported that challenges hindered their ability to implement incremental development, which included: (1) inefficient governance processes; (2) procurement delays; and (3) organizational changes associated with transitioning from a traditional software methodology that takes years to deliver a product, to incremental development, which delivers products in shorter time frames. Nevertheless, agencies reported that the certification process was beneficial because they used the information from the process to assist with identifying investments that could more effectively use an incremental approach, and using lessons learned to improve the agencies' incremental processes.
As of August 2017, only 4 of the 24 agencies had clearly defined CIO incremental development certification policies and processes that contained: descriptions of the role of the CIO in the process; how the CIO's certification will be documented; and included definitions of incremental development and time frames for delivering functionality consistent with Office of Management and Budget (OMB) guidance (see figure).
Figure: Analysis of Agencies' Policies for Chief Information Officer Certification of the Adequate Use of Incremental Development in Information Technology Investments
In addition, OMB's fiscal year 2018 capital planning guidance did not establish how agency CIOs are to make explicit statements to demonstrate compliance with FITARA's incremental provisions, while the 2017 guidance did. However, OMB's fiscal year 2019 guidance provides clear direction on reporting incremental certification and is a positive step in addressing this issue.
Why GAO Did This Study
Investments in federal IT too often result in failed projects that incur cost overruns and schedule slippages. Recognizing the severity of issues related to government-wide IT management, Congress enacted federal IT acquisition reform legislation in December 2014. Among other things, the law states that OMB require in its annual IT capital planning guidance that CIOs certify that IT investments are adequately implementing incremental development.
GAO was asked to review agencies' use of incremental development. This report addresses the number of investments certified by agency CIOs as implementing adequate incremental development and any reported challenges, and whether agencies' CIO certification policies and processes were in accordance with FITARA. GAO analyzed data for major IT investments in development, as reported by 24 agencies, and identified their reported challenges and use of certification information. GAO also reviewed the 24 agencies' policies and processes for the CIO certification of incremental development and interviewed OMB staff.
Recommendations
GAO is making 19 recommendations to 17 agencies, including 3 to improve reporting accuracy and 16 to update or establish certification policies. Eleven agencies agreed with GAO's recommendations, 1 partially agreed, and 5 did not state whether they agreed or disagreed. OMB disagreed with several of GAO's conclusions, which GAO continues to believe are valid, as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Energy | The Secretary of Energy should ensure that the CIO of Energy reports major IT investment information related to incremental development accurately in accordance with OMB guidance. (Recommendation 1) |
The Department of Energy (Energy) concurred with, and has taken steps to address, our recommendation. In July 2018, a review of the IT Dashboard found that the department had updated its major IT investment information related to incremental development in accordance with OMB guidance. Current IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Energy has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions and also helped to ensure the department's efforts to improve the use of incremental development are successful.
|
| Department of Agriculture | The Secretary of Agriculture should ensure that the CIO of U.S. Department of Agriculture (USDA) reports major IT investment information related to incremental development accurately in accordance with OMB guidance. (Recommendation 2) |
The U.S. Department of Agriculture (USDA) concurred with, and has taken steps to address, our recommendation. In November 2019, a review of the IT Dashboard found that the department had updated its major IT investment information related to incremental development in accordance with OMB guidance. Current IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, USDA has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions and also helped to ensure the department's efforts to improve the use of incremental development are successful.
|
| Social Security Administration | The Commissioner of the Social Security Administration (SSA) should ensure that the CIO of SSA reports major IT investment information related to incremental development accurately in accordance with OMB guidance. (Recommendation 3) |
The Social Security Administration (SSA) concurred with and has taken steps to address, our recommendation. Specifically, in May 2018, SSA updated its guidance, Systematic, Disciplined IT Capital Planning Process at Social Security Administration, to include a description of the agency's process for reviewing project information on a quarterly basis in order to confirm the use of incremental development prior to reporting this information to OMB. In addition, a review of SSA's incremental project data on the IT Dashboard in July 2018 found that the agency had updated this information to include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, SSA has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the agency's investments in order to make decisions and also helped to ensure the agency's efforts to improve the use of incremental development are successful.
|
| Department of Housing and Urban Development | The Secretary of Housing and Urban Development (HUD) should ensure that the CIO of HUD establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 4) |
The Department of Housing and Urban Development (HUD) concurred with, and has taken steps to address, our recommendation. Specifically, in December 2018, HUD established its guidance, Agile Methodology Policy, which includes a description of the CIO's role in the certification process, a description of how CIO certification will be documented, and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. In particular, the CIO has delegated authority for certification to the Technical Review Sub-Committee, which reviews each project's use of adequate incremental development during the project life cycle phases and documents the certification of incremental development in decision memos. HUD's guidance also defines incremental development and timeframes for delivering functionality in a manner consistent with OMB guidance. By establishing guidance for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, HUD will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of the Interior | The Secretary of the Interior should ensure that the CIO of Interior updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development, consistent with OMB guidance. (Recommendation 5) |
The Department of the Interior (Interior) concurred with, and has taken steps to address, our recommendation. Specifically, in January 2018, Interior updated its guidance, Fiscal Year 2018 Information Technology Capital Planning & Investment Control Annual Requirements, which includes a description of CIO's role in the certification process and how CIO certification will be documented, and a definition of incremental development, consistent with OMB guidance. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, Interior will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of Justice | The Attorney General of the United States should ensure that the CIO of Justice establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 6) |
The Department of Justice (Justice) concurred with, and has taken steps to address, our recommendation. Specifically, in March 2019, Justice established its guidance, Component CIO Incremental Certification Procedure, which includes a description of the CIO's role in the certification process and how CIO certification will be documented, and a definition of incremental development, consistent with OMB guidance. In particular, the CIO delegates this role to the Investment Business Manager, who validates the component CIOs certification during the fall and spring budget submissions. The component CIOs are required to sign the Component CIO Certification Resource Statement, which signifies adherence to incremental development. Justice's procedures also define incremental development and timeframes for delivering functionality in a manner consistent with OMB guidance. By establishing guidance for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, Justice will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of Labor | The Secretary of Labor should ensure that the CIO of Labor updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 7) |
The Department of Labor (Labor) has taken steps to address our recommendation. Specifically, in October 2019, Labor updated its guidance, IT Capital Planning and Investment Control (CPIC) Guide: Managing IT Investments, which includes a description of CIO's role in the certification process and how CIO certification will be documented. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, Labor will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of State | The Secretary of State should ensure that the CIO of State updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 8) |
The Department of State (State) has taken steps to address our recommendation. Specifically, in November 2017, State updated its guidance, 5 Foreign Affairs Manual 690 Incremental Development Policy, to include a description of the CIO's role in the certification process and a definition of incremental development and timeframes for delivering functionality, consistent with OMB guidance. In addition, State updated its guidance, 5 Foreign Affairs Manual 914 Responsibilities to include a description of how CIO certification will be documented. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, State will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of Agriculture | The Secretary of Agriculture should ensure that the CIO of USDA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 9) |
The U.S. Department of Agriculture (USDA) concurred with, and has taken steps to address, our recommendation. Specifically, in May 2021, USDA established a departmental regulation on IT capital planning and investment control which includes roles and responsibilities for the department CIO or designated mission area assistant CIO to certify the incremental development of its IT investments. In addition, the department updated its Integrated IT Governance Framework to include a description of how CIO certification would be documented by the CIO or designated assistant CIO using a certification email or memo. The department also updated its capital planning and investment control guidance to include a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. By establishing guidance for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, USDA will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of Veterans Affairs | The Secretary of Veterans Affairs (VA) should ensure that the CIO of VA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 10) |
The Department of Veterans Affairs (VA) concurred with our recommendation and, and has taken steps to address, our recommendation. In September 2022, we confirmed that VA had updated its guidance on the department's investment review board as well as guidance on its system used to manage and track VA agile development projects. The updated guidance includes a description of the CIO's role in the certification process and how CIO certification will be documented. In particular, the CIO reviews the status of all IT investments using incremental development on a weekly basis and during scheduled investment review board meetings. The CIO documents the certification of investments' adequate use of incremental development as part of the annual budget submission to the Office of Management and Budget. By updating its guidance for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, VA will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| Environmental Protection Agency | The Administrator of the Environmental Protection Agency (EPA) should ensure that the CIO of EPA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 11) |
The Environmental Protection Agency (EPA) concurred with, and has taken steps to address, our recommendation. Specifically, in February 2022, the EPA Chief Information Officer (CIO) finalized and signed two agency directives: Capital Planning and Investment Control Program Policy and Capital Planning and Investment Control Procedures. These directives include a description of the CIO's role in the certification process and how CIO certification will be documented, as well as a definition of incremental development and timeframes for delivering functionality, consistent with OMB guidance. In particular, the CIO performs a review of all IT investments as part of the agency's annual FITARA IT portfolio review process. This includes ensuring all acquisition strategies and acquisition plans that include IT apply adequate incremental development principles. The CIO documents the final certification of all data, including data related to incremental development, in the agency's IT portfolio management tool. EPA's directive also defines incremental development and timeframes for delivering functionality in a manner consistent with OMB guidance. By establishing an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, EPA will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| General Services Administration | The Administrator of the General Services Administration (GSA) should ensure that the CIO of GSA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 12) |
The General Services Administration (GSA) concurred, and has taken steps to address, our recommendation. Specifically, in June 2018, GSA updated its guidance, GSA IT Guide to Capital Planning and Investment Control, to include a description of CIO's role in the certification process and how CIO certification will be documented. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, GSA will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| National Aeronautics and Space Administration | The Administrator of the National Aeronautics and Space Administration (NASA) should ensure that the CIO of NASA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 13) |
The National Aeronautics and Space Administration (NASA) concurred, and has taken steps to address, our recommendation. Specifically, in September 2020, NASA updated its guidance, NPR 7120.7A NASA Information Technology Program and Project Management Requirements to include a description of the CIO's role and how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, NASA will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| National Science Foundation | The Director of the National Science Foundation (NSF) should ensure that the CIO of NSF updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 14) |
The National Science Foundation (NSF) has taken action to address our recommendation. Specifically, in August 2018, NSF issued its guidance, CIO Incremental Development Policy, that includes a description of the CIO's role in the certification process, a description of how CIO certification will be documented, and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. By updating its guidance for the CIO's certification of major IT investment' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, NSF will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| Nuclear Regulatory Commission | The Chairman of the Nuclear Regulatory Commission (NRC) should ensure that the CIO of NRC establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 15) |
The U.S. Nuclear Regulatory Commission (NRC) has taken steps to address our recommendation. Specifically, in December 2017, NRC updated its guidance, Capital Planning and Investment Control Policy and Overview, to include a description of the CIO's role in the certification process and how CIO certification will be documented. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, NRC will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| Office of Personnel Management | The Director of the Office of Personnel Management (OPM) should ensure that the CIO of OPM updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 16) |
The Office of Personnel Management (OPM) concurred with our recommendation and stated that the agency would update its guidance to address our recommendation. In February 2024, an OPM official reported that the agency was working to incorporate incremental development guidance into its revised IT Portfolio Management policy, which is expected to be finalized by July 2024. In addition, OPM is drafting an IT Portfolio Management Guide that will provide more detailed information regarding these processes and is expected by October 2024. We will continue to monitor OPM's progress on these efforts.
|
| Small Business Administration | The Administrator of the Small Business Administration (SBA) should ensure that the CIO of SBA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 17) |
The Small Business Administration (SBA) concurred with, and has taken steps to address our recommendation. Specifically, in January 2020, SBA updated its guidance, SBA Information Technology and Capital Planning and Investment Control Standard Operating Procedures, to include a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality. By establishing a policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, SBA will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| Social Security Administration | The Commissioner of the Social Security Administration should ensure that the CIO of SSA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 18) |
The Social Security Administration (SSA) concurred with and has taken steps to address, our recommendation. Specifically, in May 2018, SSA updated its guidance, Systematic, Disciplined IT Capital Planning Process at Social Security Administration, to include a description of the CIO's role in the certification process and how CIO certification will be documented. By updating its policy for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, SSA will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
| U.S. Agency for International Development | The Administrator of the U.S. Agency for International Development (USAID) should ensure that the CIO of USAID establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 19) |
The U.S. Agency for International Development (USAID) has taken steps to address our recommendation. Specifically, in May 2019, USAID established its guidance, Automated Directives System Chapter 509: Management and Oversight of Agency Information Technology Resources, which includes a description of the CIO's role in the certification process, a description of how CIO certification will be documented, and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. In particular, the CIO receives monthly reports on the status of delivered incremental functionality for each of the agency's major IT investments and uses this information to certify the adequate use of incremental development in the agency's annual IT resource statements. In addition, USAID's guidance also defines incremental development and timeframes for delivering functionality in a manner consistent with OMB guidance. By establishing guidance for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, USAID will be able to help ensure that the agency is adequately implementing and benefiting from incremental development practices.
|
Full Report
GAO Contacts
David A. Powner
Topics
Chief information officersCost growthCost overrunsFederal agenciesFederal procurementInformation technologyIT investmentsIT acquisitionsPolicies and proceduresReporting requirementsSchedule slippagesSoftware development