IT Systems Annual Assessment: DOD Needs to Improve Performance Reporting and Development Planning
Fast Facts
The Department of Defense requested about $45.2 billion for FY 2023 for its unclassified information technology, such as communications and business systems.
We reviewed the performance of DOD's 25 major IT business programs, including their software development and cybersecurity practices. Of the 25 programs, we found:
13 didn't fully report on the extent to which they achieved their goals, including 3 programs that didn't identify required metrics
11 didn't have approved plans for user training and deployment to help implement the systems
6 didn't have approved cybersecurity strategies
Our recommendations address these and other issues.
Highlights
What GAO Found
According to the Department of Defense's (DOD) fiscal year (FY) 2023 submission to the Federal IT Dashboard, DOD planned to spend about $9 billion on its portfolio of 25 major IT business programs and about $31 billion on its 723 standard IT infrastructure investments from FY 2021 through FY 2023. These two areas accounted for 30 percent of total planned spending on the department's unclassified IT portfolio (see figure).
The Department of Defense's Major IT Programs and IT Infrastructure Accounted for 30% of Total Planned Spending on Its Unclassified IT for Fiscal Years 2021–2023
Sixteen of the 25 major IT business programs reported cost or schedule changes since January 2021, including 12 that had cost increases ranging from $43 thousand to $194 million (a median of $4.6 million); 12 had schedule delays ranging from 3 to 33 months (a median of 24 months). Program officials attributed the changes to factors such as new requirements and unanticipated technical complexities.
Programs also reported performance data. As of January 2023, 22 of the 25 programs identified at least the minimum required number of operational performance metrics, consistent with Office of Management and Budget (OMB) guidance. However, the other three programs did not identify the minimum required metrics, including two that did not identify any metrics data. Additionally, 13 programs did not fully report on the extent to which they achieved their targets. By not ensuring that programs fully identify and report required performance metrics, DOD limits program accountability and its own ability to effectively oversee performance.
As of February 2023, officials for the eight programs that we identified as actively developing software reported using recommended iterative development approaches and practices that can limit risks of adverse cost and schedule outcomes. In addition, five of the eight programs reported delivering software functionality every 6 months or less as called for in OMB guidance (see table).
Department of Defense Major IT Business Programs Actively Developing Software Reported Using Iterative Development Approaches and Practices
|
Development approach or practice |
Number of programs that reported using each approach or practice |
|---|---|
|
Uses an iterative development approach |
8 of 8 |
|
Uses Agile as an approach |
6 of 8 |
|
Delivery of minimum viable product |
7 of 8 |
|
Delivery of software at least every 6 months |
5 of 8 |
Source: GAO analysis of Department of Defense program questionnaire responses, as of February 2023.| GAO-23-160117
Moreover, recognizing the importance of user involvement throughout the software development process, officials for all eight programs in active development reported involving users through collecting feedback during requirements development and refinement. In addition, most of the 25 major IT business programs in various stages reported involving users through testing and surveying them about customer experience (see table).
Department of Defense Major IT Business Programs in Various Stages of Development Reported Conducting Activities to Involve Users
|
User involvement activity |
Number of programs that reported conducting each activity |
|---|---|
|
Collecting user feedback during development |
8 of 8 |
|
Involving users in testing |
23 of 25 |
|
Surveying users about customer experience |
20 of 25 |
Source: GAO analysis of Department of Defense program questionnaire responses, as of February 2023. | GAO-23-160117
However, as of February 2023, 11 of the 25 programs did not demonstrate having approved plans for conducting user training and deployment as required by DOD. Program officials provided various reasons for not having the plans, including the system nearing retirement or predating the requirement. However, DOD officials acknowledged that programs should have user training and deployment plans and stated that they will follow up with the programs that did not have them. Without such plans, the department is at increased risk of programs not achieving required organizational changes and delivering business systems that do not meet their users' needs and are not widely adopted by users.
Further, while program officials reported conducting cybersecurity assessments and tests, six programs did not demonstrate having an approved cybersecurity strategy as required. In June 2022, GAO reported that 10 of DOD's major IT business programs did not have approved strategies and recommended the DOD Chief Information Officer (CIO) ensure programs develop them. The department concurred with the recommendation and, as of March 2023, officials stated that they were following up with the programs that did not have one. Until the department ensures that all programs develop strategies, it lacks assurance that programs are positioned to effectively manage cybersecurity risks and mitigate threats. As a result, DOD programs are at increased risk of adverse cost, schedule, and performance impacts.
Regarding legislative and policy changes, DOD has taken actions to implement the National Defense Authorization Act (NDAA) for FY 2021, which eliminated the DOD Chief Management Officer (CMO) position. This position previously had broad oversight responsibilities for the department's business systems. In September 2021, the Deputy Secretary of Defense directed an extensive realignment of the responsibilities previously assigned to the CMO. In March 2023, GAO reported on DOD's oversight of its business systems and recommended that DOD update guidance for addressing statutory requirements for initially approving and annually certifying business systems and maintain complete and accurate data for its systems, among other things. The department has efforts underway to implement changes, including plans to issue revised business systems investment management guidance. GAO will continue to monitor DOD's efforts to redistribute the roles and responsibilities formerly assigned to the CMO and to improve how the department manages its IT investments.
Why GAO Did This Study
For FY 2023, DOD requested approximately $45.2 billion for its unclassified IT investments, encompassing essential infrastructure, communications, and business systems. This includes the department's major IT programs, which are intended to help sustain key business operations such as contracting, logistics, human resources, and financial management.
The NDAA for FY 2019, as amended, includes a provision for GAO to assess selected DOD IT programs annually through March 2026. GAO's objectives were to (1) examine how DOD's portfolio of major IT business programs has performed, (2) determine the extent to which DOD has implemented key software development and cybersecurity practices for selected programs, and (3) describe actions DOD has taken to implement legislative and policy changes that could affect its IT acquisitions.
To address these objectives, GAO selected the 25 major IT business programs DOD reported in its FY 2023 submission to the Federal IT Dashboard (a public website with information on the performance of IT investments). GAO analyzed the Dashboard data to examine DOD's planned expenditures for these programs and for its standard IT infrastructure (the supporting hardware, software, and services that a business system requires to operate) from FY 2021 through FY 2023. GAO compared programs' operational performance data to OMB guidance. GAO also met with DOD OCIO officials to determine reasons for differences between how metrics data were reported and reporting guidance.
In addition, GAO administered a questionnaire to the 25 program offices to obtain information about cost and schedule changes that the programs had experienced since January 2021. The questionnaire also sought information about software development and cybersecurity practices used by the programs, including whether users were involved during the development process. GAO compared the responses to relevant guidance and leading practices to identify gaps and risks. For programs that did not demonstrate having plans, strategies, or other comparable documents, GAO followed up with DOD officials for clarification.
Further, GAO reviewed actions DOD has taken to implement its plans for addressing previously identified legislative and policy changes that could affect its IT acquisitions. This included reviewing policy, plans, and guidance associated with the department’s efforts to (1) reorganize former CMO responsibilities and (2) implement changes associated with its defense business systems investment management guidance and business enterprise architecture. GAO met with DOD officials to discuss each of the topics addressed in this report.
Recommendations
GAO is making two recommendations to DOD to ensure programs (1) identify operational performance metrics data, as appropriate, in its reporting to the Federal IT Dashboard and (2) develop plans that address conducting user training and deployment, as appropriate. GAO also reiterates the need for DOD to address previous recommendations focused on improving major IT programs.
DOD agreed with the content of GAO's report, but did not concur with the recommendations because the department believes it has already taken actions to address them. However, the department did not provide sufficient evidence indicating it had done so. As a result, GAO continues to believe the recommendations are appropriate.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | The Secretary of Defense should direct the Chief Information Officer to ensure that major IT business programs identify at least the minimum required amount of operational performance metrics, as appropriate, in the department's submission to the Federal IT Dashboard. (Recommendation 1) |
As of August 2023, the department indicated that it has taken steps to address our recommendation. The department stated that it had implemented an additional audit check to ensure operational performance metrics are provided for all major IT systems and reported to the Federal IT Dashboard, but that the checks did not work properly and it has since corrected them. The department stated that it estimates the data will be available in April 2024 as part of DOD's fiscal year 2025 submission to the Dashboard and we will continue to monitor the department's efforts to fully implement this recommendation through our annual reviews of selected DOD IT programs.
|
| Department of Defense | The Secretary of Defense should direct the Chief Information Officer to ensure that major IT business programs develop capability implementation plans or other program plans that address conducting user training and deployment, as appropriate. (Recommendation 2) |
As of August 2023, the department indicated that it has taken steps to address our recommendation. The department stated that, as of January 2020, the Secretary of Defense provided direction to ensure that major IT business programs develop capability implementation plans or other program plans that address conducting user training and deployment, as appropriate. Specifically, in accordance with DOD directives and July 2014 and July 2018 Deputy Secretary of Defense memorandums, USD(A&S) issued guidance on the acquisition of business systems to include establishing policy for the use of the business capability acquisition cycle (BCAC) for business systems requirements and acquisition and documenting user training throughout. The department also noted that the requirement to develop capability implementation plans is codified within DODI 5000.75 and, as a defense business system progresses through the BCAC, they are required to mature their user training and deployment plans at each phase/authority to proceed decision point. The department added that, per DODI 5000.75, the milestone decision authority has the ability to review the user training and deployment plans prior to progressing into the capability support phase. We will continue to monitor the department's efforts to fully implement this recommendation as part of our annual review of selected DOD IT programs, with the next report planned for issuance in summer 2024.
|