Cloud Computing: Agencies Need to Incorporate Key Practices to Ensure Effective Performance
Highlights
What GAO Found
Federal and private sector guidance highlights the importance of federal agencies using a service level agreement (SLA) in a contract when acquiring information technology (IT) services through a cloud computing services provider. An SLA defines the level of service and performance expected from a provider, how that performance will be measured, and what enforcement mechanisms will be used to ensure the specified performance levels are achieved. GAO identified ten key practices to be included in an SLA, such as identifying the roles and responsibilities of major stakeholders, defining performance objectives, and specifying security metrics. The key practices, if properly implemented, can help agencies ensure services are performed effectively, efficiently, and securely. Under the direction of the Office of Management and Budget (OMB), guidance issued to agencies in February 2012 included seven of the ten key practices described in this report that could help agencies ensure the effectiveness of their cloud services contracts.
GAO determined that the five agencies and the 21 cloud service contracts it reviewed had included a majority of the ten key practices. Specifically, of the 21 cloud service contracts reviewed from the Departments of Defense, Health and Human Services, Homeland Security, Treasury, and Veterans Affairs, 7 had fulfilled all 10 of the key practices, as illustrated in the figure. The remaining 13 contracts had incorporated 5 or more of the 10 key practices and 1 had not included any practices.
Figure 1: Number of Cloud Service Contracts That Met All 10 Key Practices
Agency officials gave several reasons for why they did not include all elements of the key practices into their cloud service contracts, including that guidance directing the use of such practices had not been created when the cloud services were acquired. Unless agencies fully implement SLA key practices into their SLAs, they may not be able to adequately measure the performance of the services, and, therefore, may not be able to effectively hold the contractors accountable when performance falls short.
Why GAO Did This Study
Cloud computing is a means for delivering computing services via IT networks. When executed effectively, cloud-based services can allow agencies to pay for only the IT services used, thus paying less for more services. An important element of acquiring cloud services is a service level agreement that specifies, among other things, what services a cloud provider is to perform and at what level.
GAO was asked to examine federal agencies' use of SLAs. GAO's objectives were to (1) identify key practices in cloud computing SLAs and (2) determine the extent to which federal agencies have incorporated such practices into their SLAs. GAO analyzed research, studies, and guidance developed by federal and private entities to develop a list of key practices to be included in SLAs. GAO validated its list with the entities, including OMB, and analyzed 21 cloud service contracts and related documentation of five agencies (with the largest fiscal year 2015 IT budgets) against the key practices to identify any variances, their causes, and impacts.
Recommendations
GAO recommends that OMB include all ten key practices in future guidance to agencies and that Defense, Health and Human Services, Homeland Security, Treasury, and Veterans Affairs implement SLA guidance and incorporate applicable key practices into their SLAs. In commenting on a draft of this report, OMB and one agency had no comment, the remaining four agencies concurred with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | To ensure that agencies are provided with more complete guidance for contracts for cloud computing services, the Director of OMB should include all ten key practices in future guidance to agencies. |
The Office of Management and Budget (OMB) has taken steps to implement our recommendation. Specifically, in June 2019, OMB issued its Federal Cloud Computing Strategy, which incorporates key practices on service level agreements that we had identified in our report related to specifying roles and responsibilities for the agency and the cloud services provider and establishing clear performance metrics. Subsequently, in January 2020, OMB staff reported that they had worked with the General Services Administration to identify best practices related to service level agreements and had made this guidance available to agencies to help improve federal acquisition of cloud-based technologies. This included International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 19086-1 on Information technology - Cloud computing -Service level agreement (SLA) framework - Part 1: Overview and concepts, which incorporates all ten key practices identified in our report. For example, ISO/IEC 19086-1 includes guidance on having clear measures for performance related to service availability and response time; incorporating requirements related to how the cloud services provider will monitor performance and report results to the agency; and specifying metrics the cloud provider must meet related to protecting agency data. The guidance also identifies the need for enforceable consequences in the event that the cloud provider fails to meet the performance measures. By providing guidance to agencies that includes all ten key practices for service level agreements, OMB has helped agencies to be better positioned to more effectively measure the performance of the services they receive, and, therefore, more likely to ensure the delivery and effective implementation of services for which they have contracted.
|
| Department of Defense | To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretary of Defense should direct the appropriate officials to ensure key practices are fully incorporated for cloud services as the contracts and associated SLAs expire. These efforts should include updating the Department of Defense memorandum on acquiring cloud services and current Defense Acquisition Regulations System to more completely include the key practices. |
The Department of Defense (Defense) agreed with our recommendation and stated that the department would update its cloud computing guidance and contracting guidance. In August 2017, Defense finalized its updated guidance on service level agreements in its Defense Acquisition Guidebook, which incorporates the key practices identified in our report. For example, the Guidebook specifying the roles and responsibilities of all parties involved in the agreement, including department personnel and service provider staff; defining clear measures for performance by the cloud service provider, such as service availability and response time; and providing how data and networks are to be managed and maintained. The guidance also identifies that enforceable consequences, such as penalties, in the case of non-compliance with the performance measures should be included. Subsequently, in May 2018, Defense provided evidence that the guidance has been incorporated into cloud service contracts. For example, a review of the department's contract documentation for milCloud found that language was included that the contractor would demonstrate that they complied with sections of the service level agreement and that the cloud provider would maintain the cloud environment in accordance with applicable department polices and guidance. By updating its guidance to address these key practices, Defense is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.
|
| Department of Homeland Security | To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire. |
The Department of Homeland Security (DHS) agreed with our recommendation and stated that the department would establish common cloud computing service level agreement guidance. In August 2017, DHS finalized its service level agreement template, which provides a consistent format for agreements between DHS and cloud service providers and incorporates the key practices identified in our report. For example, the template ensures the delivery and effective implementation of services by specifying the roles and responsibilities of all parties involved in the agreement, including department personnel and service provider staff; defining clear measures for performance by the contractor, such as service availability and quality; and providing planning and testing for disaster recovery and continuity of operations. The template also identifies a range of enforceable consequences, such as penalties, in the case of non-compliance with the performance measures by the contractors. In December 2017, DHS' Office of the Chief Procurement Officer notified all the heads of contracting activities and component acquisition executives that the template had been added to the Homeland Security Acquisition Manual, and requested that acquisition staff and financial personnel be notified of the change as well. Subsequently, in February 2018, DHS provided evidence that the template has been incorporated into cloud service contracts. For example, Immigrations and Customs Enforcement confirmed that the contractor's service level agreement for a new contract was aligned with the DHS template, and that proper parameters and commitments are in place to ensure that the DHS requirements are met. By finalizing a standard service level agreement to address these key practices, DHS is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.
|
| Department of Health and Human Services | To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire. |
The Department of Health and Human Services (HHS) concurred with, and has taken steps, to address our recommendation. In January 2023, HHS issued its Cloud Adoption Strategy, which included guidance regarding service level agreements and incorporated the key practices identified in our report. For example, the standardized language specified the roles and responsibilities of all parties involved in the agreement; defined clear measures for performance by the cloud provider, such as service availability and response time; and provided information related to disaster recovery and continuity of operations. By finalizing department documentation to address these key practices, HHS is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.
|
| Department of the Treasury | To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire. |
The Department of the Treasury (Treasury) has taken steps to address our recommendation. In November 2024, Treasury issued a policy memorandum on service level agreements for cloud procurements, which included guidance regarding service level agreements and incorporated the key practices identified in our report. For example, the standardized language defined clear measures for performance by the cloud provider, such as system performance and available; and provided information related to disaster recovery, patch management and continuous monitoring. The department also had a specific section that outlined the consequences for non-compliance for each of the SLA measures. By finalizing department documentation to address these key practices, Treasury is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.
|
| Department of Veterans Affairs | To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire. |
The Department of Veterans Affairs (VA) concurred with, and has taken steps, to address our recommendation. In June 2021, VA updated its Enterprise Cloud Technical Reference Guide, which provided guidance that linked to a standardized cloud service operational level agreement that incorporated the key practices identified in our report. For example, the standardized language ensures the delivery and effective implementation of services by specifying the roles and responsibilities of all parties involved in the agreement, including department personnel and service provider staff; defining clear measures for performance by the cloud provider, such as service availability and downtime; and providing information related to disaster recovery and continuity of operations. The language also identifies enforceable consequences, such as receiving credits, in the case of non-compliance with the performance measures by the provider. In addition, VA also provided evidence that the agreement was in use with a current cloud provider, ensuring that the key practices were incorporated as department service level agreements and other contracts expire. By finalizing department documentation to address these key practices, VA is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.
|