This is the accessible text file for GAO report number GAO-09-529 
entitled 'Office Of Personnel Management: Retirement Modernization 
Planning and Management Shortcomings Need to Be Addressed' which was 
released on April 21, 2009. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 
GAO: 

April 2009: 

Office Of Personnel Management: 

Retirement Modernization Planning and Management Shortcomings Need to 
Be Addressed: 

GAO-09-529: 

GAO Highlights: 

Highlights of GAO-09-529, a report to congressional committees. 

Why GAO Did This Study: 

For the past two decades, the Office of Personnel Management (OPM) has 
been working to modernize the paper-intensive processes and antiquated 
systems used to support the retirement of federal employees. By moving 
to an automated system, OPM intends to improve the program’s efficiency 
and effectiveness. In January 2008, GAO recommended that the agency 
address risks to successful system deployment. Nevertheless, OPM 
deployed a limited initial version of the modernized system in February 
2008. After unsuccessful efforts to address system quality issues, OPM 
suspended system operation, terminated a major contract, and began 
restructuring the modernization effort, also referred to as RetireEZ. 
For this study, GAO was asked to (1) assess the status of OPM’s efforts 
to plan and implement the RetireEZ program and (2) evaluate the 
effectiveness of the agency’s management of the modernization 
initiative. To do this, GAO reviewed OPM program documentation and 
interviewed agency and contractor officials. 

What GAO Found: 

OPM remains far from achieving the modernized capabilities it had 
planned. Specifically, the agency has partially implemented two of 
eight planned capabilities: 

* an integrated database of retirement information accessible to OPM 
and agency retirement processing personnel and; 

* enhanced customer service capabilities that support customer needs 
and provide self-service tools. 

However, the remaining six capabilities have yet to be implemented 
because they depended on deliverables that were to be provided by a 
contract that is now terminated. Examples of these missing capabilities 
include: 

* automated submission of retirement information through interfaces 
with federal agencies and; 

* Web-accessible self-service retirement information for active and 
retired federal employees. 

Further, OPM has not yet developed a complete plan that describes how 
the program is to proceed without the system that was to be provided 
under the terminated contract. Although agency documents describe 
program implementation activities, they do not include a definition of 
the program, its scope, lines of responsibility and authority, 
management processes, and a schedule. Also, modernization program 
documentation does not describe results-oriented performance goals and 
measures. Until the agency completes and uses a plan that includes all 
of the above elements to guide its efforts, it will not be properly 
positioned to move forward with its restructured retirement 
modernization initiative. 

Further, OPM has significant weaknesses in five key management areas 
that are vital for effective development and implementation of its 
modernization program: cost estimating, earned value management (a 
recognized means for measuring program progress), requirements 
management, testing, and oversight. For example, the agency has not 
developed a cost estimating plan or established a performance 
measurement baseline—prerequisites for effective cost estimating and 
earned value management. Further, although OPM is revising its 
previously developed system requirements, it has not established 
processes and plans to guide this work or addressed test activities 
even though developing processes and plans, as well as planning test 
activities early in the life cycle, are recognized best practices for 
effective requirements development and testing. Finally, although OPM’s 
Executive Steering Committee and Investment Review Board have recently 
become more active regarding RetireEZ, these bodies did not exercise 
effective oversight in the past, which has allowed the aforementioned 
management weaknesses to persist and OPM has not established guidance 
regarding how these entities are to intervene when corrective actions 
are needed. Until OPM addresses these weaknesses, many of which GAO and 
others made recommendations to correct, the agency’s retirement 
modernization initiative remains at risk of failure. Institutionalizing 
effective management is critical not only for the success of this 
initiative, but also for that of other modernization efforts within the 
agency. 

What GAO Recommends: 

To improve OPM’s effort to plan and implement its retirement 
modernization program, GAO is recommending that OPM correct management 
weaknesses. In written comments on a draft of this report, the Director 
of OPM agreed with GAO’s recommendations and described steps the agency 
is taking to address them. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/products/GAO-09-529]. For more 
information, contact Valerie C. Melvin at (202) 512-6304 or 
melvinv@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Few Planned Capabilities Have Been Implemented, and Future Efforts Are 
Not Guided by a Complete Plan: 

OPM Is Not Positioned to Effectively Manage Its Retirement 
Modernization Initiative: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Office of Personnel Management: 

Appendix III: GAO Staff Contact and Acknowledgments: 

Tables: 

Table 1: Key Retirement Modernization Contracts: 

Table 2: Retirement Modernization Management Entities: 

Table 3: Status of Implementing Retirement Modernization Capabilities: 

Table 4: Fiscal Year 2009 Initiatives: 

Figures: 

Figure 1: Estimated Federal Employees Eligible and Likely to Retire by 
2016: 

Figure 2: Simplified View of Retirement Application Process: 

Figure 3: Timeline of Retirement Modernization Initiatives: 

Abbreviations: 

CIS: Center for Information Services: 

CRIS: Center for Retirement and Insurance Services: 

CSRS: Civil Service Retirement System: 

DBTS: defined benefits technology solution: 

EVM: earned value management: 

FAPS: FERS Automated Processing System: 

FERS: Federal Employees Retirement System: 

GSA: General Services Administration: 

OMB: Office of Management and Budget: 

OPM: Office of Personnel Management: 

RSM: Retirement Systems Modernization: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

April 21, 2009: 

The Honorable Richard J. Durbin:
Chairman:
The Honorable Susan Collins:
Ranking Member:
Subcommittee on Financial Services and General Government: 
Committee on Appropriations:
United States Senate: 

The Honorable José E. Serrano:
Chairman:
The Honorable Jo Ann Emerson:
Ranking Member:
Subcommittee on Financial Services and General Government: 
Committee on Appropriations:
House of Representatives: 

For the past two decades, the Office of Personnel Management (OPM) has 
been attempting to modernize the paper-intensive processes and 
antiquated systems it uses to support the retirement of federal 
employees, in part because the current process does not provide prompt 
and complete benefit payments upon retirement. The agency's retirement 
modernization initiative, also referred to as RetireEZ, was intended to 
address this situation and improve the efficiency and effectiveness of 
the retirement program, which serves federal employees who are eligible 
to receive benefits in the future, employees who are already retired, 
and their survivors and beneficiaries. 

In January 2008, we reported on challenges that OPM faced in deploying 
RetireEZ, noted a number of concerns regarding the agency's management 
of the initiative, and recommended corrective actions.[Footnote 1] In 
late February 2008, OPM deployed a version of the modernized retirement 
system; however, the agency suspended the system's operation in May 
2008. After 5 months of attempting to address system quality issues, 
the agency terminated its contract with the primary modernization 
vendor in October 2008 and began restructuring the retirement 
modernization initiative in November 2008. 

At your request, we conducted a follow-up study of OPM's management of 
RetireEZ. Specifically, our objectives were to: 

* assess the status of OPM's efforts toward planning and implementing 
the RetireEZ program and: 

* evaluate the effectiveness of the agency's management of the 
modernization initiative. 

To accomplish our objectives, we reviewed relevant program 
documentation and interviewed appropriate OPM and contractor officials. 
Specifically, to assess the status of the program, we reviewed 
documentation such as the retirement modernization program management 
plan and supporting documents. We also visited OPM's retirement 
operations facilities in Washington, D.C., and Boyers, Pennsylvania. To 
evaluate the effectiveness of the agency's management of the 
modernization initiative, we reviewed OPM's cost estimation, earned 
value management (EVM), requirements management, test management 
activities, and oversight. We compared OPM's plans and activities to 
best practices and assessed the agency's progress toward implementing 
our prior recommendations. 

We conducted this performance audit from May 2008 through March 2009 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. See appendix II for a more 
complete description of our objectives, scope, and methodology. 

Background: 

OPM's mission is to ensure that the federal government has an effective 
civilian workforce. In this regard, one of the agency's major human 
resources tasks is to manage and administer the retirement program for 
federal employees. According to the agency, the program serves federal 
employees by providing (1) retirement compensation and (2) tools and 
options for retirement planning. OPM's Center for Retirement and 
Insurance Services administers the two defined benefit retirement plans 
that provide retirement, disability, and survivor benefits to federal 
employees. The first plan, the Civil Service Retirement System (CSRS), 
provides retirement benefits for most federal employees hired before 
1984. The second plan, the Federal Employees Retirement System (FERS), 
covers most employees hired in or after 1984 and provides benefits that 
include Social Security and a defined contribution system.[Footnote 2] 

According to OPM, there are approximately 2.9 million active federal 
employees and nearly 2.5 million retired federal employees. The 
agency's March 2008 analysis of federal employment retirement data 
estimates that nearly 1 million active federal employees will be 
eligible to retire and almost 600,000 will most likely retire by 2016. 
[Footnote 3] Figure 1 summarizes the estimated number of employees 
eligible and likely to retire. 

Figure 1: Estimated Federal Employees Eligible and Likely to Retire by 
2016: 

[Refer to PDF for image: multiple line graph] 

Fiscal year: 2009; 
Likely to retire: 184,461; 
Eligible to retire: 494,619. 

Fiscal year: 2010; 
Likely to retire: 246,209; 
Eligible to retire: 566,801. 

Fiscal year: 2011; 
Likely to retire: 307,159; 
Eligible to retire: 637,645. 

Fiscal year: 2012; 
Likely to retire: 366,707; 
Eligible to retire: 707,750. 

Fiscal year: 2013; 
Likely to retire: 424,546; 
Eligible to retire: 775,035. 

Fiscal year: 2014; 
Likely to retire: 480,464; 
Eligible to retire: 836,516. 

Fiscal year: 2015; 
Likely to retire: 534,411; 
Eligible to retire: 896,335. 

Fiscal year: 2016; 
Likely to retire: 586,339; 
Eligible to retire: 956,613. 

Source: OPM. 

[End of figure] 

Federal Employee Retirement Application Processing: 

OPM and employing agencies' human resources and payroll offices are 
responsible for processing federal employees' retirement applications. 
The process begins when an employee submits a paper retirement 
application to his or her employer's human resources office and is 
completed when the individual begins receiving regular monthly benefit 
payments (as illustrated in figure 2). 

Figure 2: Simplified View of Retirement Application Process: 

[Refer to PDF for image: illustration] 

Agency: 

Employee submits retirement application; 

Agency: Human Resources (HR): 
* Required meeting in which agency benefits officer checks paperwork, 
provides retirement counseling, and provides benefits estimate; 

* Develop retirement package and separation form; 

* Review retirement package and provide final documents to agency 
payroll; 

Agency: Payroll: 
* Receive package from HR and perform analysis; 

* Prepare individual retirement record; 

* Electronically submit summary file and mail hardcopy of complete 
package to OPM; 

OPM: 
* Agency retirement package received; 

* Interim payment calculated based on agency summary file and 
authorized; 

* Development actions taken as needed to adjudicate application; 
- Ensure package has all required information; 
- Regular benefit payment is calculated and package is submitted for
authorization; 
- Package and benefit payment reviewed and verified by second 
adjudicator; 

* Regular benefit payment authorized. 

Source: GAO analysis of OPM data. 

[End of figure] 

Once an employee submits an application, the employing agency's human 
resources office provides retirement counseling services to the 
employee and augments the retirement application with additional 
paperwork, such as a separation form that finalizes the date the 
employee will retire. Then the agency provides the retirement package 
to the employee's payroll office. After the employee separates for 
retirement, the payroll office is responsible for reviewing the 
documents for correct signatures and information, making sure that all 
required forms have been submitted, and adding any additional paperwork 
that will be necessary for processing the retirement package. Once the 
payroll office has finalized the paperwork, the retirement package is 
mailed to OPM to continue the retirement process. Payroll offices are 
expected to submit the package to OPM within 30 days of the retiree's 
separation date. 

Upon receipt of the retirement package, OPM calculates an interim 
payment based on information provided by the employing agency. The 
interim payments are partial payments that typically provide retirees 
with 80 percent of the total monthly benefit they will eventually 
receive.[Footnote 4] OPM then starts the process of analyzing the 
retirement application and associated paperwork to determine the total 
monthly benefit amount to which the retiree is entitled. This process 
includes collecting additional information from the employing agency's 
human resources and payroll offices or from the retiree to ensure that 
all necessary data are available before calculating benefits. After OPM 
completes its review and authorizes payment, the retiree begins 
receiving 100 percent of the monthly retirement benefit payments. OPM 
then stores the paper retirement folder at the Retirement Operations 
Center in Boyers, Pennsylvania. 

According to the agency's 2008 performance report, the average 
processing time from the date OPM receives the initial application to 
the time the retiree receives a full payment is 42 days. According to 
the Deputy Associate Director for the Center of Retirement and 
Insurance Services, about 200 employees are directly involved in 
processing the approximately 100,000 retirement applications OPM 
receives annually. This processing includes functions such as 
determining retirement eligibility, inputting data into benefit 
calculators, and providing customer service. The agency uses over 500 
different procedures, laws, and regulations, which are documented on 
the agency's internal Web site, to process retirement applications. For 
example, the site contains memorandums that outline new procedures for 
handling special retirement applications, such as those for disability 
or court orders. Further, OPM's retirement processing involves the use 
of over 80 information systems that have approximately 400 interfaces 
with other internal and external systems. For instance, 26 internal 
systems interface with the Department of the Treasury to provide, among 
other things, information regarding the total amount of benefit 
payments to which an employee is entitled. 

OPM has stated that the federal employee retirement process currently 
does not provide prompt and complete benefit payments upon retirement, 
and that customer service expectations for more timely payments are 
increasing. The agency also reports that a greater workload is expected 
due to an anticipated increase in the number of retirement applications 
over the next decade, yet current retirement processing operations are 
at full capacity. Further, the agency has identified several factors 
that limit its ability to process retirement benefits in an efficient 
and timely manner. Specifically, it noted that: 

* current processes are paper-based and manually intensive, resulting 
in a higher number of errors and delays in providing benefit payments; 

* the high costs, limited capabilities, and other problems with the 
existing information systems and processes pose increasing risks to the 
accuracy of benefit payments; 

* current manual capabilities restrict customer service; 

* federal employees have limited access to their retirement records, 
making planning for retirement difficult; and: 

* attracting qualified personnel to operate and maintain the antiquated 
retirement systems, which have about 3 million lines of custom 
programming, is challenging. 

OPM Has a Long History of Retirement Modernization Initiatives: 

In the late 1980s, OPM recognized the need to automate and modernize 
its retirement processing and began retirement modernization 
initiatives that have continuously called for automating its antiquated 
paper-based processes. The agency's previously established program 
management plans included the objectives of having timely and accurate 
retirement benefit payments and more efficient and flexible processes. 
For example, the agency's plans call for processing retirement 
applications and providing retirees 100 percent of their monthly 
benefit payments the day it is due versus providing interim monthly 
payments. Its initial modernization vision called for providing prompt 
and complete benefit payments by developing an integrated system and 
automated processes. However, the agency has faced significant and long-
standing challenges in doing so. 

In early 1987, OPM began a program called the FERS Automated Processing 
System (FAPS). However, after 8 years of planning, the agency decided 
it needed to reevaluate the program, and the Office of Management and 
Budget (OMB) requested that an independent board conduct a review to 
identify critical issues impeding progress and recommend ways to 
address the issues. The review identified various management 
weaknesses, including the lack of an established strategic plan, cost 
estimation methodologies, and baseline; improperly defined and 
ineffectively managed requirements; and no clear accountability for 
decision making and oversight.[Footnote 5] Accordingly, the board 
suggested areas for improvement and recommended terminating the program 
if immediate action was not taken. In mid-1996, OPM terminated the 
program. 

In 1997, OPM began planning a second modernization initiative, called 
the Retirement Systems Modernization (RSM) program. The agency 
originally intended to structure the program as an acquisition of 
commercially available hardware and software that would be modified in- 
house to meet its needs. From 1997 to 2001, OPM developed plans and 
analyses and began developing business and security requirements for 
the program. However, in June 2001, it decided to change the direction 
of the retirement modernization initiative. 

In late 2001, retaining the name RSM, the agency embarked upon its 
third initiative to modernize the retirement process and examined the 
possibility of privately sourced technologies and tools. To this end, 
OPM issued a request for information to obtain private sourcing options 
and determined that contracting was a viable alternative that would be 
cost efficient, less risky, and more likely to be completed on time and 
on budget. In 2006, the agency awarded three contracts for: (1) a 
commercially available, defined benefits technology solution (DBTS) to 
automate retirement processing; (2) services to convert paper records 
to electronic files; and (3) consulting services to support the 
redesign of its retirement operations. The contract for DBTS was 
awarded to Hewitt Associates, and the additional contracts to support 
the technology were awarded to Accenture Ltd. and Northrop Grumman 
Corporation, as reflected in table 1. 

Table 1: Key Retirement Modernization Contracts: 

Contractor: Hewitt Associates; 
Award date: May 2006; 
Contract value (in millions): $290.0; 
Deliverables: A defined benefits technology solution (DBTS) that 
enables automating retirement processes. 

Contractor: Accenture Ltd.; 
Award date: May 2006; 
Contract value (in millions): $40.0; 
Deliverables: Redesigned retirement processes to support DBTS. 

Contractor: Northrop Grumman Corporation; 
Award date: September 2006; 
Contract value (in millions): $30.7; 
Deliverables: Converting paper retirement records to electronic files. 

Source: OPM. 

[End of table] 

OPM produced a December 2007 program management plan that, among other 
things, described capabilities the agency expected to implement as 
outcomes of retirement modernization. Among these capabilities, the 
agency expected to implement retirement benefit modeling and planning 
tools for active federal employees, a standardized retirement benefit 
calculation system, and a consolidated system to support all aspects of 
retirement processing. 

In February 2008, OPM renamed the program RetireEZ and deployed a 
limited initial version of DBTS. As the foundation of the modernization 
initiative, DBTS was to be a comprehensive technology solution that 
would provide capabilities to substantially automate retirement 
processing. This technology was to be provided by the contractor for a 
period of 10 years and was intended to provide, among other things, an 
integrated database with calculation functionality for retirement 
processing. In addition to calculating retirement benefit amounts, DBTS 
was intended to provide active and retired federal employees with self- 
service, Internet-based tools for accessing accounts, updating 
retirement records, submitting transactions, monitoring the status of 
claims, and forecasting retirement income. The technology was also 
expected to enhance customer service by providing OPM and agency 
personnel with the capability to access retirement information online. 
Further, the technology was expected to be integrated with OPM and 
federal agency electronic retirement records and processes. When fully 
implemented, the modernized program was expected to serve OPM 
retirement processing personnel, federal agency human resources and 
payroll offices, active federal employees, retirees, and the 
beneficiaries of retirees. 

According to the agency, in late February 2008, the DBTS was deployed 
with limited functionality to 26,000 federal employees serviced by the 
General Services Administration's (GSA) payroll offices. In April 2008, 
OPM reported that 13 of the 37 retirement applications received from 
GSA's payroll office had been processed through DBTS with manual 
intervention and provided the retirees 100 percent of their monthly 
benefits within 30 days from their retirement date. However, a month 
later, the agency determined that DBTS had not worked as expected and 
suspended system operation. In October 2008, after 5 months of 
attempting to address system quality issues, the agency terminated the 
contract. 

In November 2008, OPM began restructuring the program and reported that 
its efforts to modernize retirement processing would continue. Figure 3 
illustrates the timeline of retirement modernization initiatives from 
1987 to the present. 

Figure 3: Timeline of Retirement Modernization Initiatives: 

[Refer to PDF for image: timeline illustration] 

FAPS: First effort: about 8 years: 1987-1996. 

RSM (in-house): Second effort: about 4 year: 1997-2001. 

RSM (privately sourced: Third effort: about 7 years: 2001-2008; 
DBTS deployed (RSM renamed RetireEZ): 2008. 

RetireEZ (restructures): Fourth effort: 2009 and beyond. 

Source: GAO analysis of OPM data. 

[End of figure] 

Current Retirement Modernization Management: 

Various entities within OPM are responsible for managing RetireEZ. 
Specifically, the management is composed of committees, a program 
office, and operational support, as reflected in table 2. 

Table 2: Retirement Modernization Management Entities: 

Title: Executive Steering Committee; 
Description: The Executive Steering Committee is responsible for 
overall program direction and decision making. The committee is chaired 
by the Director of the Office of Modernization and Human Resources Line 
of Business. 

Title: Investment Review Board; 
Description: The Investment Review Board is responsible for monitoring 
the progress of continued investment in the program and determining 
appropriate corrective action when necessary. The board is chaired by 
the agency's Chief Information Officer. 

Title: Office of Modernization and Human Resources Line of Business; 
Description: The Office of Modernization and Human Resources Line of 
Business is responsible for overseeing the program office for the 
retirement modernization. The Director of Modernization and Human 
Resources Line of Business maintains overall administration of program 
activities and resources. 

Title: Center for Information Services; 
Description: The Center for Information Services (CIS), within the 
Management Services Division, is responsible for providing support and 
oversight for acquisition, systems, contract, and security management. 
The Deputy Associate Director of CIS (also the Chief Information 
Officer) oversees the maintenance and operation of legacy retirement 
systems. 

Title: Center for Retirement and Insurance Services; 
Description: The Center for Retirement and Insurance Services (CRIS), 
within the Human Resources Products and Services Division, is 
responsible for retirement processes and is expected to assume 
ownership of the retirement modernization. The Deputy Associate 
Director of CRIS oversees current operations and allocates resources as 
necessary to support the program office. 

Source: GAO analysis of OPM data. 

[End of table] 

Prior GAO Reviews of OPM's Retirement Modernization Initiatives: 

Since 2005, we have conducted several studies of OPM's retirement 
modernization noting weaknesses in its management of the initiative. In 
February of that year, we reported that the agency lacked processes for 
retirement modernization acquisition activities, such as determining 
requirements, developing acquisition strategies, and implementing a 
risk program. Further, the agency had not established effective 
security management, change management, and program executive 
oversight.[Footnote 6] We recommended that the Director of OPM ensure 
that the retirement modernization program office expeditiously 
establish processes for effective oversight of the retirement 
modernization in the areas of system acquisition management, 
information security, organizational change management, and information 
technology (IT) investment management. In response, between 2005 and 
2007, the agency initiated steps toward establishing management 
processes for retirement modernization and demonstrated the completion 
of activities with respect to each of our nine recommendations. 

However, in January 2008, we reported that the agency still needed to 
improve its management of the program to ensure a successful outcome 
for its modernization efforts.[Footnote 7] Specifically, we reported 
that initial test results had not provided assurance that DBTS would 
perform as intended, the testing schedule increased the risk that the 
agency would not have sufficient resources or time to ensure that all 
system components were tested before deployment, and trends in 
identifying and resolving system defects had indicated a growing 
backlog of problems to be resolved prior to deployment. Further, we 
reported that although the agency had established a risk management 
process, it had not reliably estimated the program costs, and its 
progress reporting was questionable because it did not reflect the 
actual state of the program. We recommended that the Director of OPM 
address these deficiencies by conducting effective system tests and 
resolving urgent and high priority system defects prior to system 
deployment, in addition to improving program cost estimation and 
progress reporting. 

In response to our report, OPM stated that it concurred with our 
recommendations and was taking steps to address them. However, in March 
2008, we determined that the agency was moving forward with system 
deployment and had not yet implemented its planned actions.[Footnote 8] 
OPM subsequently affirmed its agreement with our recommendations in 
April 2008 and reported that it had implemented or was in the process 
of implementing each recommendation. As of March 2009, however, these 
recommendations still had not been fully addressed. 

Few Planned Capabilities Have Been Implemented, and Future Efforts Are 
Not Guided by a Complete Plan: 

OPM remains far from fully implementing the retirement modernization 
capabilities described when it documented its plans for RetireEZ in 
2007. The agency only partially implemented two of eight capabilities 
that it identified to modernize retirement processing. The remaining 
six capabilities, which were to be delivered through the DBTS contract, 
have not been implemented, and OPM's plans to continue implementing 
them are uncertain. While the agency has taken steps to restructure the 
RetireEZ program without the DBTS contract, it has not developed a plan 
to guide its future modernization efforts. 

Few Retirement Modernization Capabilities Have Been Implemented: 

OPM's retirement modernization plans from 2007 described eight 
capabilities that were to be implemented to achieve modernized 
processes and systems. As of late March 2009, the agency had partially 
implemented two of these capabilities while the remaining six had not 
been implemented (see table 3). 

Table 3: Status of Implementing Retirement Modernization Capabilities: 

Capability: An integrated database of retirement information accessible 
to OPM and agency retirement processing personnel; 
Implemented: Partially. 

Capability: Enhanced customer service that support customer needs and 
self-service tools; 
Implemented: Partially. 

Capability: Automated submission of retirement information through 
interfaces with federal agencies; 
Implemented: No. 

Capability: Web-accessible, self-service retirement information for 
active and retired federal employees; 
Implemented: No. 

Capability: A consolidated system to support all aspects of retirement 
processing; 
Implemented: No. 

Capability: Electronic case management system to manage retirement 
applications and processes; 
Implemented: No. 

Capability: Standardized systems for determining and calculating 
retirement benefit that is integrated with the retirement information 
database and is accessible to OPM and federal agencies; 
Implemented: No. 

Capability: Retirement benefit modeling and planning tools for active 
federal employees; 
Implemented: No. 

Source: GAO analysis of OPM data. 

[End of table] 

Specifically, it had achieved partial implementation of an integrated 
database of retirement information that was intended to be accessible 
to OPM and agency retirement processing personnel. In this regard, the 
agency implemented a new database, populated with images of retirement 
information, which is accessible to OPM retirement processing personnel 
online. This database contains over 8 million files which, according to 
agency officials, represent approximately 80 to 90 percent of the 
available retirement information for all active federal employees. 
However, the capability for the information in the database to be 
integrated with OPM's legacy retirement processing systems and to be 
accessible to other agency retirement processing personnel has not yet 
been implemented. 

OPM has also partially implemented enhanced customer service 
capabilities. Specifically, the agency acquired a new telephone 
infrastructure (i.e., additional lines) and hired additional customer 
service representatives to reduce wait times and abandonment rates. 
However, the agency has not yet developed the capabilities for OPM 
retirement processing personnel to provide enhanced customer support to 
active and retired federal employees through online account access and 
management. 

Moreover, six other capabilities have not been implemented--and plans 
to implement them are uncertain--because they were to be delivered 
through the now-terminated DBTS contract, which had been expected to 
provide a single system that would automate the processing of 
retirement applications, calculations, and benefit payments. 

Among the capabilities not implemented was one for other agencies' 
automated submissions of retirement information to OPM that could be 
used to process retirement applications. While OPM began developing 
this capability by establishing interfaces with other agencies as part 
of its effort to implement DBTS, it discontinued the use of the 
interfaces for processing retirement applications when the DBTS 
contract was terminated. Thus, federal agencies that submit retirement 
information to OPM continue to provide paper packages and information 
when employees are ready to retire. 

Further, OPM has not implemented a planned capability for active and 
retired federal employees to access online retirement information 
through self-service tools. While the agency provided demonstrations of 
DBTS in April 2008 that showed the ability for employees to access 
information online, including applying for retirement and modeling 
future retirement benefits, this capability was to be provided by DBTS, 
and thus, no longer exists. The contractor had also been expected to 
deliver a consolidated system to support all aspects of retirement 
processing and an electronic case management system to support 
retirement processing. In the absence of these capabilities, the agency 
continues to manage cases through paper tracking and stand-alone 
systems. Additionally, OPM and federal agencies continue to rely on 
nonstandardized systems to determine and calculate retirement benefits, 
and federal retirees currently have only limited online, self-service 
tools. 

OPM Does Not Have a Complete Plan for the Future of the RetireEZ 
Program: 

Program management principles and best practices emphasize the 
importance of using a program management plan that, among other things, 
establishes a complete description that ties together all program 
activities.[Footnote 9] An effective plan includes a description of the 
program's scope, implementation strategy, lines of responsibility and 
authority, management processes, and a schedule. Such a plan 
incorporates all the critical areas of system development and is to be 
used as a means of determining what needs to be done, by whom, and 
when. Furthermore, establishing results-oriented (i.e., objective, 
quantifiable, and measurable) goals and measures, that can be included 
in a plan, provides stakeholders with the information they need to 
effectively oversee and manage programs. 

A plan for the future of the RetireEZ program has not been completed. 
In November 2008, OPM began restructuring the program and reported it 
was continuing toward retirement modernization without the DBTS 
contract. The restructuring efforts have resulted in a wide variety of 
documentation, including multiple descriptions of the program in formal 
agency reports, budget documentation, agency briefing slides, and 
related documents. 

For example, OPM's November Fiscal Year 2008 Agency Financial Report 
described what the RetireEZ program is expected to achieve (e.g., 
provide retirement modeling tools for federal employees) once 
implemented. The agency's Annual Performance Report, dated January 
2009, outlined that the new vision for the restructured program is "to 
support benefit planning and management throughout a participant's 
lifecycle through an enhanced federal retirement program." The agency 
also presented information to OMB that identified eight fiscal year 
2009 program initiatives, as listed in table 4. 

Table 4: Fiscal Year 2009 Initiatives: 

Initiative: 

* Determine next steps for technology tools and develop an execution 
plan. 

* Increase customer contact center capacity. 

* Complete conversion of paper records. 

* Work with agencies to improve the quality, timeliness, and 
completeness of retirement data. 

* Begin activities to provide agencies access to electronic data. 

* Improve performance metrics for interim payments. 

* Partner with the Enterprise Human Resources Integration initiative[A] 
for data storage, cleansing, and transmission of recurring data. 

* Begin imaging all incoming paper records. 

Source: OPM data. 

[A] OPM's Enterprise Human Resources Integration initiative is intended 
to provide federal agencies access to personnel folders online. 

[End of table] 

The agency has developed a variety of informal program documents and 
briefing slides that describe retirement modernization activities. For 
instance, one document prepared by the program office describes a five- 
phased approach that is intended to replace its previous DBTS-reliant 
strategy. The approach includes the following activities: (1) 
collecting electronic retirement information, (2) automating the 
retirement application process, (3) integrating retirement information, 
(4) developing retirement calculation technologies and tools, and (5) 
improving post-retirement processes through a technology solution. In 
addition, briefing slides also prepared by the program office outline a 
schedule for efforts to identify new technologies to support retirement 
modernization by drafting a request for information, which OPM expects 
to issue in late April 2009. 

Regardless, OPM's various reports and documents describing its planned 
retirement modernization activities do not provide a complete plan for 
its restructured program. Specifically, although agency documents 
describe program implementation activities, they do not include a 
definition of the program, its scope, lines of responsibility and 
authority, management processes, and schedule. Also, the modernization 
program documentation does not describe results-oriented (i.e., 
objective, quantifiable, and measurable) performance goals and 
measures. According to the RetireEZ program manager, the agency is 
developing plans, but they will not be ready for release until the new 
OPM director has approved them, which is expected to occur in April 
2009. Until the agency completes and uses a plan that includes all of 
the above elements to guide its efforts, it will not be properly 
positioned to obtain agreement with relevant stakeholders (e.g., 
Congress, OMB, federal agencies, and OPM senior executives) for its 
restructured retirement modernization initiative. Further, the agency 
will also not have a key mechanism that it needs to help ensure 
successful implementation of future modernization efforts. 

OPM Is Not Positioned to Effectively Manage Its Retirement 
Modernization Initiative: 

OPM has significant management weaknesses in five areas that are 
important to the success of its retirement modernization program: cost 
estimating, EVM, requirements management, testing, and program 
oversight. For example, the agency has not performed key steps, 
including the development of a cost estimating plan or completion of a 
work breakdown structure, both of which are necessary to develop a 
reliable program cost estimate. Also, OPM has not established and 
validated a performance measurement baseline, which is essential for 
reliable EVM. Further, although OPM is revising its previously 
developed system requirements, it has not established processes and 
plans to guide this work. Nor has the agency addressed test activities, 
even though developing processes and planning test activities early in 
the life cycle are recognized best practices for effective testing. 
Furthermore, although OPM's Executive Steering Committee and Investment 
Review Board have recently become more active regarding RetireEZ, these 
bodies did not exercise effective oversight in the past, which has 
allowed the aforementioned management weaknesses to persist. Notably, 
OPM has not established guidance regarding how these entities are to 
engage with the program when corrective actions are needed. Until OPM 
addresses these weaknesses, many of which we and others made 
recommendations to correct, the agency's retirement modernization 
initiative remains at risk of failure. 

OPM Has Yet to Complete Key Steps in Developing a Reliable Cost 
Estimate for Retirement Modernization: 

The establishment of a reliable cost estimate is a necessary element 
for informed investment decision making, realistic budget formulation, 
and meaningful progress measurement. A cost estimate is the summation 
of individual program cost elements that have been developed by using 
established methods and validated data to estimate future costs. 
According to federal policy, programs must maintain current and well- 
documented estimates of program costs, and these estimates must span 
the full expected life of the program.[Footnote 10] Our Cost Estimating 
and Assessment Guide includes best practices that agencies can use for 
developing and managing program cost estimates that are comprehensive, 
well-documented, accurate, and credible, and provide management with a 
sound basis for establishing a baseline to measure program performance 
and formulate budgets.[Footnote 11] This guide identifies a cost 
estimating process that includes initial steps such as defining the 
estimate's purpose (i.e., its intended use, scope, and level of 
detail); developing the estimating plan (i.e., the estimating approach, 
team, and timeline); defining the program (e.g., technical baseline 
description); and determining the estimating structure (e.g., work 
breakdown structure). According to best practices, these initial steps 
in the cost estimating process are of the utmost importance, and should 
be fully completed in order for the estimate to be considered valid and 
reliable. 

OPM officials stated that they intend to complete a modernization 
program cost estimate by July 2009. However, the agency has not yet 
fully completed initial steps for developing the new estimate. 
Specifically, the agency has not yet fully defined the estimate's 
purpose, developed the estimating plan, defined program characteristics 
in a technical baseline description, or determined the estimating 
structure. 

* With respect to the estimate's purpose, agency officials stated that 
the estimate will inform the budget justification of RetireEZ for 
fiscal year 2011 and beyond. However, the agency has not clearly 
defined the scope or level of detail of the estimate. 

* Regarding the estimating plan, agency officials stated that they have 
created a timeline for the estimate's completion by July 2009. However, 
the agency has not documented an estimating plan that includes the 
approach and resources required to complete the estimate in the time 
period identified. 

* With respect to the technical baseline description, agency officials 
stated that they are in the advanced stages of developing a request for 
information and a concept of operations that will serve as the basis 
for a technical baseline description. These documents are expected to 
be reviewed for approval in April 2009. 

* Regarding the estimating structure, the agency has developed a work 
breakdown structure that identifies elements of the program to be 
estimated. However, the agency has not yet developed a work breakdown 
structure dictionary that clearly defines each element. 

Weaknesses in the reliability of OPM's retirement modernization cost 
estimate have been long-standing. We first reported on the agency's 
lack of a reliable cost estimate in January 2008 when we noted that 
critical activities, including documentation of a technical baseline 
description, had not been performed, and we recommended that the agency 
revise the estimate.[Footnote 12] Although OPM agreed to produce a 
reliable program cost estimate, the agency has not yet done so. Until 
OPM fully completes each of the steps, the agency increases the risk 
that it will produce an unreliable estimate and will not have a sound 
basis for measuring program performance and formulating retirement 
modernization program budgets. 

OPM Is Not Prepared to Perform Reliable Earned Value Management for 
Retirement Modernization: 

OMB and OPM policies require major IT programs to use EVM to measure 
and report program progress.[Footnote 13] EVM is a tool for measuring 
program progress by comparing the value of work accomplished with the 
amount of work expected to be accomplished. Such a comparison permits 
actual performance to be evaluated, based on variances from the planned 
cost and schedule, and future performance to be forecasted. 
Identification of significant variances and analysis of their causes 
helps program managers determine the need for corrective actions. 
Before EVM analysis can be reliably performed, developing a credible 
cost estimate is necessary. In addition to developing a cost estimate, 
an integrated baseline review must be conducted to validate a 
performance measurement baseline and attain agreement of program 
stakeholders (e.g., agency and contractor officials) before reliable 
EVM reporting can begin. The establishment of a baseline depends on the 
completion of a work breakdown structure, an integrated master 
schedule, and budgets for planned work. 

Although the agency plans to begin reporting on the restructured 
program's progress using EVM in April 2009, the agency is not yet 
prepared to do so because initial steps have not been completed and are 
dependent on decisions about the program that have not been made. 
Specifically, 

* the agency has not yet developed a reliable cost estimate for the 
program; such an estimate, which is critical for establishing reliable 
EVM, is not expected to be complete until July 2009; 

* the agency does not plan to conduct an integrated baseline review to 
establish a reliable performance measurement baseline before beginning 
EVM reporting; and: 

* the work breakdown structure and integrated master schedule that 
agency officials report they have developed may not accurately reflect 
the full scope and schedule because key program documentation, such as 
the concept of operations, has not been completed. 

This situation resembles the state of affairs that existed in January 
2008, when we reported that OPM's EVM was unreliable because an 
integrated baseline review had not been conducted to validate the 
program baseline.[Footnote 14] At that time we recommended, among other 
things, that the agency establish a basis for effective use of EVM by 
validating a program performance measurement baseline through a program-
level integrated baseline review. Although the agency stated that it 
agreed, it did not address this recommendation. 

Until the agency has developed a reliable cost estimate, performed an 
integrated baseline review, and validated a performance measurement 
baseline that reflect its program restructuring, the agency is not 
prepared to perform reliable EVM. Engaging in EVM reporting without 
first performing these fundamental steps could again render the 
agency's assessment unreliable. 

OPM Has Not Established Processes Needed to Effectively Develop and 
Manage Retirement Modernization Requirements: 

Well-defined and managed requirements are a cornerstone of effective 
system development and acquisition. According to recognized guidance, 
disciplined processes for developing and managing requirements can help 
reduce the risks of developing a system that does not meet user and 
operational needs.[Footnote 15] Such processes include (1) developing 
detailed requirements that have been derived from the organization's 
concept of operations[Footnote 16] and are complete and sufficiently 
detailed to guide system development and (2) establishing policies and 
plans, including defining roles and responsibilities, for managing 
changes to requirements and maintaining bidirectional requirements 
traceability.[Footnote 17] 

OPM's retirement modernization requirements processes include some, but 
not all, of the elements needed to effectively develop and manage 
requirements. The agency began an effort to better develop its 
retirement modernization requirements in November 2008. This effort was 
in response to the agency's recognition that its over 1,400 
requirements lacked sufficient detail, were incomplete, and required 
further development. The agency intends to complete this requirements 
development effort in April 2009. However, the requirements will not be 
derived from OPM's concept of operations because the agency is revising 
the concept of operations expected to be completed by April 2009, to 
reflect the program restructuring. Further, OPM documentation indicates 
that the agency has not yet determined the level of detail to which 
requirements should be developed. Additionally, agency officials stated 
that OPM is developing a requirements development process for 
retirement modernization. With respect to requirements management, OPM 
developed an organizational charter that outlined roles and 
responsibilities for supporting efforts to manage requirements. 
However, the agency does not yet have a requirements management plan. 

OPM's prior experience with DBTS illustrates the importance of 
effective requirements development and management. According to 
RetireEZ program officials, insufficiently detailed requirements, 
poorly controlled requirements changes, and inadequate requirements 
traceability were factors that contributed to DBTS not performing as 
expected. Moreover, these requirements development and management 
weaknesses were identified, and recommendations for improvement were 
made by OPM's independent verification and validation contractor before 
DBTS deployment. However, the agency has not yet corrected these 
weaknesses. 

Until OPM fully establishes requirements development and management 
processes, the agency increases the risk that it will (1) identify 
requirements that are neither complete nor sufficiently detailed and 
(2) not effectively manage requirements changes or maintain 
bidirectional traceability, thus further increasing agency risk that it 
will produce a system that does not meet user and operational needs. 

OPM Is Not Positioned to Effectively Manage Retirement Modernization 
Testing: 

Effective testing is an essential component of any program that 
includes developing systems. Generally, the purpose of testing is to 
identify defects or problems in meeting defined system requirements and 
satisfying user needs. To be effectively managed, testing should be 
planned and conducted in a structured and disciplined fashion that 
adheres to recognized guidance and is coordinated with the requirements 
development process.[Footnote 18] Beginning the test planning process 
in the early stages of a program life cycle can reduce rework later in 
the program. Early test planning in coordination with requirements 
development can provide major benefits. For example, planning for test 
activities during the development of requirements may reduce the number 
of defects identified later and the costs related to requirements 
rework or change requests. Further, planning test activities early in a 
program's life cycle can inform requests for proposals and help 
communicate testing expectations to potential vendors. 

OPM has not begun to plan test activities in coordination with 
developing its requirements for the RetireEZ program. According to OPM 
officials, the agency intends to begin its test planning by revising 
the previously developed DBTS test plans after requirements have been 
developed. However, the agency has not yet added test planning to its 
project schedule. 

Early test planning is especially important to avoid repeating the 
agency's experience during DBTS testing when it identified more defects 
than it could resolve before system deployment. In January 2008, we 
reported that an unexpectedly high number of defects were identified 
during testing; yet, the deployment schedule had increased the risk of 
not resolving all defects that needed to be corrected before deploying 
DBTS. According to the RetireEZ program officials, the failure to fully 
address these defects contributed to the limited number of federal 
employees who were successfully processed by the system when it was 
deployed in February 2008. 

If it does not plan test activities early in the life cycle of 
RetireEZ, OPM increases the risk that it will again deploy a system 
that does not satisfy user expectations and meet requirements (i.e., 
accurately calculate retirement benefits) because of its potential 
inability to address a higher number of defects than expected. 
Moreover, criteria used to develop requests for proposals and 
communicate testing expectations to potential vendors could be better 
informed if the agency plans RetireEZ test activities early in the life 
cycle. 

OPM Is Not Providing Effective Oversight for Retirement Modernization: 

GAO and OMB guidance calls for agencies to ensure effective oversight 
of IT projects throughout all life-cycle phases. Critical to effective 
oversight are investment management boards made up of key executives 
who regularly track the progress of IT projects such as system 
acquisitions or modernizations. These boards should maintain adequate 
oversight and track project performance and progress toward predefined 
cost and schedule goals, as well as monitor project benefits and 
exposure to risk. Another element of effective IT oversight is 
employing early warning systems that enable management boards to take 
corrective actions at the first sign of cost, schedule, and performance 
slippages. 

OPM's Investment Review Board was established to ensure that major 
investments are on track by reviewing their progress and determining 
appropriate actions when investments encounter challenges. Despite 
meeting regularly and being provided with information that indicated 
problems with the retirement modernization, the board did not ensure 
that the investment was on track, nor did it determine appropriate 
actions for course correction when needed. For example, from January 
2007 to August 2008 the board met and was presented with reports that 
described problems the retirement modernization program was facing, 
such as the lack of an integrated master schedule and earned value data 
that did not reflect the "reality or current status" of the program. 
However, meeting minutes indicate that no discussion or action was 
taken to address these problems. According to a member of the board, 
OPM guidance regarding how the board is to communicate recommendations 
and corrective actions when needed for the investments it is 
responsible for overseeing has not been established. 

In addition, OPM established an Executive Steering Committee to oversee 
retirement modernization. According to its charter, the committee is to 
provide strategic direction, oversight, and issue resolution to ensure 
that the program maintains alignment with the mission, goals, and 
objectives of the agency and is supported with required resources and 
expertise. However, the committee was inactive for most of 2008 and, 
consequently, did not exercise oversight of the program during a 
crucial period in its development. For example, from January 2008 until 
October 2008, the committee discontinued its formal meetings, and as a 
result, it was not involved in key program decisions, including the 
deployment of DBTS. Further, a member of the committee noted that OPM 
guidance for making recommendations and taking corrective actions also 
has not been provided. 

The ineffectiveness of the board and the inactivity of the committee 
allowed program management weaknesses in the areas of cost estimation, 
EVM, requirements management, and testing to persist and raise concerns 
about OPM's ability to provide meaningful oversight as the agency 
proceeds with its retirement modernization. Without fully functioning 
oversight bodies, OPM cannot monitor modernization activities and make 
the course corrections that effective boards and committees are 
intended to provide. 

Conclusions: 

OPM's retirement modernization initiative is in transition from a 
program that was highly dependent on the success of a major contract 
that no longer exists, to a restructured program that has yet to be 
fully defined. Although the agency has been able to partially implement 
a database of retirement information and improvements to customer 
service, it remains far from implementing six other key capabilities. 
Recognizing that much work remains, OPM has undertaken steps to 
restructure the retirement modernization program, but it has not yet 
produced a complete description of its planned program, including 
fundamental information about the program's scope, implementation 
strategy, lines of responsibility and authority, management processes, 
and schedule. Further, OPM's retirement modernization program 
restructuring does not yet include definitions of results-oriented 
goals and measures against which program performance can be objectively 
and quantitatively assessed. 

In addition, OPM has not overcome managerial shortcomings in key areas 
of program management, including areas that we have previously 
reported. Specifically, the agency is not yet positioned to develop a 
reliable program cost estimate or perform reliable EVM, both of which 
are critical to effective program planning and oversight. Nor has OPM 
overcome weaknesses in its management of system testing and defects, 
two activities that proved problematic as the agency was preparing to 
deploy the RetireEZ system that subsequently was terminated. Adding to 
these long-standing concerns are weaknesses in OPM's process to 
effectively develop and manage requirements for whatever system or 
service it intends to acquire or develop. Finally, these weaknesses 
have been allowed to persist by entities within the agency that were 
ineffective in overseeing the retirement modernization program. 

As a consequence, the agency is faced with significant challenges on 
two fronts: defining and transitioning to its restructured program, and 
addressing new and previously identified managerial weaknesses. Until 
OPM addresses these weaknesses, many of which were previously 
identified by GAO and others, the agency's retirement modernization 
initiative remains at risk of failure. Institutionalizing effective 
planning and management is critical not only for the success of this 
initiative, but also for that of other modernization efforts within the 
agency. 

Recommendations for Executive Action: 

To improve OPM's effort toward planning and implementing its retirement 
modernization program by addressing management weaknesses, we recommend 
that the Director of the Office of Personnel Management provide 
immediate attention to ensure the following six actions are taken: 

* Develop a complete plan for the restructured program that defines the 
scope, implementation strategy, lines of responsibility and authority, 
management processes, and schedule. Further, the plan should establish 
results-oriented (i.e., objective, quantifiable, and measurable) goals 
and associated performance measures for the program. 

* Develop a reliable cost estimate by following the best practice steps 
outlined in our Cost Estimating and Assessment Guide, including 
definition of the estimate's purpose, development of an estimating 
plan, definition of the program's characteristics, and determination of 
the estimating structure. 

* Establish a basis for reliable EVM, when appropriate, by developing a 
reliable program cost estimate, performing an integrated baseline 
review, and validating a performance measurement baseline that reflects 
the program restructuring. 

* Develop a requirements management plan and execute processes 
described in the plan to develop retirement modernization requirements 
in accordance with recognized guidance. 

* Begin RetireEZ test planning activities early in the life cycle. 

* Develop policies and procedures that would establish meaningful 
program oversight and require appropriate action to address management 
deficiencies. 

Agency Comments and Our Evaluation: 

The Director of the Office of Personnel Management provided written 
comments on a draft of this report. (The comments are reproduced in 
appendix II.) In the comments, OPM agreed with our recommendations and 
stated that it had begun to address them. To this end, the Director 
stated that the agency had, among other actions, begun revising its 
retirement modernization plans, developing a new program cost estimate, 
planning for accurate EVM reporting, incorporating recognized guidance 
in requirements management planning, and planning test activities 
during requirements development. If the recommendations are properly 
implemented, they should better position OPM to effectively manage its 
retirement modernization initiative. The agency also provided comments 
on the draft report regarding our description of the federal retirement 
application process, as well as our characterizations of OPM's EVM and 
requirements management capabilities vis-à-vis the retirement 
modernization program. In each of these instances, we made revisions as 
appropriate. 

We are sending copies of this report to the Director of the Office of 
Personnel Management, appropriate congressional committees, and other 
interested parties. In addition, the report is available at no charge 
on the GAO Web site at [hyperlink, http://www.gao.gov. 

If you or your staffs have questions about this report, please contact 
me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices 
of Congressional Relations and Public Affairs may be found on the last 
page of this report. Key contributors to this report are listed in 
appendix III. 

Signed by: 

Valerie C. Melvin: 
Director, Information Management and Human Capital Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

As requested, the objectives of our study were to (1) assess the status 
of the Office of Personnel Management's (OPM) efforts toward planning 
and implementing the RetireEZ program and (2) evaluate the 
effectiveness of the agency's management of the modernization 
initiative. 

To assess the status of OPM's efforts toward planning and implementing 
the RetireEZ program, we: 

* reviewed and analyzed program documentation, including program 
management plans, briefing slides, and project status documentation, to 
identify planned retirement modernization capabilities and determine to 
what extent these capabilities have been implemented; 

* evaluated the agency's documentation about restructuring the program 
and analyzed the extent to which the documentation describes current 
and planned RetireEZ program activities; 

* identified and evaluated the agency's program goals and measures and 
compared them to relevant guidance to determine the extent to which the 
goals and measures are described in results-oriented terms; 

* supplemented agency program documentation and our analyses by 
interviewing agency and contractor officials, including the OPM 
Director, Chief Information Officer, Chief Financial Officer, Director 
of Modernization, Associate Director for Human Resources Products and 
Services Division, and executives from Hewitt Associates and Northrop 
Grumman Corporation; and: 

* observed retirement operations and ongoing modernization activities 
at OPM and contractor facilities in Washington, D.C.; Boyers, 
Pennsylvania; and Herndon, Virginia. 

To determine the effectiveness of OPM's management of the retirement 
modernization initiative, we evaluated the agency's management of 
program cost estimating, earned value management (EVM), requirements, 
test planning, and oversight and compared the agency's work in each 
area with recognized best practices and guidance. Specifically, 

* to evaluate whether OPM effectively developed a reliable program cost 
estimate, we analyzed the agency's program documentation and determined 
to what extent the agency had completed key activities described in our 
Cost Estimating and Assessment Guide; 

* to assess OPM's implementation of EVM, we reviewed program progress 
reporting documentation and compared the agency's plans for restarting 
its EVM-based progress reporting against relevant guidance, including 
our Cost Estimating and Assessment Guide; 

* regarding requirements management, we evaluated OPM's processes for 
developing and managing retirement systems modernization requirements 
and compared the effectiveness of those processes against recognized 
guidance; 

* to determine the effectiveness of the agency's test planning for the 
retirement modernization, we reviewed program activities and test plans 
against best practices and evaluated the extent to which the agency has 
begun planning for these activities; and: 

* we reviewed and analyzed documentation from program oversight 
entities and evaluated the extent to which these entities took actions 
toward ensuring the RetireEZ program was being effectively overseen. 

We also evaluated OPM's progress toward implementing our open 
recommendations and interviewed OPM and contractor officials as noted. 

We conducted this performance audit at OPM headquarters in Washington, 
D.C., the Retirement Operations Center for OPM in Boyers, Pennsylvania, 
and contractor facilities in Herndon, Virginia, from May 2008 through 
April 2009, in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Office of Personnel Management: 

United States Office Of Personnel Management: 
The Director: 
Washington, DC 20415: 
"Our mission is to ensure the Federal Government has an effective 
civilian workforce."
[hyperlink, www.opm.gov] 
[hyperlink, www.usajobs.gov] 

April 15, 2009: 

Valerie C. Melvin: 
Director, Information Management and Human Capital Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Melvin: 

Thank you for the opportunity to provide comments in response to the 
Government Accountability Office's (GAO) draft report entitled Office 
of Personnel Management: Retirement Modernization Planning and 
Management Shortcomings Need to Be Addressed (GAO-09-529). The Office 
of Personnel (OPM) agrees with the six recommendations outlined in the 
draft report and has already begun to address them. 

As you are aware, the primary objectives of retirement business 
transformation are: 1. to migrate from a paper-based environment to an 
electronic system of retirement data management; 2. to improve 
retirement processing and post-retirement processing efficiency; and 3. 
to enhance customer service. To that end, the Program Management Office 
(PMO), Executive Steering Committee (ESC), and program stakeholders 
continue to refine the retirement systems modernization program five-
phased implementation plan to ensure each phase and its initiatives 
directly align to retirement systems modernization end-state 
capabilities. Progress continues on the fundamental requirements of the 
plan - automating data collection, conversion of data from paper 
records to electronic media and the re-orientation of agency staff 
concerning their critical role in improving data quality and improving 
the retirement experience. These critical, foundational steps are 
needed to prepare the way for retirement modernization going forward. 
Progress has also been made in the establishment of basic performance 
measures and their monthly reporting. These measures are related to the 
fundamental requirement for improvements and allow the agency to ensure 
that current investments result in better services and reductions in 
future cost increases. 

As OPM's new Director, I consider one of my first and most important 
responsibilities to get this program fixed and working so that the 
retirement experience for government employees is as simple and 
seamless as current technology allows. 

To ensure success moving forward, I will solicit the views of experts 
at Federal agencies that have undertaken similarly ambitious efforts to 
modernize federal business processes and systems. Their views and 
experiences may be useful to OPM in shaping the technology way ahead 
for retirement systems modernization. We will discuss best practices 
and pitfalls to avoid with agencies that changed their business models 
and supporting systems to deliver services that are citizen-focused, 
agile, responsive and accessible. This infusion of input from agencies 
with expertise and experience with similarly large systems, combined 
with a planned request for information to the private sector will 
enhance OPM's ability to incorporate innovative approaches to our 
efforts to modernize and improve retirement services for federal 
retirees. 

In addition, OPM has engaged agencies to be part of the solution, i.e. 
to improve the quality and completeness of retirement applications 
submitted to OPM. With them as full participants, they are far more 
likely to buy in to the concept of shared responsibility for the 
retirement process. Awareness and appreciation has increased to the 
point that it is now clearly recognized as a joint endeavor among the 
employee's agency at retirement, the employee, and OPM. One example of 
this was a Retirement Symposium OPM hosted for more than 175 human 
resource and payroll employees from throughout the federal government 
on January 28th, 2009. In addition to the Symposium, OPM continues to 
work one-on-one with agencies to improve their individual processes 
based on the data collected on their retirement applications in the 
2008 quality audit. Following an initial quality audit review meeting, 
agencies developed and submitted to OPM action plans to improve quality 
based on feedback from OPM quality audits. OPM is developing plans for 
regular retirement application quality audits to ensure continued 
feedback to agencies to help them make improvement on their retirement 
application submissions. 

OPM has reached out to over 40 organizations by conducting kick-off 
meetings and working sessions on a series of activities designed to 
trigger data gathering on the agency's systems, data, and business 
structure; discussions on secure electronic data transfer; and data 
requirements. The bottom line is that OPM has established partnerships 
with the agencies to ensure their support and buy-in moving forward. 

I agree with the characterization that meaningful oversight is needed 
and that is the first issue addressed this past November with 
designation of the new Project Manager, Deputy Project Manager and a 
reconvened Executive Steering Committee. I want to assure you, however, 
that the shortcomings identified within retirement systems 
modernization are not indicative of all acquisition programs at OPM. 
Moreover, it is important to emphasize that, late in 2007 and early in 
2008, there was a determination to directly align the day-to-day 
oversight of the program under the Director to ensure critical 
milestones could be closely monitored and met. Unfortunately, the 
desire to achieve success within an overly ambitious timeframe resulted 
in shortcuts that proved to be somewhat short-sighted. Nonetheless, 
this agency takes GAO's findings very seriously and is addressing each 
of the six recommendations with specific actions as outlined below: 

I. Complete Project Plan: 

OPM recognizes the value of strong program management plans and through 
implementation of previous GAO recommendations, the retirement systems 
modernization program developed a robust set of program planning and 
management documentation. The program recognizes that the current 
baselined versions of the program documents do not fully reflect the 
state of the program, as they were based on a single Defined Benefits 
Technology Solution (DBTS) with supporting implementing vendors. OPM 
began the process of updating the documents in December 2008. 

Program documentation for retirement systems modernization includes a 
program management plan (PMP) that defines the scope and objectives of 
retirement systems modernization. It also describes the overall 
governance structure for retirement systems modernization, including 
lines of responsibility and authority, and documents the complete set 
of management processes for the program. This set of management 
processes includes planning and earned value management, funds 
management, contract management, configuration management, and 
integration with OPM's Information Technology System Manager (ITSM) 
system development methodology. Also included are risk management, 
requirements management, quality management, communication and security 
management. In a number of these areas, such as, risk management and 
requirements management, further details are provided in separate 
documents referenced in the PMP. 

A host of other program documents are fundamentally linked to the PMP 
and provide further structure to the program, such as the program 
concept of operations (CONOPS), the Work Breakdown Structure (WBS) and 
Integrated Master Schedule. Examples of these linkages are the 
inclusion of significant milestones from the program's Integrated 
Master Schedule in the PMP, and the program's five phase implementation 
approach which is detailed in the CONOPS and described in the PMP. 

The PMP and associated documents are currently undergoing a thorough 
revision based on a change in program approach and updates to the 
processes and documents described above. The table below provides the 
list of documents that are undergoing revisions: 

Program Management Documentation: Concept of Operations; 
Date of	Projected Completion: 2/20/07; 
Current Version	of Update: May 2009 (in development). 

Program Management Documentation: Configuration Management Process 
Guide; 
Date of	Projected Completion: 6/25/07; 
Current Version	of Update: April 2009 (completed). 

Program Management Documentation: Program Management Plan; 
Date of	Projected Completion: 12/4/07; 
Current Version	of Update: June 2009 (in development). 

Program Management Documentation: Requirements Management Plan; 
Date of	Projected Completion: 6/25/07; 
Current Version	of Update: May 2009 (in coordination). 

Program Management Documentation: Risk Management Plan; 
Date of	Projected Completion: 12/4/07; 
Current Version	of Update: May 2009 (in coordination). 

Program Management Documentation: Program Cost Estimate; 
Date of	Projected Completion: 12/19/08; 
Current Version	of Update: June 2009 (in development). 

Program Management Documentation: Test Management Plan; 
Date of	Projected Completion: 10/16/07; 
Current Version	of Update: June 2009 (in development). 

Program Management Documentation: Work Breakdown Structure (WBS); 
Date of	Projected Completion: 3/3/09; 
Current Version	of Update: March 2009 (completed). 

Program Management Documentation: WBS Dictionary; 
Date of	Projected Completion: 7/17/07; 
Current Version	of Update: May 2009 (in development). 

[End of table] 

As outlined above, updates to the PMP are ongoing and are not likely to 
be completed until the other underlying documents are finalized, due to 
the linkages among them. I anticipate this process to be completed by 
June 2009. OPM is currently developing goals which tie to the 
capabilities the program is working to make a reality. Establishing 
results-oriented program goals and measures requires coordinating with 
multiple stakeholders within OPM. This ensures harmonization with 
overall agency strategic and operational goals, which are developed 
under their own timetables, and as a result, are anticipated to take 
longer. 

When the updates currently underway are completed, and with the input 
we are seeking from other agencies and the private sector, I am 
confident that the retirement systems modernization program will have a 
solid foundation to guide its efforts and be properly positioned to 
deliver modernized retirement capabilities and systems. 

II. Reliable Cost Estimate: 

OPM recognizes the importance of a comprehensive, well-documented, 
accurate and credible cost estimate for managing the retirement systems 
modernization program. Previous cost estimates were based primarily on 
the value of vendor, firm-fixed price contracts (which comprised the 
vast majority of program costs), rather than a build-up of program 
costs. This was deemed to be the most accurate approach at the time. 
However, given the substantial changes to the program in the past year, 
and recognition of the benefits of a more rigorous approach to cost 
estimation, the weaknesses in previous estimates are being actively 
addressed. Specifically, OPM is using the GAO Cost Estimating and 
Assessment Guide (published in March 2009) and on-going program efforts 
to develop a comprehensive CONOPS and the program integrated master 
schedule (IMS) as a guide to developing a comprehensive cost estimate 
for the program. 

The cost estimation work is part of a broader business case analysis 
that is re-examining all aspects of the program and engaging 
stakeholders throughout OPM to ensure that program replanning provides 
the proper foundation for the program. The GAO Cost Estimating and 
Assessment Guide are being used during every stage of the business case 
analysis and will complete each of the twelve recommended steps. OPM 
has made considerable progress in the initial stages of developing the 
cost estimate, including: 

* Define estimate purpose The purpose of the estimate is to inform 
budget planning for the program, inform the justification for continued 
investment in the program, and provide the basis for measuring program 
performance. The level of detail has been established at level 4 in the 
program work breakdown structure (WBS), and the overall scope of the 
estimate matches the scope laid out in the draft program concept of 
operations (CONOPS) and is consistent with OPM's 5 phase implementation 
approach. 

* Develop estimating plan - The cost estimating team was established in 
mid-February and has developed the master schedule and associated 
timeline for completion of the estimate. The estimating approach is 
being tailored to specific program initiatives. Preparations to conduct 
the independent cost estimate are underway. 

* Define program characteristics The program has been developing 
documentation such as an updated CONOPS and has developed an RFI, the 
results of which will help to inform the technical baseline 
description. Additionally, interviews with program stakeholders have 
collected a wealth of information on program needs, the current 
environment, alternatives for going forward, and the associated risks. 

* Determine estimating structure - The estimating structure for the 
cost estimate is the updated program WBS which was approved by the 
program change control board in early March. The WBS divides program 
work into phases and initiatives and work packages detail the full 
system development lifecycle for each initiative. Leveraging the 
approved WBS as the structure for the cost estimation will ensure an 
end-to-end cost estimate for the program that reflects dependencies 
across initiatives. Development of the program WBS dictionary is 
underway to ensure clear understanding of each element of the WBS and 
appropriate classification of work and resources within the program's 
Integrated Master Schedule (IMS). Specific technical components of each 
initiative are being outlined in the CONOPS and will provide additional 
detail for the cost estimate. 

The work underway will position us to establish a credible, 
comprehensive, and defensible cost estimate for retirement systems 
modernization in the coming months. OPM is working with internal and 
external stakeholders to ensure all parties are in agreement with the 
approach for developing the cost estimate. It is clear that a good cost 
estimate is a living artifact and that processes must be developed to 
update the cost estimate throughout the life of the program in order to 
maintain its accuracy over time. 

III. Earned Value Management: 

OPM agrees with GAO on the importance of Earned Value Management (EVM) 
for measuring and reporting program progress. As a measure of the 
importance OPM places on reliable EVM reporting, the deficiency of 
quality EVM data was a major factor in the decision to stop work on the 
DBTS contract. Retirement system modernization has invested 
considerable resources in EVM processes and has required vendors to 
provide detailed EVM data for measuring program progress. Cessation of 
work on the DBTS contract rendered it impossible to continue program-
level EVM reporting.[Footnote 19] Given these deficiencies, OPM was 
unable to conduct an integrated baseline review as recommended by GAO 
in its January 2008 report. 

OPM recognizes the value of a reliable cost estimate and is currently 
developing one in order to establish a new program baseline upon which 
to report future progress. In the interim, OPM is restarting EVM 
reporting based on an updated Integrated Master Schedule (IMS) as a 
means to capture program status information and to acclimate program 
vendors and business owners to the process of providing and statusing 
program EVM data. This is a significant effort requiring frequent 
meetings with initiative owners and other OPM stakeholders to decompose 
work packages into constituent tasks and identifying cross-program 
dependencies. OPM recognizes that the value of the resulting data will 
be limited until the cost estimate is complete and the program can 
formally establish the performance measurement baseline (PMB). As such, 
OPM will be working to baseline the program throughout the cost 
estimation process outlined above. Program EVM status for FY2009 and 
FY2010 will be statused and reported on a monthly basis while OPM 
completes the cost estimation and establishes a comprehensive, credible 
PMB. OPM plans to execute an integrated baseline review as soon as 
reasonably possible after completing the cost estimate and establishing 
the PMB. 

OPM has invested considerable effort in creating the artifacts 
necessary for accurate EVM reporting, having developed an IMS with over 
1,000 activities. At the same time, OPM has updated and refined other 
program artifacts, such as the program CONOPS. Given the concurrent 
development of these artifacts, OPM has worked to ensure that updates 
in one document are applied across the full set of program artifacts, 
through continuous communication within the project team, thus 
minimizing gaps or conflicting information. 

In addition, OPM is finalizing a briefing on the EVMS to share with 
program stakeholders and initiative owners in the next month. This 
presentation will review the expectations for EVM statusing and 
reporting and the importance of robust EVM. Additionally, the PMO will 
provide training to federal and vendor initiative owners who will be 
expected to provide status on a monthly basis and will be responsible 
for developing variance reports. 

IV. Requirements Management: 

OPM supports the implementation of recognized best practices to develop 
and manage detailed retirement modernization requirements. Previous 
requirements development efforts focused on tailoring requirements to 
the Defined Benefits Technology Solution (DBTS), resulting in 
requirements that were oriented to commercial-off-the-shelf software, 
but which lacked specific detail about complex federal retirement 
business processes. As GAO noted, lacking an adequate degree of 
specificity, previous OPM requirements efforts were unable to trace 
software specifications and functionality to the complex federal 
retirement business processes the technology solutions were designed to 
support. This lack of traceability was one factor that contributed to 
the unsuccessful DBTS technology solution. Current requirements 
management efforts are based on industry best practices and technology 
standards and clearly assign responsibilities across the organization. 
They also establish repeatable procedures for all phases of 
requirements process and ensure the management of the requirements 
baseline through standard Configuration Management (CM) and 
Configuration Control Board (CCB) processes. 

Leveraging Carnegie Mellon's Software Engineering Institute (SEI) 
Capability Maturity Model Integration (CMMI)-Development v1.2 and 
Institute of Electrical and Electronics Engineers (IEEE) Standard 
1233 - 1998, IEEE Guide for Developing System Requirements 
Specifications, OPM has developed a requirements management framework 
and procedures which will ensure future clarity, traceability, and 
accuracy. The CMMI-Development framework provides standard process 
steps for Requirements Management. Also, OPM's Information Technology 
Systems Manager (ITSM) incorporates both CMMI and IEEE requirements 
standards as part of OPM's System Development Life Cycle. These steps 
have been incorporated into the Retirement Requirements Management 
process as follows. 

* Develop Business Requirements - The operational concepts captured in 
the Concept of Operations (CONOPS) will be articulated as business 
processes, data and requirements. These requirements are not technical 
but represent the operational user's views of the business processes. 
Retirement systems cannot be developed from these requirements as they 
lack technical specificity. Once these requirements are documented and 
validated by the stakeholders, they will be further deconstructed into 
technical system requirements. 

* Develop System Specifications - User business requirements need to be 
translated into product requirements articulated in technical terms 
before system design can begin. The system specifications transform the 
user requirements into technical language, detailed enough to support 
system design. The system requirements are then organized by system 
components. An integral part of the development of these system 
specifications is the elicitation of interface requirements for 
connecting related systems with each other so that they are 
interoperable and can exchange data. 

* Testing - Test procedures are developed to verify each system 
specification and that system function meets the business need stated 
in the requirements. OPM is in the process of implementing a Test 
Management Plan to ensure that test procedures verify that developed 
systems meets the documented system and business requirements. The test 
procedures will be tracked as part of the requirements management 
process in order to ensure that the requirements they represent can be 
tested and verified. 

* Requirements Management and Traceability - The goal of Requirements 
Management is to ensure that requirements are elicited, developed and 
tracked in a systematic manner and managed through standard 
Configuration Management and Configuration Control Board processes. 
Requirements traceability provides a method to link business 
requirements and technical specifications in order to validate that the 
system design meets the user's business requirements. 

OPM acknowledges that previous requirements management efforts were 
geared specifically to the implementation of a single commercial system 
and did not describe complex federal retirement business processes and 
requirements to the necessary level of detail. Thus, the CMMI-
Development (Requirements Management) Framework and processes will 
enable OPM to develop business requirements and specifications, based 
on an approved Concept of Operations, that reliably produce modem 
systems and system components that meet the Retirement program's 
business needs. 

V. Test Planning: 

OPM recognizes that effective testing, based on established standards 
and best practices, is an essential component in the delivery of 
federal Retirement capabilities and systems. System testing can 
identify defects early enough in the development process to allow for 
corrections before deployment, verify that the developed system matches 
system requirements and specifications, and validate that the system 
meets user needs. In the case of Retirement Modernization testing, 
basic test management plans and processes were in place to guide and 
evaluate system testing. OPM test management processes generally 
conformed to established software engineering standards. However, a 
number of factors impacted the timely execution of testing and 
correction of system defects. These factors contributed to a deployed 
system which did not fully deliver required Retirement Modernization 
capabilities. 

The Retirement Modernization program issued a Retirement System 
Modernization Test Approach and User Acceptance Testing (UAT) Execution 
Plan in 2007 with updates in early 2008. In line with testing best 
practices described in Institute of Electrical and Electronics 
Engineering (IEEE) standard 829, IEEE Standard for Software Test 
Documentation, the OPM testing approach included test scope and 
methodology, timeline, test entry and exit criteria, guidance for 
defect tracking and provisions for independent validation and 
verification of test results. 

As GAO notes, the project test plans were issued midway through the 
development cycle, rather than at the inception of the process, thus 
compressing the time allocated for testing into an extremely narrow 
window. Complicating the shortened testing timeline, system 
requirements and system specifications that drove development did not 
capture, at a detailed level, a number of highly complex federal 
retirement business processes required to fully deliver clean data 
required for successful annuity calculations. Additionally, interface 
specifications and reporting requirements were still in the development 
phase three months before the system deployed, leaving the delivered 
system unable to pass data to legacy systems critical to retiree and 
annuitant services. Due to the compressed testing schedule, system 
component testing was conducted in parallel or concurrently vice 
sequentially, which did not allow developers to identify 
interoperability issues or defects among the various system components. 

As the testing cycle proceeded, using structured scenarios, system 
defects were identified and documented and defect tracking and 
assessments provided as the Test Plans required. Given the compressed 
test window and the complexity of implementing a commercial system for 
the federal sector, the system deficiencies could not be effectively 
resolved prior to the system deployment date. The DBTS was deployed 
with reduced initial operating capability, but with the hope that 
further rapid development would follow. The system was deployed and 
subsequently manual intervention was required to augment retirement 
calculations. 

OPM acknowledges that planning test activities early in the retirement 
systems modernization lifecycle in conjunction with the requirements 
development process will potentially reduce future system defects. 
While the DBTS Test Approach and User Acceptance Test Execution Plan 
were tailored specifically to the DBTS solution, OPM will release an 
updated Retirement Systems Modernization Test Management Plan, 
consistent with best practices and industry standards, and fully 
correlated with the Requirements Management Plan. Retirement systems 
modernization test activities will be included in the Project 
Management Plan, and test activities will be fully integrated with the 
requirements development process. OPM is confident that planning and 
integrating test management activities with requirements development 
will preclude the level of defects experienced in previous development 
efforts and result in retirement systems modernization capabilities 
that are fully consistent and supportive of federal retirement 
services. 

VI. Policy/Procedures for Program Oversight: 

OPM has established different types of oversight groups to ensure that 
acquisition programs remain on track to meet the requirements, cost, 
and schedule aligned to the selected programs. For retirement systems 
modernization, OPM leadership initially determined that an Executive 
Steering Committee would be the proper oversight group for the 
retirement system modernization program. Over the course of time, 
however, it was determined that the program office would report 
directly to the OPM Director to streamline the process and to 
accelerate the program. Therefore, the GAO report is correct in saying 
that the ESC was suspended during the period of January 2008 to October 
2008. 

With the change in leadership at OPM in August 2009, the new Acting 
Director who headed the agency at that time quickly established a 
senior level working group (Sunrise Group) to oversee the management of 
the retirement modernization program. To quickly get the retirement 
program back on track, the Sunrise Group established a series of 
actions and each stakeholder took responsibility for progress on their 
respective actions. Based on this oversight, many actions quickly 
occurred, including increasing the number of personnel at the call 
center, engagements with the shared service centers, and the imaging of 
documents at the Retirement Operations Center, to name a few. After an 
extensive Federal search a program manager and a deputy program manager 
were selected to take responsibility for the day-to-day operation of 
the program. One of the first duties of the program manager was to 
craft a revised charter for the Executive Steering Committee to take 
over direct oversight of the program. The committee is made up of 
senior level (SES) stakeholders that meet on monthly basis, in addition 
to the weekly meetings that continue. 

OPM also has an Information Technology (IT) Investment Review Board 
ORB) which was established according to Clinger-Cohen Act requirements 
to advise the agency head on issues related to the selection and 
evaluation of IT investments. The Board, which includes as its members 
senior executives of the agency, met consistently from 2005 to evaluate 
investments and recommend approval or disapproval of funding. In 
addition, the Board members received independent assessments of 
investment status, including the status of retirement system 
modernization. Board members, who met routinely with the agency 
Director in their capacities as her direct reports, provided feedback 
and advice to the Director, influenced by the independent assessments 
as well as the project manager's reports provided at meetings of the 
Board. Because the Clinger-Cohen Act assigns responsibility for capital 
planning and investment control and other requirements of the Act to 
the agency head, OPM's Investment Review Board advised the Director, 
but the IRB was not established to decide how the agency would resolve 
concerns and risks associated with the retirement system modernization 
program that were brought to its attention. Such decisions were made by 
the Director per the Clinger-Cohen Act. 

In addition, OPM specifically submits the following comments for GAO's 
consideration: 

1. "Employing agency's" should be added to clarify the first sentence 
on page 7 of the draft report, thus changing it to read, "Once an 
employee submits an application, the employing agency's human resources 
office provides retirement counseling services to the employee and 
augments the retirement application with additional paperwork. 

2. Regarding the statement at the bottom of page 7 of the draft report, 
"Payroll offices are required to submit the package to OPM within 30 
days of the retiree's separation date.", we recommend clarifying that 
this is currently only a timeliness standard for this information. This 
standard is being augmented by the Agency Retirement Record Improvement 
Plan which emphasizes the shared responsibility between OPM and 
agencies to have complete and accurate information on which to base the 
adjudication. 

3. Regarding the statement in the 9th line of the 2nd paragraph on page 
8, "Retirement processing includes functions such as determining 
retirement eligibility, inputting data into benefit calculators, and 
providing customer service," we wish to emphasize that the referenced 
customer service is in relation to the initial adjudication. This does 
not include the customer service group included in the call centers 
that provide service to existing annuitants and survivors. 

4. Regarding the heading on page 23, we recommend modifying the section 
header to refer to retirement systems modernization to be consistent 
with the content of this section and the scope of the report. In this 
section of the report, the authors refer to the application of Earned 
Value Management (EVM) techniques in the retirement systems 
modernization program. However, OPM has successfully managed several IT 
investments using EVM techniques. Those projects and the agency's EVM 
program as a whole were not addressed in this study and are not the 
subject of this report. Therefore, OPM requests that you change the 
section header to more explicitly articulate your concerns about the 
implementation of EVM in the retirement systems modernization program, 
since the concerns you have raised are related to retirement system 
modernization rather than the agency as a whole. 

5. Regarding the section title in the middle of page 24, this title 
similarly extrapolates from the retirement modernization analysis to an 
agency-wide issue. OPM recommends that this wording be changed to 
reflect areas of concern specifically related to requirements 
management for retirement modernization. In addition, as referenced 
above, OPM does have an established system development life cycle 
(SDLC) process called IT Systems Manager (ITSM) that has clear, 
established processes for development and management of requirements. 

6. Similarly, OPM recommends modifying the statement in the middle of 
page 25 that currently reads, "Additionally, agency officials stated 
that OPM is developing a requirements development process but did not 
specify when it will be finalized." to indicate that the requirements 
process under development is specifically related to retirement 
modernization. OPM has a requirements development process included in 
the IT Systems Manager (ITSM) that has been in existence since 2003. 

Thank you for this opportunity to comment on GAO's draft report. I am 
excited about meeting the challenges that lie ahead and look forward 
GAO's guidance in addressing them. 

Sincerely, 

Signed by: 

John Berry: 
Director: 

[End of section] 

Appendix III: GAO Staff Contact and Acknowledgments: 

GAO Contact: 

Valerie C. Melvin (202) 512-6304 or melvinv@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, key contributions to this 
report were made by Mark T. Bird, Assistant Director; Barbara S. 
Collier; Neil J. Doherty; David A. Hong; Thomas J. Johnson; Rebecca E. 
LaPaze; Lee A. McCracken; Teresa M. Neven; Melissa K. Schermerhorn; 
Donald A. Sebers; and John P. Smith. 

[End of section] 

Footnotes: 

[1] GAO, Office of Personnel Management: Improvements Needed to Ensure 
Successful Retirement Systems Modernization, GAO-08-345 (Washington, 
D.C.: Jan 31, 2008). 

[2] The Social Security Administration is responsible for administering 
Social Security, and the Federal Retirement Thrift Investment Board 
administers the defined contribution system known as the Thrift Savings 
Plan. Defined benefit plans calculate benefit amounts in advance of 
retirement based on factors such as salary level and years of service, 
and defined contribution plans calculate benefit amounts based on how 
the amount is invested by the employee and employer. 

[3] OPM, An Analysis of Federal Employee Retirement Data: Predicting 
Future Retirements and Examining Factors Relevant to Retiring from the 
Federal Service (March 2008). 

[4] OPM reported in November 2008 that it has made improvements to this 
process and is currently providing retirees with interim payments that 
are about 90 percent of the monthly payment which they are entitled. 

[5] OPM Retirement Insurance Service, FERS Automated Process System 
Information Technology Resources Board Review (Washington, D.C., July 
16, 1996). Executive Order 13011, Section 5 (July 1996), established 
that the Information Technology Resources Board was to provide 
independent assessments to assist in the development, acquisition, and 
management of selected major information systems and to provide 
recommendations to agency heads and OMB as appropriate. Executive Order 
13011 was revoked by Executive Order 13403 (May 2006). 

[6] GAO, Office of Personnel Management: Retirement Systems 
Modernization Program Faces Numerous Challenges, GAO-05-237 
(Washington, D.C.: Feb. 28, 2005). 

[7] GAO-08-345. 

[8] GAO, Comments on the Office of Personnel Management's February 20, 
2008 Report to Congress Regarding the Retirement System Modernization, 
GAO-08-576R (Washington, D.C.: Mar. 28, 2008). 

[9] Institute of Electrical and Electronics Engineers (IEEE), IEEE/EIA 
Guide for Information Technology, IEEE/EIA 12207.1-1997 (April 1998). 

[10] OMB, Circular No. A-11, Preparation, Submission, and Execution of 
the Budget (Washington, D.C., Executive Office of the President, June 
2008); Circular No. A-130 Revised, Management of Federal Information 
Resources (Washington, D.C., Executive Office of the President, Nov. 
28, 2000); and Capital Programming Guide: Supplement to Circular A-11, 
Part 7: Planning, Budgeting, and Acquisition of Capital Assets 
(Washington, D.C., Executive Office of the President, June 2006). 

[11] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, 
D.C.: March 2009). 

[12] GAO-08-345. 

[13] OMB issued policy guidance (M-05-23) to agency chief information 
officers on improving technology projects that includes requirements 
for reporting performance to OMB using EVM (August 2005). OPM, Earned 
Value Management Policy (June 2005). 

[14] GAO-08-345. 

[15] Carnegie Mellon Software Engineering Institute, Capability 
Maturity Model® Integration for Development, Version 1.2 (Pittsburgh, 
Pa., August 2006), and Software Acquisition Capability Maturity Model® 
(SA-CMM®) version 1.03, CMU/SEI-2002-TR-010 (Pittsburgh, Pa., March 
2002); and the Institute of Electrical and Electronic Engineers (IEEE), 
1362-1998, IEEE Guide for Information Technology--System Definition-- 
Concept of Operations Document (New York, N.Y.,1998). 

[16] A concept of operations is a user-oriented document that describes 
the characteristics of a proposed system from the users' viewpoint. 

[17] Maintaining bidirectional requirement traceability means that 
system-level requirements can be traced both backward to higher level 
business or operational requirements, and forward to system design 
specifications and test plans. 

[18] See for example, Carnegie Mellon Software Engineering Institute, 
Software Acquisition Planning Guidelines, CMU/SEI-2005-HB-006 
(Pittsburgh, Pa., December 2005); IEEE Standard 829-2008, Standard for 
Software and System Test Documentation (July 2008); and IEEE Standard 
1044-1993, Standard Classification for Software Anomalies (December 
1993). 

[19] Project level EV reporting for the program's PDCC vendor continued 
without interruption and continues to be used to track project 
performance. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: