This is the accessible text file for GAO report number GAO-11-737R entitled 'Financial Education and Counseling Pilot Program' which was released on July 27, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-11-737R: United States Government Accountability Office: Washington, DC 20548: July 27, 2011: The Honorable Tim Johnson: Chairman: The Honorable Richard C. Shelby: Ranking Member: Committee on Banking, Housing, and Urban Affairs: United States Senate: The Honorable Spencer Bachus: Chairman: The Honorable Barney Frank: Ranking Member: Committee on Financial Services: House of Representatives: Subject: Financial Education and Counseling Pilot Program: The federal government has numerous programs designed to improve Americans' financial literacy, some of which are targeted at helping consumers determine whether and when to purchase a home, how to manage a mortgage, and how to deal with setbacks that could limit their ability to make timely mortgage payments. However, as we have reported, little is known about the effectiveness of specific strategies for improving financial literacy.[Footnote 1] In the Housing and Economic Recovery Act of 2008 (HERA), Congress created a pilot program to provide grants to providers of financial education and counseling services to prospective homebuyers.[Footnote 2] Pursuant to HERA, the goals of this education and counseling include increasing the knowledge and decision-making capabilities of prospective homebuyers, identifying successful methods resulting in positive behavioral change for financial empowerment, and educating prospective homebuyers about options for building savings. HERA also mandated that we submit a report to Congress evaluating this grant program, which was later named the Financial Education and Counseling (FEC) Pilot Program. Accordingly, the objectives of this report are to describe (1) the characteristics of the organizations providing services under the FEC program and how they were selected, and (2) what is known about the program's impact in improving the financial situation and behavior of homeowners and prospective homebuyers who participate in the program. To address these objectives, we reviewed FEC program applications, assistance agreements, and performance reports from organizations awarded grants through the program. In addition, we reviewed the Department of the Treasury's (Treasury) process and criteria for selecting grantees. We also interviewed all nine grantees of the program and staff at Treasury, which administers the program.[Footnote 3] Finally, we assessed the reliability of the data we reported and determined that the data were sufficiently reliable for the purposes of our report. We conducted this work from January to July 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Enclosure I contains a more detailed description of our scope and methodology. Background: Section 1132 of HERA authorized the Secretary of the Treasury to make grants to eligible organizations to provide financial education and counseling to prospective homebuyers through what Treasury subsequently named the FEC Pilot Program. Treasury's Community Development Financial Institutions (CDFI) Fund administers the program, in collaboration with the department's Office of Financial Education and Financial Access (OFEFA). HERA authorizes not more than five pilot project grants, which the act states are intended to identify successful methods resulting in positive behavioral change and to establish program models for organizations to carry out effective counseling services.[Footnote 4] For fiscal year 2009, Congress appropriated $2 million to the CDFI Fund for the FEC program, and Treasury selected five organizations, each receiving $400,000, to provide services. For fiscal year 2010, Congress appropriated $4.15 million, of which Congress designated $3.15 million for an eligible organization in Hawaii. Treasury selected one eligible organization in Hawaii and three other organizations (two receiving $400,000 each and one receiving $200,000) to provide services. Under HERA, to be eligible for grant funding, an organization must be either a housing counseling agency certified by the Secretary of Housing and Urban Development (HUD); a state, local, or tribal government agency; a community development financial institution or a credit union; or any collaborative effort of any of these entities. Treasury's Notices of Funds Availability for the FEC program set certain minimum eligibility requirements related to applicants' ability and experience and stated that the program is targeted at low-income, low-wealth, and moderate-income individuals at least 18 years of age who do not currently own a home or pay a mortgage on a residence. Organizations receiving grant funds may use those funds to cover expenses such as personnel (salaries and benefits), consulting fees and contracts, and materials, supplies, and equipment, among other things. Financial education and counseling related to housing and homeownership (broadly referred to as housing counseling) can take several forms. Financial education generally provides individuals with information on topics such as the importance of saving and budgeting and may take place in a group setting. Counseling generally offers individuals personalized advice in a one-on-one setting and may include prepurchase counseling for potential homebuyers and postpurchase counseling on subjects such as home maintenance.[Footnote 5] Housing counseling can be geared toward various populations-- homebuyers, homeowners, renters, senior citizens, and other populations with particular housing goals. It can take place in person, over the telephone, via a self-study computer module, or with a workbook and can vary in length from a single session to several sessions spread over a period of weeks or months. As we recently reported, research on the impact of financial education and counseling on consumer behavior has been limited. In a recent review of the literature, we identified only one evidence-based study that evaluated the effectiveness of a housing counseling program.[Footnote 6] As a condition of receiving grant funds under the FEC program, grantees are to report on the results of five performance goals within 6 months of disbursement and annually thereafter. Two of these are output goals: number of participants served and hours of class time, counseling, and instruction provided. The other three are impact goals that the grantees select. These impact goals seek to gauge the effect of counseling activities and include measures such as the average percentage increase in participants' household savings, average percentage decrease in household debt, and percentage of participants who improve their credit scores or use formal budgeting tools. Grantees for the fiscal year 2009 funding round of the FEC program received their grant funds in August 2010 and submitted the initial results of their output and impact goals to Treasury in January 2011. Grantees of the fiscal year 2010 round are not required to report their initial results to Treasury until July 30, 2011. The Nine FEC Grantees Are Generally Experienced and Were Selected from among Hundreds of Applicants: FEC Grantees Are HUD-Approved Housing Counseling Agencies That Primarily Target Low-Income Populations: The nine grantees that have received FEC funding are experienced and geographically diverse organizations. As seen in table 1, key characteristics of the organizations providing services under the FEC program include the following: * Type of organization. Eight of the organizations are nonprofit organizations that provide financial education and housing counseling services, and one is a public housing authority. * Geographic diversity. The nine grantees are geographically diverse, representing most major areas of the country and a mix of urban and rural regions. * Financial education or housing counseling experience. Most of the grantees have had at least 10 years of experience in financial education or housing counseling. For example, the Boulder County Housing Authority reported over 20 years of experience in housing counseling. The organization with the least amount of experience, Solita's House, Inc., has been providing counseling for roughly 5 years. * Experience with federal grants. All of the grantees had recently received other federal grants, including Community Development Block Grants (CDBG) from HUD or grants from Treasury's CDFI Fund. These other grant programs typically require organizations receiving funds to periodically report on program performance and agree to audits of their financial statements. * Financial condition. Eight of the nine organizations reported no material findings from recent audits of their financial statements. One of the newer organizations had not yet conducted an annual financial audit. * HUD approval. All nine grantees are HUD-approved housing counseling agencies or part of a network of such agencies. As such, they met HUD's requirements for approval, including having provided housing counseling for at least a year and not having unresolved government audit findings.[Footnote 7] Table 1: Characteristics of Fiscal Year 2009 and Fiscal Year 2010 FEC Pilot Program Grantees: Name of grantee: Boulder County Housing Authority; Funding year and amount: FY 2009; $400,000; Type of organization: Public housing authority; Target population: Low-income and low-wealth population in Boulder County, Colorado, with a focus on Hispanic/Latino populations; Years of experience: More than 20 years of experience in housing counseling; Recent federal grant experience: HUD CDBG and housing counseling grants; Department of Agriculture grants; Department of Health and Human Services grants; Financial condition: No audit findings in 2006 and 2007; three audit findings in 2008, but none considered to be material weaknesses. Name of grantee: Georgia Financial Education Collaborative; Funding year and amount: FY 2009; $400,000; Type of organization: Collaboration of four nonprofit housing counseling agencies; Target population: Minority and low-income populations in Georgia and Alabama; Years of experience: The four agencies range from more than 20 years to more than 60 years of experience in housing counseling and financial education; Recent federal grant experience: All four agencies received funding for various homeownership counseling or foreclosure prevention programs, including HUD housing counseling grants; Financial condition: Lead applicant for the collaborative had an unqualified opinion on its 2008 financial statement audit. Name of grantee: Mission Economic Development Agency; Funding year and amount: FY 2009; $400,000; Type of organization: Nonprofit community-based, local economic development corporation; Target population: Low-and moderate-income Latino families in San Francisco, California; Years of experience: More than 10 years of experience in homeownership counseling and financial education; Recent federal grant experience: CDBG and housing counseling grants; Financial condition: Audit reports for the prior 3 years had unqualified opinions. Name of grantee: New Hampshire Housing Finance Authority; Funding year and amount: FY 2009; $400,000; Type of organization: Nonprofit public benefit corporation; Target population: Low-income, low-wealth population in New Hampshire; Years of experience: Roughly 16 years of experience in financial education and counseling services; Recent federal grant experience: Funding from HUD and U.S. Department of Agriculture; Financial condition: Annual audits with unqualified opinions from 2007 to 2009. Name of grantee: OnTrack Financial Education and Counseling; Funding year and amount: FY 2009; $400,000; Type of organization: Nonprofit United Way agency; Target population: Low-income/low-wealth residents of Asheville/ Buncombe County, a small urban area of Western North Carolina; Years of experience: More than 35 years of experience in financial education and counseling; Recent federal grant experience: HUD housing counseling grants, funding from the Department of Labor and Department of Health and Human Services; Financial condition: Audit reports for the prior 3 years had unqualified opinions. Name of grantee: Council for Native Hawaiian Advancement; Funding year and amount: FY 2010; $3.15 million; Type of organization: Certified Community Development Financial Institution; nonprofit community development corporation; Target population: Native Hawaiians and low-income populations throughout Hawaii; Years of experience: Roughly 7 years of experience in financial education and counseling; Recent federal grant experience: Various federal financial education and counseling grants, including grants from Treasury's CDFI Fund and the U.S. Small Business Administration; Financial condition: Annual audits with unqualified opinions since 2001. Name of grantee: Greater Erie Community Action Committee; Funding year and amount: FY 2010; $400,000; Type of organization: Community action agency; Target population: Low-income/low-wealth prospective homebuyers in Erie County, Pennsylvania; Years of experience: More than 30 years experience in housing assistance; Recent federal grant experience: HUD housing counseling and Section 8 grants; funding from Department of Health and Human Services; Financial condition: Audits in the prior 3 years had not indicated any findings. Name of grantee: Homewise, Inc; Funding year and amount: FY 2010; $400,000; Type of organization: Nonprofit housing organization; Target population: Prospective low-and moderate-income homeowners in Santa Fe, New Mexico; Years of experience: More than 17 years in assisting first-time homebuyers; Recent federal grant experience: CDBG, HUD housing counseling, and CDFI Fund grants; funding from NeighborWorks America; Financial condition: Audits in the prior 3 years had unqualified opinions. Name of grantee: Solita's House, Inc; Funding year and amount: FY 2010; $200,000; Type of organization: Nonprofit organization dedicated to homebuyer education and credit counseling services; Target population: Low- and moderate-income population in Hillsborough County, Florida, including the city of Tampa; Years of experience: Roughly 5 years' experience in housing and financial literacy counseling; Recent federal grant experience: HUD housing counseling grants, HUD national foreclosure mitigation grants; Financial condition: Has not yet conducted an annual audit.[A] Financial statements in prior 3 years had not shown negative change in net assets. Sources: FEC grantees, Treasury, and HUD. [A] According to its assistance agreement, Solita's House is required to provide the CDFI Fund with audited financial statements within 180 days after the close of its fiscal year. [End of table] Treasury Followed a Standardized Process to Select the Nine FEC Grantee Organizations: Treasury's CDFI Fund and OFEFA had responsibility for selecting grantees. To select the nine grantees, Treasury first solicited applications, then followed a standardized review process for each application. In October 2009, Treasury issued a Notice of Funds Availability inviting applications for the fiscal year 2009 funding round of the FEC program. Applications were due roughly 1 month later and more than 200 organizations applied. According to Treasury, the CDFI Fund and OFEFA selected staff from their own offices to review the applications, as well as staff from other federal agencies, such as HUD and the Department of Health and Human Services. Treasury officials told us that in selecting the federal agency reviewers of FEC applications, they sought individuals with experience in financial literacy and education and examined their résumés. Reviewers assessed the applications using a scoring process developed by the CDFI Fund. Following a prescreening by OFEFA staff for basic eligibility, applications were scored in two phases. In phase one, each application was evaluated by one reviewer and given a rating score ranging from 0 to 5 points for each of four main sections of an application--Implementation Plan, Proposed Impact and Effective Use, Organizational Capacity, and Budget and Program Funding. Treasury provided reviewers with specific written guidance for each rating. [Footnote 8] For example, the guidance stated that if a reviewer was "highly confident" that an applicant had chosen the most appropriate impact measures for its proposed FEC activities, that applicant should be rated "excellent" (5 points) in the section on Proposed Impact and Effective Use. The 45 applications with the highest scores in phase one advanced to phase two. In this second phase, Treasury further divided the 45 applications into four groups based on the organizations' size and delivery method. Assigned to each group were two reviewers, as well as a Treasury staff member who served as team leader. The four teams in this second phase rescored the 45 applications following the same method used in the first phase. Reviewers read different applications in the second phase than they read in the first phase. On the basis of the second review, the 2 highest-scoring applications from each of the four groups, plus the next four highest scorers from the overall pool, became the 12 finalists. Treasury officials told us that a panel consisting of three CDFI Fund staff and two OFEFA staff then independently reviewed and ranked the applications of the 12 finalists. Treasury had $2 million in funding for the fiscal year 2009 round of the FEC program and chose to award five grants of $400,000 each. In deciding which five organizations to fund, CDFI Fund and OFEFA staff selected five of the six highest-scoring applicants among the 12 finalists.[Footnote 9] For the fiscal year 2010 round of the program, Treasury officials told us that they did not have the same type of nationwide application process because of the large number of applications received in 2009. Instead, Treasury held a new application process for only Hawaii-based organizations because, as noted previously, Congress designated $3.15 million of the $4.15 million appropriated for an eligible organization in Hawaii. Treasury received two applications from Hawaii-based organizations and, according to Treasury, followed the same process for reviewing these two applications that it used for the 2009 applicants. For the non-Hawaii grants in the 2010 round of the program, Treasury awarded funds to three organizations that had been finalists in the 2009 round. Our review of documentation related to the selection process indicated that Treasury appeared to have developed reasonable selection criteria and appeared to have applied these criteria consistently, awarding grants to those applicants who received the highest ratings. Treasury officials also told us that they conducted due diligence before announcing the final FEC awards through reviews of the applicants' financial statements and consultations with HUD about the applicants' standing as HUD-approved housing counseling agencies.[Footnote 10] Treasury's standard operating procedures for the FEC program state that after grantees receive their awards, Treasury will continue oversight activities, such as reviewing the results of financial audits. Not Enough Time Has Passed to Assess the FEC Program's Impact, and Grantees May Need Guidance in Developing More Meaningful Impact Measures: Grantee organizations have been providing financial education and counseling services under the FEC program for less than a year, and therefore not enough time has passed to meaningfully assess the program's effectiveness and impact. The 2009 grantees have submitted only one performance report to date, and the 2010 grantees have not submitted any. Information for our review was therefore limited to the initial performance reports that the 2009 grantees submitted in January 2011. However, these data cover only a few months of program operations, while some desired outcomes of the FEC program, such as homeownership, can take years to realize. The five organizations that received grants under the fiscal year 2009 round of the FEC program were notified of their awards in May 2010, and Treasury disbursed funds to these organizations in August 2010. Three of the five organizations began using their grant funds to serve prospective homebuyers in August 2010, and one began in October 2010. A fifth organization is using its grant funds for a financial education website, which was still under development as of June 2011. (See enclosure II for a description of the activities by the grantee organizations of the FEC program.) Grantees are to submit their first performance report within 6 months of disbursement, so only the January 2011 reports from the 2009 grantees are available. Table 2 shows the performance goals of FEC program grantees, as approved by Treasury. Because grantees' activities and approaches are specific to each organization, the targets set forth in the goals differ. For example, Homewise has an output goal of serving 1,000 clients each year, while Solita's House has an output goal of serving 40 clients each year. According to staff at Homewise, the high goal is because the organization is using its grant in part to train staff at other organizations and it is including in the goal the clients expected to be served by these other organizations. By contrast, Solita's House's goal covers only direct services to clients and envisions a relatively high number of hours of service per client. Similarly, organizations' impact goals vary, and some organizations have established more challenging impact goals than others. For example, as seen in table 2, Georgia Financial Education Collaborative has a goal of increasing annual average household savings by 200 percent, while Boulder County Housing Authority's goal is to increase such savings by 25 percent. Such variation may be a function of differences in the starting point of the organizations' target audiences, as well as the extent of services they expect to provide to each client. In addition, some of the impact measures appear to measure interim steps (such as setting savings goals) rather than the ultimate desired outcome (such as meeting those goals). Table 2: Performance Goals of FEC Pilot Program Grantees: Grantee: Boulder County Housing Authority; Output goals: 1. serve 150 clients each year; 2. provide 11.5 hours of service per client; Impact goals: 1. 25 percent increase in annual average household savings; 2. 75 percent of clients use formal budgeting tools; 3. 75 percent of clients increase their credit scores. Grantee: Georgia Financial Education Collaborative; Output goals: 1. serve 140 clients each year; 2. provide 10 hours of service per client; Impact goals: 1. 200 percent increase in annual average household savings; 2. 75 percent of clients use formal budgeting tools; 3. 25 percent average decrease in household debt. Grantee: Mission Economic Development Agency; Output goals: 1. serve 180 clients each year; 2. provide 8 hours of service per client; Impact goals: 1. 66 percent of clients create a long-term (12 months or longer) savings goal; 2. 66 percent of clients increase their credit scores; 3. 20 percent average decrease in household debt. Grantee: New Hampshire Housing Finance Authority; Output goals: 1. serve 200 clients each year; 2. provide 8 hours of service per client; Impact goals: 1. 50 percent of clients create a long-term (12 months or longer) savings goal; 2. 100 percent of clients use formal budgeting tools; 3. 33 percent of clients increase their credit scores. Grantee: OnTrack Financial Education and Counseling; Output goals: 1. serve 100 clients each year; 2. provide 5.5 hours of service per client; Impact goals: 1. 25 percent average increase in household savings; 2. 25 percent average decrease in household debt; 3. 25 clients obtain or qualify for a home mortgage. Grantee: Council for Native Hawaiian Advancement; Output goals: 1. serve 500 clients each year; 2. provide 4 hours of service per client; Impact goals: 1. 50 percent of clients create a long-term (12 months or longer) savings goal; 2. 50 percent of clients increase their credit scores; 3. 50 percent of clients obtain or qualify for a home mortgage. Grantee: Greater Erie Community Action Committee; Output goals: 1. serve 50 clients each year; 2. provide 12 hours of service per client; Impact goals: 1. 250 percent increase in annual average household savings; 2. 75 percent of clients use formal budgeting tools; 3. 75 percent of clients increase their credit scores. Grantee: Homewise, Inc.; Output goals: 1. serve 1,000 clients each year; 2. provide 8 hours of service per client; Impact goals: 1. 50 percent of clients increase their household savings; 2. 50 percent of clients increase their credit scores; 3. 400 clients obtain or qualify for a home mortgage. Grantee: Solita's House, Inc.; Output goals: 1. serve 40 clients each year; 2. provide 30 hours of service per client; Impact goals: 1. 30 clients increase their annual savings; 2. 18 clients create a long-term (12 months or longer) savings goal; 3. 30 clients increase their credit scores. Source: grantees' assistance agreements. [End of table] As seen in table 3, the five fiscal year 2009 grantees have reported to Treasury output data--the number of clients served and hours of service provided to date. However, because no grantee had been providing services for more than a few months at the time of the January 2011 reports, not enough time has passed to compare these output data with the output goals noted above because the goals are based on activities to be performed over the course of a full year. Table 2: Output Data Reported by Fiscal Year 2009 Grantees of FEC Program for Services Provided August 2010 through December 2010: Grantee: Boulder County Housing Authority; Number of clients served: 21; Hours of class time, instruction, and counseling per client: 3.89. Grantee: Georgia Financial Education Collaborative; Number of clients served: 149; Hours of class time, instruction, and counseling per client: 8.6. Grantee: Mission Economic Development Agency; Number of clients served: 112; Hours of class time, instruction, and counseling per client: 6. Grantee: New Hampshire Housing Finance Authority; Number of clients served: 0; Hours of class time, instruction, and counseling per client: 0. Grantee: OnTrack Financial Education and Counseling; Number of clients served: 29; Hours of class time, instruction, and counseling per client: 2.5. Source: grantees' Performance Goal Reports. Note: Since the January 2011 reports, grantees have continued to build their counseling programs. As of June 2011, three grantees told us they had notable increases in the number of clients served since the January report, and four told us they had increased the average time served per client. [End of table] In the January 2011 Performance Goal Reports, two of the five fiscal year 2009 grantees did not provide any impact data to Treasury. One grantee--New Hampshire Housing Finance Authority--told us that its grant was being used to develop an interactive financial education website that had not been launched when the reports were due. Another grantee--OnTrack Financial Education and Counseling--said it had collected only baseline data at the time the performance report was due and said impact data would be collected as participants completed its financial education and counseling program in subsequent years. Three of the five grantees did provide some impact data in their January 2011 Performance Goal Report. The Mission Economic Development Agency provided information on the number of clients who created long- term savings goals; the Boulder County Housing Authority reported on the number of participants who used budgeting tools; and the Georgia Financial Education Collaborative reported the average percentage change in prospective homebuyer savings, total number of clients using budgeting tools, and the average percentage change in outstanding household debt. Because these initial impact measures are based on services provided for no more than a few months--while the goals are based on at least a full year of service--and because of measurement issues discussed below, we are not providing the specific results of these initial measures. Grantees May Need Guidance in Establishing More Meaningful Impact Measures: We found that some grantees were calculating the results of the impact measures in an erroneous or misleading way or were not fully capturing meaningful information, which may limit the usefulness of these data for assessing program effectiveness. As we have reported, successful performance measures should be reliable--for example, they should produce the same result under similar conditions.[Footnote 11] Treasury officials told us that they discussed the specific impact measures with each grantee but had not yet provided guidance to the grantees on how to calculate the results of the impact measures. Without such guidance, Treasury and the grantees may be unable to accurately assess the extent to which homeowners and prospective homebuyers have improved their financial situation and behavior after receiving services under the FEC program. We identified an instance in which a grantee was not accurately calculating the impact measure of average percentage increase in prospective homebuyer savings. According to the grantee's calculation, a participant who began a financial education and counseling program with no savings but subsequently saved $500 was shown to have a 50,000 percent increase in savings. In fact, a percentage increase cannot be meaningfully calculated from zero savings because any percentage increase on zero is infinite. As an alternative, a more accurate and useful way of calculating changes in savings might be to compute total savings at the beginning of the program across all participants, compute total savings at the end of a program across all participants and, based on these two figures, calculate an average percentage change in savings. This methodology would have the advantage of assigning more weight to larger values and less weight to smaller ones. In addition, it may be advantageous to focus on net savings-- that is, savings minus debt--to provide a more complete picture of an individual's financial situation. For example, under one scenario, an individual might increase his or her savings for a down payment on a home by reducing the amount allotted each month to pay down credit card debt. In this example, the individual would be saving more, but at the same time could be increasing his or her debt by paying off less of it each month. As a result, looking at changes in savings without taking into account changes in debt may lead to an inaccurate assessment of an individual's financial health. We also identified ways of potentially improving the usefulness of other impact measures that grantees chose to use: changes in household debt and changes in credit scores. As with savings, some grantees of the FEC program are calculating the percentage change in household debt for each participant separately, before computing the average change. As was the case with savings, it might be more accurate and useful to calculate the change by aggregating all debt at the beginning of a program across all participants, and again at the end, and then computing the average percentage change in debt. Another way to make this measure more meaningful might be to distinguish between types of debt, as lenders view some debt, such as mortgage or student loan debt, more positively than other debt, such as credit card debt. Looking at debt relative to income might also be useful, as lenders often look at a debt-to-income ratio when reviewing mortgage loans for prospective homebuyers. For example, both government-backed mortgage loans and conventional mortgage loans have specific debt-to-income ratio limits. With regard to credit scores, reporting participants' actual scores themselves, rather than just the number or percentage of participants who improve their scores, might provide a more complete picture of the participants' progress toward being ready for a mortgage. For example, borrowers must have a credit score of at least 500 to qualify for a mortgage backed by the Federal Housing Administration; individuals with scores of 580 or higher are eligible for maximum financing, while those with scores of between 500 and 579 are eligible for 90 percent loan-to-value financing. Comparing a participant's credit scores before and after receiving financial education and counseling with such benchmarks may help assess improvement toward qualifying for a mortgage. Conclusions: The organizations selected to serve prospective homebuyers under the FEC program are generally experienced in providing financial education and housing counseling, and Treasury's process for selecting these organizations appears to have been conducted using established criteria that were applied consistently. Because grantees have been providing services under the FEC program for less than a year, not enough time has passed to meaningfully assess the impact of the program or the effectiveness of individual grantees. However, the performance measures selected and the calculation of results for some impact measures may not provide fully reliable or meaningful information with which to assess the grantees and thus the FEC program's impact. In addition, in some cases the impact measures being used do not reflect the desired outcomes but instead represent interim steps, such as setting savings goals. By providing additional guidance and technical assistance to grantees on calculating the results of some impact measures and working with the grantees to improve the measures' design, Treasury can help avoid inaccurate calculations and ensure the most accurate assessment of the FEC program's impact. Recommendation for Executive Action: To help ensure that FEC program grantees' impact measures are useful and the results of these measures are accurately recorded, we recommend that the Secretary of the Treasury instruct the directors of the CDFI Fund and OFEFA to provide additional guidance or technical assistance to the grantees on how to accurately and meaningfully calculate the results of the impact measures. This guidance or technical assistance could take the form of recommendations on how best to measure and calculate changes in savings, debt, and credit scores. Agency Comments: We provided a draft of this report to the Department of the Treasury for review and comment. Treasury provided written comments, which are reprinted in enclosure III. In its comments, Treasury stated that it concurred with the observations in our report and plans to provide grantees with supplemental guidance in measuring the impact of the FEC program prior to the next reporting deadline. Treasury also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the Department of the Treasury and interested congressional committees. In addition, the report will be available at no charge on GAO's website at [hyperlink, http://www.gao.gov]. If you or your staffs have any questions about this report, please contact me at (202) 512-8678 or cackleya@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs are on the last page of this report. Major contributors to the report were Jason Bromberg, Assistant Director; Tania Calhoun; Emily Chalmers; Pamela Davidson; Ronald Ito; Winnie Tsen; and Seyda Wentworth. Signed by: Alicia Puente Cackley: Director, Financial Markets and Community Investment: Enclosures (3): [End of section] Enclosure I: Scope and Methodology: To describe the characteristics of the organizations providing services under the Financial Education and Counseling (FEC) Pilot Program, we reviewed the program applications of the five fiscal year 2009 and four fiscal year 2010 grantees of the FEC program. In addition, we reviewed the Department of the Treasury's (Treasury) fiscal year 2009 and fiscal year 2010 Notices of Funds Availability and its standard operating procedures for the FEC program. We also reviewed the scoring sheets Treasury used in making its final award decisions and interviewed Treasury officials and grantee staff about the process followed to select grantees. To determine what is known about the FEC program's impact in improving the financial situation and behavior of homeowners and prospective homebuyers, we reviewed the program applications and assistance agreements of all nine grantees--five from the fiscal year 2009 round of the program and four from the fiscal year 2010 round of the program.[Footnote 12] We also interviewed staff of each of the grantees about activities planned and under way using FEC grant funds. In addition, we reviewed the output and impact data contained in the Performance Goal Reports that grantees submit to Treasury. In reviewing these reports, we assessed the accuracy and completeness of the data in them by discussing the data collection procedures with grantee staff. We determined that the output data (number of clients served and hours of service per client) were sufficiently reliable for the purposes of this report. Further, we noted similarities and differences in the performance measures that grantees have chosen to report and assessed the usefulness of some of the measures. We also assessed ways that some of the measures were being calculated, such as by examining one grantee's data for outliers and obvious errors and performing our own calculation of a case involving an outlier. We conducted this work from January to July 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Enclosure II: Activities of Grantee Organizations under the FEC Pilot Program: Grantee organizations are using a variety of approaches to serve prospective homebuyers under the FEC program. Below is a description of the activities of the nine grantees of the program. Fiscal Year 2009 Grantees: Boulder County Housing Authority (Boulder, CO): This organization offers an initial one-on-one appointment with a housing counselor followed by a series of five "tracks"--budget, credit, debt reduction, banking and savings, and homeownership training--each of which is followed with homework assignments and small workgroups that provide peer support. Participants receive gift cards and cash savings that total $500 after completing all of the tracks. Georgia Financial Education Collaborative (Atlanta, GA): An effort of four Department of Housing and Urban Development-approved housing counseling agencies, the collaborative is using a three-phase approach to serve prospective homebuyers: (1) a full-day workshop and an initial one-on-one session with a housing counselor, (2) 10 or more group financial education workshops, and (3) counseling for clients determined to be ready for a mortgage. Mission Economic Development Agency (San Francisco, CA): This agency offers a 6-hour workshop, offered in English and Spanish, with 2 hours focused on general financial education and 4 hours on the homebuying process. The agency also provides one-on-one financial coaching to help participants develop personalized financial action plans. New Hampshire Housing Finance Authority (Bedford, NH): This organization is developing an interactive online financial education and counseling website for use by existing clients of two housing programs. The website will include interactive features and focus on household budgeting and maintaining good credit. OnTrack Financial Education and Counseling (Asheville, NC): This organization provides one-on-one counseling and group education classes on-site at five apartment complexes. Its program also features a matched savings component to help with closing costs, down payment, or paying down debt that negatively affects credit scores. The financial coaches are residents of the targeted complexes. Fiscal Year 2010 Grantees: Council for Native Hawaiian Advancement (Honolulu, HI): The council is subcontracting services through partner organizations with specific expertise or geographic focus. The council's program includes group training, one-on-one counseling, matched savings, and deposit accounts. Greater Erie Community Action Committee (Erie, PA): The committee is using grant funds to provide a combination of one-on-one counseling, matched savings accounts, and group education. Participants begin with one-on-one counseling sessions with a debt and delinquency counselor. If deemed ready for home purchase within 3 years, participants are enrolled in a matched savings program and required to attend 10 group education classes. Homewise, Inc. (Santa Fe, NM): This organization is providing one-on- one financial counseling and group classes--offered in English and Spanish--in financial literacy and the homebuying process. It is also expanding its services to prospective homebuyers in Albuquerque, New Mexico, and training additional organizations that, in turn, are to serve prospective homebuyers in Portland, Oregon; Chicago, Illinois; and Charleston, South Carolina. Solita's House, Inc. (Tampa, FL): This organization is seeking to expand its current financial education and counseling program--which is offered in English and Spanish and includes homebuyer education, prepurchase counseling, financial literacy, and credit counseling--to incorporate new services, workshops, matched savings, and a pilot program focused on credit and household budgeting. [End of section] Enclosure III: Comments from the Department of the Treasury: Department Of The Treasury: Community Development Financial Institutions Fund: 601 Thirteenth Street, NW, Suite 200 South: Washington, DC 20005: July 20, 2011: Ms. Alicia Puente Cackley: Director, Financial Markets and Community Investment: U.S. Government Accountability Office: 441 G Street N.W. Washington, D.C. 20548: Dear Ms. Cackley: Thank you for the opportunity to comment on the Government Accountability Office (GAO) draft report titled Financial Education and Counseling (FEC) Pilot Program. The Community Development Financial Institutions Fund (CDFI Fund) concurs with the observations noted in the report. We are pleased that you acknowledged our efforts to conduct a fair and reasonable review process and that you noted the challenges faced in determining the impact of the program so soon after the awards were made. While we are confident that the FEC Pilot-Program awardees selected will design and implement successful financial education programs, we agree that additional assistance to awardees in measuring the impact of the program would be beneficial. Working with the Treasury's Office of Financial Education and Financial Access, the CDFI Fund plans to provide awardees with supplemental guidance prior to the next reporting deadline. Thank you again for the opportunity to comment on the draft report. We appreciate your efforts and the collaborative relationship that you fostered during the course of your review. Sincerely, Signed by: Donna J. Gambrell: Director: [End of section] Footnotes: [1] GAO, Financial Literacy: A Federal Certification Process for Providers Would Pose Challenges, [hyperlink, http://www.gao.gov/products/GAO-11-614] (Washington, D.C.: June 28, 2011). [2] Pub. L. No. 110-289, Title I, Subtitle B, § 1132, 122 Stat. 2654, 2727 (2008). [3] For the purposes of this report, we use the term "grantee" to refer to organizations that were awarded grants under the FEC program. [4] HERA provides that, "the Secretary of the Treasury shall authorize not more than 5 pilot project grants," which Treasury has interpreted to mean not more than five pilot project grants per fiscal year. [5] According to Treasury's Notices of Funds Availability for the FEC program, examples of financial education and counseling services include assisting prospective homebuyers in developing monthly budgets, building personal savings, improving credit scores, and financing or planning for major purchases. [6] [hyperlink, http://www.gao.gov/products/GAO-11-614]. The study that evaluated a housing counseling program was Abdighani Hirad and Peter M. Zorn, A Little Knowledge Is a Good Thing: Empirical Evidence of the Effectiveness of Pre-purchase Homeownership Counseling (Cambridge, Mass: Joint Center for Housing Studies, Harvard University, 2001). The study found that borrowers who received prepurchase counseling were, on average, 13 percent less likely to become 60-day delinquent during the study period. In addition, those who received individual counseling were half as likely to become delinquent as those that received their counseling in a classroom. [7] Once a housing counseling agency is HUD-approved, it must meet certain requirements, including (1) providing counseling to at least 30 clients in a 12-month period; (2) employing staff trained in housing counseling, with at least half of the counselors having 6 months or more of experience; and (3) undergoing financial audits, with those expending $500,000 or more in federal funds in a year undergoing a single or program-specific financial audit for that year and those expending less than $500,000 in a year in federal funds undergoing an independent audit every 2 years. See 24 C.F.R. Part 214; OMB Circular A-133. [8] According to Treasury guidance, the Implementation Plan refers to traits such as the applicant's understanding of the demand for FEC services and its marketing, outreach, and delivery strategy. Proposed Impact and Effective Use refers to the three impact measures an applicant has selected and its method of evaluating and documenting behavioral change in program participants. Organizational Capacity refers to the applicant's capacity, skills, and expertise. Budget and Program Funding refers to the extent to which an applicant has adequate funding to successfully implement the FEC program. [9] According to Treasury, one of the five highest-scoring organizations was not chosen in fiscal year 2009 because HUD indicated some concerns regarding its continued status as a housing counseling agency at the time of the fiscal year 2009 award process. Treasury provided this organization an award in the fiscal year 2010 round after consultations with HUD determined that the organization was again meeting its requirements. [10] In one case where the applicant was a collaborative entity, Treasury officials told us that they reviewed the financial statements of the organization designated as the lead applicant but not of the other members of the collaborative. [11] For example, see GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22, 2002), 45-53. [12] An assistance agreement is a formal agreement between Treasury and grantees that includes the terms and conditions of the FEC grant. These agreements contain information on the output and impact goals that grantees will seek to achieve with their grant funds. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: