This is the accessible text file for GAO report number GAO-15-104R entitled 'College- and Career-Readiness: States Have Made Progress in Implementing New Standards and Assessments, but Challenges Remain' which was released on December 12, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Chairman, Committee on Health, Education, Labor, and Pensions, United States Senate: December 2014: College- and Career-Readiness: States Have Made Progress in Implementing New Standards and Assessments, but Challenges Remain: GAO-15-104R: Contents: Letter: States Use Professional Development, New Curriculum, and Communications Strategies to Implement New State Academic Standards: Time Frames, Technology, and Potential Declines in Test Scores Challenge Assessment Implementation: Education Plans to Review All Statewide Assessments and Monitors Consortia Progress in Developing Assessments under the Grant: Agency Comments: Appendix I: Briefing Slides: Appendix II: GAO Contact and Staff Acknowledgments: Abbreviations: Common Core: Common Core State Standards: Education: U.S. Department of Education: ESEA: Elementary and Secondary Education Act: PARCC: Partnership for Assessment of Readiness for College and Careers: Assessment Program: Race to the Top Assessment Program: Smarter Balanced: Smarter Balanced Assessment Consortium: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: December 12, 2014: The Honorable Tom Harkin: Chairman: Committee on Health, Education, Labor, and Pensions: United States Senate: Dear Senator Harkin: Many students in the United States are not adequately prepared for college and careers upon graduating high school. Four out of 10 new college students, for example, are required to take remedial coursework when entering college and many employers comment on the inadequate preparation of high school students.[Footnote 1] Federal law requires states to adopt academic standards that specify what students should know in each grade and implement assessments to test this knowledge, to receive funding for students in low-income schools. [Footnote 2] To address concerns about inadequately prepared students, all states are now using or developing academic standards that are explicitly tied to college and career preparation. The majority of states came together to develop common college- and career-ready standards for grades K through 12 in math and English. The Council of Chief State School Officers and the National Governors Association spearheaded this effort, which resulted in common math and English standards known as the Common Core State Standards (Common Core) published in 2010. As of the beginning of the 2014-15 school year, 44 states and the District of Columbia[Footnote 3] were using Common Core,[Footnote 4] and the remaining states were using or developing their own college- and career-ready standards. The U.S. Department of Education (Education) has taken several steps to support states in their move toward more rigorous academic college- and career-ready standards and assessments, including: * waiving certain requirements of the Elementary and Secondary Education Act of 1965 (ESEA), as amended, (such as certain improvement requirements for schools that fail to make sufficient academic progress) for states that agreed to implement certain policies, including adopting college- and career-ready standards, which could include the Common Core or an individual state's standards.[Footnote 5] * awarding approximately $360 million through its Race to the Top Assessment Program (Assessment Program) to two state consortia that are using the funds to develop computer-based statewide assessments aligned to common college- and career-ready standards.[Footnote 6] The consortia are the Smarter Balanced Assessment Consortium (Smarter Balanced) and the Partnership for Assessment of Readiness for College and Careers (PARCC). The ESEA requires that, to receive federal funding for students in low- income schools, states' academic standards and assessments meet minimum requirements, including that states: * implement a set of "high-quality" assessments that measure students' achievement against state standards in math and reading/language arts annually in grades 3 through 8 and at least once in grades 10 through 12,[Footnote 7] and: * use assessments for purposes for which they are valid and reliable. States can fulfill the federal assessment requirements with their own assessments or with the consortia assessments. * Education uses a peer review process to assess all states' compliance with ESEA's requirements for standards and assessments. * Education also provides formula grants under ESEA to states which can be used to develop and administer assessments. The total awarded in fiscal year 2014 was approximately $369 million.[Footnote 8] In light of the importance and widespread adoption of new state college- and career-ready standards and assessments, you asked us to obtain information on how all states are implementing their standards and how consortia states are implementing their assessments. We focused on: 1. What is known about states' implementation of college- and career- ready standards? 2. What key considerations are states facing as they implement assessments aligned to college- and career-ready standards? 3. How does Education plan to oversee statewide assessments, and how has it overseen the Assessment Program? To address these objectives, we: * identified 55 generalizable and nongeneralizable studies on the implementation of college- and career-ready standards from 2011 onward and reviewed 17 in more detail. We assessed the methodologies of these 17 studies and determined that they were sufficiently reliable for the purposes of our study. We selected these studies because they were published in 2013 or later and explicitly addressed the implementation of college- and career-ready standards and assessments at the national, state, or district levels. * conducted site visits or phone interviews with education officials in five states that vary by size, location, participation in the consortia, and whether they adopted Common Core or their own state standards. Specifically, we: - Conducted site visits in California and Massachusetts to discuss how they were implementing the Common Core and preparing to implement the consortia assessments. In each state we met with the State Educational Agency and three school districts that varied by size and location. As of October 2014, Massachusetts and California were members of the PARCC or Smarter Balanced consortium and used Common Core standards. - We also conducted phone interviews with State Educational Agencies in Kentucky, Texas, and Virginia, which were not members of the consortia and implemented their own assessments aligned to new standards two or more years ago. Kentucky used Common Core and Texas and Virginia developed their own standards. * held two panel discussions--one per consortium--with a total of 16 school district superintendents or their designees about their experiences implementing Common Core standards and consortia assessments. These districts varied in size and participated in consortia field tests. * reviewed relevant federal laws and regulations. We also reviewed Department of Education program guidance and other documents, such as oversight reports, technical reviews, and correspondence with the consortia. In addition, we reviewed consortia documents, including policies, procedures, guidelines, and annual reports. * interviewed officials from the Department of Education, the two consortia, and the Federal Communications Commission. The Federal Communications Commission is modernizing its E-Rate program to provide more affordable telecommunications and Internet services to schools. We also interviewed stakeholders, such as education researchers, and representatives from various associations such as the National Governor's Association, the Council of Chief State School Officers, and the State Educational Technology Directors Association. We conducted this performance audit from November 2013 to December 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. On December 2, 2014, we briefed committee staff on results of this study and this report formally conveys the information provided during this briefing (see appendix I for briefing slides). In summary: States Use Professional Development, New Curriculum, and Communications Strategies to Implement New State Academic Standards: Regardless of whether states adopted the Common Core or their own college- and career-ready standards, they use similar implementation strategies and face similar challenges. States and districts: * provide professional development to educators to help them effectively implement the new standards. However, some teachers may not receive sufficient high-quality professional development opportunities, according to various state and district officials. * develop curricula and acquire instructional materials aligned to new standards. However, new curricula takes time to develop, and many district officials and stakeholders have found that instructional materials are not always well-aligned to the new standards. * develop communication strategies to educate the public about new standards. Some district officials said that communicating to parents about the Common Core standards has been challenging. Time Frames, Technology, and Potential Declines in Test Scores Challenge Assessment Implementation: Both consortia and non-consortia states that have implemented new assessments over the past few years reported facing some similar challenges, such as technological concerns and concerns about declining assessment scores. Consortia States: * Education gave the consortia 4 years to develop the assessments. The consortia are on schedule to implement their assessments during the 2014-15 school year, as required by the terms of the grant. Stakeholders and district officials said, however, that the time frames caused challenges. For example, they said that it is difficult to use the assessments for accountability purposes when the standards to which they are aligned are not yet fully implemented. * The consortia field tested their assessments in member states in spring 2014. Although they were largely successful from a technological standpoint, all states and districts may not have sufficient Internet capacity and devices to administer the assessments via computer when they first become operational, according to a variety of education stakeholders and state and district officials. * Test scores may initially decline as students grapple with more challenging assessment content. * According to an official whose organization surveyed states that were consortia members in 2013, the consortia assessments may be less costly than previous state assessments for most member states surveyed.[Footnote 9] Non-consortia States of Kentucky, Texas and Virginia: * Officials from these states noted that: - ensuring that schools have sufficient Internet capacity and computing devices to administer computer-based statewide assessments takes time and can be challenging, and: - their states experienced significant declines in student scores--as much as 30 percent in some cases--when implementing new assessments aligned to their college- and career-ready standards. Parents were concerned about these declines in scores and officials noted the need for extensive communication to prepare them for the results. Education Plans to Review All Statewide Assessments and Monitors Consortia Progress in Developing Assessments under the Grant: * Education uses a peer review process in all states to assess compliance with ESEA requirements for standards and assessments. This includes assessing the reliability and validity of all statewide assessments. In the past we found that this process gave no assurance that state assessments included "highly, cognitively complex content." [Footnote 10] Education suspended and began revising this process in 2012 to permit states to focus on the design and implementation of new assessments. Revisions will include a focus on high-quality assessment principles outlined in Education's Assessment Program and its ESEA Flexibility initiative. Education will begin implementing the new process in late 2015. * Education regularly monitors consortia's progress with the Assessment Program grant requirements, including their progress in developing assessments.[Footnote 11] * Assessment Program guidance states that assessments are expected to generate data that can be used, among other purposes, to measure the college- and career-readiness of high school students and inform teacher and principal evaluation systems. Moreover, another assessment program document notes that the development of common assessments will enable the production of comparable data across states.[Footnote 12] Education officials told us that this applies to cross-state comparisons within a consortium. In their grant applications, the consortia were required to include research and evaluation plans and Education has monitored both consortia's development of these plans. The plans include research into uses of assessment scores. Education is also collecting data on the use of assessment scores in all states. Agency Comments: We provided a draft of this report to the Department of Education for review and comment. The agency agreed with the findings contained in the report. It also provided technical comments, which we incorporated where appropriate. We will send copies of this report to relevant congressional committees, the Secretary of Education, and other interested parties. In addition, this report will be available at no charge on GAO's website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (617) 788-0580 or nowickij@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Sincerely yours, Signed by: Jacqueline M. Nowicki: Director, Education, Workforce, and Income Security Issues: [End of section] Appendix I: Briefing Slides: College And Career Readiness: States Have Made Progress in Implementing New Standards and Assessments, but Challenges Remain: Overview: * Introduction; * Research Objectives; * Scope and Methodology; * Background; * Findings. Introduction: New Standards Are Intended to Enhance K-12 Student Learning: Many U.S. students are not adequately prepared for college and careers after high school. Forty percent of college entrants require remedial coursework, and many employers comment on high school graduates' inadequate preparation for the workforce.[A] All states are using or developing new academic standards that build toward college- and career-readiness. * 44 states and the District of Columbia[B] currently use Common Core State Standards (Common Core): - The Council of Chief State School Officers and the National Governors Association spearheaded this multi-state effort to establish common K-12 standards in math and English, which they published in 2010. * Six states have their own standards. [A] A Blueprint for Reform: The Reauthorization of the Elementary and Secondary Education Act. March 2010. U.S. Department of Education. [B] In this report we refer to the District of Columbia as a state. Introduction: College- and Career-Ready Standards by State as of September 1, 2014: Figure: 45 states are using Common Core in the 2014-15 school year: [Refer to PDF for image: illustrated U.S. map] Currently implementing Common Core: Alabama; Arizona; Arkansas; California; Colorado; Connecticut; Delaware; District of Columbia; Florida; Georgia; Hawaii; Idaho; Illinois; Indiana; Iowa; Kansas; Kentucky; Louisiana; Maine; Maryland; Massachusetts; Michigan; Minnesota[A]; Mississippi; Missouri; Montana; Nevada; New Hampshire; New Jersey; New Mexico; New York; North Carolina; North Dakota; Ohio; Oregon; Pennsylvania; Rhode Island; South Carolina; South Dakota; Tennessee; Utah; Vermont; Washington; West Virginia; Wisconsin; Wyoming. Implementing college- and career-ready standards other than Common Core: Alaska; Nebraska; Texas; Virginia. Repealed Common Core and temporarily implementing former state standards: Oklahoma. Source: Achieve, "Closing the Expectations Gap 2013 Annual Report on the Alignment of State K-12 Policies and Practice with the Demands of College and Careers" and Education Commission of the States, "State Academic Standards Activity Around the Common Core," September 1, 2014; Map Resources (map). GAO-15-104R. Note: This map shows states' use of standards in school year 2014-15; states may change their standards in future years. For example, Missouri recently enacted legislation that directs the state board of education to convene work groups to develop and recommend standards, to be implemented in the 2016-17 school year. In addition, there is litigation that could affect Louisiana's use of standards in the future. [A] Minnesota uses Common Core for English and its own standards for math. [End of figure] Introduction: The U.S. Department of Education has Encouraged States to Adopt New Standards: The U.S. Department of Education (Education) did not create any college- and career-ready standards. It has, however: * waived certain requirements of the Elementary and Secondary Education Act of 1965 (ESEA), as amended, for 42 states (as of October 9, 2014) that agreed to implement certain policies, including adopting college- and career-ready standards, which could include the Common Core;[A] * awarded approximately $360 million to two state consortia through its Race to the Top Assessment Program (Assessment Program) to develop assessments that measure student achievement against common standards.[B] The consortia are the Smarter Balanced Assessment Consortium (Smarter Balanced) and the Partnership for Assessment of Readiness for College and Careers (PARCC). They chose to use the funds to develop computer-based assessments aligned with Common Core. [A] Education refers to this as its ESEA Flexibility initiative. [B] These grants were awarded using American Recovery and Reinvestment Act of 2009 (Recovery Act) funds. This program is carried out through cooperative agreements between the consortia and Education, although we refer to them as grants in this report. Research Objectives: 1)What is known about states' implementation of college- and career-ready standards? 2)What key considerations are states facing as they implement assessments aligned to college- and career-ready standards? 3)How does Education plan to oversee statewide assessments, and how has it overseen the Assessment Program? Scope and Methodology: We identified 55 generalizable and nongeneralizable studies on how states and school districts implemented college- and career-ready standards and reviewed 17 in more detail after assessing their methodology and determining that they were sufficiently reliable for the purposes of our study. We selected these because they were published in 2013 or later and explicitly addressed the implementation of college- and career-ready standards and assessments at the national, state, or district levels. We also conducted site visits or phone interviews with education officials in five states that vary by size and location. * As of October 2014, California and Massachusetts were members of the PARCC or Smarter Balanced consortium and used Common Core standards. Kentucky, Texas, and Virginia were not members and implemented their own assessments aligned to new standards 2 or more years ago. Kentucky used Common Core and Texas and Virginia developed their own standards. In addition, we held two panel discussions– one per consortium– with a total of 16 school district superintendents or their designees. The districts varied in size and participated in field tests of the consortia assessments. Lastly, we reviewed relevant federal laws, regulations, guidance, and other documents and interviewed Education and consortia officials and stakeholders, such as education association representatives and researchers. Background: Academic Standards, Defined: In general, academic standards are learning goals that define what students are expected to know and do in each grade.[A] College- and career-ready academic standards are commonly defined as standards that prepare students for credit-bearing college coursework and ensure that students who enter the workforce after high school have the knowledge and skills to qualify for and succeed in careers or postsecondary job training. [A] Standards differ from curricula, which are the day-to-day lessons taught in classrooms. Background: Federal Law Requires States to Adopt Academic Standards and Assessments: ESEA requires that, to receive federal funding for students in low- income schools, states adopt academic standards and assessments that meet minimum federal requirements. This includes giving assessments that measure students' achievement against state standards in math and reading/language arts annually in grades 3 through 8 and at least once in grades 10 through 12.[A] * To ensure alignment between new college- and career-ready standards and assessments, states may modify their existing assessments or develop new ones. * The consortia assessments are multi-state, newly-developed assessments that states can use to satisfy federal assessment requirements. [A] Funding authorized under Title I, Part A of ESEA. For the purposes of this report, reading/language arts is referred to as English. Assessments also must be given in science at least once in grades 3 through 5, once in grades 6 through 9, and once in grades 10 through 12. Background: Education's Oversight of Assessments: Education oversees consortia compliance with the Race to the Top Assessment Program (Assessment Program), which provided about $360 million to develop the consortia assessments. The grant periods are expected to end in December 2014 for Smarter Balanced and August 2015 for PARCC. * Education uses a peer review process to evaluate state compliance with federal assessment requirements, though it suspended this process for revision in December 2012.[A] [A] Under Title VI of the ESEA, Education also provides formula grants to all states, which can be used for the development and administration of assessments. Total awarded in FY 2014 was approximately $369 million. Background: Purpose of Consortia Assessments: Education officials and other stakeholders said that consortia assessments will: * measure higher-order skills like critical thinking and problem solving; * allow for comparisons of student performance in consortium states; and; * offer more efficient scoring and provide broader access for English language learners and students with disabilities. Background: Consortia and Non-Consortia States[A]: As of November 2014: * 34 of the 44 states currently using Common Core standards belonged to the Smarter Balanced or PARCC consortium. Consortia officials expect 30 of these states to use their respective consortium's assessments when they become available in spring 2015 and said the remaining 4 may use them in later years. * 17 states were not members of either consortium. Some of these (e.g., Kentucky, Texas, Virginia) have been using new assessments aligned to new standards for 2 or more years. [A] We use the term consortia states to refer to PARCC and Smarter Balanced members and the term non-consortia states to refer to states that are not members of these consortia. Background: Map Showing Consortia and Non-Consortia States: Figure: Map Showing Consortia and Non-Consortia States: [Refer to PDF for image: illustrated U.S. map] Members of Smarter Balanced Consortium: California; Connecticut; Delaware; District of Columbia; Hawaii; Idaho; Maine; Michigan; Missouri; Montana; Nevada; New Hampshire; North Dakota; Oregon; South Dakota; Vermont; Washington; West Virginia; Wisconsin. Members of PARCC Consortium: Arkansas; Colorado; Illinois; Louisiana; Maryland; Massachusetts; Mississippi; New Jersey; New Mexico; Ohio; Rhode Island. Consortium member not likely to use assessments in the 2014-15 school year: Iowa; New York; North Carolina; Wyoming. Not a member of either consortium: Alabama; Alaska; Arizona; Florida; Georgia; Indiana; Kansas; Kentucky; Minnesota; Nebraska; Oklahoma; Pennsylvania; South Carolina; Tennessee; Texas; Utah; Virginia. Source: Smarter Balanced and PARCC; Map Resources (map); GAO-15-104R. [End of figure] Background: Consortia Administration: Both Smarter Balanced and PARCC are governed by committees comprised of representatives from member states who oversee assessment development and manage funding. Member states vote on major policy issues. At the time of our review, both consortia were moving toward a fee-for- service model in an effort to become self-sustaining. * Smarter Balanced is affiliated with UCLA. Member states will select their own vendors to provide a technological platform through which assessments will be delivered and scored. * In contrast, PARCC plans to use one vendor to provide a technological platform to deliver and score assessments in all member states and another vendor to provide ongoing project management services and support to states.[A] [A] At the time of our review, PARCC's vendor contract was being challenged in New Mexico state court. Background: Consortia Field Tests: The consortia field-tested the assessments in spring 2014. These pilots were intended to evaluate the quality, validity, and reliability of the assessments and help students, teachers, and schools gauge their technological and administrative readiness before the assessments “go live” in spring 2015. Some states piloted assessments in selected schools, while others piloted the assessments in all schools. More than 5.2 students took the piloted assessments. Finding 1: States Use Professional Development, New Curriculum, and Communications Strategies to Implement New State Academic Standards: Finding One Summary: Whether states adopted the Common Core or their own standards, they use similar implementation strategies and face similar challenges. States and districts: * provide professional development to educators to help them effectively implement the new standards. However, some teachers may not receive sufficient professional development opportunities. * develop curricula and acquire instructional materials aligned to new standards. However, new curricula takes time to develop, and some district officials and stakeholders have found that instructional materials are not always well-aligned to the new standards. * develop communication strategies to educate the public about new standards. New Standards Require Extensive Professional Development: Officials from five districts we visited told us that the Common Core, in particular, is rigorous and emphasizes critical thinking and real world application. As a result, teachers require significant professional development to change their pedagogical content and strategies. One survey, representative of pre K-12 public school teachers in 47 states, found that 86% of respondents had participated in professional development classes and workshops around new college- and career-ready standards.[A] [A] * Bill and Melinda Gates Foundation/Scholastic, Primary Sources: America's Teachers on Teaching in an Era of Change Edition (February 2014). Some Educators May Lack Sufficient Professional Development to Teach to New Standards: State and district officials raised concerns about the quantity of professional development. For example: * In one California district, officials reported needing funding to provide at least 10 additional days of professional development to adequately prepare teachers to teach to the Common Core. * In addition, state officials in Virginia, which has its own college- and career-ready standards, told us they have had to scale back on professional development for teachers due to budget cuts. In a survey of 525 responding school superintendents and administrators from 48 states, almost two-thirds noted that providing professional development for teachers has been challenging.[A] [A] AASA, The School Superintendents Association, Common Core and Other State Standards: Superintendents Feel Optimism, Concern and Lack of Support (June 2014). Survey respondents were AASA members. Survey results are not generalizable. States and Districts Collaborate to Provide Professional Development on the Common Core: Common standards reportedly increase collaboration across schools and districts on professional development. * Educators leverage resources, training, and tools to implement new standards offered by some national organizations. * A nationally representative 2014 survey of school districts in Common Core states found that 90 percent of districts collaborated on teacher professional development. Most of these efforts occurred within a state or among state educational agencies.[A] [A] ”Center for Education Policy, Common Core Standards in 2014: Districts' Perceptions, Progress, and Challenges (October 2014). Surveyed 400 school districts in 46 states and had a 65 percent response rate. Implementing Rigorous New Standards Requires Shifts in Curriculum Design: Implementing curricula aligned to new college- and career-ready standards is critical for students to learn the academic content required by the standards. Some stakeholders said that schools and districts may need more time to develop curricula aligned with the Common Core. Moreover, according to a recent survey of deputy state superintendents of education, 26 out of 40 responding states reported such challenges.[A] In one district in California, officials said that they are using curricula from the New York Department of Education until they can develop their own. [A] Center on Education Policy, Year 3 of Implementing the Common Core State Standards: An Overview of States' Progress and Challenges. (August 2013). Surveyed deputy state superintendents of education or their designees in the 46 that had adopted the Common Core as of July 2013. Instructional Materials Are Not Always Fully Aligned to New Standards: Instructional materials aligned to the new standards are important to support and reinforce the content students need to learn. Officials from four districts and multiple stakeholders said that it is difficult to find textbooks aligned to the Common Core. A stakeholder noted that this may cause delays in implementing the standards in the classroom. Local Communication Strategies Needed for Outreach to Parents and the Public: Communicating with parents and the public can help them understand how new standards differ from previous ones. Officials from several districts told us that, because the Common Core is politically sensitive, they have spent considerable time educating parents and the public about it. * In a survey of 122 district officials, many responding communications directors said they faced challenges in explaining to parents how the Common Core differs from previous standards. They also said that Common Core's complexity is difficult to explain and that they are often called on to address misinformation and controversy about the standards.[A] [A] * Council of the Great City Schools, Implementing the Common Core State Standards: Year 2 Progress Report from the Great City Schools (2013). Surveyed directors of curriculum, research, English Language Learner, special education, and communication in 48 districts. Results are not generalizable. [End of Finding 1] Finding 2: Time Frames, Technology, and Potential Declines in Student Test Scores Challenge Assessment Implementation: Finding Two Summary: For consortia states: * Education gave the consortia 4 years to develop their assessments, which stakeholders and school officials described as short given the magnitude of this multi-state effort. The consortia are on track to meet the grant requirement that the assessments be operational during the 2014-15 school year. * The field tests were largely successful from a technological standpoint, but it is unclear whether all states and districts have sufficient Internet capacity and devices to administer the assessments in 2015 as planned. * Test scores may initially decline as students grapple with the more challenging content and assessments. * Consortia assessments may be less costly than previous state assessments for many states. Some of these issues are not unique to consortia states. Non-consortia states that have implemented new assessments over the past few years reported facing similar technological challenges and concerns as consortia states about declining assessment scores. Education's Time Frame for Consortia Assessment Development: The initial grant period for assessment development was 4 years and the assessments are on track to be implemented, as required, during the 2014-15 school year. Education also extended the grant period to allow the consortia time to complete activities unrelated to implementing the assessments, such as developing tools to help students prepare for them. Figure: refer to PDF for image: timeline] 2010: Consortia awarded Assessment Programs grants. 2011: Consortia issue written guidance on assessment question design. 2012: Consortia begin developing assessment questions; Original Assessment Program grant period: September 2010-September 2014. 2013: Consortia first test of questions on students in member states; continue to develop assessment questions. 2014: Consortia field-test their assessment on more than 5.2 million students in member states; Consortia review field-test results; continue to refine assessments. Smarter Balance Extension: through December 31, 2014. 2015: PARCC Extension: through August 1, 2015; Students scheduled to take assessments in spring. Source: GAO analysis of consortia information. GAO-15-104R. [End of figure] Time Frames Challenge Consortia Assessment Development: Several stakeholders said Education's timeframes may be typical for developing single-state assessments, but that developing the consortia assessments is more challenging because they: * incorporate new technology; and; * involve collaboration and agreement across multiple states. For example, all states within a consortium had to agree on the types of accommodations to offer students with disabilities and English language learners. Education officials said they consulted with stakeholders to determine the time frames for assessment development. They also said that the assessment grants were funded by the Recovery Act, and any funds not spent by September 2015 will be returned to the Treasury.[A] [A] Education may approve a project period of up to 60 months under 34 C.F.R. § 75.250(a). Time Frames Challenged Consortia Assessment Implementation: Stakeholders and district officials described the timeline for implementing the assessments as short and challenging. For example, they said that it is difficult to use the assessments for accountability purposes when the standards to which they are aligned are not yet fully implemented. * A nationally representative 2014 survey of school districts in Common Core states found that more than half of the districts are not expected to complete important milestones, such as implementing Common Core-aligned curricula, until after the assessments are operational.[A] [A] Center for Education Policy, Common Core Standards in 2014: Districts' Perceptions, Progress, and Challenges (October 2014). Surveyed 400 school districts in 46 states and had a 65 percent response rate. Some States and Districts are Concerned about Technological Readiness for Consortia Assessments: While the consortia field tests were largely successful technologically, concerns still exist about whether all states and districts have sufficient Internet capacity and computing devices to administer the assessments in the 2014-15 school year. A 2014 survey found that 40 percent of districts reported that they either did not expect to have the technological infrastructure in place to administer the assessments until the 2015-16 school year, or were not sure when they would have it.[A] Further, stakeholders and state and district officials told us: * although school districts participating in the field tests were generally better equipped than districts that did not participate, some had difficulty scheduling all students to take the assessments on a limited number of computing devices; and; * the field tests may not accurately predict technological success because more students will be online than during the field tests. [A] Center for Education Policy, Common Core State Standards in 2014: District Implementation of Consortia-Developed Assessments (October 2014). The Consortia Provide Flexibility and Federal Initiatives Assist Schools in Obtaining Technology: The consortia and the federal government recognize that modernizing technological infrastructure in schools may be challenging. Therefore: * the consortia plan to offer paper-and-pencil based versions of the assessments while states transition, * the Federal Communications Commission is modernizing its E-Rate program to provide more affordable telecommunications and Internet services to schools, and, * the Administration's ConnectED initiative leverages federal funding and solicits private donations to improve Internet capacity for schools. States and Districts Are Concerned about Potential Declines in Consortia Assessment Scores: Several stakeholders and officials from two states told us that student scores may drop the first few years the consortia assessments are operational. This is because: * the Common Core is rigorous and the assessments are more challenging than most states' previous assessments, and; * students in schools where technology is not regularly integrated into classroom learning may be less adept at navigating the technological aspects of computer-based assessments. Parents may be concerned about declining assessment scores. Officials from two states told us they were concerned about communicating with parents about decreases in student scores. Consortia Assessments May Cost Less in Many Member States Than Previous Assessments: A 2012 report showed that in 44 states and the District of Columbia, state per-student spending on assessments ranged from $7 to $114, and averaged $27.[A] * Smarter Balanced officials estimate that assessments will cost approximately $22.50 per student. * PARCC officials said that assessments will cost $23.90 per student for computer-based assessments. - These costs reflect the prices of administering and scoring the individual assessments and do not include other tools the consortia offer. The Assessment Solutions Group surveyed 36 states that were consortia members in 2013. An ASG official told us that the results showed that 24 of these states would pay less for one or both of the consortia assessments than their own state assessments and 12 would pay more.[B] [A] Brown Center on Education Policy, Strength in Numbers: State Spending on K-12 Assessment Systems . (November 2012). [B] At the time of this survey, 2 of the 36 states belonged to both consortia; 22 belonged to Smarter Balanced, and 12 belonged to PARCC. Non-Consortia States Faced Similar Concerns about Technology and Potential Declines in Scores: Officials in Kentucky, Texas, and Virginia said that: * ensuring schools have sufficient Internet capacity and computing devices to administer computer-based statewide assessments takes time and can be challenging. - In Kentucky and Texas, for example, districts can choose to give computer-based assessments to high school students, but officials said many schools do not have the technological infrastructure to do so. * their states experienced significant declines in student scores—as much as 30 percent in some cases—when implementing new assessments aligned to their college- and career-ready standards. - Parents in Virginia were concerned about these declines in scores, and officials noted the need for extensive communication to prepare them for the results. [End of Finding 2] Finding 3: Education Plans to Review All Statewide Assessments and Monitors Consortia Progress with Assessment Development under the Grant: Finding Three Summary: Education is revising its peer review process to better evaluate the reliability and validity of all state assessments. Because the peer review applies to all state assessments, both consortia and non- consortia states will undergo these reviews. Education regularly monitors consortia progress with assessment development under the grant. As required by the terms of the grant, the consortia developed research plans-—which include research into the use of assessment scores. Education has monitored both consortia's development of these plans. Education is also collecting data in all states on their use of assessment scores. Education's Revised Peer Review Process Aims to Better Evaluate Reliability and Validity of All Assessments: Since 2012, Education has been revising its peer review process to permit states to focus on the design and implementation of new assessments; that is, consortia and non-consortia assessments. * Education plans to finalize the process in early 2015 and begin implementing it in late 2015. GAO examined Education's former peer review process in 2009[A] and found that this process gave no assurance that states include “highly, cognitively complex content” in their assessments. * “Highly, cognitively complex content” refers to activities demonstrating higher order thinking, such as critiquing experimental designs or combining ideas into new concepts. Several education stakeholders said that the old process focused too much on procedures and not enough on assessment quality. [A] GAO, [hyperlink, http://www.gao.gov/products/GAO-09-911], (Washington, D.C.: Sept. 24, 2009). No Child Left Behind Act: Enhancements in the Department of Education's Review Process Could Improve State Academic Assessments. Education's New Peer Review Process Is to Focus on High-Quality Assessment Principles: Under federal law, states are required to implement “high-quality” assessments.[A] The revised peer review process will include a focus on Education's requirements for assessments under the Assessment Program and ESEA Flexibility Initiative, including: * covering the full range of a state's college- and career-ready standards, including standards against which student achievement has traditionally been difficult to measure. * producing data that can be used to: - determine whether students are—or are on track to become— college- and career-ready; and; - inform educator evaluation systems[B] and school effectiveness determinations for ESEA accountability purposes. [A] See 20 U.S.C. § 6311(b) for ESEA assessment requirements. [B] Referred to in program guidance as making determinations of teacher and principal effectiveness for purposes of evaluation. Education Regularly Monitors Consortia Progress in Developing Assessments Under the Grant: Education's grant monitoring includes: * Approval procedures for changes to project plans. Education may approve changes if they meet certain criteria; for example, proposed changes may not violate grant terms and conditions or alter the overall scope and objectives. * Annual on-site program reviews. Education's reviews are intended to examine all aspects of assessment development and compliance with Assessment Program requirements. * Annual technical reviews. Stakeholders examine consortia approaches to developing assessment questions and ensuring valid assessment results. * Oversight reports. These describe what the consortia do well and where they need to improve. Education regularly followed up to ensure that the consortia were addressing areas needing improvement. Examples of Education Monitoring of Consortia Progress with Assessment Development under the Grant: Example 1: Education's monitoring of consortia developed assessment questions: * The terms of the grant require that assessments measure student knowledge and skills against the full range of standards. * In oversight reports, Education described how the consortia developed assessment questions. * Education described and monitored areas for improvement, including the action steps that each consortium proposed. * In its latest reports, Education commended both PARCC and Smarter Balanced on their processes for developing questions. Example 2: Education's approval of consortia requests to change their project plans: * Education responded in 21 letters to written requests from the consortia to alter their initial project plans. It approved all requests, sometimes after requiring additional information. - The majority of Education's responses pertained to requests about transferring money between budget areas. - PARCC proposed assessing students twice at the end of the year instead of four times throughout the year. Education approved this change after PARCC provided additional information and indicated that the goals and scope of its assessment would not change. The Consortia Plan to Evaluate the Validity of Their Assessments, as Required by the Grant: Assessment Program guidance states that assessments are expected to generate data that, among other purposes, can be used to measure college- and career-readiness of students and inform educator evaluation systems. * Another Assessment Program document notes that development of common assessments will enable the production of comparable data across states.[A] Education officials told us that this applies to cross-state comparisons within a consortium. Whether assessment data can be effectively used in these ways cannot be fully known until research, using assessment scores, can be conducted. In their grant applications, the consortia were required to include research and evaluation plans. The consortia plan to conduct research on assessment validity, including the use of assessment scores, once the assessments are operational. Education monitors the development of these plans and is also collecting data on the use of assessment scores in all states. Using Assessment Scores to Measure College- and Career-Readiness Requires Coordinating with Institutions of Higher Education: For purposes of the Assessment Program, high school students are considered college- and career-ready if they are prepared for success, without remediation, in credit-bearing entry-level courses in an institution of higher education. The consortia are taking actions to encourage institutions of higher education to use consortia assessment scores by: * coordinating with them to determine what high school assessment score should indicate college- and career-readiness, and; * planning empirical research on how well students who achieve college- and career-ready assessment scores perform in post-secondary, credit-bearing courses. Using Scores to Compare Student Performance across States Requires Research: How states and districts administer the assessments and use scores can affect comparability of student results.[A] For example: * Assessments can be taken on a variety of computing devices – some of which may be easier for students to navigate than others. * How states choose to use assessment scores (e.g., for teacher evaluation or student grade promotion) might, according to stakeholders, affect how seriously students and teachers take the tests, which can affect scores. The consortia have begun researching comparability of assessment scores using field test results and plan additional research once the assessments are operational. [A] It may also be difficult to compare student proficiency across consortium states because, according to Education officials, members of a consortium could use differing scores on their consortium's assessment to indicate student proficiency for purposes of Title I, Part A of ESEA. Education Is Assessing State Educator Evaluation Systems: To be approved for a waiver under its ESEA Flexibility initiative, Education requires states to implement educator evaluation systems that use multiple measures, including data on student growth from state assessments. Consortia assessment scores can help fulfill this purpose. * In August 2014, Education offered states transitioning to new assessments an additional year to incorporate student growth data into their evaluation systems. Education is sponsoring state and district surveys about ESEA implementation, including educator evaluation systems under the Flexibility Initiative and expects to begin reporting results in 2015. [End of Finding 3] [End of briefing slides section] Appendix II: GAO Contact and Staff Acknowledgments: GAO Contact: Jacqueline M. Nowicki, (617) 788-0580 or nowickij@gao.gov: Staff Acknowledgments: In addition to the contact named above, Scott Spicer (Assistant Director), Nancy Cosentino (Analyst-in-Charge), Divya Bali, and William Colvin made significant contributions to this report. Assistance, expertise, and guidance were provided by Sarah C. Cornetto, Angela L. Gallagher, Stuart M. Kaufman, Mimi Nguyen, Kathleen van Gelder, and Charlie E. Willson. [End of section] Footnotes: [1] United States Department of Education, A Blueprint for Reform: The Reauthorization of the Elementary and Secondary Education Act, March 2010. [2] To be eligible for federal funding under Title I, Part A of the Elementary and Secondary Education Act of 1965 (ESEA), as amended, states are required to adopt challenging academic content and student achievement standards and implement a set of high-quality, yearly student academic assessments. 20 U.S.C. § 6311(b). In this briefing, we use the terms "state" and "school district" to include "state educational agencies" and "local educational agencies" as used in the ESEA. [3] In this report we refer to the District of Columbia as a state. [4] Minnesota uses Common Core for English, but its own standards for math. Moreover, although the 45 states are using Common Core in the 2014-15 school year, some may change to different standards in future years. For example, Missouri recently enacted legislation that directs the state board of education to convene work groups to develop and recommend new standards, to be implemented in the 2016-17 school year. In addition, there is litigation that could affect Louisiana's standards in the future. [5] Education is authorized to waive ESEA requirements in certain circumstances. See 20 U.S.C. § 7861. As of October 9, 2014, Education had granted 42 states waivers under its ESEA Flexibility initiative. [6] These grants were funded by the American Recovery and Reinvestment Act of 2009, Pub. L. No. 111-5, tit. XIV, 123 Stat. 115, 279-86. The Assessment Program is carried out through cooperative agreements between the consortia and Education, although we refer to them as grants in this briefing. The Assessment Program requires that the consortia develop assessments that measure student achievement against a common set of college- and career-ready standards. Both consortia chose to develop assessments that align with the Common Core. [7] 20 U.S.C. § 6311(b)(3). In this report we refer to reading/language arts as English. States are also required to give assessments in science at least once in grades 3 through 5, once in grades 6 through 9, and once in grades 10 through 12. [8] These grants are authorized under Title VI of the ESEA. See 20 U.S.C. § 7301. [9] The PARCC consortium has established the cost of their assessments and the Smarter Balanced consortium has estimated the cost of their assessments. [10] "Highly, cognitively complex content" refers to activities that demonstrate higher order thinking, such as critiquing experimental designs or combining ideas into new concepts. [11] The grant periods are expected to end in December 2014 and August 2015 for Smarter Balanced and PARCC, respectively. These end dates do not affect the grant requirement that the consortia assessments be operational in the 2014-15 school year. [12] 75 Fed. Reg.18,171 (Apr. 9, 2010). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]