2020 Census: Bureau Needs to Better Leverage Information to Achieve Goals of Reengineered Address Canvassing
Fast Facts
For the 2020 Census, the Census Bureau originally planned to phase in technology and third-party data sources to build its address list—reducing the need to go door-to-door. The Bureau estimated it could update 75% of its address list this way, reducing costs by $900 million.
However, the Bureau has not evaluated the accuracy of its approach, which involves comparing aerial images to data in the master address list. It also suspended the operation's second phase—resolving mismatches—without detailing the cost, value, or accuracy of the new approach.
We made 3 recommendations to help the Bureau tackle these issues prior to the 2020 Census.
The Bureau’s initial plan for in-office and in-field canvassing to update its Master Address File
Flow chart illustrating Bureau's two-phased in-office address canvassing approach.
Highlights
What GAO Found
To achieve cost savings, the Census Bureau (Bureau) decided in September 2014 to use a reengineered approach for building its address list for the 2020 Census without having to go door-to-door (or “in-field”) across the country, as it has in prior decennial censuses. Rather, some areas (known as “blocks”) might only need a review of their address and map information using computer imagery and third-party data sources – what the Bureau calls “in-office” procedures. The Bureau planned to use a two-phase, in-office address canvassing operation to update 75 percent of all housing units in its master address list, and initially estimated this approach could reduce 2020 Census costs by $900 million.
The Bureau has not completed evaluations of results from tests and activities it carried out in 2016 to assess the effectiveness of in-office address canvassing. Specifically, the Bureau has not completed its evaluations designed to evaluate the accuracy of in-office address canvassing. Similarly, citing budget constraints, the Bureau also cancelled 2017 fieldwork that could have provided additional data to evaluate the reengineered approach. Further, the Bureau has changed the design of the in-office address canvasing operation with limited information on the cost and quality implications of the changes. Citing budget uncertainty, in March 2017 the Bureau suspended its second phase of in-office address canvassing, which was designed to resolve address coverage issues identified in the first phase. Bureau officials indicated that this decision would increase the in-field address canvassing workload from 25 percent to at least 30 percent, but they could not provide details of the cost or quality tradeoffs of this decision. Using its remaining evaluations to determine cost and quality implications would better position the Bureau to justify future decisions for its reengineered approach.
GAO has previously reported on the need for the Bureau to rethink its approach to testing and evaluating the census. The Bureau agreed, and its early plans called for the use of incremental, small-scale testing throughout the decade. As the Bureau begins planning for future census operations, small-scale testing by the Bureau may be less susceptible to budgetary constraints and allow it to better demonstrate the effectiveness of its operational innovations.
The Bureau manages the day-to-day production of in-office address canvassing using measures that do not fully reflect the effectiveness of the approach in reducing fieldwork. For example, the Bureau regularly tracks the number of blocks for which it has conducted an initial round of review. However, the workload the Bureau identifies as completed includes blocks that have been put on hold and that await availability of better information before being reviewed again. Other assumptions about how quickly the workflow proceeds have not been met. In testing the Bureau's productivity assumptions using actual production data through March 2017, GAO found that even small differences in measurement can have significant impacts on estimates of the remaining workload and the time to complete it. Without measuring how much in-office work is truly completed and how much fieldwork has been reduced, the Bureau risks underestimating the time and resources needed to complete the in-office operation, and may require more in-field address canvassing—at greater costs—than the Bureau currently projects.
Why GAO Did This Study
With a lifecycle cost of $12.3 billion (in 2020 dollars), the 2010 Census was the most expensive in U.S. history. Reengineering how the Bureau builds its address list is one of the ways the Bureau intends to reduce the per–housing unit cost of the 2020 count.
GAO was asked to evaluate the Bureau's reengineered approach for 2020 address canvassing. This report (1) describes the Bureau's design for 2020 address canvassing, (2) evaluates the extent to which the Bureau assessed the cost and quality implications of its reengineered address canvassing approach, and (3) assesses the status of the Bureau's efforts to reduce the in-field address canvassing workload. GAO reviewed relevant design and testing documentation and interviewed cognizant Bureau officials. GAO also reviewed Bureau address canvassing production and payroll data in December 2016 and March 2017.
Recommendations
GAO recommends that the Bureau (1) use its remaining evaluations before 2020 to determine cost and quality implications and to justify future decisions for its reengineered approach; (2) plan and execute smaller, more flexible tests needed to support key address canvassing design decisions in future census operations; and (3) use productivity measures that track the progress of the in-office address canvassing and its effectiveness in reducing fieldwork. The Department of Commerce had no disagreements with GAO's findings and recommendations. In addition, the Secretary noted that the Department was conducting its own review.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce |
Priority Rec.
The Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to use the Bureau's evaluations before 2020 to determine the implications of in-office address canvassing on the cost and quality of address canvassing, and use this information to justify decisions related to its re-engineered address canvassing approach. |
In October 2018 the Bureau provided preliminary results from evaluation efforts of its address canvassing for the 2018 Census Test, and in particular for the second phase of in-office address canvassing (referred to as active block resolution), which it decided not to use for the 2020 Census. The documents provided suggest specific technical aspects of canvassing to further explore, and make recommendations. Bureau officials stated that these results would inform its decisions about what else to look at with its 2020 Census canvasing and how possibly to further improve its in-office address canvassing for the 2030 Census. Taking these actions have potential to improve the effectiveness or cost efficiency of address canvassing in the next census.
|
| Department of Commerce | Early in the next decennial cycle, the Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to plan and execute more flexible, and perhaps smaller, address canvassing test and evaluation activity needed to support key design decisions having significant effect on the cost and quality of the census. |
In March 2024, Bureau officials updated us on their planned reliance on machine-learning tools to help improve detection of changes to the address list during the decade. They expect the tools to help provide "rolling" updates that would eliminate the need for manual review of potential additions and changes to the address list, notably reducing staff time preparing the address list for the 2030 Census. Officials had previously noted that as other decennial enhancements are researched and tested later in the decade, they expect the demand for large-scale field testing of address canvassing activity will be lower. In order to fully implement this recommendation, the Bureau will need to demonstrate how its key design decisions for the 2030 Census are less dependent on large, costly, or time-consuming tests, such as of address canvassing.
|
| Department of Commerce | The Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to use productivity measures that track the progress in completing in-office address canvassing workload and the effectiveness of in-office address canvassing in reducing fieldwork in order to make informed decisions on allocating resources to current and future address canvassing workload so that the operation is completed in a timely and cost-effective manner. |
In June 2018, the Bureau provided us an estimation methodology it was using to project the remaining in-office address canvassing workload, including that work which had been previously put on-hold and needed additional rounds of review. This document also quantified potential implications of the additional rounds of in-office review on the in-field address canvassing workload totals. In September 2018, the Bureau also provided us a workforce needs estimate that incorporated the additional rounds of in-office review when projecting how many clerical staff would be required to complete the in-office address canvassing operation. We believe these actions demonstrate the Bureau's follow through as recommended.
|