VA Disability Benefits: Board of Veterans' Appeals Should Address Gaps in Its Quality Assurance Process
Fast Facts
If the VA denies a veteran's benefit request, the veteran can bring the case to the Board of Veterans' Appeals. Congress has raised questions about the quality of the Board's decisions.
We testified that as the Board faces large workloads and time pressure, quality assurance activities such as case reviews and decision monitoring are key tools for minimizing rework.
But the Board could better use evidence to improve these activities. For example, it hasn't fully assessed the underlying causes of common errors, which could lead to better solutions. We recommended doing so—and using other evidence—to address gaps in the Board's quality assurance.
Highlights
What GAO Found
The Board of Veterans' Appeals (Board) has a quality assurance (QA) process and a related accuracy measure for its decisions. Specifically, its QA process involves: 1) checking a random sample of draft decisions for certain types of Board-defined errors each month through its case review process; and 2) monitoring outcomes of Board decisions that were further appealed to the U.S. Court of Appeals for Veterans Claims (CAVC). The Board uses results from these activities to provide various interventions, such as individual feedback to veteran law judges (VLJs) or training. The Board also calculates and publishes an accuracy rate that represents error-free adjudications. However, GAO found shortfalls in the Board's process for calculating this measure. Contrary to federal internal control standards, GAO found that the Board did not have:
(1) written policies or procedures for calculating its accuracy rate or managing case review error data, or
(2) checks on its accuracy rate calculation.
As a result, the Board lost data and GAO could not verify accuracy rates provided by the Board in 2 of 4 fiscal years. Until the Board develops written policies and procedures, the Board will likely continue to have difficulty supporting the accuracy of its publicly reported measure.
GAO found gaps in the Board's efforts to build and use evidence—such as a lack of data and analysis—to assess its QA process and related activities. GAO analyses of CAVC and Board data show that over the past 3 fiscal years, CAVC remanded (sent back) about 80 percent of appealed Board decisions, often because CAVC found the Board's explanation of its findings to be inadequate. However, GAO found that the Board lacked evidence to better understand and address these and other issues and set priorities to help improve its QA process. Specifically:
- Board officials told GAO it has no comprehensive, written plan outlining how it will accomplish the mission and goals of its QA process.
- Board officials told GAO they had not fully analyzed trends or underlying causes of the most common Board-identified errors or CAVC remands.
- The Board has not systematically or comprehensively built or used evidence to better understand and improve its interventions, such as collecting feedback about training.
- The Board does not assess VLJ decisions for consistency, such as whether common misunderstandings of policy or law exist in decisions.
GAO's prior work has identified key practices for evidence-based decision-making. These practices involve building and assessing evidence and using it to foster a culture of continuous improvement. Absent such a process and a plan to guide it, the Board is not positioned to fully understand and address underlying causes of the most common errors and remands or understand consistency—evidence which is needed to target and implement effective interventions and foster continuous improvement of its QA efforts.
Why GAO Did This Study
Each year, the Board adjudicates thousands of cases in which a veteran was dissatisfied with the Department of Veterans Affairs' (VA) initial decision on their claim for benefits. However, researchers and Congress have raised questions about how the Board ensures the quality of its adjudications.
GAO was asked to review the Board's QA process. This testimony examines (1) how the Board assures and measures the quality of its adjudicative decisions, and (2) the extent to which the Board builds and uses evidence to assess its QA process.
GAO reviewed relevant federal laws and Board documents. GAO analyzed QA data provided by the Board for fiscal years 2019 through 2022 and assessed its reliability. GAO also reviewed relevant quality assurance literature and interviewed VA officials, attorneys, VLJs, and subject matter experts, including researchers and veterans service organizations. GAO also conducted group discussions with randomly selected Board decision-writing attorneys and VLJs.
Recommendations
GAO is making four recommendations, including that the Board develop policies for calculating its accuracy rate; develop and implement an evidence-based decision-making process for its QA efforts; and study decision-making consistency. The Board generally agreed with three recommendations and disagreed with the fourth about consistency. GAO continues to believe that the recommendation is valid as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Board of Veterans Appeals | 1. The Chairman of the Board of Veterans' Appeals should develop written policies and procedures related to its accuracy rate measure, to require that OAI (1) involves more than one official in the calculation process; (2) documents its calculation of monthly and fiscal year accuracy rates; and (3) manages related error data. (Recommendation 1) |
The Board agreed in principle with this recommendation. Board officials stated the recommendation should clarify the Board's need for a policy and procedure to manage data: the loss of data was specific to fiscal years 2019 and 2020. However, we note that the data loss stemmed from not having policies and procedures and without them, these types of data management issues could occur again. We continue to believe that having policies and procedures are an important part of internal control. Without written policies and procedures, the Board could face further challenges such as additional data loss, improperly storing data, or using inconsistent methodologies.
|
| Board of Veterans Appeals | 2. The Chairman of the Board of Veterans' Appeals should monitor how veteran law judges choose to incorporate the feedback they receive from the case review process—including whether errors are corrected—and use this data to inform decision making related to the case review process. (Recommendation 2) |
The Board agreed with this recommendation. We will monitor the agency's progress to implement it.
|
| Board of Veterans Appeals | 3. The Chairman of the Board of Veterans' Appeals should develop and implement an evidence-based decision-making process that includes a plan outlining how it will build evidence to assess the underlying causes for the most common errors identified by the case review process and the most common reasons for CAVC remands. The Board should use this evidence to better target its interventions and assess their results. One option is to fold the development of this process into the Board's planned evaluation. (Recommendation 3) |
The Board agreed in principle with this recommendation. We will monitor the agency's efforts to implement it.
|
| Board of Veterans Appeals | 4. The Chairman of the Board of Veterans' Appeals should study how to evaluate VLJ adjudicative decisional consistency. One option is to fold the development of this study into the Board's planned evaluation. (Recommendation 4) |
The Board disagreed with this recommendation. Board officials stated that it would be inappropriate to force consistency in VLJ decisions in a way that is inconsistent with codes of judicial conduct and other standards applicable to VLJs. They stated that there will and should be variance in how legal authority is applied to the individual facts and circumstances of each case appealed to the Board. Board officials further stated that VLJs should be independent and not subject to pressure and influence. They also stated that, as part of evaluating individual VLJs' adherence to performance standards, the Board evaluates the total number of decisions each VLJ has adjudicated and the manner in which they have adjudicated them, among other things. In addition, Board officials noted that the number of decisions remanded does not demonstrate whether such remands were warranted, and that the number of CAVC remands or reversals does not necessarily correlate with productivity, legal acumen, or even with performance. Finally, they said that the concept of "consistency" of decision-making among individual judges evaluating sets of facts and circumstances of each individual case is a difficult one to address at all levels of adjudication, not only at the Board, but also at CAVC. We acknowledge the importance of judicial independence, that some degree of variance is expected given that independence, and that variation is not necessarily an indicator of poor quality decision-making. In addition, the Board acknowledges that consistency among VLJs in the use of appropriate legal authority is appropriate. However, without studying consistency, the Board will remain unaware of whether systemic inconsistencies in VLJ decision-making exist, and the Board will be unable to target interventions, as appropriate. We continue to believe that the results of systematic study of VLJ adjudicative decisions for consistency could provide a basis for targeting interventions, such as training, to assist VLJs. As such, this recommendation is not intended to "force" consistency in any VLJ decisions, but rather is meant to help the Board assist VLJs. Our recommendation is worded to allow the Board the necessary latitude to implement it in a way that allows for the retention of judicial independence while meeting other goals it deems appropriate.
|