EOIR's Data Release on Asylum So Deficient Public Should Not Rely on Accuracy of Court Records
From a June 3 press release by TRAC
TRAC Immigration has concluded that the data updated through April 2020 it has just received on asylum and other applications for relief to the Immigration Courts are too unreliable to be meaningful or to warrant publication. We are therefore discontinuing updating our popular Immigration Court Asylum Decisions app, and will take other steps to highlight this problem[1]. We also wish to alert the public that any statistics EOIR (Executive Office for Immigration Review) has recently published on this topic may be equally suspect, as will be any future reports the agency publishes until these major data deficiencies are explained and rectified[2].
The EOIR's apparent reckless deletion of potentially irretrievable court records raises urgent concerns that without immediate intervention the agency's sloppy data management practices could undermine its ability to manage itself, thwart external efforts at oversight, and leave the public in the dark about essential government activities. Left unaddressed, the number of deleted records will compound each month and could trigger an expensive data crisis at the agency. And here the missing records are the actual applications for asylum, and how the court is handling them. This is a subject on which there is widespread public interest and concern.
EOIR Data Irregularities Approaching Point of No Return
Despite TRAC's appeals to the EOIR, Immigration Court records continue to disappear each month. TRAC initially reported 1,507 missing applications for relief in our October 2019 report, which grew to 3,799 missing applications the following month. We wrote EOIR Director James McHenry providing a copy of the 1,507 missing applications asking for answers on why these records were missing from their files. We wrote again when the number of missing applications more than doubled the following month. These letters were met with silence. Not only have these cases disappeared entirely, they have not been restored in any subsequent data releases and the number of missing relief applications continue to grow. (See the final section for a short explanation of TRAC's methodology.)
Alarmingly, the number of relief applications that were present in the March 2020 data release but were missing in the April release jumped to 68,282. This is just the number of records that disappeared over a single month. It does not include the ever growing number of applications that had previously disappeared month-by-month. As was true in past months, roughly four out of five of the records in the March 2020 release that disappeared from April's release concerned applications on which the court had rendered its decision, including many cases in which the immigration judge had granted asylum as well as other forms of relief.
To put that into perspective, the number of missing cases just last month is more than the 63,734 asylum applications received by the Immigration Courts during all of FY 2015. If these applications are missing because they have been deleted from the Court's own master files, the magnitude of the task of restoring just this single month's destruction—assuming this is even possible—is enormous. To go back and restore the cumulative number of relief applications that went missing during previous months will obviously be even greater.
In fact, so many asylum decisions were dropped from EOIR's April release that the cumulative number of asylum decisions went down, not up, despite asylum decisions continuing to be made. The volume of disappearing records has reached a scale that little faith can be placed in the factual accuracy of reports published by the EOIR based on its data.
The EOIR's escalating data problems should raise dire concerns for Congress, policymakers and the public who routinely put their faith in federal agencies to provide complete and accurate information about their work. Indeed, the management of the court system itself, including the quota system recently imposed on immigration judges, presupposes the accuracy of the court's own records. It is deeply worrisome that the EOIR and the Department of Justice appear unconcerned with ensuring that their own records are accurate and uncommitted to providing the public with accurate and reliable data about the Court's operations.
TRAC Urges EOIR to Take Immediate Action
To date, the EOIR has not responded to TRAC's requests for an explanation of these disappearances, nor has the EOIR responded to TRAC's FOIA requests for records that would shed light on this matter.
Therefore, TRAC has written a third letter to Director McHenry reporting our findings of 68,282 new disappearances and we are again seeking a commitment from him to take the steps needed to address the problem. More urgently, we are asking that the EOIR immediately preserve—rather than destroy—all back-up tapes or other media in the hopes that records apparently improperly deleted from the Court's master files might be restored. We assured Director McHenry that we would be more than happy to work cooperatively with the agency to help them better ensure that going forward the public is provided with more accurate and reliable data about the Immigration Court's operations.
How EOIR's Data Mismanagement Impacts TRAC's Immigration Court Tools
TRAC's mission is to provide the public with accurate, reliable, unbiased, and timely data on the operations of the federal government, and to ensure that the public is informed about changes that impact our data.
The EOIR's disappearing records fall under the data related to applications for relief. The record on the existence of the court case itself is present, but for a growing number of these cases there now is no record that the immigrant ever applied for relief, or the court's decision on that application. One of the key moments in the life of the case—including applications for asylum—is missing entirely. As a direct consequence TRAC does not have the information needed to provide reliable or meaningful updates on the court's handling of applications for asylum and must therefore discontinue updating its asylum decision app.
While each of the other files in EOIR's monthly data releases also have the same problem of records disappearing, the magnitude of these disappearances has not reached the levels seen with applications for relief. While still worrisome, these levels have not yet climbed to where we believe we can no longer use the information we receive. Thus, we are continuing to update the rest of our other Immigration Court apps. We continue to closely monitor the situation, while we urge EOIR to explain why records keep disappearing. We further continue to ask the agency to take the steps needed to rectify the situation.
TRAC will continue to retain all previous and future EOIR data shipments for research purposes.
How did TRAC Identify the EOIR's Data Irregularities?
The Executive Office for Immigration Review (EOIR) oversees the nationwide Immigration Court system, including more than 60 physical Immigration Court locations (as well as many more remote hearing locations including teleconference sites and ad hoc "tent" courts), hundreds of Immigration Judges, and millions of immigration cases that pass through the court system. The EOIR records information on each case and tracks various proceedings, filings, hearings and other aspects of each case in a large database. This database is central to the Court's ability to manage its workload, prepare and publish reports for the public, and respond to queries from Congress about its operations. It is also used in implementing new practices, including the recent decision to impose new evaluation criteria for Immigration Judges.
As a result of TRAC's ongoing FOIA requests, the EOIR releases a large batch of anonymized Immigration Court data each month that provides a snapshot of a great deal of the information recorded in this database on the handling of each case. In short, TRAC does not create data on the EOIR; rather, TRAC's uses the EOIR's own data. This data is the foundation for TRAC's Immigration Court data tools which help ensure transparency and accountability for the American public.
TRAC used this data to precisely identify deleted records. While the information TRAC receives does not identify individuals, EOIR's computer system assigns a unique computer sequence number to each case that identifies it. Because TRAC receives comprehensive data shipments from the EOIR each month that include these unique computer-assigned tracking numbers, TRAC can match each record received in the previous month with the same corresponding record in the following month's release. Each release is also cumulative. That means it should include every record from the previous month plus every new record that has been added to the database over the course of the current month. As a rule, records should therefore never disappear[3].
When a record that was present is not included in the next month's release, TRAC refers to these as missing or disappearing records. Because humans maintain most databases including EOIR's, mistakes will occur. Therefore no database is ever perfect. So a few disappearing records might be expected. However, as is the situation here, concern is warranted whenever significant numbers of records disappear. Indeed, alarm bells should ring as the number of disappearing records grow. This situation means the data can no longer be trusted to reliably track the court's proceedings.
Footnotes
[1] EOIR monthly releases consist of a series of tables covering different aspects of its workload. While each of these tables continue to have disappearing records each month, the magnitude of these missing records varies by table. For example, in the table that tracks each case before the court there were 228 cases present in March that disappeared from the April release, compared with 41,233 new cases that were added. While the problem of disappearing case records remains very troubling for the case table along with each of the other EOIR tables, TRAC believes that their magnitudes do not rise to the same level as the problem for applications for relief where the data now are so unreliable and misleading that they do not warrant the public placing any trust in them. At this time, we therefore are continuing to update our other Immigration Court apps while alerting the public to this continuing serious problem that affects the reliability of EOIR data releases more generally.
[2] For an example of a recent EOIR publication that may contain significant data errors, see the graph and table reporting total asylum applications through March 2020, which was generated using data from April 2020: https://www.justice.gov/eoir/page/file/1106366/download.
[3] Even when a data entry error is made, the database has special codes to indicate that a record should be disregarded because it was a data entry error so that rarely is it necessary to actually delete records.
TRAC is a nonpartisan, nonprofit data research center affiliated with the Newhouse School of Public Communications and the Whitman School of Management, both at Syracuse University. For more information, to subscribe, or to donate, contact trac@syr.edu or call 315-443-3563.