Exposing Error in Poverty Management Technology: A Method for Auditing Government Benefits Screening Tools
Assistant to Lead Investigator
Lead Investigators: Nikola Banovic and Nel Escher
In PACM HCI. 4, CSCW1, Article 64 (May 2020) (CSCW 2020). ACM, New York, NY, USA, 20 pages.
Public benefits programs help people afford necessities like food, housing, and healthcare.
In the US, such programs are means-tested: applicants must complete long forms to prove financial distress before
receiving aid. Online benefits screening tools provide a gloss of such forms, advising households about their eligibility
prior to completing full applications. If incorrectly implemented, screening tools may discourage qualified households from
applying for benefits. Unfortunately, errors in screening tools are difficult to detect because they surface one at a time and
difficult to contest because unofficial determinations do not generate a paper trail. We introduce a method for auditing such tools
in four steps: 1) generate test households, 2) automatically populate screening questions with household information and retrieve determinations,
3) translate eligibility guidelines into computer code to generate ground truth determinations, and 4) identify conflicting determinations to
detect errors. We illustrated our method on a real screening tool with households modeled from census data. Our method exposed major
errors with corresponding examples to reproduce them. Our work provides a necessary corrective to an already arduous benefits application process.