BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Kids Endangered By Predictive Analytics? Child Advocate Says Yes

Following
This article is more than 8 years old.

A recently released report from the National Coalition for Child Protection Reform (NCCPR) questions the use of predictive analytics in child welfare. The report asserts that recommendations for child protection practice under consideration by one federal commission would lead to unnecessary trauma for children and increase their risk of abuse.

Parents don’t often kill their children. An estimated 1580 children in the US died due to child abuse or neglect in 2014, about 2 deaths per 100,000 children. Everyone wants to prevent these worst cases of child abuse. The question is, what’s the right way to go about it?

Enter the young industry of predictive analytics solutions for child welfare. Software developers, among them SAS and the nonprofit Eckerd, are producing and promoting solutions geared to mathematical assessment of child abuse risk. Draft recommendations from the Commission to Eliminate Child Abuse and Neglect Fatalities (CECANF) call for all states to use these solutions. In fact, the draft recommendations specifically refer to “Rapid Safety Feedback,” a term trademarked by Eckerd.

Richard Wexler, Executive Director of NCCPR, and author of the NCCPR report, casts doubt on the idea that predictive models can identify specific children at greatest risk. He points out that very few parents kill their children, and the available data is insufficient to accurately identify specific children who are likely to be victims. He’s particularly concerned about “false positives,” that is child welfare agencies treating too many cases as high risk.

What’s the harm in approaching a child as high risk for fatality if the child is not at risk ? Wexler focuses on this CECANF recommendation: ”Investigate all CPS hotline calls for children under age 5.” Wait, wouldn’t we want all hotline calls investigated? Maybe not.

CECANF itself points out that 40% of hotline calls are screened out. That is, social workers screen these calls and decide against further action, which may include visiting the home, questioning family members and searching the child’s body for signs of physical abuse. Screening does not count as investigation, only the more intrusive actions do.

So, picture this. You’re a parent of a young child. Someone, anyone, calls a hotline and accuses you of abusing your child. Today, child protective services staff can examine the information available and may decide that the complaint does not warrant coming to your home, questioning you and your child, or making your child undress and submit to physical examination by a stranger. But, under the proposed CECANF recommendations, every call would lead to an intrusive investigation. So anyone could make a call, for any reason, and you’d have child abuse investigators at the door.

If the recommendation is only partially accepted, risk scores created by predictive modeling software would be used to justify greater numbers of child abuse investigations. Would that be better? After all, these tools use real data and mathematics.

Wexler has critics. Yesterday, The Chronicle of Social Change posted an op-ed piece by Joshua New, a policy analyst at the Center for Data Innovation. New accused Wexler and a number of other child welfare advocates of sabotaging child protection efforts, describing them as “advocates more fearful of data than they are concerned about the welfare of children.”

Wexler is not fearful of data. When we spoke, he referred me to a myriad of research studies, each loaded with data and thoughtful analysis. He was quick to point out known risk factors, and the data sources behind them. He also explained subtle issues with seemingly impartial risk models, and how they may mask race or income discrimination.

Richard Wexler is not a statistician. But I am.

As a statistician, I take the view that the math means less than your ability to use it as a basis for appropriate action. Technical excellence in analytics (whether you call it predictive analytics, statistics, data science or by any other name) is not the goal. The goal of predictive analytics is always rational action that benefits people.

Questioning analytics should always be part of the decision-making process. We should always ask about underlying data, its source, and its limitations. We should not make more of the math than it really is – just a very good tool for providing information, if you know how to use it well.

Predictions are often wrong, even when they are made using excellent data and excellent mathematical technique. That matters little when the prediction is used, say, to decide whether a particular family should receive a new credit card promotion or solicitation for a charitable contribution. Those are just ads, and the recipient can take the offer or leave it. Predictions used to single out families for intrusive criminal investigations are another matter, and deserve greater scrutiny.

The NCCPR report discusses the costs of false positives – flagging cases as high risk when they are not. They include the expense of added investigations, diverting staff from cases of real need, traumatizing children and even increasing their risk by placing too many kids in foster care, where the risk of abuse may be higher than in the child’s home.

Costs of additional investigations alone are pretty staggering. I asked Wexler how he’d suggest using all that money to protect children, if not on investigation. He was quick with suggestions of other preventive measures – housing support, daycare, intensive family preservation services. As a parent, I found his suggestions humane. As a statistician, I found them data-driven. He rattled off information about studies and findings to support each item, to the point that I could not write fast enough to make notes on them all.

Analysts and decision makers would do right to follow Wexler’s lead, consider the cost of false positives as part of any analytics project, and look beyond the math to find humane and sustainable solutions for our problems.

Follow me on LinkedInCheck out my website