Many WMDs create feedback loops that perpetuate injustice. Recidivism models and predictive policing algorithms—programs that send officers to patrol certain locations based on crime data—are rife with the potential for harmful feedback loops. For example, a recidivism model may ask about the person’s first encounter with law enforcement. Due to racist policing practices such as stop and frisk, black people are likely to have that first encounter earlier than white people. If the model takes this measure into account, it will probably deem a black person more likely But they are harmful even beyond their potential to be racist. O’Neil writes, A person who scores as ‘high risk’ is likely to be unemployed and to come from a neighborhood where many of his friends and family have had run-ins with the law. Thanks in part to the resulting high score on the evaluation, he gets a longer sentence, locking him away for more years in a prison where he’s surrounded by fellow criminals—which raises the likelihood that he’ll return to prison. He is finally released into the same poor neighborhood, this time with a criminal record, which makes it that much harder to find a job. If he commits another crime, the recidivism model can claim another success. But in fact the model itself contributes to a toxic cycle and helps to sustain it. O’Neil’s book is important in part because, as she points out, an insidious aspect of WMDs is the fact that they are invisible to those of us with more power and privilege in this society. As a white person living in a relatively affluent neighborhood, I am not targeted with ads for predatory payday lenders while I browse the web or harassed by police officers who are patrolling “sketchy” neighborhoods because an algorithm sends them there. People like me need to know that these things are happening to others and learn more about how to fight them. - https://blogs.scientificamerican.com/roots-of-unity/review-weapons-of-math-destruction/