Quotessence
Home / Authors / Cathy O'Neil Books
Cathy O'Neil

Cathy O'Neil Books

Author

Related Quotes

“From a mathematical point of view, however, trust is hard to quantify. That's a challenge for people building models. Sadly, it's far easier to keep counting arrests, to build models that assume we're birds of a feather and treat us as such. Innocent people surrounded by criminals get treated badly, and criminals surrounded by law-abiding public get a pass. And because of the strong correlation between poverty and reported crime, the poor continue to get caught up in the digital dragnets. The rest of us barely have to think about them.”

CrimeInjusticeInequalityFairnessBig DataFeedback Loop
Book:Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Create image

“Will those insights be tested,or simply used to justify the status quo and reinforce prejudices? When I consider the sloppy and self-serving ways that companies use data, I'm often reminded of phrenology, a pseudoscience that was briefly the rage in the nineteenth century. Phrenologists would run their fingers over the patient's skull, probing for bumps and indentations. Each one, they thought, was linked to personality traits that existed in twenty-seven regions of the brain. Usually the conclusion of the phrenologist jibed with the observations he made. If the patient was morbidly anxious or suffering from alcoholism, the skull probe would usually find bumps and dips that correlated with that observation - which, in turn, bolstered faith in the science of phrenology. Phrenology was a model that relied on pseudoscientific nonsense to make authoritative pronouncements, and for decades it went untested. Big Data can fall into the same trap. Models like the ones that red-lighted Kyle Behm and black-balled foreign medical students and St. George's can lock people out, even when the "science" inside them is little more than a bundle of untested assumptions.”

ScienceFairnessPseudosciencePhrenologyBig DataFeedback Loop
Book:Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Create image

“By construction, the world of big data is siloed and segmented and segregated so that successful people, like myself - technologists, well-educated white people, for the most part - benefit from big data, and it's the people on the other side of the economic spectrum, especially people of color, who suffer from it. They suffer from it individually, at different times, at different moments. They never get a clear explanation of what actually happened to them because all these scores are secret and sometimes they don't even know they're being scored.”

PeopleKnowsWorldWellsDifferentSometimesMomentsBigsSufferingSidesWhiteSecretSuccessfulClearHappenedEconomicColorBenefitsEducatedDataExplanationScoreConstructionSuccessful PeopleSpectrumDifferent TimesWell Educated
Author:Cathy O'Neil
Create image

“Because of my experience in Occupy, instead of asking the question, "Who will benefit from this system I'm implementing with the data?" I started to ask the question, "What will happen to the most vulnerable?" Or "Who is going to lose under this system? How will this affect the worst-off person?" Which is a very different question from "How does this improve certain people's lives?"”

PeoplePersonsDoeDifferentHappensCertainAsksLosesWorstBenefitsAskingVulnerableDataImplementing
Author:Cathy O'Neil
Create image

“With recidivism algorithms, for example, I worry about racist outcomes. With personality tests [for hiring], I worry about filtering out people with mental health problems from jobs. And with a teacher value-added model algorithm [used in New York City to score teachers], I worry literally that it's not meaningful. That it's almost a random number generator.”

PeopleProblemJobsUsedValuesNumbersCitiesWorryTeacherNew YorkExamplePersonalityModelsTestsMental HealthMeaningfulOutcomesScoreRacistNew York CityHiringAlgorithmsHealth ProblemsGeneratorRecidivism
Author:Cathy O'Neil
Create image

“We don't let a car company just throw out a car and start driving it around without checking that the wheels are fastened on. We know that would result in death; but for some reason we have no hesitation at throwing out some algorithms untested and unmonitored even when they're making very important life-and-death decisions.”

KnowsImportantReasonDecisionResultsCompanyCarDrivingWheelsLife And DeathThrowingHesitationAlgorithmsImportant Life
Author:Cathy O'Neil
Create image

“The Facebook algorithm designers chose to let us see what our friends are talking about. They chose to show us, in some sense, more of the same. And that is the design decision that they could have decided differently. They could have said, "We're going to show you stuff that you've probably never seen before." I think they probably optimized their algorithm to make the most amount of money, and that probably meant showing people stuff that they already sort of agreed with, or were more likely to agree with.”

PeopleThinkingSaidShowsStuffDecisionTalkingDesignAmountDecidedAgreeDesignerAlgorithms
Author:Cathy O'Neil
Create image