I read the book “Weapons of Math Destruction” by C. O’Neil and I am glad I did. The book at times reads like a rant, certainly, that is the idea you can get if you read only the short excerpts available online. But it is much more than that. O’Neil has assembled an informed and complete critique of the contemporary uses and misuses of statistical models in various aspects of everyday life.
O’Neil starts with a basic thesis that she stays faithful to throughout the book: despite their legitimate uses, statistical models when applied to domains outside the realm of pure science and technology, have adverse side effects. Her argument is an argument of justice, there is always a minority (???) of people affected for whom we should, and can, do better.
Statistical inference is at the heart of statistical science; you collect some data and based on them you try to estimate the probability of some event. This is not news for e.g. medicine where statistics has been put to good use for ages. For example, what is the fraction of people that will develop cancer at some point in their lives given their smoking and eating habits? Based on a study like this scientists can identify factors that affect our health and make informed public health decisions.
The legitimate example O’Neil provides in the book is about the use of statistics in sports e.g. baseball. There, by meticulously collecting ample and relevant data (emphasis on “relevant” data, we will return to that later) you can rank players and optimize your roster. No objection.
But what happens when direct data is not, easily, available and you have to resort to the use of “proxies” (I am using the vocabulary used in the book). Is it fair to dispute your ability to pay your bills only because you grew up and live in a neighborhood with a high percentage of high school dropouts? Certainly, a lot of kids that do not finish high school have trouble finding and keeping a good job. But, are we allowed to inflict higher insurance premiums to people who have been assigned to a similar bucket only because other people in the same bucket cannot pay their credit card? Fairness and common sense require bypassing the use of such proxies and using the actual data of payment history.
This brings us to the second point the author brings consistently: opacity. How would you like to be evaluated on a set of rules that are not clear to you? This is exactly what happens in many cases where statistical models are used to make decisions. For example, having a perfect SAT score does not guarantee college admission. If so, what does? Well nobody knows. We all tend to think of higher education as a place that promotes academic achievement but academic achievement alone is not a deciding factor. Or, take the example of teacher evaluation at schools. It certainly resonates well with most of us that if we evaluate teachers and reward those that excel accordingly, then the quality of education will improve. But how do we evaluate teachers? Could it be through the performance of their students in standardized tests? If so, how do we correct from improvement from year to year or the educational level of the parents?
A fair critique one can mount against the author’s arguments is one of consistency as well. When it comes to the use of big data in the insurance industry, statistical tools classify people in different buckets of risk. As a result, the industry raises premiums accordingly. This, O’Neil claims, violates the essence of insurance since it transforms it from a bet against a worst-case scenario to a rearrangement of payments for that scenario across time. I can understand this argument but then how is it fair towards the risk-averse fraction of us who deserve a lower premium than the rest?
Clearly, there are no easy answers when it comes to important social problems like the ones the author tackles. O’Neil’s critique rests on two solid pillars that are consistent with our legal tradition: fairness and transparency (I prefer the positive term to the negative ‘opacity’ used in the book). It is reasonable for us to demand that any adoption of the use of big data conforms to these constants of our civilization.
You can hear the author talk about her book here.
No comments:
Post a Comment