Blog 3: Exception to Data Driven Rules
Published on:
Case Study:
The Right to Be an Exception to a Data-Driven Rule
Today’s case study aims to provide a framework that can be used in deciding whether a data-driven rule should be employed rather than a human one in critical decision making. The case study focuses on three main components in coming up with the new framework: individualization, uncertainty, and harm. My goal in this blog is to address what all these terms mean in the context of decision making.
First things first, what do they mean by a data-driven rule, you might wonder? A data-driven rule is a directive that results from pre-trained algorithms that often give a score to a question, like, is somebody more likely to commit a crime again? based on previvous data that it has been fed. Like any rule created by mankind, there are always exceptions which occur when the rule does not apply to a specific subject because theu subject possesses way more covariates than what the algorithm has been fed with. Something to note, however, is that an exception is not the same as an error because error means that the algorithm is giving straight up wrong information based on preconceived data whereas in the case of an exception the algorithm does not even have enough data to judge a specific subject.
The alternative to a data-driven rule is human decision. Human decisions are way more context-dependent than data-driven ones will ever be. For example, a human recruiter is very quick to see the attractiveness of a resume and can use that to judge if a candidate is suitable or not for the role of UX/UI design. It would be, on the other way, harder to make an algorithm judge whether a resume is more attractive than another. Data-driven decisions, on another note, if accurate and precise, can be applied a large set of subjects in an instant of time in the way that human ones cannot. For example, if I am a recruiter and I am looking for a backend developer who has basic Python programming skills, it is easy to build an algorithm that scan the thousands of resumes I have gotten and rank them from longer to shorter Python programming experience. A human would take months to go through all thousands of resumes and rank them.
One of the critical requirements for a fair data-driven rule is individualization. Its benefits include the reduction of bias in the data-driven decisions that are being made. It forces the decision makers to really consider a vast majority of covariates before attempting to assign a general rule to a diverse population. However, its downside that is not really discussed is the risk of transforming really fast algorithms into really slow ones because of the increase of factors that the algorithms now have to take into consideration.
Equally critical is the concept of uncertainty, which means that a data-driven rule should only be used if the levels of individualization and certainty are high enough to justify any harm that could result from automated decision-making. I feel like when the stakes are high, whatever evaluation metric we use to justify the use of a data-driven rule better have a success rate of 100%. So far I have not come across such a metric so I would say no for the sake of preserving the spirit of the case study that states that each individual should have the right to be an exception. Uncertainty is so critical when the decision is irreversible in the future, for example, in a death sentence case.
Through writing this blog, I have come to realize how much sacrifice authorities are willing to make to come up with fast solutions rather than ethical ones. This can be seen in the individualization concept that makes algorithms really slow. Now I am left wondering, is this a product of capitalism or just a human nature and is there any political system out there that prioritizes safety more than profit or dominion? I think this question helps the reader ponder on what makes us human and how should we take care of our neighbors whose voices are not heard but still who need safety as much as we do.
All in all, this blog was a great way to come to terms with how technology can be biased in the name of simplicity and how that bias can lead to irreversible damage. It has made me to be more compassionate especially to the ones I would normally consider as the 'other’\