Blog 8: Harms in Machine Learning
Published on:
Case Study:
Understanding Potential Sources of Harm throughout the Machine Learning Life Cycle
In today’s blog I reflect on perhaps the most critical case study so far. The reason is because it goes over the root of all of the AI systems we see today. Machine learning is at the core of AI develoment and involves a lengthy process of training based on human data. One of the topics the authors of the case study discuss is the sources of harm in the Machine Learning pipeline (the process of designing, developing and deploying a machine learning model).
At the beginning of the process, there is always a risk of historical bias in the data we are using to train a ML model. The world as it is is heavily biased and every real-world data comes with those biases embedded in it. One of those biases is about reinforcing certain preexisting stereotypes against a particular group. For example, imagine you want to train a machine learning model that detects the geographical location of a place based on an outdoor picture. The resulting model is more likely to be accurate when it comes to pinpointing locations in the western world because most people here have access to a smartphone to capture high quality pictures. However, the ML model might not be so lucky in localizing pictures in remote mountainous areas in Africa and instead might guess that the picture is in some caribbean islands, some of the most-visited touristic destinations. The reason for this is because the majority of the population in remote areas in Africa do not have access to smartphones that take pictures so the only pictures that the dataset might get are the ones in urban areas which do not resemble the whole Africa. This example highlights how ML models can oversimplify complex geographies simply because the data that it has been fed represent only one group of a certain location.
A second source of bias is from the fact that the training data can underrepresent certain groups of population. Let us take for example an AI poem generator. Because of a lack of poem samples from non-western countries online, the AI might struggle to generate poems resembling non-western genres but will definitely have no problem when asked to generate poems resembling one of Moliere’s poems. One of the way we can address this source of bias is to increase the amount of data that we extract from non-western countries.
A third source of bias is when a label is used to approximate a very complex social situation. As a result of this, there is an oversimplification of the said social construct and the predictor. An example is the use of standardized tests scores to gauge how well a student will do in law school. The problem with standardized tests is that they are very predictable so anybody who spend time preparing for them can do them. On the other hand, law is such an ever-evolving discipline that requires constant education so the one-time test does not really matter in the long run. A way to curb the effects that come with this bias is to be extremely specific with what we are measuring with the tools that we have. For example, we can say the LSAT measures your aptitude to practice law but does not really indicate anything more than that.
A fourth source of bias is when a more general model is used for representing a population that contain different subsets, hence needs a particular consideration. For example, imagine if we were to build a predictor of how well STEM majors do after school based on how many years after graduation they got their first job. Although this model can work for certain STEM majors that can easily get a job after undergraduate studies, there is a whole other world of STEM specializations that require further studies in order to begin working in the industry so the proposed model would be heavily biased against the latter. Same as the measurement bias, being specific can go a long way in addressing the aggregation bias. Avoiding one-size fits all helps realize differences in subgroups of a certain group of individuals and therefore come up with solutions that satisfy as many people as possible.
A fifth source of bias comes from the learning phase of a model. When it is tuned to focus on a certain metric while missing out on another. For example, if a model is being trained on data that represents MPG and Horsepower of cars, it might assume at first that there is a linear regression of those two variables even though the horsepower can be increased without necessarily touching the fuel system of a car. An additional way to detect this source of harm is to become wary everytime there is two or more variables that we are looking at. When more than one variable is at our disposition and we are sure there is no correlation whatsoever, it is better to fine-tune the model from the get-go that there is no relationship between them in order to avoid the faulty correlation assumptions later.
A sixth source of bias is a result of the narrowness of evaluation benchmarks that are used to evaluate different AI models. The bias can be tied to the representation bias discussed earlier. An example is imagine we have two models that are tasked with ranking two different pieces of architecture. If we want to figure out which model performs better, we have to evaluate them against somehow similar benchmarcks, which is fine in my opinion. The problem arises because those benchmarks are rarely neutral. They might give much more importance to gothic-styled houses compared to other styles of architecture which means the model that was trained on gothic-styled houses is going to perform better. However, this does not really make sense as each style architecture is good depending on who is looking at. A simple way to mitigate the harms of this source of bias is to avoid comparing models that are different in how they have been trained and what they give as output.
The seventh source of bias is when there is a mismatch between the problem that the model is intended to solve and what it ends up being used for in the end. An example would be using a model that was built for recognizing diseases based on symptoms to now offer treatment solutions for the said disease. Although the model might give you answers to what you are asking it to do, it will not know for sure what it is talking about simply because it was not trained to offer treatment solutions. One way to mitigate this source of bias would be to install barriers in the models for them to know their limitations. This would make sure the models know when to answer and when not to depending on what they are good at.
With the enormous amount of biases discussed above, I think there should be a checklist that anybody with an ML project in mind should use in order to diminish the harm that is caused by those biases. The first thing to always check is how narrow is my training dataset?, the second is what specific solution am I looking for and for who? The third is how many subgroups are there in those for whom I am making the model for and will they all benefit equally from the solution? Lastly, it is imperative to do away with comparing models that are meant for different scenarios so making sure each model is treated differently and is only measured based on the goals of whoever made it would be a nice precaution to always take.
Looking forward, it is very hard to imagine an AI model that is completely free from bias. Despite this, there is always this talk about AGI, Artificial General Intelligence, which basically can accomplish whatever task a human can. How feasibe is the idea of AGI and is it still wishful thinking or something that will happen in the near future? I am really curious about this question because I think it invites the reader to pose and think about the future of AI and whether it can trully replace humans.
All in all, the case study for today’s blog was nothing short of informative. I learned a lot in so few lines and at the same time I feel like I am left stranded and I need to figure out more. This blog has left me questioning everything I thought I knew about AI and what direction it is headed in the near future.