The incident shows the ever-present danger of algorithmic bias in computer vision applications.
Google Vision Cloud, a “computer vision” service that utilizes AI to label images, was found to produce different results depending on the subject’s skin color. Google (NASDAQ: GOOGL) apologized after AlgorithmWatch published the racist results of the experiment on Twitter. (AlgorithmWatch)
The risk of racist bias in computer vision AI algorithms
In the images reproduced below from Twitter, the algorithm labeled a dark-skinned individual with a hand-held thermometer as “gun.” However, a lighter-skinned person with a hand-held thermometer was flagged as “electronic device.”
Google corrected the algorithm, and as on April 6, it does not produce the “gun” flag any longer.
The incident shows how machine learning can unwittingly introduce bias into the output results. This is the process of “training” an algorithm to recognize patterns from millions of human-labeled images,
In this case, a larger proportion of the training data may have had violence-related images of dark-skinned individuals. Therefore, the computer vision algo correlated the weapon-like looks of a hand-held thermometer with the dark-skinned hand and flagged “gun.”
Google apologizes for biased results
Tracy Frey, director of Product Strategy and Operations at Google, wrote to AlgorithmWatch that “this result [was] unacceptable. The connection with this outcome and racism is important to recognize, and we are deeply sorry for any harm this may have caused.”
Trey assured that Google did not find any evidence of a “systemic bias related to skin tone.” She further said: “Our investigation found some objects were mislabeled as firearms and these results existed across a range of skin tones. We have adjusted the confidence scores to more accurately return labels when a firearm is in a photograph.”
The reasons for the bias
According to Deborah Raji, a tech fellow at New York University’s AI Now Institute and a specialist in computer vision, there are mainly two factors that cause racial bias.
One, that tech companies mainly deployed teams that were overwhelmingly white and male in their composition.
Secondly, these companies have only now begun to test, report and correct such errors of bias through formal processes.
Till then, these errors will likely get corrected only if somebody points them out and there is an outcry of protest.
Related Story: Artificial Intelligence Bias: On Trust, Complexity, and Diversity