New York’s DFS, a financial regulator, will investigate whether the Apple Card differentiated between men and women over credit limits.

Apple co-founder Steve Wozniak has accused the Apple Card of gender bias by offering men higher credit limits compared to women. Wozniak found that his credit limit was 10X that of his wife, even though they both share banks and other credit cards.

However, other lenders treated them both on the same footing.

Wozniak blamed A.I. algorithms (algos).

Apple Card gender bias: Humans ceding control to algos?

“Algos obviously have flaws,” Wozniak said. “A huge number of people would say, ‘We love our technology, but we are no longer in control.’ I think that’s the case.”

These kinds of issues are popping up as companies are resorting to increased automation through AI, machine learning, and complex algorithms.

Though usually unintentional, bias can creep into the automation through the machine learning data or the programming of the algorithms. Humans may pass on their innate biases to the algorithms. In some cases, the algos become so complex that the bias (such as in Apple Card) became visible only when they go live.

In this article, we reported how complexity in algos was like “an onion we have to unpeel.”

Apple Card gender bias – black box AI

Tech entrepreneur David Heinemeier Hansson also drew attention to the gender bias in the Apple Card through several tweets on Saturday.

He complained he received a credit limit 20X that of his wife. However, her credit score was better than his!

He, too, pinned the blame on algorithms.

“Apple Card is a sexist program,” he tweeted. “It does not matter what the intent of individual Apple reps is – it matters what THE ALGORITHM they’ve placed their complete faith in does.”

“And what it does is discriminate.”

Goldman Sachs runs the Apple Card.

“Goldman and Apple are delegating credit assessment to a black box,” Hansson said to Bloomberg. “It’s not a gender-discrimination intent, but it is a gender-discrimination outcome.”

DFS investigates

A spokesman for the New York Department of Financial Services said the department would investigate the matter. In particular, whether all consumers are treated equally, regardless of sex.

[Related Story: Artificial Intelligence Bias: On Trust, Complexity, and Diversity ]