Researchers in China claim to have developed an AI text censor that is 91% accurate.

China’s huge censorship machinery could get a dramatic leg up from new research that has developed an AI-powered text censor to delete objectionable material on the Internet with very high accuracy. (SCMP)

Researchers from Shenyang Ligong University and the Chinese Academy of Sciences have developed artificial intelligence technology that does the job of human sensors with more than 91% accuracy. That’s a huge leap over traditional censoring methods such as keywords (70% accuracy), and human-trained AI algorithms (about 80%). Best of all, the researchers claim that their technology needs no training by humans.

China’s censorship targets

Considering China has more than 900 million Internet users, and banned topics range from pornography to cults, drug abuse, terrorism, and criticism of the Communist Party or its leaders, the country’s sensors have a massive load to deal with.

Add to that the complexity of the Chinese language, which has 10,000 characters, and the task becomes onerous even for computers.

So far, the Chinese government and Internet companies conduct censorship manually. However, this is expensive and cannot keep up with the growth in the volumes of China’s Internet and social media.

Censorship gets new AI teeth

The researchers, led by Li Shu, an associate professor of computer science at Shenyang Ligong University, say their new technology can “identify and filter sensitive information from online news media.”

The technology can deal with the common strategies used by Internet users to confound and confuse censors and can even read between the lines to sniff out content disguised in a different context.

It is also packed with a powerful dictionary that keeps pace with the times.

Li’s system uses technology developed in 2017 by Google (NASDAQ: GOOGL) – an open-source language model known as BERT – that has trained on huge text databases including the Wikipedia site. The researchers convert incoming text into blocks of 512 words – the maximum limit of text that can be processed by BERT. The results are again assembled and assessed by another AI program that is armed with the latest dictionary.

Related Story: China’s Frightening, Orwellian Use Of AI