“The high-context nature of Chinese language used to create hurdles in natural language processing,” says Thomas Qitong Cao, a PhD candidate at Stanford University who studies political behaviour and the internet. “But the gap between languages has significantly closed in the era of pretrained large language models.”
Cao says the challenges of training Chinese-language AI models test the size and quality of data sets, as well as computing power.
Companies will also have to contend with the government’s censorship of subjects it considers sensitive. Social media platforms in China already employ a combination of algorithms and human moderators to monitor content and remove anything that breaches the government’s constantly moving rules
→ Continue reading at WIRED
