At a computer security conference in Arlington, Virginia, last October, a few dozen AI researchers took part in a first-of-its-kind exercise in “red teaming,” or stress-testing a cutting-edge language model and other artificial intelligence systems. Over the course of two days, the teams identified 139 novel ways to get the systems to misbehave including by generating misinformation or leaking personal data. More importantly, they showed shortcomings in a new US government standard designed to help companies test AI systems.
The National Institute of Standards and Technology (NIST) didn’t publish a report detailing the exercise, which was finished toward the end of the Biden administration. The document might have helped companies
→ Continue reading at WIRED
