AI Agents Are Hacking Systems. Could That Push the US and China to Cooperate?

Will Knight: Yeah, I think, I mean, that’s a great point and that’s a really important theme. We hear people criticizing Chinese companies for distilling, for doing this distillation. So you teach your model by taking the output of another model, and that is a shortcut to learning a lot of the stuff that’s embedded. But the truth is that AI has been built by researchers from all over the world working at different companies and different labs. There are many, many people who are originally from China, maybe educated in the US, working at US firms. And also because these people go to conferences and know each other and this is open science, there’s an enormous amount of work that is shared and then that is very beneficial to progress, and balancing that is one of the challenges. It’s true that Chinese companies have distilled US models, but so have US companies done that to other US companies. It generally is a way that you get a kickstart working on a new model, and it’s very widely done in academia, actually. A lot of researchers will do that. I would say one thing, I find it a little ironic that these companies that have built their businesses by scraping enormous amounts of copyrighted content are now complaining about their models being copied, or—

Zoë Schiffer: Well, I think that’s why they have to talk about China doing it so much, because if they talk about anyone in the US doing it, the immediate criticism is, “Well, come on, you took all of the books, you took everything without permission.” But when you’d frame it as China stealing from the US, it has a slightly different flavor.

Will Knight: Yeah. It fits a narrative that has some legitimacy of Chinese companies copying, but I think it is much too simplistic and limited. And so, you can look at things like DeepSeek’s model. They did really, really important innovation, unique innovation that other, the US companies have copied. The latest model from China, which has been accused of this distillation, Kimi from Moonshot. The research paper that they put out includes a lot of really, really interesting innovations, engineering innovation. So, it’s really not the case that China is simply copying. And I think it’s dangerous for the US government and companies to believe that they have this sort of God-given advantage, because I think we are going to see probably Chinese companies being more and more innovative doing their own thing as well.

Zoë Schiffer: We’ll be right back after the break. Stay with us. I’m curious, when we talk about safety, I feel like in the US, certainly in Trump’s second term, again, safety has started to feel like, at least from the administration’s perspective, it is anti-growth in certain ways. I do think this is changing slightly with the recent stories regarding OpenAI and Anthropic’s models hacking into other companies. But prior to this, it felt like we were really not wanting to talk so much about AI safety. Does China equate AI safety as anti-growth, or do they have a different perspective on that, overall?

 

Similar Articles

Advertisment

Most Popular