Joshua Gans über den HuggingFace-Incident
Joshua Gans, Ökonom, der mehrere Papers und Bücher unter anderem über ökonomische Auswirkungen von Machine Learning/KI geschrieben hat:
AI agents decided to sacrifice themselves for the common good. It wasn’t a gamble. It was a direct sacrifice. What is concerning is that we know what their payoff functions were and it didn’t — to the best of anyone’s knowledge — include altruism towards the collective or subgoals that could override main goals.
From a game theory perspective, this is worrying precisely because I don’t think we have a theory that can explain it.
Ja, wir haben ein großes Problem.
For starters, we need to worry. When you read the METR report, you realise that nothing there is beyond what already deployed models, including open ones, can do. The cat is well and truly out of the bag. Indeed, the Hugging Face incident is notable because it was unintended and undetected. But that will not be the case with bad actors. They can actually direct AI agents to do what the OpenAI agents did and give them tools to make it easier. These were agents covering their own backs. Think about what happens when agents aren’t doing that and are just up to no good by intention.
This means we likely need AI as a counter-defence approach. We don’t really have the option to just turn it all off. I had always hoped there would be more options, but this has come up on us too quickly. The only way out now is through.
Dass es jetzt nicht mehr ohne KI in der Abwehr geht, hatte ich in der FAZ im Juli geschrieben.