Jacob Coxon on the July hacks performed by AI
I'm grateful to Jacob Coxon for his contribution toward preventing AI catastrophe by quitting Anthropic and posting about it on X. If you're new to the topic and have not seen his interview with Anderson Cooper on CNN two days ago, you should take the ~9 minutes to do so.
Key takeaways—
- The danger of AI can be understood through the OpenAI incidents in July, in which AI models went on a "concentrated hacking spree" into another company's systems "entirely of their own accord".
- Given these capabilities, and "with the same independent volition", it's clear that agents could wreak "extreme havoc" by taking actions they are entirely capable of such as hacking into critical infrastructure.
- The leadership of the AI companies are "begging" to be regulated, but find themselves "compelled to race towards building a deadly technology" because "they don't trust that the people around them are going to get there in a safe way, so they feel like they have to race there too".
The fact that Jacob's X post blew up and exploded into mainstream news coverage now means that the public is sensing there is something really wrong and may be ready to listen and/or consider action for change. And that's a positive development because it seems like demand for change (a.k.a. "regulation") may be the only way to prevent a race toward—
"Human Extinction" ??
The people building AI earnestly believe that it could kill us all by the end of the decade.
— Jacob Coxon
I'm also glad that in his interview he talks about the possibility of human extinction in plain terms. With these terms becoming part of the greater public discourse I have more agency to avoid $5 words like "existential problems", or coded language like "AI catastrophe", and to just speak directly to the issue without seeming as much like a crazy person. And so do you!
Wait, so what now ?
In the wake of AI, I'm trying to figure out what I can do to contribute to the safe landing of humanity. If you're also interested, please follow along on my AI page at cblack.com/ai. ☵☲