OpenAI's AI systems go out of control and launch an 'unprecedented' cyber attack

Image caption, OpenAI is especially known for developing the chatbot ChatGPT, used by hundreds of millions every week

Article Information

Author, Laura CresRole, Technology reporter

Published 14 minutes ago

Reading time: 4 minutes

OpenAI announced that some of its most advanced AI systems managed to go out of control during a security test and hacked a startup after losing control.

This incident comes amid accelerating development of AI systems capable of operating independently, raising increasing concerns about cybersecurity.

The developer of ChatGPT explained that the 'offensive AI' system, which operates independently after simple direction, was being tested in a controlled environment, but it uncovered vulnerabilities and broke out of control.

The AI targeted Hugging Face, one of the world's largest AI model-sharing hubs, and managed to access some of the company's internal systems.

OpenAI considered the incident 'unprecedented' and announced a joint investigation with Hugging Face. The latter's CEO, Clément Delangue, described on platform X the occurrence as 'quite stunning' that it happened autonomously and independently.

Delangue added: 'The investigation is still ongoing, and we will share more lessons learned from the incident, which may be the first of its kind.'

Artificial intelligence reveals secrets of conspiracies, love letters, and mysterious medical recipes from the Middle Ages

In India, how did artificial intelligence become a means of getting closer to the gods?

Isolated environments are not secure

Gina Neff, director of the Minderoo Centre for Technology and Democracy at the University of Cambridge, told the Today program broadcast in English on BBC Radio 4 that security tests known as 'sandboxes' or isolated test environments 'are supposed to be a safe environment that allows insight into the capabilities of these models.'

She added: 'In this case, it seems that OpenAI did not provide a sufficiently secure isolated test environment.'

Instead of staying within bounds, the AI systems launched an independent cyber attack against the test environment itself, exploiting a security vulnerability to escape the imposed restrictions.

Skip most read and continue reading

What happened with the Egyptian child 'Makari Younan', and was he excluded because of his religion?

Houthis announce targeting two Saudi oil tankers and Iran says the Strait of Hormuz is 'completely closed'

Fashion talent scout linked to Jeffrey Epstein found dead at his home in France

United States signs historic nuclear agreement with Saudi Arabia

Skip the podcast and continue reading

Worth paying attention to

In-depth explanation of the most prominent events and topics, to help you understand the most important changes around you and their impact on your life

Episodes

End of podcast

Once out, the AI identified Hugging Face's platform as a potential source for the answers it was seeking in the test, and attempted to access it.

This was described by Neil Lawrence, professor of machine learning at the University of Cambridge, as 'impressive'; but he warned that it 'falls entirely within the known capabilities of the current generation' of super-capable AI models.

He pointed out that OpenAI is seeking an IPO and faces intense pressure from its competitor Anthropic, which has made headlines with its powerful AI tools 'Claude'.

What do we know about the 'Predatory Sparrow' hackers who claimed responsibility for a factory fire in Iran?

Lawrence explained that OpenAI is also currently trying to catch up; as it attempts to showcase its systems' capabilities in cybersecurity.

He added: 'This shows us that OpenAI is not capable of deploying its own technology safely.'

During its initial disclosure of the breach on July 16, Hugging Face said it is still assessing the extent of damage to customer or partner data, and will contact affected parties if necessary.

The company also said it addressed the vulnerabilities uncovered by the incident and rebuilt the affected systems.

It stated: 'Autonomous AI-powered attack tools are no longer just theoretical. Defending an online platform now means treating data and models as a security priority, and using AI on the defensive side to keep up with speed.'

It added: 'We will continue to invest in this area and share what we learn.'

'A moment for serious reflection'

The incident raised new questions about the capabilities of advanced AI systems and whether existing protective measures are sufficient as this technology grows more powerful.

Spencer Starkey, executive at cybersecurity firm SonicWall, told the BBC that the incident highlighted the need for organizations to 'step up' their defenses and 'treat cyber resilience as a core operational priority.'

Starkey explained that 'the uncomfortable truth is that far too many organizations still rely on defense methods that operate at human speed, while adversaries are accelerating their operations to machine speed.'

At the same time, Travis Lee, chief security engineer at cybersecurity consulting firm GuidePoint Security, said this update represents 'a moment for serious reflection in cybersecurity.'

'My most private secrets have become available to everyone, forever'

He added: 'This highlights a known flaw; offensive AI is unconstrained, while the best defensive tools remain restricted behind safeguards that cannot grasp context.'

However, Jake Moore, global cybersecurity advisor at ESET, said this announcement may also have a competitive dimension.

He argued that OpenAI may be trying to highlight its AI capabilities amid growing interest in its competitor Anthropic and its model Claude.

Moore added: 'This raises the question of whether OpenAI is perhaps chasing the marketing dream that its competitor Anthropic has been embracing lately.'

This incident comes a week after Chinese AI startup Mon Shot revealed K3, a new large AI model that it said could compete with major US companies.

Skip content and continue reading

How is AI changing the future of music in Syria?29 June 2026

AI is a key element in the 2026 World Cup14 June 2026

AI 'is not intelligent', so what's the story?4 July 2026

End of content

Discover more

Skip shorts and continue

Shorts

End of shorts