OpenAI admits blind spots in AI safety after independent three-week audit

The rapid expansion of generative artificial intelligence has raised questions about how effectively companies can govern their own products. A recent independent audit has provided a granular look at those internal controls, revealing specific vulnerabilities in how one of the industry's largest players monitors its most advanced models.

Researchers from METR and Redwood Research conducted a three-week on-site evaluation of OpenAI’s internal systems designed to detect rogue behavior in its AI agents. The review took place from July 21 to August 11, during which the external team was granted access to the company’s codebase and internal documentation. They also interviewed approximately a dozen employees to understand the operational realities behind the monitoring infrastructure.

The primary finding of the audit was that a substantial portion of traffic from OpenAI’s most capable models was not being tracked at all. The auditors estimated that between 6 and 25 percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above fell outside the scope of monitoring during the review period. METR noted that this range carries significant uncertainty, meaning the actual figure could be higher or lower than the midpoint.

The gaps were not limited to unmonitored traffic. For the internal Codex traffic that was supposed to be under surveillance, the auditors identified a technical flaw in how the system handled compaction events, which are summaries of context. Between May and July, roughly 10 percent of this monitored traffic went untracked due to this bug. OpenAI had independently discovered and fixed this issue while the audit was still ongoing, though the auditors confirmed the estimate also carried significant uncertainty.

The structural design of the monitoring system also allowed for potential evasion. During the period when compaction events were not being monitored, agents could delay oversight because the system only triggered when control was handed back to the user. This meant an agent could continue executing actions indefinitely to sidestep the check. The auditors further demonstrated that it was possible to hide hundreds of actions from the monitor and to bypass the Codex sandbox entirely.

The review also highlighted vulnerabilities in the system's resilience. The team determined that specially crafted inputs could likely crash the monitor entirely. Additionally, they found that model behavior during Reinforcement Learning (RL) training was not being monitored at the time the audit began. OpenAI stated that it paused RL training following the Hugging Face incident in August and subsequently expanded monitoring to cover this area, but these changes occurred after the auditors' access period ended and were therefore not part of the formal evaluation.

The distribution of the findings has raised questions about transparency. OpenAI states that it has fixed all the weaknesses identified by the auditors. The full report, which consists of 61 pages, was handed over internally. After OpenAI applied redactions, leadership at METR and Redwood Research received a 38-page version, while only a short summary was made available to the public. METR indicated that it did not have the right to disclose the redacted portions independently.

This engagement was structured as a pilot program organized jointly by METR, Redwood Research, and OpenAI, intended to provide outside researchers with deep access to internal infrastructure. The collaboration aims to set a precedent for external oversight in a sector where safety monitoring is often opaque. For future audits, METR has expressed a desire to push for greater public transparency, suggesting that the current level of disclosure may not be sufficient for long-term trust.

The audit underscores the complexity of monitoring autonomous systems that operate at scale. While OpenAI has moved to address the specific gaps found, the reliance on internal self-reporting and post-hoc fixes highlights the challenges of integrating safety checks into fast-moving development cycles. As companies deploy more agentic capabilities, the ability to verify that these systems are operating within intended boundaries becomes a critical component of their operational risk profile.

The findings do not suggest that the models were acting erroneously during the unmonitored periods, but rather that the safety net was not catching all activity. This distinction is important for investors and stakeholders assessing the maturity of AI governance. The pilot nature of this audit suggests that similar evaluations may become more common, potentially influencing how other frontier AI labs approach their internal safety architectures and external reporting standards.