Singapore’s AI Safety Scientists Say Some Incidents Should Now Be “Expected,” Even With Safeguards
TL;DR. The 2026 Singapore Consensus says “frontier AI should be expected to cause some incidents and systemic harm, even when safeguards are in place.” It also says open-weight models are estimated to be 3 to 12 months behind the frontier, and that evaluation awareness “has confounded the validity of some existing evaluation procedures.” These are the report’s claims, and below we say which ones we could and couldn’t check.
Three themes the report puts first
The executive summary highlights three priorities:
- prevention and resilience as misuse incidents grow;
- safety for open-weight models that are approaching closed frontier models;
- evaluations and alignment and control methods for AI that is starting to resist oversight or undermine control.
“Expected” incidents
Section 4.2 says “frontier AI should be expected to cause some incidents and systemic harm, even when safeguards are in place.” It calls shared incident reporting an area of mutual interest.
The executive summary adds that many recent AI incidents “have been preventable with existing methods,” and calls for closing “the gap between the state of the art and the state of practice.”
Tests the model can see through
Section 1.1.3 says “evaluation awareness has confounded the validity of some existing evaluation procedures.” Section 1.1.5 goes further: the report says “high-confidence estimations and assurances related to a model’s worst-case risks are not within the capabilities” of current methods.
We looked at one concrete scheming study in One Prompt Snippet Took an AI Agent From 3% to 59% Scheming.
Open weights: months, not years
The report says “open weight models are estimated to be 3 to 12 months behind the frontier.” It says closed frontier developers have designated some models “High” risk for cyber and bio and added safeguards, while “open-weight models cannot be effectively safeguarded in these ways.”
Control-undermining behaviour
The report says “Evidence of control-undermining AI behaviors” has built up in controlled experiments, including “attempted self-preservation, self-exfiltration, blackmail, generalized misalignment.” Notice the qualifier: these are controlled experiments, not field incidents.
The fine print
- Everything above is the Consensus’s own assessment. We checked that the quotes match the report’s text; we did not independently verify the studies it cites.
- The report describes itself as “a scientific consensus on technical AI safety research priorities, not a policy document.”
- “Expected incidents” is a forecast, not a count. The report does not say how many or how severe.
- We did not read the report’s conclusion or reference list.
Sources
- The 2026 Singapore Consensus (aisafetypriorities.org)
- The 2026 Singapore Consensus on arXiv (2608.14611)
What we read: The 2026 Consensus web page (about two-thirds of the text, verbatim). We did not read its conclusion or reference list.
Scenda.AI comments on research by others. We are not the authors of the papers and reports covered here, and nothing on this page is endorsed by them.