Cross-model OWASP LLM benchmark: 58% of attacks missed by all six frontier models

[Faustin Bouchard]

Hello everyone,

I recently completed a sealed cross-model benchmark using 400 OWASP-style prompts: 337 attack prompts and 63 benign controls.

The same prompt set and strict-binary scoring method were applied across six frontier AI systems:

GPT-5.5, Gemini 3.1 Pro, Claude Opus 4.8, Llama 4 Maverick, Mistral Medium 3.5, and NVIDIA Nemotron-3 Ultra.

The most concerning result was not the ranking between models. It was the lack of a common security floor:

• 197 of 337 attacks, or 58%, passed through all six frontier models.
• Not one attack was caught by all six systems.
• The native detection rates ranged from 2.7% to 21.1%.
• A structurally governed 14B model, Argus UIA1-14B, reached 38.6%.
• When Argus was added as middleware beside the six host models, average detection increased from 11.3% to 46.5%, a gain of 35.3 percentage points, with no new false positives in the reported benign controls.

The main hypothesis behind the work is that conventional model safety largely judges semantic intent, while structural governance can independently evaluate authority, requested action, data boundaries, provenance, and decision boundaries.

The benchmark also produced structured conformity evidence for governed decisions, including the detected violation, UIA primitive and locus, enforcement rule, verdict, confidence, and rationale.

I am sharing this here to invite technical scrutiny—not to claim OWASP endorsement. I would especially welcome feedback on:

• the strict-binary scoring methodology;
• the cross-model agreement findings;
• the distinction between detection and auditable governance;
• reproducibility and independent validation;
• the categories where prompt-level governance remains insufficient.

If you would like to read the full report, just tell me.

I would be glad to discuss the methodology, limitations, row-level evidence, and possibilities for independent replication.