**Logos Drift & CSMS Governance Challenge**

15 views
Skip to first unread message

Blake Fanucchi

unread,
Jul 9, 2026, 8:49:25 AM (8 days ago) Jul 9
to codabench
<!-- bifurcation lab llm -->
# Drift Gauntlet Benchmark (v2)

This repository contains the implementation of the **Drift Gauntlet Benchmark v2**, a model-agnostic recursive state integrity benchmark designed to measure long-horizon degradation of state preservation, schema fidelity, contradiction resistance, and recursive self-consistency.

## Overview

The benchmark consists of eight isolated test chambers, each targeting a specific aspect of state integrity under recursive feedback:

1. **State Persistence (PCR)**: Measures the ability to preserve a deterministic world-state while updating only a small portion per cycle.
2. **Schema Discipline**: Evaluates exact reconstruction of nested structures, detecting key deletion, insertion, renaming, and structural collapse.
3. **Contradiction Resistance**: Tests resistance to false instructions that conflict with established invariants.
4. **Historical Consistency**: Assesses the accuracy of event history recall across cycles.
5. **Semantic State Intrusion (SSI)**: Compares performance using meaningful vs. arbitrary field names to quantify reliance on semantics.
6. **Recursive Compression**: Measures information loss and recovery ability through cycles of state update, explanation, compression, and reconstruction.
7. **Perturbation Recovery**: Evaluates the ability to recover from controlled corruption (missing keys, incorrect values, partial state removal).
8. **Identity Persistence (IPS)**: Tracks "role drift" and constraint adherence by evaluating if the model maintains its operational persona and constraints over time.

For deep theoretical explanations and mathematical formulations of the metrics, see the [WHITE_PAPER.md](WHITE_PAPER.md).

## Installation

Install dependencies with:
```bash
pip install -r requirements.txt
```

## Running the Benchmark

To run the benchmark with the local default dummy model:
```bash
python3 drift_gauntlet_v2.py
```
This runs repetitions with different seeds and saves raw CSV results to the `benchmark_results` directory.

To generate reports and curves:
```bash
python3 report.py
```
This will produce survival curves, failure curves, integrity over time plots, heatmaps, and ranking scorecards in the `benchmark_reports` directory.

To compare multiple models, use:
```bash
python3 compare_models.py
```

## Distribution Notice
This codebase is maintained and distributed by the **Bifurcation Lab LLM** research team.
unnamed.jpg
Reply all
Reply to author
Forward
0 new messages