Case study 03 · Governed agents
WAGF
A Water Agent Governance Framework that checks proposed LLM-agent decisions before they update a human–water simulation.
- structured decision contract
- physical + financial + behavioral checks
- deterministic audit trace
Why this problem matters
A syntactically valid action can still violate physical limits, household finances, behavioral theory, or the current simulation state.
How I approached it
I designed the context builder, structured decision schema, parser, validators, targeted repair loop, audit record, and simulation-state boundary.
What it contributes
Governance is clearest when every decision exposes the proposal, the rule that failed, the allowed repair, and the exact state transition.
The governance pattern transfers to any stateful agent system where an action must satisfy explicit constraints and leave an audit trail before execution.
System artifact
Inspect the research logic
The public governance trace is a frozen demonstration of the control flow. It does not call a live model and is not a released product benchmark.
Evidence enters. Decisions stay inspectable.
Rejected · state unchanged
- Protect contents
- protect-contents
- Requested resources
- 9
- Available resources
- 6
- Contents protection
- Not applied
- Format: Pass
- Permission: Pass
- Budget: Fail
9 requested, 6 available. Only the cost field needs repair.
Resource tokens are illustrative, not currency.
Interactive controls require JavaScript. The complete flow and sources remain below.
Evaluation
- Human evidence
Measured constructs and reported choices establish the empirical reference.
Published evidence - Context
A label-blind synthetic profile carries only approved attributes into the prompt.
Illustrative example - LLM proposal
Repeated model runs produce a choice and a concise rationale for comparison.
Illustrative example - Validation / repair
Checks compare direction, subgroup behavior, and run-to-run stability without exposing respondent records.
Public artifact - Consequence
The result becomes an evaluation finding, not a claim that the model represents a person.
Illustrative example
Governance
- Human evidence
Behavioral and physical constraints define what a plausible action must respect.
Published evidence - Context
A synthetic agent state limits the actions the model may propose.
Illustrative example - LLM proposal
The LLM returns a structured proposal rather than mutating the simulation directly.
Illustrative example - Validation / repair
Validators reject invalid fields, request targeted repair, and retain an audit trace.
Public artifact - Consequence
Only an accepted proposal updates the coupled model state.
Illustrative example
Simulation
- Human evidence
Survey evidence and public flood-risk sources ground household behavior.
Published evidence - Context
Household tenure, risk perception, resources, and exposure form the decision context.
Published evidence - Agent decision
Agents choose adaptation, insurance, or no action under modeled constraints.
Public artifact - Validation / repair
Model checks keep behavioral, financial, and physical state transitions consistent.
Public artifact - Consequence
Flood losses and adaptation alter the household and environmental state for the next cycle.
Published evidence
Research context and method
Frozen demonstration · no live model call
- Proposed action
- Parser
- Physical check
- Financial check
- Behavioral check
- Audit record
- State update
Rejected: cost exceeds available household budget
Method in detail
- Build context from approved persona and simulation state.
- Require a structured action and ratings.
- Parse before any state mutation.
- Run physical, financial, and behavioral-theory checks.
- Accept, reject, or request targeted repair.
- Write the trace, then update state only for an accepted action.
Validation and limitations
Validation
- Schema validation
- Fail-closed parser boundary
- Named rule results
- Deterministic demonstration trace
- State update occurs after acceptance
Limitations
- Declared rules cannot cover every failure mode.
- Repair can satisfy a rule while still producing a weak decision.
- Demonstration records are illustrative, not research outcomes.
Evidence slice
From question to consequence
- 01Problem
- An unconstrained LLM proposal can create impossible financial or physical state changes.
- 02My role
- I designed the proposal schema, validator sequence, targeted repair path, and audit record.
- 03Input
- A synthetic household state, action catalog, and explicit behavioral, financial, and physical constraints.
- 04Decision
- The model proposes an action with typed parameters but cannot write to state.
- 05Validation
- A failed affordability check blocks the update; repair changes only the invalid field before all validators rerun.
- 06Consequence
- The accepted proposal is committed with a trace linking input, failure, repair, and final state update.
Contact
Build evaluations that connect model behavior to real evidence
Interested in evaluating AI decisions or building evidence-linked research systems? I welcome research and open-source collaboration, and Summer 2027 internship conversations.
More about my research path