Human respondents provide evidence. Simulated household agents and AI research agents are models and software, not people. These projects share a research approach, not one integrated published system.
01
Human-Grounded LLM Evaluation
Can AI-generated flood-insurance answers reflect how real households make decisions?How I approached it I compared real household surveys with profile-based LLM responses using measurement checks, decision-pathway analysis, and repeated runs.What it contributes A way to detect differences in household variation and decision relationships that average answers alone can hide.evaluation
Methods
Evaluation design · Psychometrics and SEM · Stability analysis
Boundaries
The interactive examples use synthetic records, not individual survey responses or research results.
How do household choices change flood losses and the resources available next year?How I approached it I built the household decision model and connected adaptation choices to catastrophe, insurance, and financial mechanics.What it contributes A traceable link from household adaptation choices to losses, insurance outcomes, and resources in later years.simulation
Methods
Agent-based modeling · Coupled simulation · Risk systems
Boundaries
Simulated household agents model adaptation; they are not an integrated LLM system.
What stops an invalid AI proposal from changing a simulation?How I approached it I designed the validation and repair path that checks an agent proposal before the simulation state can change.What it contributes An inspectable proposal-to-action boundary: invalid actions are blocked, repaired, and checked before execution.governance
Two applications, one method: compare with human evidence, check the proposal, and inspect what changes. Select an object or a step to follow its source.
Decision Observatory
Evidence enters. Decisions stay inspectable.
Synthetic example
Interactive controls require JavaScript. The complete flow and sources remain below.
Evaluation
Human evidence
Measured constructs and reported choices establish the empirical reference.
Published evidence
Context
A label-blind synthetic profile carries only approved attributes into the prompt.
Illustrative example
LLM proposal
Repeated model runs produce a choice and a concise rationale for comparison.
Illustrative example
Validation / repair
Checks compare direction, subgroup behavior, and run-to-run stability without exposing respondent records.
Public artifact
Consequence
The result becomes an evaluation finding, not a claim that the model represents a person.
Research infrastructure with inspectable failure behavior
Selected repositories support research operations, agent orchestration, and multilingual learning. Counts are refreshed through a reviewed build-time data workflow.
awesome-agentic-ai-zh
Maintained trilingual roadmap for agentic AI, with automated cross-language checks.
Updated: 2026-10-31 · Source: GitHub REST API snapshot
Public updates
Ideas shared in public
Selected posts I shared on LinkedIn, with links to the original context.
LinkedIn
Beyond predicting a choice: is the decision process coherent?
My perspective on testing the factors behind a simulated preference, rather than treating a higher predicted probability as sufficient evidence of human-like behavior.
Why I built awesome-agentic-ai-zh: connect scattered resources into a trilingual path through LLM foundations, tools, memory, evaluation, and multi-agent systems.
Connecting research tools with Model Context Protocol
A public account of my research-tool setup: connecting AI coding environments, literature retrieval, project files, and manuscript preparation through MCP.
Build evaluations that connect model behavior to real evidence
Interested in evaluating AI decisions or building evidence-linked research systems? I welcome research and open-source collaboration, and Summer 2027 internship conversations.
