Turn customer friction into replayable test cases.

Expand the cases your test plan can see

Conventional AI evaluations often begin with prompts written by the product team. AnthroSim adds a defined synthetic audience and a moderated discussion, so teams can observe how people with different communication styles interpret the same policy or response.

The current product does not connect directly to a live bot. Add product documentation, support policies, and representative responses as project materials. The room’s questions and misunderstandings can then be converted into cases for your existing evaluation harness.

Example: support escalation review

A product team is revising an account-recovery assistant. It adds the recovery policy, a set of sample assistant replies, and the conditions under which a human agent must take over. A 20-person synthetic room reviews the flow from a customer’s perspective.

The saved transcript reveals which identity-check instructions were read in conflicting ways and where a sample reply cited a policy passage that did not answer the customer’s question. The team writes regression cases from those moments and runs them against the production candidate.

Inspectable output

The run record contains each generated profile, the moderated transcript, participation events, source-use excerpts with provenance, final ballots, and operator-visible structured decision traces. The whole run can be downloaded as JSON.

Use that record to broaden a test set and organize expert review. Product quality, safety, and launch decisions still depend on evaluations of the real system with real users and domain reviewers.

See the recorded fields →