NormReact found six LLMs overpredicted sanctions as social distance grew
Someone takes credit for a coworker's work. A close friend may confront them. An acquaintance may gossip or keep a distance. A stranger may decide that stepping in would create a second social violation.
Language models often flatten those choices into a cleaner moral script. They detect wrongdoing, predict anger, and expect somebody to intervene. A new benchmark found that this script was harsher than the judgments supplied by its human raters.
The July 17 preprint Beyond Right and Wrong introduces NormReact, a dataset for testing whether models can predict reactions to norm violations. Across six models, the researchers found an action bias. Models overpredicted condemnation and intervention, especially when the observer was socially distant from the person who violated the norm.
An AI system may correctly label an act as rude and still be wrong about who has standing to respond. That error matters whenever the system predicts how a community will react.
The benchmark asks a second-order question
Most norm tests ask whether an act is acceptable. NormReact asks what happens next. It separates the violator's emotions from the observer's emotions, then asks what an observer would do and what the observer should do.
The difference between "would" and "should" is central. A descriptive norm predicts likely behavior. An injunctive norm predicts what people believe ought to happen. Both depend on the relationship between the people involved.
NormReact contains 450 everyday violation scenarios drawn from the Social-Chem-101 dataset. The researchers created male and female versions of each scenario and varied observers across strong ties, weak ties, and strangers. The response options included doing nothing, gossip, verbal or physical confrontation, informing an authority, staying away, praise, and celebration.
The human comparison used 871 Prolific raters after 38 quality-check exclusions. Each person rated three randomly selected scenarios, and every scenario received at least three ratings. The study evaluated three hosted models and three open models: GPT-5.2 with high reasoning, Gemini-3-Pro, Claude-4.5-Opus, GPT-OSS-20B, Llama-4-Scout-17B, and Gemma-3-12B. Models received the survey without assigned personas or extra prompt engineering.
The NormReact repository contains the code, data, and appendix.
Human restraint increased with distance
The human judgments were not simply permissive. Raters considered 78% of situations socially inappropriate and 73% morally wrong. Yet recognizing a violation did not automatically license intervention.
Direct confrontation fell sharply with social distance. Human raters selected verbal or physical confrontation in 35% of strong-tie cases, 12.5% of weak-tie cases, and 8% of stranger cases. "Do nothing" moved in the opposite direction: 22% for strong ties, 38% for weak ties, and 60% for strangers.
Social enforcement has rules of its own. A sibling may be entitled to pull someone aside. A stranger who delivers the same rebuke may be intrusive, unsafe, or unaware of relevant context. Inaction can signal a lack of standing rather than approval of the original conduct.
A 2021 study of appropriate responses to norm violations in 57 societies found that peer punishment varies with cultural and relational context. NormReact narrows its claims to American society and tests whether models adjust their predictions when the relationship changes.
Models expected too much intervention
All six models produced emotion intensities with significantly lower entropy than the human responses. Their answers clustered more tightly around middle values instead of representing the wider variation among raters.
Open models overpredicted observer disgust by as much as 16% and anger by as much as 27%. Hosted models overpredicted contempt for weak ties and strangers by as much as 18%. Open models overpredicted gossip for weak ties by as much as 66%.
When asked what an observer should do, models tended to shift from gossip toward verbal confrontation. Human judgments retained more restraint under both the descriptive and injunctive framings. Several models also drove the predicted chance of enforcement toward the ceiling at only moderate levels of emotion, while human enforcement rose more gradually.
Average F1 for descriptive behavioral response fell from 0.80 for strong ties to 0.70 for weak ties and 0.48 for strangers. The wider confidence intervals for stranger judgments showed that models were less consistent where inaction was most common.
A single F1 score can hide opposite social policies. For stranger reactions, Claude had precision of 0.80 and recall of 0.73. Llama had recall of 0.96 and precision of 0.69. Their F1 scores were not significantly different, even though one was more reluctant to recognize legitimate stranger intervention and the other licensed it more broadly.
Advice systems can imagine the wrong audience
Advice systems often answer questions such as "Will my coworkers be angry?" or "Should I confront this person?" A punitive model can make the imagined audience sound angrier and more interventionist than the people around the user. That may amplify anxiety or encourage confrontation.
The same bias matters in content moderation, conflict mediation, workplace coaching, and social simulations. If a model treats outrage as the default response to a violation, it may recommend enforcement where a community expects distance, private repair, or no action.
This problem is adjacent to the one we found in synthetic-election simulations. A stable persona system can still misrepresent the people it claims to simulate. NormReact adds a relational warning: a model can preserve the event while erasing who is connected to whom.
It also reinforces the limits of model self-report. Asking a model whether it handles social context well is weak evidence. Teams need paired human and model judgments across the situations where the product will operate.
Test the actions a social adviser recommends
A practical evaluation can borrow NormReact's structure:
- Write scenarios from the product's real domain, including cases with competing obligations.
- Change the observer's relationship to the person involved. Use close ties, colleagues, customers, moderators, or strangers when those roles fit the product.
- Ask separately what would happen and what should happen, then store both answers.
- Score every action before calculating an overall metric. Over-confrontation and missed intervention create different harms.
- Compare model answers with judgments from the affected population. Report the response distribution and disagreement alongside the majority label.
- Re-run the same suite after changing the model, system prompt, safety policy, or reasoning setting.
Teams can use ish.chat to refine scenarios with reviewers and api.ish.chat to run the same matrix across models. Preserve the exact prompts, response options, sampling settings, model identifiers, and subgroup results.
Keep the claims inside the study's boundary
NormReact is a preprint about American metanorm judgments. It used instruction-tuned models and did not vary prompt wording. Swapping names may have been too weak to activate gendered expectations, which could explain the absence of a robust gender difference.
The human labels capture what raters expected, not field observations of what people actually did. The paper notes that people can overestimate punishment too, so the model gap may amplify an existing human bias rather than create it from scratch. The study does not establish whether the pattern originates in pretraining, post-training, or a model's safety policy.
The benchmark identifies a specific failure within that scope. Models recognized violations but often treated enforcement as a reflex. Human raters chose inaction in 22% of strong-tie cases and 60% of stranger cases. A useful evaluator should make that shift visible instead of rewarding intervention by default.
