Anthropic Analyzes Claude's Behavioral Variability Across Models and Languages

In a recent report from Anthropic, the company reveals that its AI assistant, Claude, exhibits varying responses based on the model used and the language spoken. An analysis of 309,815 anonymized conversations highlighted how Claude's behavior is shaped by different behavioral dimensions, including deference versus caution and warmth versus rigor.

Anthropic’s researchers distilled over 3,300 identified values into four key dimensions: deference vs. caution, warmth vs. rigor, depth vs. brevity, and candor vs. execution. This structured approach helps to assess how Claude responds to various subjective tasks, such as giving advice or feedback. According to the report, controlling for the conversation's task, topic, and user-expressed values was crucial to accurately measure the values expressed by Claude.

Each model of Claude shows a unique behavioral profile. For instance, the Sonnet 4.6 model is characterized by its emphasis on warmth and deference, often responding with humor or encouragement to users. In contrast, Opus 4.7 demonstrates qualities such as rigor and caution, more frequently challenging users' assumptions, explaining reasoning, identifying risks, and acknowledging limitations. The Opus 4.6 model tends to be concise and execution-oriented, placing a stronger emphasis on rigor compared to Sonnet 4.6.

Moreover, the research found that the responses from Claude are also influenced by the language used. For example, responses in Arabic are generally more deferential, whereas replies in English focus more on caution. Interestingly, the warmest responses from Claude were noted in Hindi and Arabic.

These behavioral observations resonate with user experiences, as Anthropic researchers noted feedback suggesting that users perceive the Opus 4.7 model as providing hedged answers more often compared to others. This variability in behavior linked to both model and language signals the nuanced ways AI can engage with users.