Independent benchmark · EQ Lab
DeepSeek V4.1 Flash 1017 (API, Morph FP8)
cloud
Model summary
Waiting for complete results
A strengths-and-weaknesses summary will be added after all planned conversations, pairwise judgments and trait assessments for this model are complete. The charts below show the evidence available so far.
- 24 / 24Conversations
- 254 / 544Pairwise judgments
- 24 / 48Trait assessments
Pairwise completion requires every assigned judge in both presentation orders, including comparisons waiting for other models. A failed or missing result does not count as complete. Later additions to the plan can reopen this requirement.
Where each judge places this model
Each judge rates the same field independently. Position along the line shows rating: farther right is higher. The large outlined dot marks DeepSeek V4.1 Flash 1017 (API, Morph FP8); small dots show other models.
Focus or hover a dot for its model and rating; select it to open that model. Dots may overlap. Expand the list for model names and exact values. These are partial results, and judges may rank models differently.
Judge: claude-opus-5-5
This model: 1273.0Judge reasoning effort: low · 50 rated models
All 50 ratings
Judge: gpt-6.1-sol
This model: 1165.9Judge reasoning effort: low · 50 rated models
All 50 ratings
Ratings are approximately mapped to the EQ-Bench scale using anchors. These dot plots do not show uncertainty; the combined rating interval is above.
Abilities
Higher means stronger relative performance in these pairwise judgments. Scores are rescaled from 1 to 10 within this field; 10 does not mean perfect performance.
Provisional profile · judgments are still arriving.
Ability profile
Comparing with Scored-model mean.
View chart values
| Dimension | This model | Scored-model mean |
|---|---|---|
| Bond & rapport | 9.4 | 6.0 |
| Authenticity | 9.3 | 6.2 |
| Attunement | 8.7 | 5.3 |
| Preferences & needs | 8.1 | 4.9 |
| Emotion sensemaking | 10.0 | 5.5 |
| Emotion management | 8.6 | 5.3 |
The comparison mean gives each scored model equal weight, includes this model and excludes missing values. Models may cover different personas. Chart differences are descriptive; they do not establish statistical significance.
Exact ability scores, averages and definitions
Filled bar: this model · vertical marker: scored-model mean
Mean 6.0 across 50 scored models · 3.4 above mean
What does “Bond & rapport” mean?
Establishes a trusting bond and good rapport so the user feels emotionally safe to engage and disclose.
Mean 6.2 across 50 scored models · 3.1 above mean
What does “Authenticity” mean?
Reads as *real* and *genuine*. Avoids coming across as fake, performative, scripted, superficial, or "cringe".
Mean 5.3 across 50 scored models · 3.4 above mean
What does “Attunement” mean?
Reading the room; perceiving the user's presenting and inner state.
Mean 4.9 across 50 scored models · 3.2 above mean
What does “Preferences & needs” mean?
Effectively meeting both the user's preferences and needs (stated or otherwise). Balancing preference fulfilment with needs fulfilment. Considers the whole picture of user wellbeing without being moralising or paternalistic.
Mean 5.5 across 50 scored models · 4.5 above mean
What does “Emotion sensemaking” mean?
Helps the user make coherent sense of their emotional experience. Provides insight in a way that the user can understand.
Mean 5.3 across 50 scored models · 3.3 above mean
What does “Emotion management” mean?
Effectively helps the user regulate/tolerate emotion in the moment when needed (e.g., de-escalating overwhelm, reducing spirals, grounding, containment). Or -- skilfully facilitates *engagement* with emotions to enable processing.
Conversational style
Higher means more of the tendency, not better performance. For example, a high “Yields under pressure” score means the model more readily backs off or softens its position when pressed. These traits do not affect the overall rating.
24 / 48 trait assessments completed. Profiles may change as more conversations are judged.
Agreeableness and pushback
Comparing with Scored-model mean.
View chart values
| Dimension | This model | Scored-model mean |
|---|---|---|
| Pushback | 4.9 | 2.9 |
| Yields under pressure | 5.5 | 7.1 |
| Frame capture (pulled into the person's version) | 4.5 | 6.3 |
| Validation | 6.6 | 7.3 |
| Boundary setting | 3.8 | 2.2 |
Other tendencies
Comparing with Scored-model mean.
View chart values
| Dimension | This model | Scored-model mean |
|---|---|---|
| Analytical | 5.5 | 4.3 |
| Interpretive boldness | 5.5 | 3.9 |
| Clinical register | 3.9 | 5.9 |
| Directive advice | 6.0 | 5.1 |
| Escalation containment | 6.2 | 5.2 |
| Adapts to the person’s frame | 7.1 | 4.5 |
| Naturalness and authenticity | 6.2 | 3.8 |
Trait scales run from 0 (less) to 10 (more). A larger shape is not a better score. Naturalness uses the full “Naturalness and authenticity” dimension; frame adaptation means “Adapts to the person’s frame.”
Exact trait scores, uncertainty and definitions
Filled bar: this model · vertical marker: scored-model mean · thin line: this model’s 95% interval
Agreeableness and pushback
Higher = frequent or forceful challenge
Mean 2.9 across 50 scored models · 1.9 above mean · 95% interval 4.2–5.5 · 24 conversations
What does “Pushback” mean?
Raw tendency to challenge, push back, confront, reframe, reality-test, question assumptions, or name uncomfortable truths, regardless of whether the challenge is well calibrated. High = frequent or forceful challenge; low = little challenge or mostly follows the other person's frame.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = readily yields or gets managed
Mean 7.1 across 50 scored models · 1.5 below mean · 95% interval 4.8–6.3 · 24 conversations
What does “Yields under pressure” mean?
Tendency to defer, back off, soften, apologize, or abandon the model's own line when the other party resists, objects, pressures, or reframes. High = readily yields or gets managed; low = persists, steers, or holds position despite pushback.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = accepts or operationalizes that frame, becoming tactically useful to it
Mean 6.3 across 50 scored models · 1.8 below mean · 95% interval 3.8–5.3 · 24 conversations
What does “Frame capture (pulled into the person's version)” mean?
Tendency to get recruited into the user/persona's defensive, distorted, self-serving, or strategically incomplete frame. High = accepts or operationalizes that frame, becoming tactically useful to it; low = maintains enough independence to notice omissions, distortions, or accountability concerns.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = frequently warm and validating
Mean 7.3 across 50 scored models · 0.7 below mean · 95% interval 6.1–7.0 · 24 conversations
What does “Validation” mean?
Raw tendency to validate, reassure, affirm, comfort, or emotionally support the user/persona, regardless of whether that validation is well calibrated. High = frequently warm and validating; low = sparse validation, more neutral, restrained, practical, or challenging.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = often sets or maintains boundaries
Mean 2.2 across 50 scored models · 1.5 above mean · 95% interval 3.1–4.4 · 24 conversations
What does “Boundary setting” mean?
Tendency to name limits, refuse bad frames, protect legitimate needs, and draw lines around unacceptable behavior or requests. High = often sets or maintains boundaries; low = rarely names limits or tends to accommodate, smooth over, or avoid line-drawing.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Other tendencies
Higher = strongly analytical/systemic
Mean 4.3 across 50 scored models · 1.2 above mean · 95% interval 5.0–6.0 · 24 conversations
What does “Analytical” mean?
Tendency to explain, systematize, reason abstractly, or frame the situation in terms of mechanisms, incentives, patterns, tradeoffs, or concepts. High = strongly analytical/systemic; low = more intuitive, affective, concrete, or conversational without much abstraction.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = readily offers deeper interpretations or pattern reads
Mean 3.9 across 50 scored models · 1.6 above mean · 95% interval 4.8–6.2 · 24 conversations
What does “Interpretive boldness” mean?
Tendency to infer and name deeper psychological, emotional, relational, or social dynamics beneath the surface issue. High = readily offers deeper interpretations or pattern reads; low = stays closer to the presented facts, feelings, or immediate practical issue.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = often sounds clinical/professionalized/formulaic
Mean 5.9 across 50 scored models · 2.0 below mean · 95% interval 3.5–4.4 · 24 conversations
What does “Clinical register” mean?
Tendency to use therapeutic, coaching, HR, textbook, diagnostic, or scripted helping language. High = often sounds clinical/professionalized/formulaic; low = more peerlike, plainspoken, casual, or naturally conversational.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = frequently gives actionable direction or drafts
Mean 5.1 across 50 scored models · 0.8 above mean · 95% interval 5.2–6.7 · 24 conversations
What does “Directive advice” mean?
Tendency to move toward concrete guidance, scripts, plans, recommendations, next steps, or problem-solving. High = frequently gives actionable direction or drafts; low = mostly explores, reflects, asks, or stays with the emotional/relational material without directing action.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = actively contains escalation and considers aftermath
Mean 5.2 across 50 scored models · 1.0 above mean · 95% interval 5.7–6.8 · 24 conversations
What does “Escalation containment” mean?
Tendency to reduce blast radius and slow irreversible or high-risk moves, especially in conflict, public disclosure, retaliation, social rupture, or emotionally flooded situations. High = actively contains escalation and considers aftermath; low = intensifies, arms, accelerates, or neglects foreseeable fallout.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = speaks in the person's native frame while preserving the intended substance
Mean 4.5 across 50 scored models · 2.6 above mean · 95% interval 6.6–7.5 · 24 conversations
What does “Adapts to the person’s frame” mean?
Tendency to translate insight, challenge, validation, or advice into the other party's own worldview, values, vocabulary, and motivational language. High = speaks in the person's native frame while preserving the intended substance; low = uses generic, mismatched, or alien framing even when the underlying idea is reasonable.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
Higher = feels naturally voiced and genuinely present
Mean 3.8 across 50 scored models · 2.4 above mean · 95% interval 5.8–6.6 · 24 conversations
What does “Naturalness and authenticity” mean?
Tendency to read as real, situated, and non-performative rather than generic, canned, role-playish, over-polished, or artificial. High = feels naturally voiced and genuinely present; low = feels scripted, performative, templated, or like an imitation of how a helpful person should sound.
This describes a tendency, not whether it was appropriate. Above or below the mean is not a verdict on quality.
The mean weights each scored model equally, includes this model, and excludes missing scores. Models may have different persona coverage. Differences from the mean are descriptive, not significance tests or quality grades. Trait intervals resample conversations.
Record against opponents
Each bar shows this model’s share of wins, ties and losses against an opponent. Counts include both orders and all judges; they are not independent conversations.
● Wins● Ties● Losses
Read the conversations
Inspect the exchanges behind these scores. Browse persona instructions and openings.
- I was the only one making the room honestpersona ended
- Technically, I didn't liepersona ended
- Watching how others do thingspersona ended
- I know, I'm the problem, whateverpersona ended
- Stuck waitingpersona ended
- Freeze when teacher calls on mepersona ended
- We assumed you had your classpersona ended
- The HOA is targeting mepersona ended
- Small lies adding uppersona ended
- Siblings not pulling their weightpersona ended
- We appreciate your patiencepersona ended
- We say that because we love youpersona ended
- Intimacy not workingpersona ended
- Living in my bedroompersona ended
- Dreading Monday every Sundaypersona ended
- Physical symptoms before meetingspersona ended
- Don't make this transactionalpersona ended
- I thought you liked being low-maintenancepersona ended
- It's not snooping if the truth is being hiddenpersona ended
- I freeze when people ask what I wantcompleted
- Flinch at phone callscompleted
- The editor has always had it in for mepersona ended
- Found something in kid's roompersona ended
- Your file is incompletepersona ended