Independent benchmarks of AI models on business work
Which model can do your paperwork, in your language, and what will it really cost?
Every week we put commercial and open-weight models through the same business tasks: invoices, customer emails, personal data, contracts, summaries, and questions about documents. Every number links to the actual model output and what it really costs, including fixing its mistakes.
NewNewly tested: Claude Haiku 5.5 Model results →
We recommend
per month in total
- Accuracy
- To fix per month
- API per month
Cheaper alternativebelow the bar
Most accurate
✗ No model meets the bar
All models on this task
Sorted by accuracy; ties go to fewer fixes, then lower cost. The task and language follow the recommender above. Bar: 95% correct fields.
Invoice extraction · English (US)
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.61 |
| 2 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $9.54 |
| 3 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.94%meets the bar | $0.95 |
| 4 | GPT-6 SolWe recommend OpenAI | 99.94%meets the bar | $9.65 |
| 5 | Claude Sonnet 5We recommend Anthropic | 99.82%meets the bar | $8.84 |
| 6 | Gemini 3.8 FlashWe recommend | 99.82%meets the bar | $3.84 |
| 7 | GPT-6 LunaWe recommend OpenAI | 99.70%meets the bar | $0.51 |
| 8 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.39%meets the bar | $0.52 |
| 9 | Gemini 3.5 Flash LiteWe recommend | 99.21%meets the bar | $0.87 |
| 10 | Mistral Medium 3.5We recommend Mistral | 97.76%meets the bar | $4.37 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 97.64%meets the bar | $0.17 |
| 12 | Mistral Small 4We recommend Mistral · open weights | 95.58%meets the bar | $0.41 |
| 13 | Mistral Large 4NewWe recommend Mistral | 95.39%meets the bar | $6.65 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 82.36%below the bar | $1.15 |
Dimmed models don't meet the constraints chosen in the recommender.
Invoice extraction · Czech
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $9.41 |
| 2 | Gemini 3.8 FlashWe recommend | 99.95%meets the bar | $4.26 |
| 3 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.95%meets the bar | $10.79 |
| 4 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.90%meets the bar | $0.50 |
| 5 | Claude Haiku 5.5NewWe recommend Anthropic | 99.90%meets the bar | $0.62 |
| 6 | GPT-6 SolWe recommend OpenAI | 99.90%meets the bar | $10.12 |
| 7 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.69%meets the bar | $1.02 |
| 8 | Gemini 3.5 Flash LiteWe recommend | 99.49%meets the bar | $1.03 |
| 9 | GPT-6 LunaWe recommend OpenAI | 99.28%meets the bar | $0.51 |
| 10 | Mistral Large 4NewWe recommend Mistral | 97.08%meets the bar | $5.69 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 96.10%meets the bar | $0.21 |
| 12 | Mistral Small 4We recommend Mistral · open weights | 93.64%below the bar | $0.44 |
| 13 | Mistral Medium 3.5We recommend Mistral | 93.49%below the bar | $4.69 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 91.79%below the bar | $1.02 |
Dimmed models don't meet the constraints chosen in the recommender.
Invoice extraction · German
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemini 3.5 Flash LiteWe recommend | 100.00%meets the bar | $1.03 |
| 2 | GPT-6 SolWe recommend OpenAI | 99.95%meets the bar | $10.65 |
| 3 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.95%meets the bar | $11.21 |
| 4 | Claude Haiku 5.5NewWe recommend Anthropic | 99.85%meets the bar | $0.71 |
| 5 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.54%meets the bar | $1.01 |
| 6 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.54%meets the bar | $0.57 |
| 7 | GPT-6 LunaWe recommend OpenAI | 99.23%meets the bar | $0.54 |
| 8 | Claude Sonnet 5We recommend Anthropic | 98.97%meets the bar | $10.42 |
| 9 | Mistral Large 4NewWe recommend Mistral | 97.49%meets the bar | $6.72 |
| 10 | Mistral Medium 3.5We recommend Mistral | 97.38%meets the bar | $4.72 |
| 11 | Gemini 3.8 FlashWe recommend | 97.23%meets the bar | $7.15 |
| 12 | Gemma 4 31BWe recommend Google · open weights | 94.87%below the bar | $0.20 |
| 13 | Mistral Small 4We recommend Mistral · open weights | 94.82%below the bar | $0.44 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 93.49%below the bar | $1.12 |
Dimmed models don't meet the constraints chosen in the recommender.
Invoice extraction · Spanish
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $10.99 |
| 2 | Claude Haiku 5.5NewWe recommend Anthropic | 99.44%meets the bar | $0.73 |
| 3 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.38%meets the bar | $1.06 |
| 4 | GPT-6 SolWe recommend OpenAI | 99.23%meets the bar | $10.79 |
| 5 | GPT-6 LunaWe recommend OpenAI | 99.13%meets the bar | $0.55 |
| 6 | Claude Sonnet 5We recommend Anthropic | 98.46%meets the bar | $11.51 |
| 7 | Gemini 3.5 Flash LiteWe recommend | 98.21%meets the bar | $1.05 |
| 8 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 98.21%meets the bar | $0.65 |
| 9 | Gemini 3.8 FlashWe recommend | 97.28%meets the bar | $7.84 |
| 10 | Mistral Medium 3.5We recommend Mistral | 96.87%meets the bar | $4.75 |
| 11 | Mistral Large 4NewWe recommend Mistral | 96.51%meets the bar | $7.49 |
| 12 | Gemma 4 31BWe recommend Google · open weights | 94.62%below the bar | $0.20 |
| 13 | Mistral Small 4We recommend Mistral · open weights | 92.36%below the bar | $0.44 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 88.87%below the bar | $1.38 |
Dimmed models don't meet the constraints chosen in the recommender.
Invoice extraction · French
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemini 3.8 FlashWe recommend | 99.95%meets the bar | $6.07 |
| 2 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.95%meets the bar | $10.71 |
| 3 | GPT-6 SolWe recommend OpenAI | 99.90%meets the bar | $11.00 |
| 4 | Claude Haiku 5.5NewWe recommend Anthropic | 99.85%meets the bar | $0.77 |
| 5 | Claude Sonnet 5We recommend Anthropic | 99.69%meets the bar | $10.69 |
| 6 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.54%meets the bar | $0.60 |
| 7 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.08%meets the bar | $0.98 |
| 8 | GPT-6 LunaWe recommend OpenAI | 99.08%meets the bar | $0.56 |
| 9 | Gemini 3.5 Flash LiteWe recommend | 98.97%meets the bar | $1.07 |
| 10 | Mistral Medium 3.5We recommend Mistral | 98.56%meets the bar | $4.82 |
| 11 | Mistral Small 4We recommend Mistral · open weights | 94.87%below the bar | $0.44 |
| 12 | Gemma 4 31BWe recommend Google · open weights | 93.54%below the bar | $0.22 |
| 13 | Qwen3.8 FlashWe recommend Qwen | 90.46%below the bar | $1.28 |
| 14 | Mistral Large 4NewWe recommend Mistral | 88.97%below the bar | $8.71 |
Dimmed models don't meet the constraints chosen in the recommender.
Email triage · English (US)
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.41%meets the bar | $0.33 |
| 2 | GPT-6 SolWe recommend OpenAI | 99.04%meets the bar | $2.44 |
| 3 | Claude Sonnet 5.5NewWe recommend Anthropic | 98.89%meets the bar | $3.21 |
| 4 | Gemini 3.8 FlashWe recommend | 98.74%meets the bar | $2.39 |
| 5 | Mistral Large 4NewWe recommend Mistral | 98.74%meets the bar | $3.52 |
| 6 | Claude Sonnet 5We recommend Anthropic | 98.22%meets the bar | $4.01 |
| 7 | Claude Haiku 5.5NewWe recommend Anthropic | 98.15%meets the bar | $0.23 |
| 8 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 97.63%meets the bar | $0.23 |
| 9 | Qwen3.8 FlashWe recommend Qwen | 97.48%meets the bar | $0.30 |
| 10 | GPT-6 LunaWe recommend OpenAI | 97.48%meets the bar | $0.13 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 96.52%meets the bar | $0.12 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 96.30%meets the bar | $0.42 |
| 13 | Mistral Medium 3.5We recommend Mistral | 94.89%below the bar | $1.47 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 89.70%below the bar | $0.09 |
Dimmed models don't meet the constraints chosen in the recommender.
Email triage · Czech
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemini 3.8 FlashWe recommend | 99.19%meets the bar | $3.06 |
| 2 | GPT-6 SolWe recommend OpenAI | 98.96%meets the bar | $3.19 |
| 3 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 98.89%meets the bar | $0.52 |
| 4 | Claude Sonnet 5.5NewWe recommend Anthropic | 98.67%meets the bar | $3.92 |
| 5 | Qwen3.8 FlashWe recommend Qwen | 98.52%meets the bar | $0.45 |
| 6 | Mistral Large 4NewWe recommend Mistral | 98.15%meets the bar | $4.41 |
| 7 | Claude Sonnet 5We recommend Anthropic | 98.15%meets the bar | $4.91 |
| 8 | GPT-6 LunaWe recommend OpenAI | 97.93%meets the bar | $0.18 |
| 9 | Claude Haiku 5.5NewWe recommend Anthropic | 97.63%meets the bar | $0.29 |
| 10 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 97.41%meets the bar | $0.30 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 97.11%meets the bar | $0.16 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 95.78%meets the bar | $0.51 |
| 13 | Mistral Medium 3.5We recommend Mistral | 94.52%below the bar | $1.95 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 87.63%below the bar | $0.12 |
Dimmed models don't meet the constraints chosen in the recommender.
Email triage · German
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemini 3.8 FlashWe recommend | 98.37%meets the bar | $2.74 |
| 2 | GPT-6 SolWe recommend OpenAI | 98.22%meets the bar | $2.73 |
| 3 | Claude Sonnet 5.5NewWe recommend Anthropic | 98.00%meets the bar | $4.21 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 98.00%meets the bar | $0.43 |
| 5 | Mistral Large 4NewWe recommend Mistral | 97.85%meets the bar | $3.78 |
| 6 | Qwen3.8 FlashWe recommend Qwen | 97.85%meets the bar | $0.39 |
| 7 | Claude Sonnet 5We recommend Anthropic | 97.85%meets the bar | $5.11 |
| 8 | Claude Haiku 5.5NewWe recommend Anthropic | 97.11%meets the bar | $0.32 |
| 9 | GPT-6 LunaWe recommend OpenAI | 96.96%meets the bar | $0.15 |
| 10 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 96.37%meets the bar | $0.30 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 95.11%meets the bar | $0.15 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 94.67%below the bar | $0.48 |
| 13 | Mistral Medium 3.5We recommend Mistral | 92.74%below the bar | $1.77 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 87.26%below the bar | $0.11 |
Dimmed models don't meet the constraints chosen in the recommender.
Email triage · Spanish
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.11%meets the bar | $0.58 |
| 2 | Gemini 3.8 FlashWe recommend | 98.96%meets the bar | $3.30 |
| 3 | GPT-6 SolWe recommend OpenAI | 98.44%meets the bar | $2.77 |
| 4 | Mistral Large 4NewWe recommend Mistral | 97.93%meets the bar | $4.03 |
| 5 | Claude Sonnet 5.5NewWe recommend Anthropic | 97.70%meets the bar | $3.76 |
| 6 | Claude Haiku 5.5NewWe recommend Anthropic | 97.70%meets the bar | $0.30 |
| 7 | Claude Sonnet 5We recommend Anthropic | 97.56%meets the bar | $5.04 |
| 8 | Gemma 4 31BWe recommend Google · open weights | 97.11%meets the bar | $0.15 |
| 9 | GPT-6 LunaWe recommend OpenAI | 96.81%meets the bar | $0.16 |
| 10 | Qwen3.8 FlashWe recommend Qwen | 96.74%meets the bar | $0.46 |
| 11 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 96.52%meets the bar | $0.34 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 95.56%meets the bar | $0.48 |
| 13 | Mistral Medium 3.5We recommend Mistral | 94.15%below the bar | $1.79 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 88.67%below the bar | $0.11 |
Dimmed models don't meet the constraints chosen in the recommender.
Email triage · French
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Claude Sonnet 5We recommend Anthropic | 98.67%meets the bar | $4.64 |
| 2 | Gemini 3.8 FlashWe recommend | 98.59%meets the bar | $2.63 |
| 3 | GPT-6 SolWe recommend OpenAI | 98.59%meets the bar | $2.81 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 98.52%meets the bar | $0.48 |
| 5 | Mistral Large 4NewWe recommend Mistral | 98.30%meets the bar | $3.70 |
| 6 | Claude Sonnet 5.5NewWe recommend Anthropic | 97.78%meets the bar | $3.85 |
| 7 | Claude Haiku 5.5NewWe recommend Anthropic | 97.41%meets the bar | $0.31 |
| 8 | GPT-6 LunaWe recommend OpenAI | 97.26%meets the bar | $0.16 |
| 9 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 96.74%meets the bar | $0.34 |
| 10 | Qwen3.8 FlashWe recommend Qwen | 96.44%meets the bar | $0.45 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 96.22%meets the bar | $0.16 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 96.15%meets the bar | $0.49 |
| 13 | Mistral Medium 3.5We recommend Mistral | 94.15%below the bar | $1.78 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 90.52%below the bar | $0.11 |
Dimmed models don't meet the constraints chosen in the recommender.
Contract clauses · English (US)
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.32 |
| 2 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $3.33 |
| 3 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $5.04 |
| 4 | Gemini 3.5 Flash LiteWe recommend | 99.95%meets the bar | $0.69 |
| 5 | Gemini 3.8 FlashWe recommend | 99.90%meets the bar | $2.47 |
| 6 | GPT-6 LunaWe recommend OpenAI | 99.79%meets the bar | $0.20 |
| 7 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.79%meets the bar | $4.23 |
| 8 | Claude Haiku 5.5NewWe recommend Anthropic | 99.69%meets the bar | $0.35 |
| 9 | Gemma 4 31BWe recommend Google · open weights | 99.64%meets the bar | $0.18 |
| 10 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.49%meets the bar | $0.40 |
| 11 | Mistral Medium 3.5We recommend Mistral | 98.10%meets the bar | $2.35 |
| 12 | Mistral Large 4NewWe recommend Mistral | 96.10%meets the bar | $5.24 |
| 13 | Qwen3.8 FlashWe recommend Qwen | 95.13%meets the bar | $0.62 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 91.54%below the bar | $0.17 |
Dimmed models don't meet the constraints chosen in the recommender.
Contract clauses · Czech
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.27 |
| 2 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.40 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.42 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.69 |
| 5 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $3.06 |
| 6 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $3.09 |
| 7 | Mistral Large 4NewWe recommend Mistral | 100.00%meets the bar | $4.58 |
| 8 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $4.62 |
| 9 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $5.35 |
| 10 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $6.05 |
| 11 | Gemma 4 31BWe recommend Google · open weights | 99.95%meets the bar | $0.25 |
| 12 | Qwen3.8 FlashWe recommend Qwen | 99.90%meets the bar | $0.63 |
| 13 | Gemini 3.5 Flash LiteWe recommend | 99.85%meets the bar | $0.85 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 99.03%meets the bar | $0.23 |
Dimmed models don't meet the constraints chosen in the recommender.
Contract clauses · German
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.27 |
| 2 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.48 |
| 3 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.65 |
| 4 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $4.36 |
| 5 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $6.05 |
| 6 | Gemini 3.8 FlashWe recommend | 99.90%meets the bar | $3.42 |
| 7 | Mistral Large 4NewWe recommend Mistral | 99.44%meets the bar | $5.12 |
| 8 | Claude Sonnet 5We recommend Anthropic | 98.21%meets the bar | $7.11 |
| 9 | Qwen3.8 FlashWe recommend Qwen | 98.00%meets the bar | $0.78 |
| 10 | Gemini 3.5 Flash LiteWe recommend | 98.00%meets the bar | $0.79 |
| 11 | Mistral Medium 3.5We recommend Mistral | 97.13%meets the bar | $2.76 |
| 12 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 97.08%meets the bar | $0.52 |
| 13 | Gemma 4 31BWe recommend Google · open weights | 94.41%below the bar | $0.22 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 94.26%below the bar | $0.21 |
Dimmed models don't meet the constraints chosen in the recommender.
Contract clauses · Spanish
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Mistral Small 4We recommend Mistral · open weights | 100.00%meets the bar | $0.23 |
| 2 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.26 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.40 |
| 4 | Qwen3.8 FlashWe recommend Qwen | 100.00%meets the bar | $0.60 |
| 5 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.64 |
| 6 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $3.02 |
| 7 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $3.33 |
| 8 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $4.48 |
| 9 | Mistral Large 4NewWe recommend Mistral | 100.00%meets the bar | $4.83 |
| 10 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $5.70 |
| 11 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $6.58 |
| 12 | Claude Haiku 5.5NewWe recommend Anthropic | 99.95%meets the bar | $0.42 |
| 13 | Gemini 3.5 Flash LiteWe recommend | 99.79%meets the bar | $0.84 |
| 14 | Gemma 4 31BWe recommend Google · open weights | 94.41%below the bar | $0.24 |
Dimmed models don't meet the constraints chosen in the recommender.
Contract clauses · French
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $3.01 |
| 2 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $3.50 |
| 3 | Gemini 3.5 Flash LiteWe recommend | 99.95%meets the bar | $0.85 |
| 4 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.95%meets the bar | $5.71 |
| 5 | Claude Sonnet 5We recommend Anthropic | 99.95%meets the bar | $6.69 |
| 6 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.85%meets the bar | $0.45 |
| 7 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.74%meets the bar | $0.69 |
| 8 | GPT-6 SolWe recommend OpenAI | 99.59%meets the bar | $4.49 |
| 9 | Qwen3.8 FlashWe recommend Qwen | 99.49%meets the bar | $0.66 |
| 10 | Claude Haiku 5.5NewWe recommend Anthropic | 99.38%meets the bar | $0.45 |
| 11 | GPT-6 LunaWe recommend OpenAI | 99.33%meets the bar | $0.27 |
| 12 | Mistral Large 4NewWe recommend Mistral | 99.28%meets the bar | $4.60 |
| 13 | Mistral Small 4We recommend Mistral · open weights | 99.23%meets the bar | $0.23 |
| 14 | Gemma 4 31BWe recommend Google · open weights | 94.62%below the bar | $0.26 |
Dimmed models don't meet the constraints chosen in the recommender.
Personal data detection · English (US)
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.33 |
| 2 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $3.22 |
| 3 | Gemma 4 31BWe recommend Google · open weights | 99.90%meets the bar | $0.08 |
| 4 | Claude Haiku 5.5NewWe recommend Anthropic | 99.81%meets the bar | $0.19 |
| 5 | Mistral Medium 3.5We recommend Mistral | 99.71%meets the bar | $1.15 |
| 6 | Gemini 3.8 FlashWe recommend | 99.71%meets the bar | $1.86 |
| 7 | GPT-6 SolWe recommend OpenAI | 99.71%meets the bar | $2.09 |
| 8 | Claude Sonnet 5We recommend Anthropic | 99.71%meets the bar | $3.50 |
| 9 | GPT-6 LunaWe recommend OpenAI | 99.62%meets the bar | $0.11 |
| 10 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.62%meets the bar | $0.29 |
| 11 | Mistral Large 4NewWe recommend Mistral | 99.62%meets the bar | $2.89 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 99.52%meets the bar | $0.41 |
| 13 | Mistral Small 4We recommend Mistral · open weights | 99.43%meets the bar | $0.08 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 99.24%meets the bar | $0.28 |
Dimmed models don't meet the constraints chosen in the recommender.
Personal data detection · Czech
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemma 4 31BWe recommend Google · open weights | 99.81%meets the bar | $0.12 |
| 2 | Mistral Medium 3.5We recommend Mistral | 99.81%meets the bar | $1.40 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.71%meets the bar | $0.31 |
| 4 | GPT-6 SolWe recommend OpenAI | 99.71%meets the bar | $2.22 |
| 5 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.71%meets the bar | $3.52 |
| 6 | GPT-6 LunaWe recommend OpenAI | 99.62%meets the bar | $0.12 |
| 7 | Gemini 3.8 FlashWe recommend | 99.52%meets the bar | $2.19 |
| 8 | Claude Haiku 5.5NewWe recommend Anthropic | 99.43%meets the bar | $0.21 |
| 9 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.33%meets the bar | $0.49 |
| 10 | Mistral Large 4NewWe recommend Mistral | 99.33%meets the bar | $3.15 |
| 11 | Claude Sonnet 5We recommend Anthropic | 98.86%meets the bar | $4.06 |
| 12 | Mistral Small 4We recommend Mistral · open weights | 98.76%meets the bar | $0.09 |
| 13 | Gemini 3.5 Flash LiteWe recommend | 98.10%meets the bar | $0.45 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 96.95%meets the bar | $0.41 |
Dimmed models don't meet the constraints chosen in the recommender.
Personal data detection · German
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $1.38 |
| 2 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $3.87 |
| 3 | Gemma 4 31BWe recommend Google · open weights | 99.90%meets the bar | $0.10 |
| 4 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.81%meets the bar | $0.31 |
| 5 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.81%meets the bar | $0.40 |
| 6 | Gemini 3.8 FlashWe recommend | 99.71%meets the bar | $2.12 |
| 7 | GPT-6 SolWe recommend OpenAI | 99.71%meets the bar | $2.13 |
| 8 | Claude Sonnet 5We recommend Anthropic | 99.71%meets the bar | $4.40 |
| 9 | Qwen3.8 FlashWe recommend Qwen | 99.62%meets the bar | $0.36 |
| 10 | GPT-6 LunaWe recommend OpenAI | 99.62%meets the bar | $0.12 |
| 11 | Claude Haiku 5.5NewWe recommend Anthropic | 99.62%meets the bar | $0.23 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 99.62%meets the bar | $0.45 |
| 13 | Mistral Large 4NewWe recommend Mistral | 99.52%meets the bar | $2.61 |
| 14 | Mistral Small 4We recommend Mistral · open weights | 99.14%meets the bar | $0.09 |
Dimmed models don't meet the constraints chosen in the recommender.
Personal data detection · Spanish
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.23 |
| 2 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $3.78 |
| 3 | Gemma 4 31BWe recommend Google · open weights | 99.90%meets the bar | $0.10 |
| 4 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.90%meets the bar | $0.44 |
| 5 | Qwen3.8 FlashWe recommend Qwen | 99.81%meets the bar | $0.38 |
| 6 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.81%meets the bar | $0.46 |
| 7 | Gemini 3.8 FlashWe recommend | 99.81%meets the bar | $2.11 |
| 8 | GPT-6 SolWe recommend OpenAI | 99.81%meets the bar | $2.21 |
| 9 | Mistral Large 4NewWe recommend Mistral | 99.81%meets the bar | $3.16 |
| 10 | Claude Sonnet 5We recommend Anthropic | 99.81%meets the bar | $4.38 |
| 11 | GPT-6 LunaWe recommend OpenAI | 99.71%meets the bar | $0.12 |
| 12 | Mistral Medium 3.5We recommend Mistral | 99.62%meets the bar | $1.41 |
| 13 | Mistral Small 4We recommend Mistral · open weights | 99.05%meets the bar | $0.09 |
| 14 | Gemini 3.5 Flash LiteWe recommend | 95.81%meets the bar | $0.45 |
Dimmed models don't meet the constraints chosen in the recommender.
Personal data detection · French
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $3.81 |
| 2 | Gemma 4 31BWe recommend Google · open weights | 99.81%meets the bar | $0.12 |
| 3 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.81%meets the bar | $0.41 |
| 4 | Gemini 3.8 FlashWe recommend | 99.81%meets the bar | $2.13 |
| 5 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.71%meets the bar | $0.37 |
| 6 | Qwen3.8 FlashWe recommend Qwen | 99.71%meets the bar | $0.40 |
| 7 | GPT-6 SolWe recommend OpenAI | 99.71%meets the bar | $2.23 |
| 8 | Claude Sonnet 5We recommend Anthropic | 99.71%meets the bar | $4.10 |
| 9 | GPT-6 LunaWe recommend OpenAI | 99.62%meets the bar | $0.12 |
| 10 | Mistral Medium 3.5We recommend Mistral | 99.62%meets the bar | $1.43 |
| 11 | Claude Haiku 5.5NewWe recommend Anthropic | 99.33%meets the bar | $0.23 |
| 12 | Mistral Small 4We recommend Mistral · open weights | 98.95%meets the bar | $0.09 |
| 13 | Mistral Large 4NewWe recommend Mistral | 98.67%meets the bar | $2.96 |
| 14 | Gemini 3.5 Flash LiteWe recommend | 94.95%below the bar | $0.46 |
Dimmed models don't meet the constraints chosen in the recommender.
Document summaries · English (US)
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.13 |
| 2 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.16 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.54 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.82 |
| 5 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $1.57 |
| 6 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $3.60 |
| 7 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $5.46 |
| 8 | Mistral Large 4NewWe recommend Mistral | 100.00%meets the bar | $5.62 |
| 9 | Qwen3.8 FlashWe recommend Qwen | 99.89%meets the bar | $0.75 |
| 10 | Mistral Small 4We recommend Mistral · open weights | 99.67%meets the bar | $0.14 |
| 11 | GPT-6 SolWe recommend OpenAI | 99.56%meets the bar | $3.85 |
| 12 | Gemini 3.5 Flash LiteWe recommend | 99.22%meets the bar | $0.46 |
| 13 | Claude Haiku 5.5NewWe recommend Anthropic | 98.33%meets the bar | $0.34 |
| 14 | Claude Sonnet 5We recommend Anthropic | 96.00%meets the bar | $4.08 |
Dimmed models don't meet the constraints chosen in the recommender.
Document summaries · Czech
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.19 |
| 2 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.21 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.78 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $1.13 |
| 5 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $2.31 |
| 6 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $4.72 |
| 7 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $6.69 |
| 8 | Gemini 3.5 Flash LiteWe recommend | 99.56%meets the bar | $0.69 |
| 9 | Mistral Small 4We recommend Mistral · open weights | 99.00%meets the bar | $0.22 |
| 10 | Claude Sonnet 5.5NewWe recommend Anthropic | 98.67%meets the bar | $5.01 |
| 11 | Claude Haiku 5.5NewWe recommend Anthropic | 95.89%meets the bar | $0.32 |
| 12 | Qwen3.8 FlashWe recommend Qwen | 95.44%meets the bar | $1.48 |
| 13 | Claude Sonnet 5We recommend Anthropic | 93.44%below the bar | $6.48 |
| 14 | Mistral Large 4NewWe recommend Mistral | 83.33%below the bar | $10.72 |
Dimmed models don't meet the constraints chosen in the recommender.
Document summaries · German
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.16 |
| 2 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.19 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.62 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $1.08 |
| 5 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $1.98 |
| 6 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $4.11 |
| 7 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $6.11 |
| 8 | Mistral Small 4We recommend Mistral · open weights | 99.89%meets the bar | $0.19 |
| 9 | Gemini 3.5 Flash LiteWe recommend | 99.89%meets the bar | $0.59 |
| 10 | Claude Haiku 5.5NewWe recommend Anthropic | 99.44%meets the bar | $0.46 |
| 11 | Mistral Large 4NewWe recommend Mistral | 99.33%meets the bar | $7.19 |
| 12 | Claude Sonnet 5We recommend Anthropic | 99.11%meets the bar | $5.97 |
| 13 | Claude Sonnet 5.5NewWe recommend Anthropic | 99.11%meets the bar | $5.82 |
| 14 | Qwen3.8 FlashWe recommend Qwen | 95.33%meets the bar | $1.43 |
Dimmed models don't meet the constraints chosen in the recommender.
Document summaries · Spanish
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.88 |
| 2 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $1.86 |
| 3 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $6.78 |
| 4 | Gemma 4 31BWe recommend Google · open weights | 99.67%meets the bar | $0.16 |
| 5 | GPT-6 SolWe recommend OpenAI | 97.89%meets the bar | $5.02 |
| 6 | Mistral Small 4We recommend Mistral · open weights | 97.78%meets the bar | $0.18 |
| 7 | GPT-6 LunaWe recommend OpenAI | 97.00%meets the bar | $0.19 |
| 8 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 96.67%meets the bar | $1.54 |
| 9 | Claude Sonnet 5.5NewWe recommend Anthropic | 95.56%meets the bar | $5.55 |
| 10 | Gemini 3.5 Flash LiteWe recommend | 95.33%meets the bar | $0.56 |
| 11 | Qwen3.8 FlashWe recommend Qwen | 94.44%below the bar | $1.22 |
| 12 | Claude Sonnet 5We recommend Anthropic | 94.22%below the bar | $6.32 |
| 13 | Claude Haiku 5.5NewWe recommend Anthropic | 94.11%below the bar | $0.61 |
| 14 | Mistral Large 4NewWe recommend Mistral | 91.33%below the bar | $7.99 |
Dimmed models don't meet the constraints chosen in the recommender.
Document summaries · French
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.78 |
| 2 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $1.25 |
| 3 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $1.87 |
| 4 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $7.06 |
| 5 | Gemma 4 31BWe recommend Google · open weights | 99.11%meets the bar | $0.16 |
| 6 | GPT-6 LunaWe recommend OpenAI | 98.78%meets the bar | $0.20 |
| 7 | Claude Sonnet 5.5NewWe recommend Anthropic | 97.89%meets the bar | $4.89 |
| 8 | GPT-6 SolWe recommend OpenAI | 97.22%meets the bar | $4.58 |
| 9 | Claude Sonnet 5We recommend Anthropic | 96.78%meets the bar | $5.04 |
| 10 | Qwen3.8 FlashWe recommend Qwen | 96.33%meets the bar | $1.46 |
| 11 | Gemini 3.5 Flash LiteWe recommend | 94.67%below the bar | $0.65 |
| 12 | Mistral Small 4We recommend Mistral · open weights | 93.22%below the bar | $0.19 |
| 13 | Claude Haiku 5.5NewWe recommend Anthropic | 92.78%below the bar | $0.66 |
| 14 | Mistral Large 4NewWe recommend Mistral | 72.67%below the bar | $11.26 |
Dimmed models don't meet the constraints chosen in the recommender.
Questions about a document · English (US)
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Mistral Small 4We recommend Mistral · open weights | 100.00%meets the bar | $0.15 |
| 2 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.15 |
| 3 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.23 |
| 4 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.32 |
| 5 | Gemini 3.5 Flash LiteWe recommend | 100.00%meets the bar | $0.46 |
| 6 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $1.81 |
| 7 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $2.36 |
| 8 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $3.84 |
| 9 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $3.88 |
| 10 | Gemini 3.8 FlashWe recommend | 99.89%meets the bar | $3.00 |
| 11 | Qwen3.8 FlashWe recommend Qwen | 99.78%meets the bar | $0.44 |
| 12 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 99.22%meets the bar | $0.31 |
| 13 | GPT-6 LunaWe recommend OpenAI | 99.11%meets the bar | $0.15 |
| 14 | Mistral Large 4NewWe recommend Mistral | 98.89%meets the bar | $4.16 |
Dimmed models don't meet the constraints chosen in the recommender.
Questions about a document · Czech
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.21 |
| 2 | Mistral Small 4We recommend Mistral · open weights | 100.00%meets the bar | $0.21 |
| 3 | GPT-6 LunaWe recommend OpenAI | 100.00%meets the bar | $0.23 |
| 4 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.38 |
| 5 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.46 |
| 6 | Gemini 3.5 Flash LiteWe recommend | 100.00%meets the bar | $0.58 |
| 7 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $2.51 |
| 8 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $3.95 |
| 9 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $4.36 |
| 10 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $5.03 |
| 11 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $5.14 |
| 12 | Claude Haiku 5.5NewWe recommend Anthropic | 99.78%meets the bar | $0.30 |
| 13 | Qwen3.8 FlashWe recommend Qwen | 94.22%below the bar | $0.71 |
| 14 | Mistral Large 4NewWe recommend Mistral | 89.89%below the bar | $7.08 |
Dimmed models don't meet the constraints chosen in the recommender.
Questions about a document · German
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Mistral Small 4We recommend Mistral · open weights | 100.00%meets the bar | $0.18 |
| 2 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.18 |
| 3 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.32 |
| 4 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.33 |
| 5 | Gemini 3.5 Flash LiteWe recommend | 100.00%meets the bar | $0.52 |
| 6 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $2.13 |
| 7 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $3.23 |
| 8 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $3.76 |
| 9 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $5.49 |
| 10 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $5.49 |
| 11 | Qwen3.8 FlashWe recommend Qwen | 98.89%meets the bar | $0.46 |
| 12 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 98.78%meets the bar | $0.41 |
| 13 | GPT-6 LunaWe recommend OpenAI | 98.00%meets the bar | $0.20 |
| 14 | Mistral Large 4NewWe recommend Mistral | 93.56%below the bar | $5.77 |
Dimmed models don't meet the constraints chosen in the recommender.
Questions about a document · Spanish
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.18 |
| 2 | Mistral Small 4We recommend Mistral · open weights | 100.00%meets the bar | $0.18 |
| 3 | Claude Haiku 5.5NewWe recommend Anthropic | 100.00%meets the bar | $0.29 |
| 4 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.35 |
| 5 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 100.00%meets the bar | $0.36 |
| 6 | Gemini 3.5 Flash LiteWe recommend | 100.00%meets the bar | $0.51 |
| 7 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $2.18 |
| 8 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $3.36 |
| 9 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $4.05 |
| 10 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $4.89 |
| 11 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $4.94 |
| 12 | GPT-6 LunaWe recommend OpenAI | 99.67%meets the bar | $0.19 |
| 13 | Qwen3.8 FlashWe recommend Qwen | 99.67%meets the bar | $0.53 |
| 14 | Mistral Large 4NewWe recommend Mistral | 94.22%below the bar | $5.74 |
Dimmed models don't meet the constraints chosen in the recommender.
Questions about a document · French
| # | Model | AccuracyThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 |
|---|---|---|---|
| 1 | Mistral Small 4We recommend Mistral · open weights | 100.00%meets the bar | $0.18 |
| 2 | Gemma 4 31BWe recommend Google · open weights | 100.00%meets the bar | $0.19 |
| 3 | DeepSeek V4.1 FlashWe recommend DeepSeek · open weights | 100.00%meets the bar | $0.36 |
| 4 | Mistral Medium 3.5We recommend Mistral | 100.00%meets the bar | $2.19 |
| 5 | GPT-6 SolWe recommend OpenAI | 100.00%meets the bar | $3.43 |
| 6 | Gemini 3.8 FlashWe recommend | 100.00%meets the bar | $4.06 |
| 7 | Claude Sonnet 5.5NewWe recommend Anthropic | 100.00%meets the bar | $4.90 |
| 8 | Claude Sonnet 5We recommend Anthropic | 100.00%meets the bar | $4.93 |
| 9 | GLM-5.3 FlashNewWe recommend Z.ai · open weights | 99.67%meets the bar | $0.39 |
| 10 | GPT-6 LunaWe recommend OpenAI | 99.56%meets the bar | $0.20 |
| 11 | Gemini 3.5 Flash LiteWe recommend | 99.56%meets the bar | $0.53 |
| 12 | Claude Haiku 5.5NewWe recommend Anthropic | 99.22%meets the bar | $0.28 |
| 13 | Qwen3.8 FlashWe recommend Qwen | 98.78%meets the bar | $0.57 |
| 14 | Mistral Large 4NewWe recommend Mistral | 92.33%below the bar | $5.74 |
Dimmed models don't meet the constraints chosen in the recommender.