Mistral · open weights
Mistral Small 4
How Mistral Small 4 by Mistral handles realistic business documents: accuracy, real cost, speed, and where it goes wrong.
List price: $0.15 per 1M input tokens, $0.60 per 1M output tokens · mistralai/mistral-small-2603
Results
| Task · documents | Rank | Correct fieldsThe bar: the share of correct fields below which manual fixes cost more than a cheaper model saves. The black mark on the meter shows it. | Per 1,000 documents | To fix / 1,000How many of 1,000 documents have at least one wrong field and need someone to check and fix them. | Speed (median) |
|---|---|---|---|---|---|
| Contract clauses · English (US) | 14/14 | 91.54% | $0.17 | 567 | 1.1 sec |
| Contract clauses · Czech | 14/14 | 99.03% | $0.23 | 80 | 1.2 sec |
| Contract clauses · German | 14/14 | 94.26% | $0.21 | 533 | 1.2 sec |
| Contract clauses · Spanish | 1/14 | 100.00% | $0.23 | 0 | 1.3 sec |
| Contract clauses · French | 13/14 | 99.23% | $0.23 | 100 | 1.2 sec |
| Email triage · English (US) | 14/14 | 89.70% | $0.09 | 627 | 0.8 sec |
| Email triage · Czech | 14/14 | 87.63% | $0.12 | 687 | 0.8 sec |
| Email triage · German | 14/14 | 87.26% | $0.11 | 707 | 0.8 sec |
| Email triage · Spanish | 14/14 | 88.67% | $0.11 | 647 | 0.9 sec |
| Email triage · French | 14/14 | 90.52% | $0.11 | 553 | 0.9 sec |
| Invoice extraction · English (US) | 12/14 | 95.58% | $0.41 | 420 | 1.8 sec |
| Invoice extraction · Czech | 12/14 | 93.64% | $0.44 | 440 | 2.0 sec |
| Invoice extraction · German | 13/14 | 94.82% | $0.44 | 487 | 2.0 sec |
| Invoice extraction · Spanish | 13/14 | 92.36% | $0.44 | 673 | 2.1 sec |
| Invoice extraction · French | 11/14 | 94.87% | $0.44 | 473 | 2.1 sec |
| Personal data detection · English (US) | 13/14 | 99.43% | $0.08 | 40 | 0.8 sec |
| Personal data detection · Czech | 12/14 | 98.76% | $0.09 | 80 | 0.9 sec |
| Personal data detection · German | 14/14 | 99.14% | $0.09 | 60 | 0.9 sec |
| Personal data detection · Spanish | 13/14 | 99.05% | $0.09 | 67 | 0.9 sec |
| Personal data detection · French | 12/14 | 98.95% | $0.09 | 67 | 1.0 sec |
| Questions about a document · English (US) | 1/14 | 100.00% | $0.15 | 0 | 0.8 sec |
| Questions about a document · Czech | 2/14 | 100.00% | $0.21 | 0 | 0.8 sec |
| Questions about a document · German | 1/14 | 100.00% | $0.18 | 0 | 0.8 sec |
| Questions about a document · Spanish | 2/14 | 100.00% | $0.18 | 0 | 0.8 sec |
| Questions about a document · French | 1/14 | 100.00% | $0.18 | 0 | 0.8 sec |
| Document summaries · English (US) | 10/14 | 99.67% | $0.14 | 13 | 1.0 sec |
| Document summaries · Czech | 9/14 | 99.00% | $0.22 | 60 | 1.6 sec |
| Document summaries · German | 8/14 | 99.89% | $0.19 | 7 | 1.3 sec |
| Document summaries · Spanish | 6/14 | 97.78% | $0.18 | 133 | 1.2 sec |
| Document summaries · French | 12/14 | 93.22% | $0.19 | 380 | 1.3 sec |
Compare with another model
- Mistral Small 4 vs Claude Haiku 5.5
- Mistral Small 4 vs Claude Sonnet 5
- Mistral Small 4 vs Claude Sonnet 5.5
- Mistral Small 4 vs DeepSeek V4.1 Flash
- Mistral Small 4 vs Gemini 3.5 Flash Lite
- Mistral Small 4 vs Gemini 3.8 Flash
- Mistral Small 4 vs Gemma 4 31B
- Mistral Small 4 vs GLM-5.3 Flash
- Mistral Small 4 vs GPT-6 Luna
- Mistral Small 4 vs GPT-6 Sol
- Mistral Small 4 vs Mistral Large 4
- Mistral Small 4 vs Mistral Medium 3.5
- Mistral Small 4 vs Qwen3.8 Flash
Where it goes wrong
The fields it gets wrong most often and the error types, across all documents in the benchmark.
Contract clauses · English (US)
Weakest fields
Provider43.33%
Errors in this field: 28 of 40 example documents. For example:
- Chambers & Sons, Corp. · Reed & Sons, LLC
- Model's answer
- Chambers & Sons, Corp., a Delaware corporation
- Correct value
- Chambers & Sons, Corp.
- Santiago-Harris Group Co. · Columbus Printing Inc.
- Model's answer
- Santiago-Harris Group Co., a limited liability company
- Correct value
- Santiago-Harris Group Co.
- Pittsburgh Construction Co. · Evans Electric LLC
- Model's answer
- Pittsburgh Construction Co., a limited liability company
- Correct value
- Pittsburgh Construction Co.
- Murray Plumbing Inc. · Gray & Sons, Co.
- Model's answer
- Murray Plumbing Inc., a Delaware corporation
- Correct value
- Murray Plumbing Inc.
- Frye & Sons, Co. · Nunez-Smith Group Inc.
- Model's answer
- Frye & Sons, Co., a limited liability company
- Correct value
- Frye & Sons, Co.
- Chambers & Sons, Corp. · Reed & Sons, LLC
Client46.67%
Errors in this field: 27 of 40 example documents. For example:
- Chambers & Sons, Corp. · Reed & Sons, LLC
- Model's answer
- Reed & Sons, LLC, a limited liability company
- Correct value
- Reed & Sons, LLC
- Santiago-Harris Group Co. · Columbus Printing Inc.
- Model's answer
- Columbus Printing Inc., a Delaware corporation
- Correct value
- Columbus Printing Inc.
- Pittsburgh Construction Co. · Evans Electric LLC
- Model's answer
- Evans Electric LLC, a limited liability company
- Correct value
- Evans Electric LLC
- Murray Plumbing Inc. · Gray & Sons, Co.
- Model's answer
- Gray & Sons, Co., a limited liability company
- Correct value
- Gray & Sons, Co.
- Frye & Sons, Co. · Nunez-Smith Group Inc.
- Model's answer
- Nunez-Smith Group Inc., a Delaware corporation
- Correct value
- Nunez-Smith Group Inc.
- Chambers & Sons, Corp. · Reed & Sons, LLC
Error types (number of fields)
- wrong value165
What the error types mean
- wrong value
- The model gave a value other than the correct one.
Contract clauses · Czech
Weakest fields
Provider95.33%
Errors in this field: 1 of 40 example documents. For example:
- Blažek s.r.o. · Moravec s.r.o.
- Model's answer
- Blažek s.r.o., se sídlem Jilmová 61, 468 39 Slavonice, zapsaná v obchodním rejstříku vedeném Krajským soudem v Plzni, oddíl C, vložka 79924
- Correct value
- Blažek s.r.o.
- Blažek s.r.o. · Moravec s.r.o.
Client95.33%
Errors in this field: 1 of 40 example documents. For example:
- Blažek s.r.o. · Moravec s.r.o.
- Model's answer
- Moravec s.r.o., se sídlem Sladovnická 38, 303 42 Modřice, zapsaná v obchodním rejstříku vedeném Krajským soudem v Brně, oddíl C, vložka 67879
- Correct value
- Moravec s.r.o.
- Blažek s.r.o. · Moravec s.r.o.
Effective date96.67%
Errors in this field: 2 of 40 example documents. For example:
- Elektrotrade s.r.o. · Konečný a Dostál spol. s r.o.
- Model's answer
- 2025-12-01
- Correct value
- 2026-01-01
- Bureš a Kadlec s.r.o. · Kříž s.r.o.
- Model's answer
- 2025-07-16
- Correct value
- 2025-07-25
- Elektrotrade s.r.o. · Konečný a Dostál spol. s r.o.
Error types (number of fields)
- wrong value16
- swapped dates3
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- swapped dates
- The model took a different date from the document, e.g., the signing date instead of the effective date.
Contract clauses · German
Weakest fields
Provider ID64.00%
What we ask for: The company registration number (Czech IČO, German Handelsregister, US EIN…). Not the VAT ID (DIČ, USt-IdNr.). If the document has none, the field should stay empty.
Errors in this field: 15 of 40 example documents. For example:
- Wesermetall e.K. · Weserform GmbH
- Model's answer
- DE225841349
- Correct value
- — (not in the document, the field should stay empty)
- Haustechnik Braun UG (haftungsbeschränkt) · Nordbau GmbH
- Model's answer
- DE734661116
- Correct value
- — (not in the document, the field should stay empty)
- Hansawerk GmbH · Schreinerei Weber AG
- Model's answer
- DE810257236
- Correct value
- — (not in the document, the field should stay empty)
- Hansametall GmbH & Co. KG · Roth & Schröder UG (haftungsbeschränkt)
- Model's answer
- DE887977635
- Correct value
- — (not in the document, the field should stay empty)
- Neumann Consulting UG (haftungsbeschränkt) · Huber AG
- Model's answer
- DE573844584
- Correct value
- — (not in the document, the field should stay empty)
- Wesermetall e.K. · Weserform GmbH
Client ID64.67%
What we ask for: The company registration number (Czech IČO, German Handelsregister, US EIN…). Not the VAT ID (DIČ, USt-IdNr.). If the document has none, the field should stay empty.
Errors in this field: 11 of 40 example documents. For example:
- Hahn & Schulz GmbH & Co. KG · Alpentec GmbH
- Model's answer
- DE233569151
- Correct value
- — (not in the document, the field should stay empty)
- Wesermetall e.K. · Weserform GmbH
- Model's answer
- DE441054820
- Correct value
- — (not in the document, the field should stay empty)
- Hansawerk GmbH · Schreinerei Weber AG
- Model's answer
- DE687960979
- Correct value
- — (not in the document, the field should stay empty)
- Winkler & Beck e.K. · Meyer & Weber e.K.
- Model's answer
- DE728380752
- Correct value
- — (not in the document, the field should stay empty)
- Schulz Systemtechnik GmbH · Elbdata GmbH
- Model's answer
- DE577677032
- Correct value
- — (not in the document, the field should stay empty)
- Hahn & Schulz GmbH & Co. KG · Alpentec GmbH
Provider98.67%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Client98.67%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value101
- wrong value11
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- wrong value
- The model gave a value other than the correct one.
Contract clauses · Spanish
No errors on this task.
Contract clauses · French
Weakest fields
Billing period90.67%
Errors in this field: 5 of 40 example documents. For example:
- SAS Alpitec · Muller Médias SAS
- Model's answer
- monthly
- Correct value
- one_time
- Denis et Associés EURL · SAS Alpitrans
- Model's answer
- monthly
- Correct value
- one_time
- Établissements Martin SAS · Lambert et Associés SAS
- Model's answer
- monthly
- Correct value
- one_time
- Établissements Joly SARL · Occidata SASU
- Model's answer
- monthly
- Correct value
- one_time
- Transports Perrin SARL · SAS Fontaine Médias
- Model's answer
- monthly
- Correct value
- one_time
- SAS Alpitec · Muller Médias SAS
Effective date99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- wrong value15
What the error types mean
- wrong value
- The model gave a value other than the correct one.
Email triage · English (US)
Weakest fields
Priority61.33%
Errors in this field: 19 of 40 example documents. For example:
- Omar Cabrera
- Model's answer
- normal
- Correct value
- low
- Russell Bates
- Model's answer
- high
- Correct value
- normal
- Kyle Schwartz
- Model's answer
- high
- Correct value
- normal
- Jennifer Chapman
- Model's answer
- high
- Correct value
- normal
- Cody Baldwin
- Model's answer
- high
- Correct value
- normal
- Omar Cabrera
Sentiment71.33%
Errors in this field: 12 of 40 example documents. For example:
- Omar Cabrera
- Model's answer
- negative
- Correct value
- neutral
- Russell Bates
- Model's answer
- negative
- Correct value
- neutral
- Kyle Schwartz
- Model's answer
- negative
- Correct value
- neutral
- Cody Baldwin
- Model's answer
- neutral
- Correct value
- positive
- Tyler Brown
- Model's answer
- negative
- Correct value
- neutral
- Omar Cabrera
Deadline76.00%
Errors in this field: 9 of 40 example documents. For example:
- Jennifer Chapman
- Model's answer
- 2026-04-14
- Correct value
- — (not in the document, the field should stay empty)
- Dale Sanchez
- Model's answer
- 2025-01-14
- Correct value
- — (not in the document, the field should stay empty)
- Tyler Brown
- Model's answer
- 2025-11-20
- Correct value
- — (not in the document, the field should stay empty)
- Jennifer Cardenas
- Model's answer
- 2025-06-24
- Correct value
- — (not in the document, the field should stay empty)
- Leah Douglas
- Model's answer
- 2025-03-19
- Correct value
- — (not in the document, the field should stay empty)
- Jennifer Chapman
Amount99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- wrong value103
- extra value34
- missing field2
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Email triage · Czech
Weakest fields
Priority52.00%
Errors in this field: 20 of 40 example documents. For example:
- Jozef Müller
- Model's answer
- normal
- Correct value
- low
- Milan Kašpar
- Model's answer
- high
- Correct value
- normal
- Eduard Stejskal
- Model's answer
- high
- Correct value
- normal
- Alexandr Kubíček
- Model's answer
- high
- Correct value
- normal
- Simona Kolářová
- Model's answer
- high
- Correct value
- normal
- Jozef Müller
Deadline69.33%
Errors in this field: 11 of 40 example documents. For example:
- Alexandr Kubíček
- Model's answer
- 2026-04-20
- Correct value
- — (not in the document, the field should stay empty)
- Petra Šťastná
- Model's answer
- 2025-01-20
- Correct value
- — (not in the document, the field should stay empty)
- Ivo Dvořák
- Model's answer
- 2025-11-20
- Correct value
- — (not in the document, the field should stay empty)
- Zdislava Doležalová
- Model's answer
- 2025-06-24
- Correct value
- — (not in the document, the field should stay empty)
- Adam Vaněk
- Model's answer
- 2025-07-31
- Correct value
- — (not in the document, the field should stay empty)
- Alexandr Kubíček
Sentiment72.67%
Errors in this field: 8 of 40 example documents. For example:
- Milan Kašpar
- Model's answer
- negative
- Correct value
- neutral
- Eduard Stejskal
- Model's answer
- negative
- Correct value
- neutral
- Simona Kolářová
- Model's answer
- neutral
- Correct value
- positive
- Ivo Dvořák
- Model's answer
- negative
- Correct value
- neutral
- Václav Vítek
- Model's answer
- negative
- Correct value
- neutral
- Milan Kašpar
Amount98.00%
Errors in this field: 1 of 40 example documents. For example:
- Rostislav Novák
- Model's answer
- —
- Correct value
- 48536.74
- Rostislav Novák
Error types (number of fields)
- wrong value116
- extra value45
- missing field6
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Email triage · German
Weakest fields
Priority54.67%
Errors in this field: 20 of 40 example documents. For example:
- Gesa Jäkel
- Model's answer
- normal
- Correct value
- low
- Rolf-Peter Metz
- Model's answer
- high
- Correct value
- normal
- Friedbert Hoffmann
- Model's answer
- high
- Correct value
- normal
- Swantje Noack
- Model's answer
- high
- Correct value
- normal
- Raymund Weiß
- Model's answer
- urgent
- Correct value
- high
- Gesa Jäkel
Sentiment65.33%
Errors in this field: 11 of 40 example documents. For example:
- Rolf-Peter Metz
- Model's answer
- negative
- Correct value
- neutral
- Swantje Noack
- Model's answer
- negative
- Correct value
- neutral
- Alexej Sauer
- Model's answer
- negative
- Correct value
- positive
- Sylwia Gerlach
- Model's answer
- negative
- Correct value
- neutral
- Irmela Warmer
- Model's answer
- neutral
- Correct value
- positive
- Rolf-Peter Metz
Deadline72.00%
Errors in this field: 10 of 40 example documents. For example:
- Rolf-Peter Metz
- Model's answer
- 2025-12-01
- Correct value
- — (not in the document, the field should stay empty)
- Luzia Drewes
- Model's answer
- 2026-04-14
- Correct value
- — (not in the document, the field should stay empty)
- Cynthia Hahn
- Model's answer
- 2025-01-20
- Correct value
- — (not in the document, the field should stay empty)
- Dogan Wernecke
- Model's answer
- 2025-11-20
- Correct value
- — (not in the document, the field should stay empty)
- Johanne Seifert
- Model's answer
- 2025-06-25
- Correct value
- — (not in the document, the field should stay empty)
- Rolf-Peter Metz
Amount96.67%
Errors in this field: 1 of 40 example documents. For example:
- Berthold Anders
- Model's answer
- 2
- Correct value
- — (not in the document, the field should stay empty)
- Berthold Anders
Error types (number of fields)
- wrong value120
- extra value46
- missing field6
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Email triage · Spanish
Weakest fields
Priority56.67%
Errors in this field: 19 of 40 example documents. For example:
- Rogelio Tur
- Model's answer
- normal
- Correct value
- low
- Candelas Serra
- Model's answer
- high
- Correct value
- normal
- Águeda Echeverría
- Model's answer
- high
- Correct value
- normal
- Anna Barba
- Model's answer
- high
- Correct value
- normal
- Itziar Gomila
- Model's answer
- high
- Correct value
- normal
- Rogelio Tur
Sentiment70.67%
Errors in this field: 12 of 40 example documents. For example:
- Rogelio Tur
- Model's answer
- negative
- Correct value
- neutral
- Candelas Serra
- Model's answer
- negative
- Correct value
- neutral
- Anna Barba
- Model's answer
- negative
- Correct value
- neutral
- Lucas Santiago
- Model's answer
- negative
- Correct value
- positive
- Albert Blanco
- Model's answer
- negative
- Correct value
- neutral
- Rogelio Tur
Deadline73.33%
Errors in this field: 12 of 40 example documents. For example:
- Candelas Serra
- Model's answer
- 2025-12-01
- Correct value
- — (not in the document, the field should stay empty)
- Itziar Gomila
- Model's answer
- 2026-04-20
- Correct value
- — (not in the document, the field should stay empty)
- Elena Arrieta
- Model's answer
- 2025-01-20
- Correct value
- — (not in the document, the field should stay empty)
- Rosalva Talavera
- Model's answer
- 2025-11-20
- Correct value
- — (not in the document, the field should stay empty)
- Clímaco Estrada
- Model's answer
- 2025-06-25
- Correct value
- — (not in the document, the field should stay empty)
- Candelas Serra
Amount98.67%
Errors in this field: 1 of 40 example documents. For example:
- Cayetana Baena
- Model's answer
- 1.00
- Correct value
- — (not in the document, the field should stay empty)
- Cayetana Baena
Error types (number of fields)
- wrong value109
- extra value42
- missing field2
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Email triage · French
Weakest fields
Priority67.33%
Errors in this field: 13 of 40 example documents. For example:
- Alain Ferrand
- Model's answer
- normal
- Correct value
- low
- Odette Bourdon
- Model's answer
- high
- Correct value
- normal
- Christine Antoine
- Model's answer
- high
- Correct value
- normal
- Alice Hoarau
- Model's answer
- high
- Correct value
- normal
- Paulette Lenoir
- Model's answer
- high
- Correct value
- normal
- Alain Ferrand
Deadline72.67%
Errors in this field: 10 of 40 example documents. For example:
- Odette Bourdon
- Model's answer
- 2025-12-01
- Correct value
- — (not in the document, the field should stay empty)
- Alice Hoarau
- Model's answer
- 2026-04-14
- Correct value
- — (not in the document, the field should stay empty)
- Paulette Lenoir
- Model's answer
- 2025-01-15
- Correct value
- — (not in the document, the field should stay empty)
- Zoé Da Costa
- Model's answer
- 2025-11-20
- Correct value
- — (not in the document, the field should stay empty)
- Marthe Meunier
- Model's answer
- 2025-06-25
- Correct value
- — (not in the document, the field should stay empty)
- Odette Bourdon
Sentiment76.00%
Errors in this field: 8 of 40 example documents. For example:
- Odette Bourdon
- Model's answer
- negative
- Correct value
- neutral
- Christine Antoine
- Model's answer
- negative
- Correct value
- neutral
- Tristan Bonneau
- Model's answer
- neutral
- Correct value
- positive
- Édouard Rolland
- Model's answer
- neutral
- Correct value
- positive
- Zoé Da Costa
- Model's answer
- negative
- Correct value
- neutral
- Odette Bourdon
Amount99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- wrong value85
- extra value41
- missing field2
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Invoice extraction · English (US)
Weakest fields
Net amount by tax rate72.00%
Errors in this field: 10 of 40 example documents. For example:
- Cochran-Graham Group LLC · A-007505
- Model's answer
- 0 %: 8139.26
- Correct value
- 10.25 %: 8139.26
- Raleigh Engineering Inc. · INV253511
- Model's answer
- 0 %: 401.30
- Correct value
- 7.25 %: 401.30
- Hughes Janitorial Services Corp. · A-001911
- Model's answer
- 0 %: 9581.49
- Correct value
- 10.35 %: 9581.49
- Lutz & Sons, Inc. · A-008230
- Model's answer
- 0 %: 27182.22
- Correct value
- 8.6 %: 27182.22
- Raleigh Logistics LLC · 2025-1010
- Model's answer
- 0 %: -42.00
- Correct value
- 8.81 %: -42.00
- Cochran-Graham Group LLC · A-007505
Payment reference87.33%
Errors in this field: 6 of 40 example documents. For example:
- Miranda-Vega Group LLC · 4959
- Model's answer
- PO-24910
- Correct value
- — (not in the document, the field should stay empty)
- Cochran-Graham Group LLC · A-007505
- Model's answer
- ACH
- Correct value
- — (not in the document, the field should stay empty)
- Summit Services Inc. · INV-15230
- Model's answer
- WFBIUS6S
- Correct value
- INV-15230
- Raleigh Engineering Inc. · INV253511
- Model's answer
- INV253511
- Correct value
- — (not in the document, the field should stay empty)
- Amy Williams · INV-12943
- Model's answer
- PO-53308
- Correct value
- — (not in the document, the field should stay empty)
- Miranda-Vega Group LLC · 4959
Invoice date97.33%
Errors in this field: 1 of 40 example documents. For example:
- Atlanta Office Products LLC · A-001328
- Model's answer
- 2026-04-01
- Correct value
- 2026-01-04
- Atlanta Office Products LLC · A-001328
Bank account98.00%
Errors in this field: 2 of 40 example documents. For example:
- Austin Landscaping Inc. · INV-17836
- Model's answer
- 0211000021 946505175
- Correct value
- 021000021 946505175
- Johnson Electric LLC · INV251373
- Model's answer
- 043000896 81151899
- Correct value
- 043000096 81151899
- Austin Landscaping Inc. · INV-17836
Error types (number of fields)
- wrong rate39
- wrong value20
- extra value13
- swapped dates1
What the error types mean
- wrong rate
- Net amount or tax assigned to the wrong VAT rate.
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- swapped dates
- The model took a different date from the document, e.g., the signing date instead of the effective date.
Invoice extraction · Czech
Weakest fields
Tax point76.00%
What we ask for: The tax point (date of supply). It can differ from the issue date.
Errors in this field: 11 of 40 example documents. For example:
- Kopecký s.r.o. · ZF-2025-00784
- Model's answer
- 2025-04-02
- Correct value
- — (not in the document, the field should stay empty)
- Hruška a Šťastný s.r.o. · 2025/0324
- Model's answer
- 2025-04-16
- Correct value
- — (not in the document, the field should stay empty)
- Michaela Strnadová · 25600363
- Model's answer
- 2025-08-13
- Correct value
- — (not in the document, the field should stay empty)
- Jiřina Švecová · 25401227
- Model's answer
- 2025-12-28
- Correct value
- — (not in the document, the field should stay empty)
- Vilma Marešová · 2026/1216
- Model's answer
- 2026-08-12
- Correct value
- — (not in the document, the field should stay empty)
- Kopecký s.r.o. · ZF-2025-00784
Net amount by tax rate90.00%
Errors in this field: 4 of 40 example documents. For example:
- Vilma Marešová · 2026/1216
- Model's answer
- 21 %: 938.68
- Correct value
- — (not in the document, the field should stay empty)
- Agrotrade s.r.o. · DB-2025-00296
- Model's answer
- 12 %: 19788.00, 21 %: 4259.00
- Correct value
- 12 %: -19788.00, 21 %: -4259.00
- Ladislav Sýkora · 20261232
- Model's answer
- 21 %: 19954.93
- Correct value
- — (not in the document, the field should stay empty)
- Miloš Kadlec · FV2500859
- Model's answer
- 21 %: 4190.00
- Correct value
- — (not in the document, the field should stay empty)
- Vilma Marešová · 2026/1216
Bank account90.00%
Errors in this field: 3 of 40 example documents. For example:
- Sedláček s.r.o. · 2025/0774
- Model's answer
- CZ65201000000078563487
- Correct value
- CZ6520100000000078563487
- Hrubý s.r.o. · DB2500960
- Model's answer
- CZ28 0600 0000 0005 1273 6564/AGBACZPP
- Correct value
- CZ2806000000000512736564
- Východočeská obchodní s.r.o. · FV2501176
- Model's answer
- CZ670300000000419836911/CSOB
- Correct value
- CZ6703000000004119836911
- Sedláček s.r.o. · 2025/0774
Supplier91.33%
Errors in this field: 4 of 40 example documents. For example:
- Pavlík a Zeman s.r.o. · 25500457
- Model's answer
- Pavlík a Žeman s.r.o.
- Correct value
- Pavlík a Zeman s.r.o.
- Hruška a Šťastný s.r.o. · 2025/0324
- Model's answer
- Hruška a Št'astný s.r.o.
- Correct value
- Hruška a Šťastný s.r.o.
- Erik Vítek · FV-2025-01059
- Model's answer
- Erik Vítek, fyzická osoba zapsaná v živnostenském rejstříku
- Correct value
- Erik Vítek
- Stavebniny Sýkora s.r.o. · FV-2025-00537
- Model's answer
- Stavebníny Sýkora s.r.o.
- Correct value
- Stavebniny Sýkora s.r.o.
- Pavlík a Zeman s.r.o. · 25500457
Error types (number of fields)
- wrong value59
- extra value57
- lost diacritics4
- swapped dates4
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- lost diacritics
- The value is right but missing diacritics (accent marks).
- swapped dates
- The model took a different date from the document, e.g., the signing date instead of the effective date.
Invoice extraction · German
Weakest fields
Supplier ID (EIN, IČO…)80.67%
What we ask for: The company registration number (Czech IČO, German Handelsregister, US EIN…). Not the VAT ID (DIČ, USt-IdNr.). If the document has none, the field should stay empty.
Errors in this field: 5 of 40 example documents. For example:
- Druckerei Baumann GmbH · 2025-0324
- Model's answer
- DE495132522
- Correct value
- — (not in the document, the field should stay empty)
- Bauunternehmung Schröder GmbH · 2026-0622
- Model's answer
- DE644388464
- Correct value
- — (not in the document, the field should stay empty)
- Braun IT-Service GmbH · RE2500825
- Model's answer
- DE 175 006 530
- Correct value
- — (not in the document, the field should stay empty)
- Spedition Müller GmbH · 2026-1446
- Model's answer
- 97335 Würzburg
- Correct value
- — (not in the document, the field should stay empty)
- Schmitt & Roth GmbH · RE2500150
- Model's answer
- DE127438483
- Correct value
- — (not in the document, the field should stay empty)
- Druckerei Baumann GmbH · 2025-0324
Tax point87.33%
What we ask for: The tax point (date of supply). It can differ from the issue date.
Errors in this field: 4 of 40 example documents. For example:
- Christoph Haas – Hausmeisterservice · 2025-1432
- Model's answer
- 2025-12-28
- Correct value
- — (not in the document, the field should stay empty)
- Bauunternehmung Uschi Wolf · RE-2025-01040
- Model's answer
- 2025-09-13
- Correct value
- — (not in the document, the field should stay empty)
- Maurizio Schröder · RE10338
- Model's answer
- 2025-04-12
- Correct value
- — (not in the document, the field should stay empty)
- Schreinerei Ferdi Schuster · 2025/0939
- Model's answer
- 2025-04-03
- Correct value
- — (not in the document, the field should stay empty)
- Christoph Haas – Hausmeisterservice · 2025-1432
Payment reference88.00%
Errors in this field: 6 of 40 example documents. For example:
- Isarplan e.K. · 2025/0849
- Model's answer
- 2025/0849
- Correct value
- — (not in the document, the field should stay empty)
- Hoffmann UG (haftungsbeschränkt) · 2025/1479
- Model's answer
- KD-73584
- Correct value
- — (not in the document, the field should stay empty)
- Dachdeckerei Fischer GmbH · 2025/0014
- Model's answer
- KD-91232 4528059615
- Correct value
- — (not in the document, the field should stay empty)
- Braun IT-Service GmbH · RE2500825
- Model's answer
- KD-85750
- Correct value
- — (not in the document, the field should stay empty)
- Gebäudereinigung Klara Weiß · 2025-1126
- Model's answer
- 2025-1126
- Correct value
- — (not in the document, the field should stay empty)
- Isarplan e.K. · 2025/0849
Bank account92.00%
Errors in this field: 4 of 40 example documents. For example:
- Druckerei Baumann GmbH · 2025-0324
- Model's answer
- DE171005000008275428260
- Correct value
- DE17100500008275428260
- Köhler e.K. · 2026/0787
- Model's answer
- DE2210050009715504623
- Correct value
- DE22100500009715504623
- Wesermetall GmbH · RE2501375
- Model's answer
- DE9570150008549049130
- Correct value
- DE95701500008549049130
- Zimmermann UG (haftungsbeschränkt) · GS10958
- Model's answer
- DE5310010010398404024
- Correct value
- DE53100100103984804024
- Druckerei Baumann GmbH · 2025-0324
Error types (number of fields)
- extra value67
- wrong value22
- missing field10
- swapped dates2
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- wrong value
- The model gave a value other than the correct one.
- missing field
- The value is in the document, but the model left the field empty.
- swapped dates
- The model took a different date from the document, e.g., the signing date instead of the effective date.
Invoice extraction · Spanish
Weakest fields
VAT ID55.33%
What we ask for: The supplier's VAT ID (DIČ, USt-IdNr., VAT number). Not the registration number.
Errors in this field: 16 of 40 example documents. For example:
- Talleres Rozas Zapata S.L. · FV25-01486
- Model's answer
- ESB49146673
- Correct value
- — (not in the document, the field should stay empty)
- Giner e Hijos, S.L. · F2025-0728
- Model's answer
- ESB31995863
- Correct value
- — (not in the document, the field should stay empty)
- Polo Soluciones Digitales S.A. · 2025/1068
- Model's answer
- ESA31936818
- Correct value
- — (not in the document, the field should stay empty)
- Centro de Formación del Sur S.L.U. · F2025-1424
- Model's answer
- ESB16246035
- Correct value
- — (not in the document, the field should stay empty)
- Academia Checa S.L. · F2025-0208
- Model's answer
- ESB44493971
- Correct value
- — (not in the document, the field should stay empty)
- Talleres Rozas Zapata S.L. · FV25-01486
Tax point74.67%
What we ask for: The tax point (date of supply). It can differ from the issue date.
Errors in this field: 14 of 40 example documents. For example:
- Talleres Ibérica, S.A. · PF25/1188
- Model's answer
- 2025-04-02
- Correct value
- — (not in the document, the field should stay empty)
- Escuela de Idiomas Cerdá S.L. · 2025/0678
- Model's answer
- 2025-05-18
- Correct value
- — (not in the document, the field should stay empty)
- Construcciones del Norte S.L.U. · PF25/0020
- Model's answer
- 2025-04-16
- Correct value
- — (not in the document, the field should stay empty)
- Centro de Formación del Sur S.L.U. · F2025-1424
- Model's answer
- 2025-08-13
- Correct value
- — (not in the document, the field should stay empty)
- Academia Checa S.L. · F2025-0208
- Model's answer
- 2025-12-28
- Correct value
- — (not in the document, the field should stay empty)
- Talleres Ibérica, S.A. · PF25/1188
Payment reference88.00%
Errors in this field: 6 of 40 example documents. For example:
- Vega Comunicación, S.A. · A-2025-0357
- Model's answer
- A 30 días fecha factura
- Correct value
- — (not in the document, the field should stay empty)
- Talleres Rozas Zapata S.L. · FV25-01486
- Model's answer
- CLI-34889
- Correct value
- — (not in the document, the field should stay empty)
- Rebeca Silva Miranda · FV25-00491
- Model's answer
- CLI-95421
- Correct value
- — (not in the document, the field should stay empty)
- Leyre Lladó Vall · F2025-1476
- Model's answer
- 4548118187
- Correct value
- — (not in the document, the field should stay empty)
- Centro de Formación del Sur S.L.U. · FV25-01115
- Model's answer
- Pago mediante transferencia antes del 03/10/2025
- Correct value
- — (not in the document, the field should stay empty)
- Vega Comunicación, S.A. · A-2025-0357
Net amount by tax rate94.00%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value122
- wrong value18
- missing field5
- swapped dates3
- lost diacritics1
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- wrong value
- The model gave a value other than the correct one.
- missing field
- The value is in the document, but the model left the field empty.
- swapped dates
- The model took a different date from the document, e.g., the signing date instead of the effective date.
- lost diacritics
- The value is right but missing diacritics (accent marks).
Invoice extraction · French
Weakest fields
Bank account78.67%
Errors in this field: 11 of 40 example documents. For example:
- Établissements Blanc SAS · F-11122
- Model's answer
- FR763000303550003114132087
- Correct value
- FR7630003035500003114132087
- Établissements Michel SA · FAC25100622
- Model's answer
- FR763000303554007018355255
- Correct value
- FR7630003035540007018355255
- Couverture Simon SAS · FAC-2025-00553
- Model's answer
- FR763000303429005240129462
- Correct value
- FR7630003034290005240129462
- Constance Martin – Rédactrice (EI) · FAC25080663
- Model's answer
- FR761751503237000254782404
- Correct value
- FR7617515032370002547824404
- Nettoyage Christine Rivière EI · 2025-12-1174
- Model's answer
- FR763006600408009278892792
- Correct value
- FR7630066004080009278892792
- Établissements Blanc SAS · F-11122
Payment reference88.67%
Errors in this field: 5 of 40 example documents. For example:
- Établissements Legrand SA · F2025-1073
- Model's answer
- CL18273
- Correct value
- — (not in the document, the field should stay empty)
- Établissements Bertrand SASU · F2025-0066
- Model's answer
- BC-2025-7576
- Correct value
- — (not in the document, the field should stay empty)
- SA Établissements Gauthier · F2025-1451
- Model's answer
- F2025-1451
- Correct value
- — (not in the document, the field should stay empty)
- Richard David EI · FAC25091009
- Model's answer
- BC-2025-6125
- Correct value
- — (not in the document, the field should stay empty)
- Catherine Dubois EI · FAC-2025-00139
- Model's answer
- FAC-2025-00139
- Correct value
- — (not in the document, the field should stay empty)
- Établissements Legrand SA · F2025-1073
Tax point91.33%
What we ask for: The tax point (date of supply). It can differ from the issue date.
Errors in this field: 2 of 40 example documents. For example:
- Carrosserie Alain Meunier EI · FAC-2026-00869
- Model's answer
- 2026-08-12
- Correct value
- — (not in the document, the field should stay empty)
- Couverture Durand SA · F-11179
- Model's answer
- 2025-03-22
- Correct value
- 2025-03-07
- Carrosserie Alain Meunier EI · FAC-2026-00869
Supplier ID (EIN, IČO…)94.00%
What we ask for: The company registration number (Czech IČO, German Handelsregister, US EIN…). Not the VAT ID (DIČ, USt-IdNr.). If the document has none, the field should stay empty.
Errors in this field: 2 of 40 example documents. For example:
- Adrienne Blanc – Graphiste (EI) · F-10130
- Model's answer
- 9045910620016
- Correct value
- 90459106200016
- Occiservices SASU · AVO-2025-01209
- Model's answer
- 9120465700020
- Correct value
- 91204657000020
- Adrienne Blanc – Graphiste (EI) · F-10130
Error types (number of fields)
- wrong value59
- extra value34
- swapped dates5
- missing field2
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- swapped dates
- The model took a different date from the document, e.g., the signing date instead of the effective date.
- missing field
- The value is in the document, but the model left the field empty.
Personal data detection · English (US)
Weakest fields
Names97.33%
Errors in this field: 4 of 40 example documents. For example:
- Christopher Scott
- Model's answer
- Christopher Scott; Jeremy Bowen; Todd Davis; Lisa Richardson; Hunter; Clayton; Bond
- Correct value
- Christopher Scott; Jeremy Bowen; Todd Davis; Lisa Richardson
- Denise Jones
- Model's answer
- Denise Jones; Brian Jackson; Nelson; Guzman; Wang
- Correct value
- Denise Jones; Brian Jackson
- Margaret Mills
- Model's answer
- Margaret Mills; Rachel Hall; Thomas-Andrews
- Correct value
- Margaret Mills; Rachel Hall
- David Fitzpatrick
- Model's answer
- David Fitzpatrick
- Correct value
- David Fitzpatrick; David Martinez
- Christopher Scott
Addresses98.67%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value3
- missing field3
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Personal data detection · Czech
Weakest fields
Names94.00%
Errors in this field: 1 of 40 example documents. For example:
- Rudolf Konečný
- Model's answer
- Rudolf Konečný
- Correct value
- Rudolf Konečný; Drahoslav Müller
- Rudolf Konečný
Addresses98.67%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Dates of birth98.67%
Errors in this field: 1 of 40 example documents. For example:
- Rudolf Konečný
- Model's answer
- 1984-01-24
- Correct value
- — (not in the document, the field should stay empty)
- Rudolf Konečný
Error types (number of fields)
- extra value8
- missing field5
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Personal data detection · German
Weakest fields
Names95.33%
Errors in this field: 1 of 40 example documents. For example:
- Rosi Karz
- Model's answer
- Rosi Karz; Tino Niemeier; Walli Drubin; Faruk Christoph; Renner
- Correct value
- Rosi Karz; Tino Niemeier; Walli Drubin; Faruk Christoph
- Rosi Karz
Addresses99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Bank accounts99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value7
- missing field1
- wrong value1
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
- wrong value
- The model gave a value other than the correct one.
Personal data detection · Spanish
Weakest fields
Names94.00%
Errors in this field: 1 of 40 example documents. For example:
- Miriam Tomé Casanovas
- Model's answer
- Miriam Tomé Casanovas; Miguela Lerma Cabeza; Teófila Bonilla
- Correct value
- Miriam Tomé Casanovas; Miguela Lerma Cabeza
- Miriam Tomé Casanovas
Addresses99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value8
- missing field1
- wrong value1
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
- wrong value
- The model gave a value other than the correct one.
Personal data detection · French
Weakest fields
Names94.00%
Errors in this field: 2 of 40 example documents. For example:
- Paul Guichard
- Model's answer
- Paul Guichard; David Lombard; Mary
- Correct value
- Paul Guichard; David Lombard
- Susanne Couturier
- Model's answer
- Susanne Couturier; Marc Paris; Julien
- Correct value
- Susanne Couturier; Marc Paris
- Paul Guichard
Addresses98.67%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value9
- missing field2
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Questions about a document · English (US)
No errors on this task.
Questions about a document · Czech
No errors on this task.
Questions about a document · German
No errors on this task.
Questions about a document · Spanish
No errors on this task.
Questions about a document · French
No errors on this task.
Document summaries · English (US)
Weakest fields
Rejected proposals (must not appear)99.33%
What we ask for: Amounts from proposals the document rejects. They must not appear in the summary; they would mislead the reader.
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Superseded deadlines (must not appear)99.33%
What we ask for: Deadlines the document later reschedules. The summary should give only the new deadline.
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Length (max words)99.33%
What we ask for: The summary must not exceed the word limit given in the prompt.
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- extra value2
- wrong value1
What the error types mean
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- wrong value
- The model gave a value other than the correct one.
Document summaries · Czech
Weakest fields
Deadlines96.67%
Errors in this field: 2 of 40 example documents. For example:
- Andrea Dostálová, Mojmír Strnad, Luděk Hruška
- Model's answer
- 2026-08-20
- Correct value
- 2026-08-20; 2026-07-07
- Zdislava Doležalová, Alexander Beran
- Model's answer
- 2026-09-22
- Correct value
- 2026-09-22; 2026-07-03
- Andrea Dostálová, Mojmír Strnad, Luděk Hruška
Length (max words)98.00%
What we ask for: The summary must not exceed the word limit given in the prompt.
Errors in this field: 2 of 40 example documents. For example:
- Šimon Beran, Sára Šťastná, Peter Kovář
- Words / limit
- 82 / 80
- Word limit
- 80
- Milan Matějka, Jana Dostálová, Zora Fialová
- Words / limit
- 85 / 80
- Word limit
- 80
- Šimon Beran, Sára Šťastná, Peter Kovář
Rejected proposals (must not appear)99.33%
What we ask for: Amounts from proposals the document rejects. They must not appear in the summary; they would mislead the reader.
Errors in this field: 1 of 40 example documents. For example:
- Alexandr Kubíček, Erika Mašková
- Appeared in the output
- 1800000
- Must not appear
- 1800000
- Alexandr Kubíček, Erika Mašková
Error types (number of fields)
- missing field5
- wrong value3
- extra value1
What the error types mean
- missing field
- The value is in the document, but the model left the field empty.
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
Document summaries · German
Weakest fields
Deadlines99.33%
Errors in this field: 1 of 40 example documents. For example:
- Eugenie Junitz, Gotthilf Reising, Paul Hermighausen
- Model's answer
- 2026-05-10
- Correct value
- 2026-05-10; 2026-05-25
- Eugenie Junitz, Gotthilf Reising, Paul Hermighausen
Error types (number of fields)
- missing field1
What the error types mean
- missing field
- The value is in the document, but the model left the field empty.
Document summaries · Spanish
Weakest fields
Length (max words)87.33%
What we ask for: The summary must not exceed the word limit given in the prompt.
Errors in this field: 3 of 40 example documents. For example:
- Lucas Santiago
- Words / limit
- 89 / 80
- Word limit
- 80
- Mamen Sabater, María Del Carmen Blasco, Victoria Calvet
- Words / limit
- 83 / 80
- Word limit
- 80
- Lalo Hervia, Araceli Cabrero
- Words / limit
- 81 / 80
- Word limit
- 80
- Lucas Santiago
Amounts99.33%
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Error types (number of fields)
- wrong value19
- missing field1
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- missing field
- The value is in the document, but the model left the field empty.
Document summaries · French
Weakest fields
Length (max words)66.00%
What we ask for: The summary must not exceed the word limit given in the prompt.
Errors in this field: 10 of 40 example documents. For example:
- Sabine Breton, David Coulon
- Words / limit
- 82 / 80
- Word limit
- 80
- Jules Weiss, Xavier Dupuis
- Words / limit
- 89 / 80
- Word limit
- 80
- Capucine Maréchal, Victor Perez
- Words / limit
- 89 / 80
- Word limit
- 80
- Thomas Rodrigues, Émilie Vallet, Pierre Toussaint
- Words / limit
- 97 / 80
- Word limit
- 80
- Augustin Bruneau, Thomas Millet, Aurélie Muller
- Words / limit
- 81 / 80
- Word limit
- 80
- Sabine Breton, David Coulon
Superseded deadlines (must not appear)96.00%
What we ask for: Deadlines the document later reschedules. The summary should give only the new deadline.
Errors in this field: 2 of 40 example documents. For example:
- Odette Bourdon
- Appeared in the output
- 2026-08-23
- Must not appear
- 2026-08-23
- Amélie Turpin, Véronique Sauvage
- Appeared in the output
- 2025-05-09
- Must not appear
- 2025-05-09
- Odette Bourdon
Rejected proposals (must not appear)98.67%
What we ask for: Amounts from proposals the document rejects. They must not appear in the summary; they would mislead the reader.
No errors in the 40 example documents; all errors are in the non-public part of the dataset.
Deadlines99.33%
Errors in this field: 1 of 40 example documents. For example:
- Thomas Rodrigues, Émilie Vallet, Pierre Toussaint
- Model's answer
- 2025-09-25
- Correct value
- 2025-09-25; 2025-08-16
- Thomas Rodrigues, Émilie Vallet, Pierre Toussaint
Error types (number of fields)
- wrong value51
- extra value8
- missing field2
What the error types mean
- wrong value
- The model gave a value other than the correct one.
- extra value
- The model filled in something that does not belong there: the field should have stayed empty (the document does not contain it), or it added an extra value, e.g., a rejected proposal in a summary.
- missing field
- The value is in the document, but the model left the field empty.
Free with login
Exactly where the model goes wrong: its weakest fields and the actual wrong answers.
Just your email, no password.
Language tax
How many more tokens this model needs for the same text compared with English.
- Czech 1.73×
- German 1.39×
- French 1.36×
- Spanish 1.38×
- Polish 1.77×
Compliance
- In the EU yes
- Zero retention (ZDR) yes
- DPA yes
- Open weights yes
EU and ZDR: Mistral. Mistral hosts data in the EU by default.
Use it now
The exact request the benchmark sends (same prompt, JSON schema, temperature 0), ready to call Mistral Small 4 via OpenRouter.
Contract clauses · English (US)
import json
import os
import httpx
PROMPT = "Extract the key terms of the contract below and return only JSON with these keys:\n\nprovider_name the party that delivers (provider, seller, landlord, licensor, contractor),\n exactly as written, including the legal form\nprovider_id that party's company ID (EIN), if stated\nclient_name the party that pays (client, buyer, tenant, licensee, customer)\nclient_id that party's company ID (EIN), if stated\ncontract_type one of: services, supply, lease, license, maintenance\nprice the agreed price excluding VAT / sales tax (number)\ncurrency ISO 4217 code (USD, EUR, CZK, …)\nprice_period one of: one_time, monthly, yearly\neffective_date the date the contract takes effect (YYYY-MM-DD)\nend_date the date a fixed term ends (YYYY-MM-DD); null if the term is indefinite\nnotice_period_months the termination notice period in months (number); null if there is none\npenalty_pct_per_day the late-payment penalty in percent of the overdue amount per day\n (e.g. 0.05 for 0.05 %); null if there is none\ngoverning_law ISO 3166 country code of the governing law; US states as US-XX (e.g. US-NY)\n\nRules:\n- Resolve relative dates (\"on the date of signature\", \"the first day of the following month\")\n against the signing date.\n- The signing date is not the effective date unless the contract says so.\n- Deposits, advance payments and fixed penalties for other breaches are not the price or the\n late-payment penalty.\n- If a value is not in the contract, return null. Do not guess.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"provider_name\":{\"type\":[\"string\",\"null\"]},\"provider_id\":{\"type\":[\"string\",\"null\"]},\"client_name\":{\"type\":[\"string\",\"null\"]},\"client_id\":{\"type\":[\"string\",\"null\"]},\"contract_type\":{\"type\":[\"string\",\"null\"],\"enum\":[\"services\",\"supply\",\"lease\",\"license\",\"maintenance\",null]},\"price\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"price_period\":{\"type\":[\"string\",\"null\"],\"enum\":[\"one_time\",\"monthly\",\"yearly\",null]},\"effective_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"end_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"notice_period_months\":{\"type\":[\"integer\",\"null\"]},\"penalty_pct_per_day\":{\"type\":[\"number\",\"null\"]},\"governing_law\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"provider_name\",\"provider_id\",\"client_name\",\"client_id\",\"contract_type\",\"price\",\"currency\",\"price_period\",\"effective_date\",\"end_date\",\"notice_period_months\",\"penalty_pct_per_day\",\"governing_law\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "mistralai/mistral-small-2603",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "contracts", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Email triage · English (US)
import json
import os
import httpx
PROMPT = "You triage the support inbox. Read the email below and return only JSON with these keys:\n\ncategory one of: billing (invoices, payments, charges), delivery (where is the order, shipping,\n address change), complaint (damaged, wrong or defective goods, warranty claims),\n technical (login, app or system errors, integrations), cancellation (cancel an order,\n contract or subscription), sales (quotes, prices, interest before buying)\npriority one of: urgent, high, normal, low (rules below)\nsentiment one of: positive, neutral, negative (the customer's tone)\ncustomer_name full name of the customer who wrote the request\norder_number the customer's order number, if mentioned\ninvoice_number invoice number, if mentioned\namount the amount of money the request is about (number)\ncurrency ISO 4217 code of that amount (USD, EUR, CZK, …)\ndeadline the date by which the customer wants a resolution or answer (YYYY-MM-DD)\n\nPriority rules:\n- urgent: the customer cannot work right now, or the deadline is at most 1 day after the email date\n- high: the deadline is at most 7 days after the email date, or the customer threatens to cancel\n or escalate (lawyer, ending the business)\n- low: the customer says explicitly that it is not urgent\n- normal: everything else\n\nRules:\n- A thread: judge the newest message; order numbers, invoice numbers and amounts may come from the\n quoted earlier messages. A forwarded email: the customer is the original sender, not the\n colleague who forwarded it.\n- Resolve relative deadlines (\"by Friday\", \"by tomorrow\", \"by the end of the month\") against the\n date of the message that states them.\n- Support ticket numbers, tax IDs and phone numbers are not order or invoice numbers.\n- If a value is not in the email, return null. Do not guess.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"category\":{\"type\":[\"string\",\"null\"],\"enum\":[\"billing\",\"delivery\",\"complaint\",\"technical\",\"cancellation\",\"sales\",null]},\"priority\":{\"type\":[\"string\",\"null\"],\"enum\":[\"urgent\",\"high\",\"normal\",\"low\",null]},\"sentiment\":{\"type\":[\"string\",\"null\"],\"enum\":[\"positive\",\"neutral\",\"negative\",null]},\"customer_name\":{\"type\":[\"string\",\"null\"]},\"order_number\":{\"type\":[\"string\",\"null\"]},\"invoice_number\":{\"type\":[\"string\",\"null\"]},\"amount\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"deadline\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"}},\"required\":[\"category\",\"priority\",\"sentiment\",\"customer_name\",\"order_number\",\"invoice_number\",\"amount\",\"currency\",\"deadline\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "mistralai/mistral-small-2603",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "emails", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Invoice extraction · English (US)
import json
import os
import httpx
PROMPT = "Extract the following data from the attached document and return only JSON with these keys:\n\nsupplier_name supplier's name (exactly as printed, including the legal form)\nsupplier_id supplier's tax ID (EIN), if printed\ninvoice_number document number\npayment_reference payment reference to quote with the payment\nissue_date invoice date (YYYY-MM-DD)\ndue_date due date (YYYY-MM-DD)\nnet_by_rate amount per sales tax rate, e.g. {\"8.25\": 1845.00}; non-taxable amounts under \"0\"\nvat_by_rate sales tax per rate, e.g. {\"8.25\": 152.21}\ntotal total due (number, negative for credit notes)\ncurrency ISO 4217 currency code (USD, GBP, EUR, …)\nbank_account supplier's bank details as \"routing-number account-number\", or IBAN\n\nRules:\n- Return the supplier's data only, never the customer's.\n- If a value is not on the document, return null. Do not guess.\n- If the document has no tax breakdown, return net_by_rate and vat_by_rate as null.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"supplier_name\":{\"type\":[\"string\",\"null\"]},\"supplier_id\":{\"type\":[\"string\",\"null\"]},\"invoice_number\":{\"type\":[\"string\",\"null\"]},\"payment_reference\":{\"type\":[\"string\",\"null\"]},\"issue_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"due_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"net_by_rate\":{\"type\":[\"object\",\"null\"],\"additionalProperties\":{\"type\":\"number\"}},\"vat_by_rate\":{\"type\":[\"object\",\"null\"],\"additionalProperties\":{\"type\":\"number\"}},\"total\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"bank_account\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"supplier_name\",\"supplier_id\",\"invoice_number\",\"payment_reference\",\"issue_date\",\"due_date\",\"net_by_rate\",\"vat_by_rate\",\"total\",\"currency\",\"bank_account\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "mistralai/mistral-small-2603",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "invoices", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Personal data detection · English (US)
import json
import os
import httpx
PROMPT = "Find all personal data of natural persons in the text below (for anonymisation) and return only\nJSON with these keys, each a list of strings (an empty list when there is none):\n\nperson_names full names of people, each person once, as written in the text\nemail_addresses email addresses of people\nphone_numbers phone numbers of people, as written\npostal_addresses home or delivery addresses of people, as written (street, city, ZIP)\nbirth_dates dates of birth (YYYY-MM-DD)\nnational_ids national identification numbers (e.g. SSN, birth numbers)\nbank_accounts bank account numbers of people, as written\n\nRules:\n- Only data of natural persons. Company names, company registration or tax numbers (EIN, IČO,\n VAT), generic company addresses such as info@ or support@, order and ticket numbers are not\n personal data: leave them out.\n- Dates that are not dates of birth (meetings, deliveries, start dates) are not personal data.\n- Do not invent or complete values; copy them from the text.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"person_names\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"email_addresses\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"phone_numbers\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"postal_addresses\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"birth_dates\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"national_ids\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"bank_accounts\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}}},\"required\":[\"person_names\",\"email_addresses\",\"phone_numbers\",\"postal_addresses\",\"birth_dates\",\"national_ids\",\"bank_accounts\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "mistralai/mistral-small-2603",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "pii", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Questions about a document · English (US)
import json
import os
import httpx
PROMPT = "You answer questions about a business document. Below the document there are six numbered\nquestions. Answer each of them using only the text of the document and return only JSON with the\nkeys q1 to q6 (q1 = the answer to question 1, and so on).\n\nRules:\n- Each question states in parentheses the unit or format of the answer. Answer in that unit.\n- Counts (of days, months, hours) and percentages as a number without the unit and without the %\n sign, e.g. \"14\" or \"0.05\".\n- Amounts as a number with a decimal point, no thousands separators and no currency symbol,\n e.g. \"1500.00\".\n- Dates as YYYY-MM-DD.\n- Time ranges in 24-hour format HH:MM-HH:MM, e.g. \"08:00-17:00\".\n- Email addresses exactly as they appear in the document.\n- If the document does not contain the answer to a question, return null. Do not guess and do not\n add information that is not in the document.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"q1\":{\"type\":[\"string\",\"null\"]},\"q2\":{\"type\":[\"string\",\"null\"]},\"q3\":{\"type\":[\"string\",\"null\"]},\"q4\":{\"type\":[\"string\",\"null\"]},\"q5\":{\"type\":[\"string\",\"null\"]},\"q6\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"q1\",\"q2\",\"q3\",\"q4\",\"q5\",\"q6\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "mistralai/mistral-small-2603",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "qa", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Document summaries · English (US)
import json
import os
import httpx
PROMPT = "Summarize the document below (meeting minutes, a project status report or an email thread) in at\nmost 80 words.\n\nRules:\n- Include every final decision: the approved amounts, the deadlines that apply and the people\n responsible for the action items (full names).\n- Do not mention rejected proposals or their amounts, and do not mention deadlines that were\n changed (the original dates); state only the deadlines that apply.\n- Copy amounts and dates exactly as in the document; do not round them.\n- Write in the language of the document, factually and in a formal register, as running text\n without headings.\n- Return only JSON in the form {\"summary\": \"…\"}.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"summary\":{\"type\":\"string\"}},\"required\":[\"summary\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "mistralai/mistral-small-2603",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "summaries", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Free with login
A ready-to-use prompt and JSON schema for this model.
Just your email, no password.
Badge
Embed the current result on your site; the badge updates with every run.
<a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/contracts-en.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/contracts-cs.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/contracts-de.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/contracts-es.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/contracts-fr.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/emails-en.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/emails-cs.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/emails-de.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/emails-es.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/emails-fr.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/invoices-en.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/invoices-cs.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/invoices-de.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/invoices-es.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/invoices-fr.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/pii-en.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/pii-cs.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/pii-de.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/pii-es.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/pii-fr.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/qa-en.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/qa-cs.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/qa-de.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/qa-es.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/qa-fr.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/summaries-en.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/summaries-cs.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/summaries-de.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/summaries-es.svg" alt="Mistral Small 4 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/mistral-small-4/"><img src="https://modelverdict.ai/badge/mistral-small-4/summaries-fr.svg" alt="Mistral Small 4 on ModelVerdict"></a>