Gemma 4 31B vs Mistral Small 4

Same documents, same prompt: where each model is right, what it really costs and how fast it answers.

At a glance

Task · documents	English (US)	Czech	German	Spanish	French
Invoice extraction	97.6% / 95.6%	96.1% / 93.6%	94.9% / 94.6%	95.4% / 91.9%	93.6% / 94.2%
Email triage	96.5% / 89.7%	97.1% / 87.7%	95.1% / 87.1%	—	—
Personal data detection	99.9% / 99.4%	99.8% / 98.8%	99.9% / 99.1%	—	—
Contract clauses	99.6% / 91.5%	100.0% / 99.0%	94.4% / 94.0%	—	—

Correct fields: Gemma 4 31B / Mistral Small 4; the better one in bold. Click a cell to see the details.

Invoice extraction · English (US) (150 documents)

Gemma 4 31B is more accurate by 2.1 pp. Gemma 4 31B is 2.4× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	97.6%	95.6%
Error-free documents	81%	58%
Per 1,000 documents	$0.17	$0.41
To fix / 1,000	187	420
Speed (median)	8.9 sec	1.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Supplier	99.3%	100.0%
Supplier ID (EIN, IČO…)	98.0%	99.3%
Invoice number	100.0%	100.0%
Payment reference	99.3%	87.3%
Invoice date	98.7%	97.3%
Due date	98.7%	98.7%
Net by tax rate	95.3%	72.0%
Tax by rate	98.7%	99.3%
Total	100.0%	100.0%
Currency	100.0%	99.3%
Bank account	86.0%	98.0%

Documents where they differ

Harmon Logistics LLC · INV256017 · Scan · Gemma 4 31B 8/11, Mistral Small 4 11/11
Cochran-Graham Group LLC · A-007505 · Clean image · Gemma 4 31B 11/11, Mistral Small 4 9/11
Raleigh Engineering Inc. · INV253511 · Scan · Gemma 4 31B 11/11, Mistral Small 4 9/11
Atlanta Office Products LLC · A-001328 · PDF text · Gemma 4 31B 11/11, Mistral Small 4 9/11
Reid Office Products LLC · A-003849 · Scan · Gemma 4 31B 11/11, Mistral Small 4 10/11
Miranda-Vega Group LLC · 4959 · PDF text · Gemma 4 31B 11/11, Mistral Small 4 10/11
Hughes Janitorial Services Corp. · A-001911 · Scan · Gemma 4 31B 11/11, Mistral Small 4 10/11
Jennifer Moore · 5113 · Scan · Gemma 4 31B 10/11, Mistral Small 4 11/11

Invoice extraction · Czech (150 documents)

Gemma 4 31B is more accurate by 2.5 pp. Gemma 4 31B is 2.1× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	96.1%	93.6%
Error-free documents	67%	56%
Per 1,000 documents	$0.21	$0.44
To fix / 1,000	333	440
Speed (median)	10.4 sec	2.0 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Supplier	96.7%	91.3%
Supplier ID (EIN, IČO…)	93.3%	92.7%
VAT ID	96.7%	94.0%
Invoice number	100.0%	100.0%
Payment reference	96.0%	97.3%
Invoice date	96.0%	98.0%
Tax point	95.3%	76.0%
Due date	94.0%	97.3%
Net by tax rate	98.0%	90.0%
Tax by rate	97.3%	94.7%
Total	99.3%	98.7%
Currency	100.0%	97.3%
Bank account	86.7%	90.0%

Documents where they differ

Sedláček s.r.o. · 2025/0774 · Scan · Gemma 4 31B 13/13, Mistral Small 4 11/13
Pavlík a Zeman s.r.o. · 25500457 · Clean image · Gemma 4 31B 13/13, Mistral Small 4 11/13
Tesař s.r.o. · 20250237 · Scan · Gemma 4 31B 13/13, Mistral Small 4 11/13
Bednář spol. s r.o. · 20250610 · Clean image · Gemma 4 31B 11/13, Mistral Small 4 13/13
Stejskal, v.o.s. · FV2600175 · Photo · Gemma 4 31B 11/13, Mistral Small 4 13/13
Vilma Marešová · 2026/1216 · Clean image · Gemma 4 31B 13/13, Mistral Small 4 11/13
Erik Vítek · FV-2025-01059 · PDF text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Agrotrade s.r.o. · DB-2025-00296 · Clean image · Gemma 4 31B 13/13, Mistral Small 4 11/13

Invoice extraction · German (150 documents)

Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 2.2× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	94.9%	94.6%
Error-free documents	64%	50%
Per 1,000 documents	$0.20	$0.44
To fix / 1,000	360	500
Speed (median)	7.7 sec	2.0 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Supplier	98.7%	98.6%
Supplier ID (EIN, IČO…)	99.3%	80.0%
VAT ID	94.0%	92.9%
Invoice number	99.3%	100.0%
Payment reference	98.7%	87.1%
Invoice date	87.3%	100.0%
Tax point	92.0%	87.9%
Due date	90.7%	98.6%
Net by tax rate	94.7%	94.3%
Tax by rate	97.3%	99.3%
Total	99.3%	100.0%
Currency	99.3%	100.0%
Bank account	82.7%	91.4%

Documents where they differ

Spedition Müller GmbH · 2026-1446 · Scan · Gemma 4 31B 8/13, Mistral Small 4 12/13
Zimmermann & Schulz GmbH · RE11274 · Scan · Gemma 4 31B 10/13, Mistral Small 4 13/13
Wesermetall GmbH · RE2501375 · Scan · Gemma 4 31B 9/13, Mistral Small 4 12/13
Maurizio Schröder · RE10338 · PDF text · Gemma 4 31B 13/13, Mistral Small 4 10/13
Schmidt & Bauer GmbH & Co. KG · RE-2025-01397 · Scan · Gemma 4 31B 10/13, Mistral Small 4 13/13
Druckerei Baumann GmbH · 2025-0324 · Scan · Gemma 4 31B 13/13, Mistral Small 4 11/13
Dachdeckerei Fischer GmbH · 2025/0014 · Scan · Gemma 4 31B 10/13, Mistral Small 4 12/13
Braun IT-Service GmbH · RE2500825 · Photo · Gemma 4 31B 13/13, Mistral Small 4 11/13

Invoice extraction · Spanish (87 documents)

Gemma 4 31B is more accurate by 3.5 pp. Gemma 4 31B is 2.2× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	95.4%	91.9%
Error-free documents	54%	31%
Per 1,000 documents	$0.20	$0.44
To fix / 1,000	460	691
Speed (median)	8.7 sec	2.2 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Supplier	95.4%	97.6%
Supplier ID (EIN, IČO…)	93.1%	98.8%
VAT ID	98.9%	53.6%
Invoice number	100.0%	100.0%
Payment reference	95.4%	88.1%
Invoice date	98.9%	100.0%
Tax point	100.0%	69.1%
Due date	97.7%	100.0%
Net by tax rate	81.6%	96.4%
Tax by rate	100.0%	97.6%
Total	100.0%	98.8%
Currency	100.0%	100.0%
Bank account	79.3%	95.2%

Documents where they differ

Centro de Formación Mediterránea S.L. · 2026/0876 · Scan · Gemma 4 31B 13/13, Mistral Small 4 11/13
Aguirre Formación S.L. · 2025/0652 · Clean image · Gemma 4 31B 13/13, Mistral Small 4 11/13
Centro de Formación del Sur S.L.U. · FV25-01115 · PDF text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Escuela de Idiomas Cerdá S.L. · 2025/0678 · PDF text · Gemma 4 31B 13/13, Mistral Small 4 12/13
Construcciones del Norte S.L.U. · PF25/0020 · Clean image · Gemma 4 31B 12/13, Mistral Small 4 11/13
Climatización Levante S.L. · F2025-1305 · Scan · Gemma 4 31B 12/13, Mistral Small 4 13/13
Giner e Hijos, S.L. · F2025-0728 · PDF text · Gemma 4 31B 13/13, Mistral Small 4 12/13
Polo Soluciones Digitales S.A. · 2025/1068 · Scan · Gemma 4 31B 13/13, Mistral Small 4 12/13

Invoice extraction · French (87 documents)

Mistral Small 4 is more accurate by 0.7 pp. Gemma 4 31B is 2× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	93.6%	94.2%
Error-free documents	51%	50%
Per 1,000 documents	$0.22	$0.45
To fix / 1,000	494	500
Speed (median)	7.2 sec	2.1 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Supplier	96.6%	96.1%
Supplier ID (EIN, IČO…)	77.0%	93.4%
VAT ID	95.4%	93.4%
Invoice number	97.7%	100.0%
Payment reference	96.6%	85.5%
Invoice date	96.6%	100.0%
Tax point	94.3%	92.1%
Due date	96.6%	100.0%
Net by tax rate	93.1%	94.7%
Tax by rate	100.0%	100.0%
Total	100.0%	100.0%
Currency	98.9%	94.7%
Bank account	73.6%	75.0%

Documents where they differ

Lefèvre & Fils SARL · FAC-2026-00198 · Photo · Gemma 4 31B 8/13, Mistral Small 4 13/13
Guérin & Fils EURL · 2026-05-688 · Clean image · Gemma 4 31B 10/13, Mistral Small 4 13/13
Établissements Bonnet EURL · AV-10599 · Scan · Gemma 4 31B 11/13, Mistral Small 4 13/13
Carrosserie Alain Meunier EI · FAC-2026-00869 · Clean image · Gemma 4 31B 12/13, Mistral Small 4 10/13
Occiservices SASU · AVO-2025-01209 · Clean image · Gemma 4 31B 12/13, Mistral Small 4 10/13
Catherine Dubois EI · FAC-2025-00139 · PDF text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Gilbert Marie – Multiservices (EI) · FAC-2025-00924 · Clean image · Gemma 4 31B 13/13, Mistral Small 4 11/13
Établissements Michel SA · FAC25100622 · Clean image · Gemma 4 31B 13/13, Mistral Small 4 12/13

Email triage · English (US) (150 documents)

Gemma 4 31B is more accurate by 6.8 pp. Mistral Small 4 is 1.3× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	96.5%	89.7%
Error-free documents	72%	37%
Per 1,000 documents	$0.12	$0.09
To fix / 1,000	280	627
Speed (median)	2.2 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Category	100.0%	100.0%
Priority	85.3%	61.3%
Sentiment	83.3%	71.3%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	99.3%
Currency	100.0%	99.3%
Deadline	100.0%	76.0%

Documents where they differ

Robert Lewis · Email · Gemma 4 31B 9/9, Mistral Small 4 6/9
Omar Cabrera · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Russell Bates · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Kyle Schwartz · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Jennifer Chapman · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Cody Baldwin · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Dale Sanchez · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Leah Douglas · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9

Email triage · Czech (150 documents)

Gemma 4 31B is more accurate by 9.4 pp. Mistral Small 4 is 1.4× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	97.1%	87.7%
Error-free documents	75%	32%
Per 1,000 documents	$0.16	$0.12
To fix / 1,000	253	685
Speed (median)	2.5 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Category	100.0%	100.0%
Priority	85.3%	52.4%
Sentiment	88.7%	72.5%
Customer name	100.0%	98.7%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	98.0%
Currency	100.0%	98.0%
Deadline	100.0%	69.8%

Documents where they differ

Rostislav Novák · Email · Gemma 4 31B 9/9, Mistral Small 4 6/9
Milan Matějka · Email · Gemma 4 31B 9/9, Mistral Small 4 6/9
Milan Kašpar · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Eduard Stejskal · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Alexandr Kubíček · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Simona Kolářová · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Petra Šťastná · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Adam Vaněk · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9

Email triage · German (150 documents)

Gemma 4 31B is more accurate by 8 pp. Mistral Small 4 is 1.4× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	95.1%	87.1%
Error-free documents	62%	29%
Per 1,000 documents	$0.15	$0.11
To fix / 1,000	380	706
Speed (median)	3.6 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Category	100.0%	100.0%
Priority	78.0%	55.2%
Sentiment	78.0%	65.0%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	96.5%
Currency	100.0%	96.5%
Deadline	100.0%	70.6%

Documents where they differ

Berthold Anders · Email · Gemma 4 31B 8/9, Mistral Small 4 5/9
Rolf-Peter Metz · Email · Gemma 4 31B 8/9, Mistral Small 4 6/9
Swantje Noack · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Luzia Drewes · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Sibille Lehmann · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Antonietta Wende · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Augusta Lange · Email · Gemma 4 31B 9/9, Mistral Small 4 7/9
Gesa Jäkel · Email · Gemma 4 31B 9/9, Mistral Small 4 8/9

Personal data detection · English (US) (150 documents)

Gemma 4 31B is more accurate by 0.5 pp. Mistral Small 4 is 1.1× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	99.9%	99.4%
Error-free documents	99%	96%
Per 1,000 documents	$0.08	$0.08
To fix / 1,000	7	40
Speed (median)	2.4 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Names	100.0%	97.3%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	99.3%	98.7%
Dates of birth	100.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Christopher Scott · Text · Gemma 4 31B 7/7, Mistral Small 4 6/7
Denise Jones · Text · Gemma 4 31B 7/7, Mistral Small 4 6/7
Margaret Mills · Text · Gemma 4 31B 7/7, Mistral Small 4 6/7
David Fitzpatrick · Text · Gemma 4 31B 7/7, Mistral Small 4 6/7

Personal data detection · Czech (150 documents)

Gemma 4 31B is more accurate by 1.1 pp. Mistral Small 4 is 1.3× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	99.8%	98.8%
Error-free documents	99%	92%
Per 1,000 documents	$0.12	$0.09
To fix / 1,000	13	81
Speed (median)	2.9 sec	0.9 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Names	100.0%	94.0%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	98.7%	98.7%
Dates of birth	100.0%	98.7%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Rudolf Konečný · Text · Gemma 4 31B 7/7, Mistral Small 4 5/7

Personal data detection · German (150 documents)

Gemma 4 31B is more accurate by 0.8 pp. Mistral Small 4 is 1.1× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	99.9%	99.1%
Error-free documents	99%	94%
Per 1,000 documents	$0.10	$0.09
To fix / 1,000	7	65
Speed (median)	1.9 sec	0.9 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Names	100.0%	95.0%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	99.3%	99.3%
Dates of birth	100.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	99.3%

Documents where they differ

Rosi Karz · Text · Gemma 4 31B 7/7, Mistral Small 4 6/7
Gunnar Zorbach · Text · Gemma 4 31B 6/7, Mistral Small 4 7/7

Contract clauses · English (US) (150 documents)

Gemma 4 31B is more accurate by 8.1 pp. Mistral Small 4 is 1.1× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	99.6%	91.5%
Error-free documents	97%	43%
Per 1,000 documents	$0.18	$0.17
To fix / 1,000	27	567
Speed (median)	2.5 sec	1.1 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Provider	98.0%	43.3%
Provider ID	100.0%	100.0%
Client	98.0%	46.7%
Client ID	100.0%	100.0%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	99.3%	100.0%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Chambers & Sons, Corp. · Reed & Sons, LLC · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Santiago-Harris Group Co. · Columbus Printing Inc. · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Pittsburgh Construction Co. · Evans Electric LLC · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Murray Plumbing Inc. · Gray & Sons, Co. · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Frye & Sons, Co. · Nunez-Smith Group Inc. · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Kim Electric Co. · Ramos & Sons, Co. · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Seattle Printing Inc. · Gutierrez Landscaping LLC · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Blue Ridge Services Corp. · Bishop-Kim Group LLC · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13

Contract clauses · Czech (150 documents)

Gemma 4 31B is more accurate by 0.9 pp. Mistral Small 4 is 1.1× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	100.0%	99.0%
Error-free documents	99%	92%
Per 1,000 documents	$0.25	$0.23
To fix / 1,000	7	80
Speed (median)	6.0 sec	1.2 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Provider	100.0%	95.3%
Provider ID	100.0%	100.0%
Client	100.0%	95.3%
Client ID	100.0%	100.0%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	99.3%	96.7%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Blažek s.r.o. · Moravec s.r.o. · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Bureš a Kadlec s.r.o. · Kříž s.r.o. · Contract text · Gemma 4 31B 13/13, Mistral Small 4 12/13

Contract clauses · German (150 documents)

Gemma 4 31B is more accurate by 0.4 pp. Mistral Small 4 is 1.1× cheaper.

Metric	Gemma 4 31B	Mistral Small 4
Correct fields	94.4%	94.0%
Error-free documents	27%	45%
Per 1,000 documents	$0.22	$0.21
To fix / 1,000	727	553
Speed (median)	2.6 sec	1.2 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Small 4
Provider	100.0%	98.6%
Provider ID	100.0%	62.4%
Client	100.0%	98.6%
Client ID	100.0%	63.1%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	100.0%	99.3%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	27.3%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Richter & Scholz e.K. · Köhler AG · Contract text · Gemma 4 31B 13/13, Mistral Small 4 11/13
Hahn & Schulz GmbH & Co. KG · Alpentec GmbH · Contract text · Gemma 4 31B 13/13, Mistral Small 4 12/13
Hahn GmbH · Bauer & Klein GmbH · Contract text · Gemma 4 31B 12/13, Mistral Small 4 13/13
Wesermetall e.K. · Weserform GmbH · Contract text · Gemma 4 31B 12/13, Mistral Small 4 11/13
Hansawerk GmbH · Schreinerei Weber AG · Contract text · Gemma 4 31B 12/13, Mistral Small 4 11/13
Kraus Consulting GmbH · Lange Consulting GmbH & Co. KG · Contract text · Gemma 4 31B 12/13, Mistral Small 4 13/13
Schulz Systemtechnik GmbH · Elbdata GmbH · Contract text · Gemma 4 31B 12/13, Mistral Small 4 11/13
Ostlogistik GmbH & Co. KG · Neumann GmbH · Contract text · Gemma 4 31B 12/13, Mistral Small 4 13/13