Claude Sonnet 5 vs Gemma 4 31B

Same documents, same prompt: where each model is right, what it really costs and how fast it answers.

At a glance

Task · documents	English (US)	Czech	German	Spanish	French
Invoice extraction	99.8% / 97.6%	100.0% / 96.1%	99.0% / 94.9%	93.6% / 95.4%	98.7% / 93.6%
Email triage	98.2% / 96.5%	98.2% / 97.1%	97.9% / 95.1%	—	—
Personal data detection	99.7% / 99.9%	98.9% / 99.8%	99.7% / 99.9%	—	—
Contract clauses	100.0% / 99.6%	100.0% / 100.0%	98.2% / 94.4%	—	—

Correct fields: Claude Sonnet 5 / Gemma 4 31B; the better one in bold. Click a cell to see the details.

Invoice extraction · English (US) (150 documents)

Claude Sonnet 5 is more accurate by 2.2 pp. Gemma 4 31B is 52.8× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	99.8%	97.6%
Error-free documents	99%	81%
Per 1,000 documents	$8.84	$0.17
To fix / 1,000	13	187
Speed (median)	3.2 sec	8.9 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Supplier	100.0%	99.3%
Supplier ID (EIN, IČO…)	100.0%	98.0%
Invoice number	100.0%	100.0%
Payment reference	100.0%	99.3%
Invoice date	100.0%	98.7%
Due date	100.0%	98.7%
Net by tax rate	98.7%	95.3%
Tax by rate	99.3%	98.7%
Total	100.0%	100.0%
Currency	100.0%	100.0%
Bank account	100.0%	86.0%

Documents where they differ

Harmon Logistics LLC · INV256017 · Scan · Claude Sonnet 5 11/11, Gemma 4 31B 8/11
Austin Landscaping Inc. · INV-17836 · Scan · Claude Sonnet 5 11/11, Gemma 4 31B 9/11
Gomez & Sons, LLC · 7855 · Scan · Claude Sonnet 5 11/11, Gemma 4 31B 9/11
Summit Services Inc. · INV-15230 · Scan · Claude Sonnet 5 11/11, Gemma 4 31B 10/11
Jennifer Moore · 5113 · Scan · Claude Sonnet 5 11/11, Gemma 4 31B 10/11
Mason-Nelson Group LLC · 2025-3952 · Photo · Claude Sonnet 5 11/11, Gemma 4 31B 10/11
Sarah Villanueva · 2025-7876 · Clean image · Claude Sonnet 5 11/11, Gemma 4 31B 10/11
Johnson Electric LLC · INV251373 · Photo · Claude Sonnet 5 11/11, Gemma 4 31B 10/11

Invoice extraction · Czech (150 documents)

Claude Sonnet 5 is more accurate by 3.9 pp. Gemma 4 31B is 44.4× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	100.0%	96.1%
Error-free documents	100%	67%
Per 1,000 documents	$9.41	$0.21
To fix / 1,000	0	333
Speed (median)	3.6 sec	10.4 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Supplier	100.0%	96.7%
Supplier ID (EIN, IČO…)	100.0%	93.3%
VAT ID	100.0%	96.7%
Invoice number	100.0%	100.0%
Payment reference	100.0%	96.0%
Invoice date	100.0%	96.0%
Tax point	100.0%	95.3%
Due date	100.0%	94.0%
Net by tax rate	100.0%	98.0%
Tax by rate	100.0%	97.3%
Total	100.0%	99.3%
Currency	100.0%	100.0%
Bank account	100.0%	86.7%

Documents where they differ

Jiřina Švecová · 25401227 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Bednář spol. s r.o. · 20250610 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Stejskal, v.o.s. · FV2600175 · Photo · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Ekolog s.r.o. · FV2600132 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Hruška a Šťastný s.r.o. · 2025/0324 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Moravská energetická spol. s r.o. · 25101373 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Michaela Strnadová · 25600363 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Sedlák s.r.o. · 20250158 · Photo · Claude Sonnet 5 13/13, Gemma 4 31B 12/13

Invoice extraction · German (150 documents)

Claude Sonnet 5 is more accurate by 4.1 pp. Gemma 4 31B is 52.3× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	99.0%	94.9%
Error-free documents	87%	64%
Per 1,000 documents	$10.42	$0.20
To fix / 1,000	133	360
Speed (median)	3.8 sec	7.7 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Supplier	100.0%	98.7%
Supplier ID (EIN, IČO…)	100.0%	99.3%
VAT ID	100.0%	94.0%
Invoice number	100.0%	99.3%
Payment reference	100.0%	98.7%
Invoice date	99.3%	87.3%
Tax point	100.0%	92.0%
Due date	100.0%	90.7%
Net by tax rate	89.3%	94.7%
Tax by rate	100.0%	97.3%
Total	100.0%	99.3%
Currency	100.0%	99.3%
Bank account	98.0%	82.7%

Documents where they differ

Spedition Müller GmbH · 2026-1446 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 8/13
Wesermetall GmbH · RE2501375 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 9/13
Dachdeckerei Fischer GmbH · 2025/0014 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 10/13
Schmidt & Bauer GmbH & Co. KG · RE-2025-01397 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 10/13
Zimmermann & Schulz GmbH · RE11274 · Scan · Claude Sonnet 5 12/13, Gemma 4 31B 10/13
Richter GmbH & Co. KG · 2025-0509 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Nordtec GmbH & Co. KG · RE-2025-00909 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Berger UG (haftungsbeschränkt) · GS10609 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13

Invoice extraction · Spanish (100 documents)

Gemma 4 31B is more accurate by 1.8 pp. Gemma 4 31B is 58.3× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	93.6%	95.4%
Error-free documents	77%	54%
Per 1,000 documents	$11.89	$0.20
To fix / 1,000	230	460
Speed (median)	4.0 sec	8.7 sec
Valid answers	95.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Supplier	94.0%	95.4%
Supplier ID (EIN, IČO…)	95.0%	93.1%
VAT ID	95.0%	98.9%
Invoice number	95.0%	100.0%
Payment reference	94.0%	95.4%
Invoice date	95.0%	98.9%
Tax point	95.0%	100.0%
Due date	95.0%	97.7%
Net by tax rate	79.0%	81.6%
Tax by rate	95.0%	100.0%
Total	95.0%	100.0%
Currency	95.0%	100.0%
Bank account	95.0%	79.3%

Documents where they differ

Losa Comunicación S.L. · A-2026-0862 · PDF text · Claude Sonnet 5 0/13, Gemma 4 31B 13/13
Centro de Formación del Sur S.L. · F2026-0167 · PDF text · Claude Sonnet 5 0/13, Gemma 4 31B 13/13
Coloma Servicios Informáticos S.L.U. · PRO-2025-0066 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Talavera y Aller Asociados, S.L. · 2025/0610 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Distribuciones Mayo Santana, S.L.U. · A-2025-0563 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Talleres Rozas Zapata S.L. · FV25-01486 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Construcciones del Norte S.L.U. · PF25/0020 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Centro de Formación del Sur S.L.U. · F2025-1424 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 12/13

Invoice extraction · French (97 documents)

Claude Sonnet 5 is more accurate by 5.1 pp. Gemma 4 31B is 49.7× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	98.7%	93.6%
Error-free documents	96%	51%
Per 1,000 documents	$11.02	$0.22
To fix / 1,000	41	494
Speed (median)	4.4 sec	7.2 sec
Valid answers	99.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Supplier	99.0%	96.6%
Supplier ID (EIN, IČO…)	99.0%	77.0%
VAT ID	99.0%	95.4%
Invoice number	99.0%	97.7%
Payment reference	97.9%	96.6%
Invoice date	99.0%	96.6%
Tax point	99.0%	94.3%
Due date	99.0%	96.6%
Net by tax rate	96.9%	93.1%
Tax by rate	99.0%	100.0%
Total	99.0%	100.0%
Currency	99.0%	98.9%
Bank account	97.9%	73.6%

Documents where they differ

Lefèvre & Fils SARL · FAC-2026-00198 · Photo · Claude Sonnet 5 13/13, Gemma 4 31B 8/13
Guérin & Fils EURL · 2026-05-688 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 10/13
Établissements Bertrand SASU · F2025-0066 · Scan · Claude Sonnet 5 11/13, Gemma 4 31B 13/13
Nettoyage Christine Rivière EI · 2025-12-1174 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Établissements Bonnet EURL · AV-10599 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
SARL Garcia Informatique · F2025-0111 · Clean image · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Bernard et Associés SASU · FAC-2025-01115 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 11/13
Carrosserie Gauthier SASU · F-11079 · Scan · Claude Sonnet 5 13/13, Gemma 4 31B 12/13

Email triage · English (US) (150 documents)

Claude Sonnet 5 is more accurate by 1.7 pp. Gemma 4 31B is 33.2× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	98.2%	96.5%
Error-free documents	84%	72%
Per 1,000 documents	$4.01	$0.12
To fix / 1,000	160	280
Speed (median)	3.8 sec	2.2 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Category	100.0%	100.0%
Priority	97.3%	85.3%
Sentiment	86.7%	83.3%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	100.0%
Currency	100.0%	100.0%
Deadline	100.0%	100.0%

Documents where they differ

Sheila Evans · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Gene Everett · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Laurie Knight · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Jason Davis · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Daniel Stewart · Email · Claude Sonnet 5 8/9, Gemma 4 31B 7/9
Allison Floyd · Email · Claude Sonnet 5 8/9, Gemma 4 31B 7/9
Christine Copeland · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Angela Harmon · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9

Email triage · Czech (150 documents)

Claude Sonnet 5 is more accurate by 1 pp. Gemma 4 31B is 30.9× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	98.2%	97.1%
Error-free documents	83%	75%
Per 1,000 documents	$4.91	$0.16
To fix / 1,000	167	253
Speed (median)	4.7 sec	2.5 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Category	100.0%	100.0%
Priority	96.0%	85.3%
Sentiment	87.3%	88.7%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	100.0%
Currency	100.0%	100.0%
Deadline	100.0%	100.0%

Documents where they differ

Alois Sedlák · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Petra Šťastná · Email · Claude Sonnet 5 8/9, Gemma 4 31B 9/9
Natálie Kratochvílová · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Václav Vítek · Email · Claude Sonnet 5 8/9, Gemma 4 31B 9/9
Šimon Beran · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Magdalena Sedláčková · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Ctibor Dušek · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Růžena Sedláková · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9

Email triage · German (150 documents)

Claude Sonnet 5 is more accurate by 2.7 pp. Gemma 4 31B is 34.5× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	97.9%	95.1%
Error-free documents	81%	62%
Per 1,000 documents	$5.11	$0.15
To fix / 1,000	187	380
Speed (median)	4.3 sec	3.6 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Category	100.0%	100.0%
Priority	96.0%	78.0%
Sentiment	84.7%	78.0%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	100.0%
Currency	100.0%	100.0%
Deadline	100.0%	100.0%

Documents where they differ

Rolf-Peter Metz · Email · Claude Sonnet 5 7/9, Gemma 4 31B 8/9
Alexej Sauer · Email · Claude Sonnet 5 8/9, Gemma 4 31B 7/9
Sylwia Gerlach · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Benedikt Speer · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Cristina Krause · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Cynthia Hahn · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Roselinde Heinz · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9
Linda Bähr · Email · Claude Sonnet 5 9/9, Gemma 4 31B 8/9

Personal data detection · English (US) (150 documents)

Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 41.4× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	99.7%	99.9%
Error-free documents	98%	99%
Per 1,000 documents	$3.50	$0.08
To fix / 1,000	20	7
Speed (median)	2.5 sec	2.4 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Names	100.0%	100.0%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	98.0%	99.3%
Dates of birth	100.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Both models got the same number of fields right on every example document.

Personal data detection · Czech (150 documents)

Gemma 4 31B is more accurate by 0.9 pp. Gemma 4 31B is 34.1× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	98.9%	99.8%
Error-free documents	92%	99%
Per 1,000 documents	$4.06	$0.12
To fix / 1,000	80	13
Speed (median)	3.2 sec	2.9 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Names	96.7%	100.0%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	97.3%	98.7%
Dates of birth	98.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Růžena Tichá · Text · Claude Sonnet 5 6/7, Gemma 4 31B 7/7
Jonáš Toman · Text · Claude Sonnet 5 6/7, Gemma 4 31B 7/7
Rudolf Konečný · Text · Claude Sonnet 5 6/7, Gemma 4 31B 7/7
Zikmund Sedláček · Text · Claude Sonnet 5 6/7, Gemma 4 31B 7/7

Personal data detection · German (150 documents)

Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 43.2× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	99.7%	99.9%
Error-free documents	98%	99%
Per 1,000 documents	$4.40	$0.10
To fix / 1,000	20	7
Speed (median)	3.7 sec	1.9 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Names	98.7%	100.0%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	100.0%	99.3%
Dates of birth	99.3%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Gunnar Zorbach · Text · Claude Sonnet 5 7/7, Gemma 4 31B 6/7

Contract clauses · English (US) (150 documents)

Claude Sonnet 5 is more accurate by 0.4 pp. Gemma 4 31B is 28× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	100.0%	99.6%
Error-free documents	100%	97%
Per 1,000 documents	$5.04	$0.18
To fix / 1,000	0	27
Speed (median)	3.4 sec	2.5 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Provider	100.0%	98.0%
Provider ID	100.0%	100.0%
Client	100.0%	98.0%
Client ID	100.0%	100.0%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	100.0%	99.3%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Both models got the same number of fields right on every example document.

Contract clauses · Czech (150 documents)

Both are equally accurate. Gemma 4 31B is 23.8× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	100.0%	100.0%
Error-free documents	100%	99%
Per 1,000 documents	$6.05	$0.25
To fix / 1,000	0	7
Speed (median)	3.6 sec	6.0 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Provider	100.0%	100.0%
Provider ID	100.0%	100.0%
Client	100.0%	100.0%
Client ID	100.0%	100.0%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	100.0%	99.3%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Elektrotrade s.r.o. · Konečný a Dostál spol. s r.o. · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13

Contract clauses · German (150 documents)

Claude Sonnet 5 is more accurate by 3.8 pp. Gemma 4 31B is 32.3× cheaper.

Metric	Claude Sonnet 5	Gemma 4 31B
Correct fields	98.2%	94.4%
Error-free documents	87%	27%
Per 1,000 documents	$7.11	$0.22
To fix / 1,000	133	727
Speed (median)	4.3 sec	2.6 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Claude Sonnet 5	Gemma 4 31B
Provider	100.0%	100.0%
Provider ID	87.3%	100.0%
Client	100.0%	100.0%
Client ID	89.3%	100.0%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	100.0%	100.0%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	27.3%
Governing law	100.0%	100.0%

Documents where they differ

Hahn GmbH · Bauer & Klein GmbH · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Wesermetall e.K. · Weserform GmbH · Contract text · Claude Sonnet 5 11/13, Gemma 4 31B 12/13
Haustechnik Braun UG (haftungsbeschränkt) · Nordbau GmbH · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Hansawerk GmbH · Schreinerei Weber AG · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Maintec GmbH · Schreinerei Wagner GmbH · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Winkler & Beck e.K. · Meyer & Weber e.K. · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Kraus Consulting GmbH · Lange Consulting GmbH & Co. KG · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13
Hansametall GmbH & Co. KG · Roth & Schröder UG (haftungsbeschränkt) · Contract text · Claude Sonnet 5 13/13, Gemma 4 31B 12/13