ModelVerdict

Comparison

Gemma 4 31B vs Qwen3.8 Flash

Same documents, same prompt: where each model is right, what it really costs and how fast it answers.

At a glance

Task · documentsEnglish (US)CzechGermanSpanishFrench
Invoice extraction97.6% / 82.4%96.1% / 91.8%94.9% / 93.5%95.4% / 87.9%93.6% / 90.6%
Email triage96.5% / 97.5%97.1% / 98.5%95.1% / 97.9%——
Personal data detection99.9% / 99.2%99.8% / 97.0%99.9% / 99.6%——
Contract clauses99.6% / 95.1%100.0% / 99.9%94.4% / 98.0%——

Correct fields: Gemma 4 31B / Qwen3.8 Flash; the better one in bold. Click a cell to see the details.

Invoice extraction · English (US) (150 documents)

Gemma 4 31B is more accurate by 15.3 pp. Gemma 4 31B is 6.9× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields97.6%82.4%
Error-free documents81%57%
Per 1,000 documents$0.17$1.15
To fix / 1,000187427
Speed (median)8.9 sec17.4 sec
Valid answers100.0%90.7%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Supplier99.3%85.3%
Supplier ID (EIN, IČO…)98.0%88.7%
Invoice number100.0%85.3%
Payment reference99.3%79.3%
Invoice date98.7%85.3%
Due date98.7%85.3%
Net by tax rate95.3%70.0%
Tax by rate98.7%76.0%
Total100.0%85.3%
Currency100.0%84.0%
Bank account86.0%81.3%

Documents where they differ

Invoice extraction · Czech (150 documents)

Gemma 4 31B is more accurate by 4.3 pp. Gemma 4 31B is 4.8× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields96.1%91.8%
Error-free documents67%77%
Per 1,000 documents$0.21$1.02
To fix / 1,000333233
Speed (median)10.4 sec17.0 sec
Valid answers100.0%96.7%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Supplier96.7%93.3%
Supplier ID (EIN, IČO…)93.3%94.0%
VAT ID96.7%94.0%
Invoice number100.0%94.7%
Payment reference96.0%94.0%
Invoice date96.0%94.7%
Tax point95.3%94.7%
Due date94.0%94.7%
Net by tax rate98.0%80.0%
Tax by rate97.3%80.7%
Total99.3%94.0%
Currency100.0%94.0%
Bank account86.7%90.7%

Documents where they differ

Invoice extraction · German (150 documents)

Gemma 4 31B is more accurate by 1.4 pp. Gemma 4 31B is 5.6× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields94.9%93.5%
Error-free documents64%87%
Per 1,000 documents$0.20$1.12
To fix / 1,000360127
Speed (median)7.7 sec19.0 sec
Valid answers100.0%94.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Supplier98.7%94.0%
Supplier ID (EIN, IČO…)99.3%94.0%
VAT ID94.0%94.0%
Invoice number99.3%92.7%
Payment reference98.7%88.7%
Invoice date87.3%94.0%
Tax point92.0%94.0%
Due date90.7%94.0%
Net by tax rate94.7%94.0%
Tax by rate97.3%94.0%
Total99.3%94.0%
Currency99.3%94.0%
Bank account82.7%94.0%

Documents where they differ

Invoice extraction · Spanish (87 documents)

Gemma 4 31B is more accurate by 7.5 pp. Gemma 4 31B is 6.8× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields95.4%87.9%
Error-free documents54%74%
Per 1,000 documents$0.20$1.40
To fix / 1,000460260
Speed (median)8.7 sec22.3 sec
Valid answers100.0%90.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Supplier95.4%86.0%
Supplier ID (EIN, IČO…)93.1%89.0%
VAT ID98.9%86.0%
Invoice number100.0%89.0%
Payment reference95.4%85.0%
Invoice date98.9%89.0%
Tax point100.0%87.0%
Due date97.7%89.0%
Net by tax rate81.6%86.0%
Tax by rate100.0%90.0%
Total100.0%89.0%
Currency100.0%89.0%
Bank account79.3%89.0%

Documents where they differ

Invoice extraction · French (87 documents)

Gemma 4 31B is more accurate by 2.9 pp. Gemma 4 31B is 5.5× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields93.6%90.6%
Error-free documents51%82%
Per 1,000 documents$0.22$1.23
To fix / 1,000494180
Speed (median)7.2 sec19.8 sec
Valid answers100.0%95.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Supplier96.6%91.0%
Supplier ID (EIN, IČO…)77.0%91.0%
VAT ID95.4%91.0%
Invoice number97.7%90.0%
Payment reference96.6%86.0%
Invoice date96.6%91.0%
Tax point94.3%92.0%
Due date96.6%89.0%
Net by tax rate93.1%91.0%
Tax by rate100.0%94.0%
Total100.0%91.0%
Currency98.9%91.0%
Bank account73.6%90.0%

Documents where they differ

Email triage · English (US) (150 documents)

Qwen3.8 Flash is more accurate by 1 pp. Gemma 4 31B is 2.5× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields96.5%97.5%
Error-free documents72%79%
Per 1,000 documents$0.12$0.30
To fix / 1,000280213
Speed (median)2.2 sec8.9 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Category100.0%100.0%
Priority85.3%97.3%
Sentiment83.3%80.7%
Customer name100.0%100.0%
Order number100.0%100.0%
Invoice number100.0%100.0%
Amount100.0%100.0%
Currency100.0%100.0%
Deadline100.0%99.3%

Documents where they differ

Email triage · Czech (150 documents)

Qwen3.8 Flash is more accurate by 1.4 pp. Gemma 4 31B is 2.8× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields97.1%98.5%
Error-free documents75%87%
Per 1,000 documents$0.16$0.45
To fix / 1,000253133
Speed (median)2.5 sec13.0 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Category100.0%100.0%
Priority85.3%99.3%
Sentiment88.7%88.7%
Customer name100.0%99.3%
Order number100.0%99.3%
Invoice number100.0%100.0%
Amount100.0%100.0%
Currency100.0%100.0%
Deadline100.0%100.0%

Documents where they differ

Email triage · German (150 documents)

Qwen3.8 Flash is more accurate by 2.7 pp. Gemma 4 31B is 2.6× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields95.1%97.9%
Error-free documents62%81%
Per 1,000 documents$0.15$0.39
To fix / 1,000380187
Speed (median)3.6 sec12.1 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Category100.0%100.0%
Priority78.0%96.0%
Sentiment78.0%84.7%
Customer name100.0%100.0%
Order number100.0%100.0%
Invoice number100.0%100.0%
Amount100.0%100.0%
Currency100.0%100.0%
Deadline100.0%100.0%

Documents where they differ

Personal data detection · English (US) (150 documents)

Gemma 4 31B is more accurate by 0.7 pp. Gemma 4 31B is 3.3× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields99.9%99.2%
Error-free documents99%99%
Per 1,000 documents$0.08$0.28
To fix / 1,000713
Speed (median)2.4 sec7.5 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Names100.0%99.3%
Email addresses100.0%99.3%
Phone numbers100.0%99.3%
Addresses99.3%98.7%
Dates of birth100.0%99.3%
National IDs100.0%99.3%
Bank accounts100.0%99.3%

Documents where they differ

Both models got the same number of fields right on every example document.

Personal data detection · Czech (150 documents)

Gemma 4 31B is more accurate by 2.9 pp. Gemma 4 31B is 3.4× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields99.8%97.0%
Error-free documents99%89%
Per 1,000 documents$0.12$0.41
To fix / 1,00013113
Speed (median)2.9 sec9.6 sec
Valid answers100.0%99.3%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Names100.0%90.7%
Email addresses100.0%98.0%
Phone numbers100.0%97.3%
Addresses98.7%96.7%
Dates of birth100.0%98.0%
National IDs100.0%99.3%
Bank accounts100.0%98.7%

Documents where they differ

Personal data detection · German (150 documents)

Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 3.5× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields99.9%99.6%
Error-free documents99%99%
Per 1,000 documents$0.10$0.36
To fix / 1,00077
Speed (median)1.9 sec12.2 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Names100.0%99.3%
Email addresses100.0%99.3%
Phone numbers100.0%99.3%
Addresses99.3%99.3%
Dates of birth100.0%100.0%
National IDs100.0%100.0%
Bank accounts100.0%100.0%

Documents where they differ

Contract clauses · English (US) (150 documents)

Gemma 4 31B is more accurate by 4.5 pp. Gemma 4 31B is 3.4× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields99.6%95.1%
Error-free documents97%69%
Per 1,000 documents$0.18$0.62
To fix / 1,00027307
Speed (median)2.5 sec14.1 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Provider98.0%69.3%
Provider ID100.0%100.0%
Client98.0%69.3%
Client ID100.0%100.0%
Contract type100.0%100.0%
Price (net)100.0%100.0%
Currency100.0%100.0%
Billing period100.0%100.0%
Effective date99.3%99.3%
End date100.0%100.0%
Notice period (months)100.0%100.0%
Late-payment penalty (% per day)100.0%99.3%
Governing law100.0%99.3%

Documents where they differ

Contract clauses · Czech (150 documents)

Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 2.5× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields100.0%99.9%
Error-free documents99%99%
Per 1,000 documents$0.25$0.63
To fix / 1,000713
Speed (median)6.0 sec13.6 sec
Valid answers100.0%100.0%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Provider100.0%100.0%
Provider ID100.0%100.0%
Client100.0%100.0%
Client ID100.0%100.0%
Contract type100.0%98.7%
Price (net)100.0%100.0%
Currency100.0%100.0%
Billing period100.0%100.0%
Effective date99.3%100.0%
End date100.0%100.0%
Notice period (months)100.0%100.0%
Late-payment penalty (% per day)100.0%100.0%
Governing law100.0%100.0%

Documents where they differ

Contract clauses · German (150 documents)

Qwen3.8 Flash is more accurate by 3.6 pp. Gemma 4 31B is 3.5× cheaper.

MetricGemma 4 31BQwen3.8 Flash
Correct fields94.4%98.0%
Error-free documents27%87%
Per 1,000 documents$0.22$0.78
To fix / 1,000727127
Speed (median)2.6 sec19.7 sec
Valid answers100.0%99.3%

Accuracy per field

FieldGemma 4 31BQwen3.8 Flash
Provider100.0%99.3%
Provider ID100.0%94.7%
Client100.0%99.3%
Client ID100.0%94.7%
Contract type100.0%99.3%
Price (net)100.0%99.3%
Currency100.0%99.3%
Billing period100.0%99.3%
Effective date100.0%99.3%
End date100.0%99.3%
Notice period (months)100.0%99.3%
Late-payment penalty (% per day)27.3%99.3%
Governing law100.0%91.3%

Documents where they differ