Gemma 4 31B vs Mistral Small 4
Same documents, same prompt: where each model gets it right, what it really costs, and how fast it responds.
Verdict
Gemma 4 31B is more accurate; Mistral Small 4 is cheaper in 15 of 30 combinations.
More accurate
Gemma 4 31B
wins 21 of 30
Cheaper
Mistral Small 4
cheaper in 15 of 30
Faster
Mistral Small 4
1.0 sec vs 2.9 sec
Fewer fixes
Gemma 4 31B
27 vs 100 documents per 1,000
Across 30 task and document-language combinations (speed and fixes: median).
At a glance
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | Mistral Small 4 | Difference | Gemma 4 31B | Mistral Small 4 | Difference |
| Invoice extraction | 97.64% | 95.58% | ▼ 2.06 | $0.17 | $0.41 | +143 % |
| Email triage | 96.52% | 89.70% | ▼ 6.82 | $0.12 | $0.09 | −21 % |
| Personal data detection | 99.90% | 99.43% | ▼ 0.47 | $0.08 | $0.08 | −9 % |
| Contract clauses | 99.64% | 91.54% | ▼ 8.1 | $0.18 | $0.17 | −6 % |
| Document summaries | 100.00% | 99.67% | ▼ 0.33 | $0.13 | $0.14 | +12 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.15 | $0.15 | −2 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | Mistral Small 4 | Difference | Gemma 4 31B | Mistral Small 4 | Difference |
| Invoice extraction | 96.10% | 93.64% | ▼ 2.46 | $0.21 | $0.44 | +106 % |
| Email triage | 97.11% | 87.63% | ▼ 9.48 | $0.16 | $0.12 | −26 % |
| Personal data detection | 99.81% | 98.76% | ▼ 1.05 | $0.12 | $0.09 | −22 % |
| Contract clauses | 99.95% | 99.03% | ▼ 0.92 | $0.25 | $0.23 | −11 % |
| Document summaries | 100.00% | 99.00% | ▼ 1 | $0.19 | $0.22 | +17 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.21 | $0.21 | +2 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | Mistral Small 4 | Difference | Gemma 4 31B | Mistral Small 4 | Difference |
| Invoice extraction | 94.87% | 94.82% | ▼ 0.05 | $0.20 | $0.44 | +119 % |
| Email triage | 95.11% | 87.26% | ▼ 7.85 | $0.15 | $0.11 | −27 % |
| Personal data detection | 99.90% | 99.14% | ▼ 0.76 | $0.10 | $0.09 | −11 % |
| Contract clauses | 94.41% | 94.26% | ▼ 0.15 | $0.22 | $0.21 | −6 % |
| Document summaries | 100.00% | 99.89% | ▼ 0.11 | $0.16 | $0.19 | +15 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.18 | $0.18 | −3 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | Mistral Small 4 | Difference | Gemma 4 31B | Mistral Small 4 | Difference |
| Invoice extraction | 94.62% | 92.36% | ▼ 2.26 | $0.20 | $0.44 | +119 % |
| Email triage | 97.11% | 88.67% | ▼ 8.44 | $0.15 | $0.11 | −24 % |
| Personal data detection | 99.90% | 99.05% | ▼ 0.85 | $0.10 | $0.09 | −9 % |
| Contract clauses | 94.41% | 100.00% | ▲ 5.59 | $0.24 | $0.23 | −2 % |
| Document summaries | 99.67% | 97.78% | ▼ 1.89 | $0.16 | $0.18 | +16 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.18 | $0.18 | +1 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | Mistral Small 4 | Difference | Gemma 4 31B | Mistral Small 4 | Difference |
| Invoice extraction | 93.54% | 94.87% | ▲ 1.33 | $0.22 | $0.44 | +101 % |
| Email triage | 96.22% | 90.52% | ▼ 5.7 | $0.16 | $0.11 | −29 % |
| Personal data detection | 99.81% | 98.95% | ▼ 0.86 | $0.12 | $0.09 | −21 % |
| Contract clauses | 94.62% | 99.23% | ▲ 4.61 | $0.26 | $0.23 | −9 % |
| Document summaries | 99.11% | 93.22% | ▼ 5.89 | $0.16 | $0.19 | +16 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.19 | $0.18 | −5 % |
Difference: Mistral Small 4 vs. Gemma 4 31B; accuracy in percentage points, price in percent. The better value is in bold. Click a task to see the details.
Invoice extraction · English (US) (150 documents)
Gemma 4 31B is more accurate by 2.1 pp. Gemma 4 31B is 2.4× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 97.64% | 95.58% |
| Error-free documents | 81.33% | 58.00% |
| Per 1,000 documents | $0.17 | $0.41 |
| To fix / 1,000 | 187 | 420 |
| Speed (median) | 8.9 sec | 1.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Czech (150 documents)
Gemma 4 31B is more accurate by 2.5 pp. Gemma 4 31B is 2.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 96.10% | 93.64% |
| Error-free documents | 66.67% | 56.00% |
| Per 1,000 documents | $0.21 | $0.44 |
| To fix / 1,000 | 333 | 440 |
| Speed (median) | 10.4 sec | 2.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · German (150 documents)
Both are equally accurate. Gemma 4 31B is 2.2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 94.87% | 94.82% |
| Error-free documents | 64.00% | 51.33% |
| Per 1,000 documents | $0.20 | $0.44 |
| To fix / 1,000 | 360 | 487 |
| Speed (median) | 7.7 sec | 2.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Spanish (150 documents)
Gemma 4 31B is more accurate by 2.3 pp. Gemma 4 31B is 2.2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 94.62% | 92.36% |
| Error-free documents | 50.67% | 32.67% |
| Per 1,000 documents | $0.20 | $0.44 |
| To fix / 1,000 | 493 | 673 |
| Speed (median) | 7.4 sec | 2.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · French (150 documents)
Mistral Small 4 is more accurate by 1.3 pp. Gemma 4 31B is 2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 93.54% | 94.87% |
| Error-free documents | 50.67% | 52.67% |
| Per 1,000 documents | $0.22 | $0.44 |
| To fix / 1,000 | 493 | 473 |
| Speed (median) | 6.5 sec | 2.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · English (US) (150 documents)
Gemma 4 31B is more accurate by 6.8 pp. Mistral Small 4 is 1.3× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 96.52% | 89.70% |
| Error-free documents | 72.00% | 37.33% |
| Per 1,000 documents | $0.12 | $0.09 |
| To fix / 1,000 | 280 | 627 |
| Speed (median) | 2.2 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Czech (150 documents)
Gemma 4 31B is more accurate by 9.5 pp. Mistral Small 4 is 1.3× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 97.11% | 87.63% |
| Error-free documents | 74.67% | 31.33% |
| Per 1,000 documents | $0.16 | $0.12 |
| To fix / 1,000 | 253 | 687 |
| Speed (median) | 2.5 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · German (150 documents)
Gemma 4 31B is more accurate by 7.8 pp. Mistral Small 4 is 1.4× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 95.11% | 87.26% |
| Error-free documents | 62.00% | 29.33% |
| Per 1,000 documents | $0.15 | $0.11 |
| To fix / 1,000 | 380 | 707 |
| Speed (median) | 3.6 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Spanish (150 documents)
Gemma 4 31B is more accurate by 8.4 pp. Mistral Small 4 is 1.3× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 97.11% | 88.67% |
| Error-free documents | 74.00% | 35.33% |
| Per 1,000 documents | $0.15 | $0.11 |
| To fix / 1,000 | 260 | 647 |
| Speed (median) | 2.2 sec | 0.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · French (150 documents)
Gemma 4 31B is more accurate by 5.7 pp. Mistral Small 4 is 1.4× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 96.22% | 90.52% |
| Error-free documents | 68.67% | 44.67% |
| Per 1,000 documents | $0.16 | $0.11 |
| To fix / 1,000 | 313 | 553 |
| Speed (median) | 2.4 sec | 0.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.5 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.90% | 99.43% |
| Error-free documents | 99.33% | 96.00% |
| Per 1,000 documents | $0.08 | $0.08 |
| To fix / 1,000 | 7 | 40 |
| Speed (median) | 2.4 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Czech (150 documents)
Gemma 4 31B is more accurate by 1 pp. Mistral Small 4 is 1.3× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.81% | 98.76% |
| Error-free documents | 98.67% | 92.00% |
| Per 1,000 documents | $0.12 | $0.09 |
| To fix / 1,000 | 13 | 80 |
| Speed (median) | 2.9 sec | 0.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · German (150 documents)
Gemma 4 31B is more accurate by 0.8 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.90% | 99.14% |
| Error-free documents | 99.33% | 94.00% |
| Per 1,000 documents | $0.10 | $0.09 |
| To fix / 1,000 | 7 | 60 |
| Speed (median) | 1.9 sec | 0.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Spanish (150 documents)
Gemma 4 31B is more accurate by 0.8 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.90% | 99.05% |
| Error-free documents | 99.33% | 93.33% |
| Per 1,000 documents | $0.10 | $0.09 |
| To fix / 1,000 | 7 | 67 |
| Speed (median) | 3.6 sec | 0.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · French (150 documents)
Gemma 4 31B is more accurate by 0.9 pp. Mistral Small 4 is 1.3× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.81% | 98.95% |
| Error-free documents | 98.67% | 93.33% |
| Per 1,000 documents | $0.12 | $0.09 |
| To fix / 1,000 | 13 | 67 |
| Speed (median) | 4.3 sec | 1.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · English (US) (150 documents)
Gemma 4 31B is more accurate by 8.1 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.64% | 91.54% |
| Error-free documents | 97.33% | 43.33% |
| Per 1,000 documents | $0.18 | $0.17 |
| To fix / 1,000 | 27 | 567 |
| Speed (median) | 2.5 sec | 1.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Czech (150 documents)
Gemma 4 31B is more accurate by 0.9 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.95% | 99.03% |
| Error-free documents | 99.33% | 92.00% |
| Per 1,000 documents | $0.25 | $0.23 |
| To fix / 1,000 | 7 | 80 |
| Speed (median) | 6.0 sec | 1.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · German (150 documents)
Gemma 4 31B is more accurate by 0.2 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 94.41% | 94.26% |
| Error-free documents | 27.33% | 46.67% |
| Per 1,000 documents | $0.22 | $0.21 |
| To fix / 1,000 | 727 | 533 |
| Speed (median) | 2.6 sec | 1.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Spanish (150 documents)
Mistral Small 4 is more accurate by 5.6 pp.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 94.41% | 100.00% |
| Error-free documents | 27.33% | 100.00% |
| Per 1,000 documents | $0.24 | $0.23 |
| To fix / 1,000 | 727 | 0 |
| Speed (median) | 5.0 sec | 1.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · French (150 documents)
Mistral Small 4 is more accurate by 4.6 pp. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 94.62% | 99.23% |
| Error-free documents | 30.67% | 90.00% |
| Per 1,000 documents | $0.26 | $0.23 |
| To fix / 1,000 | 693 | 100 |
| Speed (median) | 5.3 sec | 1.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 99.67% |
| Error-free documents | 100.00% | 98.67% |
| Per 1,000 documents | $0.13 | $0.14 |
| To fix / 1,000 | 0 | 13 |
| Speed (median) | 2.2 sec | 1.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Czech (150 documents)
Gemma 4 31B is more accurate by 1 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 99.00% |
| Error-free documents | 100.00% | 94.00% |
| Per 1,000 documents | $0.19 | $0.22 |
| To fix / 1,000 | 0 | 60 |
| Speed (median) | 3.4 sec | 1.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · German (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 99.89% |
| Error-free documents | 100.00% | 99.33% |
| Per 1,000 documents | $0.16 | $0.19 |
| To fix / 1,000 | 0 | 7 |
| Speed (median) | 2.9 sec | 1.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Spanish (150 documents)
Gemma 4 31B is more accurate by 1.9 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.67% | 97.78% |
| Error-free documents | 98.00% | 86.67% |
| Per 1,000 documents | $0.16 | $0.18 |
| To fix / 1,000 | 20 | 133 |
| Speed (median) | 2.6 sec | 1.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · French (150 documents)
Gemma 4 31B is more accurate by 5.9 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 99.11% | 93.22% |
| Error-free documents | 94.67% | 62.00% |
| Per 1,000 documents | $0.16 | $0.19 |
| To fix / 1,000 | 53 | 380 |
| Speed (median) | 3.2 sec | 1.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · English (US) (150 documents)
Both are equally accurate.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.15 | $0.15 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.1 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Czech (150 documents)
Both are equally accurate.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.21 | $0.21 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 1.9 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · German (150 documents)
Both are equally accurate.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.18 | $0.18 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.1 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Spanish (150 documents)
Both are equally accurate.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.18 | $0.18 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 1.9 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · French (150 documents)
Both are equally accurate. Mistral Small 4 is 1.1× cheaper.
| Metric | Gemma 4 31B | Mistral Small 4 |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.19 | $0.18 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.1 sec | 0.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.