Gemini 3.5 Flash Lite vs Gemma 4 31B
Same documents, same prompt: where each model gets it right, what it really costs, and how fast it responds.
Verdict
Gemma 4 31B is more accurate and cheaper in 30 of 30 combinations.
More accurate
Gemma 4 31B
wins 17 of 30
Cheaper
Gemma 4 31B
cheaper in 30 of 30
Faster
Gemini 3.5 Flash Lite
0.9 sec vs 2.9 sec
Fewer fixes
Gemma 4 31B
27 vs 67 documents per 1,000
Across 30 task and document-language combinations (speed and fixes: median).
At a glance
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference |
| Invoice extraction | 99.21% | 97.64% | ▼ 1.57 | $0.87 | $0.17 | −81 % |
| Email triage | 96.30% | 96.52% | ▲ 0.22 | $0.42 | $0.12 | −71 % |
| Personal data detection | 99.52% | 99.90% | ▲ 0.38 | $0.41 | $0.08 | −79 % |
| Contract clauses | 99.95% | 99.64% | ▼ 0.31 | $0.69 | $0.18 | −74 % |
| Document summaries | 99.22% | 100.00% | ▲ 0.78 | $0.46 | $0.13 | −73 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.46 | $0.15 | −66 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference |
| Invoice extraction | 99.49% | 96.10% | ▼ 3.39 | $1.03 | $0.21 | −79 % |
| Email triage | 95.78% | 97.11% | ▲ 1.33 | $0.51 | $0.16 | −69 % |
| Personal data detection | 98.10% | 99.81% | ▲ 1.71 | $0.45 | $0.12 | −74 % |
| Contract clauses | 99.85% | 99.95% | ▲ 0.1 | $0.85 | $0.25 | −70 % |
| Document summaries | 99.56% | 100.00% | ▲ 0.44 | $0.69 | $0.19 | −73 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.58 | $0.21 | −64 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference |
| Invoice extraction | 100.00% | 94.87% | ▼ 5.13 | $1.03 | $0.20 | −81 % |
| Email triage | 94.67% | 95.11% | ▲ 0.44 | $0.48 | $0.15 | −69 % |
| Personal data detection | 99.62% | 99.90% | ▲ 0.28 | $0.45 | $0.10 | −78 % |
| Contract clauses | 98.00% | 94.41% | ▼ 3.59 | $0.79 | $0.22 | −72 % |
| Document summaries | 99.89% | 100.00% | ▲ 0.11 | $0.59 | $0.16 | −72 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.52 | $0.18 | −65 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference |
| Invoice extraction | 98.21% | 94.62% | ▼ 3.59 | $1.05 | $0.20 | −81 % |
| Email triage | 95.56% | 97.11% | ▲ 1.55 | $0.48 | $0.15 | −70 % |
| Personal data detection | 95.81% | 99.90% | ▲ 4.09 | $0.45 | $0.10 | −77 % |
| Contract clauses | 99.79% | 94.41% | ▼ 5.38 | $0.84 | $0.24 | −72 % |
| Document summaries | 95.33% | 99.67% | ▲ 4.34 | $0.56 | $0.16 | −72 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.51 | $0.18 | −65 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference | Gemini 3.5 Flash Lite | Gemma 4 31B | Difference |
| Invoice extraction | 98.97% | 93.54% | ▼ 5.43 | $1.07 | $0.22 | −79 % |
| Email triage | 96.15% | 96.22% | ▲ 0.07 | $0.49 | $0.16 | −67 % |
| Personal data detection | 94.95% | 99.81% | ▲ 4.86 | $0.46 | $0.12 | −74 % |
| Contract clauses | 99.95% | 94.62% | ▼ 5.33 | $0.85 | $0.26 | −70 % |
| Document summaries | 94.67% | 99.11% | ▲ 4.44 | $0.65 | $0.16 | −75 % |
| Questions about a document | 99.56% | 100.00% | ▲ 0.44 | $0.53 | $0.19 | −64 % |
Difference: Gemma 4 31B vs. Gemini 3.5 Flash Lite; accuracy in percentage points, price in percent. The better value is in bold. Click a task to see the details.
Invoice extraction · English (US) (150 documents)
Gemini 3.5 Flash Lite is more accurate by 1.6 pp. Gemma 4 31B is 5.2× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.21% | 97.64% |
| Error-free documents | 92.00% | 81.33% |
| Per 1,000 documents | $0.87 | $0.17 |
| To fix / 1,000 | 80 | 187 |
| Speed (median) | 1.7 sec | 8.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Czech (150 documents)
Gemini 3.5 Flash Lite is more accurate by 3.4 pp. Gemma 4 31B is 4.9× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.49% | 96.10% |
| Error-free documents | 93.33% | 66.67% |
| Per 1,000 documents | $1.03 | $0.21 |
| To fix / 1,000 | 67 | 333 |
| Speed (median) | 1.8 sec | 10.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · German (150 documents)
Gemini 3.5 Flash Lite is more accurate by 5.1 pp. Gemma 4 31B is 5.2× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 100.00% | 94.87% |
| Error-free documents | 100.00% | 64.00% |
| Per 1,000 documents | $1.03 | $0.20 |
| To fix / 1,000 | 0 | 360 |
| Speed (median) | 1.5 sec | 7.7 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Spanish (150 documents)
Gemini 3.5 Flash Lite is more accurate by 3.6 pp. Gemma 4 31B is 5.2× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 98.21% | 94.62% |
| Error-free documents | 78.00% | 50.67% |
| Per 1,000 documents | $1.05 | $0.20 |
| To fix / 1,000 | 220 | 493 |
| Speed (median) | 1.6 sec | 7.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · French (150 documents)
Gemini 3.5 Flash Lite is more accurate by 5.4 pp. Gemma 4 31B is 4.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 98.97% | 93.54% |
| Error-free documents | 88.00% | 50.67% |
| Per 1,000 documents | $1.07 | $0.22 |
| To fix / 1,000 | 120 | 493 |
| Speed (median) | 1.5 sec | 6.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 3.5× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 96.30% | 96.52% |
| Error-free documents | 68.67% | 72.00% |
| Per 1,000 documents | $0.42 | $0.12 |
| To fix / 1,000 | 313 | 280 |
| Speed (median) | 0.7 sec | 2.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Czech (150 documents)
Gemma 4 31B is more accurate by 1.3 pp. Gemma 4 31B is 3.2× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 95.78% | 97.11% |
| Error-free documents | 64.00% | 74.67% |
| Per 1,000 documents | $0.51 | $0.16 |
| To fix / 1,000 | 360 | 253 |
| Speed (median) | 0.7 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · German (150 documents)
Gemma 4 31B is more accurate by 0.4 pp. Gemma 4 31B is 3.2× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 94.67% | 95.11% |
| Error-free documents | 56.67% | 62.00% |
| Per 1,000 documents | $0.48 | $0.15 |
| To fix / 1,000 | 433 | 380 |
| Speed (median) | 0.8 sec | 3.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Spanish (150 documents)
Gemma 4 31B is more accurate by 1.5 pp. Gemma 4 31B is 3.3× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 95.56% | 97.11% |
| Error-free documents | 65.33% | 74.00% |
| Per 1,000 documents | $0.48 | $0.15 |
| To fix / 1,000 | 347 | 260 |
| Speed (median) | 0.8 sec | 2.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · French (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 3.1× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 96.15% | 96.22% |
| Error-free documents | 68.67% | 68.67% |
| Per 1,000 documents | $0.49 | $0.16 |
| To fix / 1,000 | 313 | 313 |
| Speed (median) | 0.8 sec | 2.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.4 pp. Gemma 4 31B is 4.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.52% | 99.90% |
| Error-free documents | 97.33% | 99.33% |
| Per 1,000 documents | $0.41 | $0.08 |
| To fix / 1,000 | 27 | 7 |
| Speed (median) | 0.8 sec | 2.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Czech (150 documents)
Gemma 4 31B is more accurate by 1.7 pp. Gemma 4 31B is 3.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 98.10% | 99.81% |
| Error-free documents | 88.00% | 98.67% |
| Per 1,000 documents | $0.45 | $0.12 |
| To fix / 1,000 | 120 | 13 |
| Speed (median) | 0.8 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · German (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 4.5× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.62% | 99.90% |
| Error-free documents | 97.33% | 99.33% |
| Per 1,000 documents | $0.45 | $0.10 |
| To fix / 1,000 | 27 | 7 |
| Speed (median) | 0.8 sec | 1.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Spanish (150 documents)
Gemma 4 31B is more accurate by 4.1 pp. Gemma 4 31B is 4.3× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 95.81% | 99.90% |
| Error-free documents | 73.33% | 99.33% |
| Per 1,000 documents | $0.45 | $0.10 |
| To fix / 1,000 | 267 | 7 |
| Speed (median) | 0.9 sec | 3.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · French (150 documents)
Gemma 4 31B is more accurate by 4.9 pp. Gemma 4 31B is 3.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 94.95% | 99.81% |
| Error-free documents | 67.33% | 98.67% |
| Per 1,000 documents | $0.46 | $0.12 |
| To fix / 1,000 | 327 | 13 |
| Speed (median) | 0.8 sec | 4.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · English (US) (150 documents)
Gemini 3.5 Flash Lite is more accurate by 0.3 pp. Gemma 4 31B is 3.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.95% | 99.64% |
| Error-free documents | 99.33% | 97.33% |
| Per 1,000 documents | $0.69 | $0.18 |
| To fix / 1,000 | 7 | 27 |
| Speed (median) | 1.0 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Czech (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 3.4× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.85% | 99.95% |
| Error-free documents | 98.00% | 99.33% |
| Per 1,000 documents | $0.85 | $0.25 |
| To fix / 1,000 | 20 | 7 |
| Speed (median) | 0.9 sec | 6.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · German (150 documents)
Gemini 3.5 Flash Lite is more accurate by 3.6 pp. Gemma 4 31B is 3.6× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 98.00% | 94.41% |
| Error-free documents | 84.00% | 27.33% |
| Per 1,000 documents | $0.79 | $0.22 |
| To fix / 1,000 | 160 | 727 |
| Speed (median) | 1.0 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Spanish (150 documents)
Gemini 3.5 Flash Lite is more accurate by 5.4 pp. Gemma 4 31B is 3.5× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.79% | 94.41% |
| Error-free documents | 98.00% | 27.33% |
| Per 1,000 documents | $0.84 | $0.24 |
| To fix / 1,000 | 20 | 727 |
| Speed (median) | 1.1 sec | 5.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · French (150 documents)
Gemini 3.5 Flash Lite is more accurate by 5.3 pp. Gemma 4 31B is 3.3× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.95% | 94.62% |
| Error-free documents | 99.33% | 30.67% |
| Per 1,000 documents | $0.85 | $0.26 |
| To fix / 1,000 | 7 | 693 |
| Speed (median) | 1.0 sec | 5.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.8 pp. Gemma 4 31B is 3.6× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.22% | 100.00% |
| Error-free documents | 95.33% | 100.00% |
| Per 1,000 documents | $0.46 | $0.13 |
| To fix / 1,000 | 47 | 0 |
| Speed (median) | 0.9 sec | 2.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Czech (150 documents)
Gemma 4 31B is more accurate by 0.4 pp. Gemma 4 31B is 3.7× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.56% | 100.00% |
| Error-free documents | 97.33% | 100.00% |
| Per 1,000 documents | $0.69 | $0.19 |
| To fix / 1,000 | 27 | 0 |
| Speed (median) | 1.0 sec | 3.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · German (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 3.6× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.89% | 100.00% |
| Error-free documents | 99.33% | 100.00% |
| Per 1,000 documents | $0.59 | $0.16 |
| To fix / 1,000 | 7 | 0 |
| Speed (median) | 0.9 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Spanish (150 documents)
Gemma 4 31B is more accurate by 4.3 pp. Gemma 4 31B is 3.5× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 95.33% | 99.67% |
| Error-free documents | 72.00% | 98.00% |
| Per 1,000 documents | $0.56 | $0.16 |
| To fix / 1,000 | 280 | 20 |
| Speed (median) | 0.9 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · French (150 documents)
Gemma 4 31B is more accurate by 4.4 pp. Gemma 4 31B is 4× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 94.67% | 99.11% |
| Error-free documents | 71.33% | 94.67% |
| Per 1,000 documents | $0.65 | $0.16 |
| To fix / 1,000 | 287 | 53 |
| Speed (median) | 0.9 sec | 3.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · English (US) (150 documents)
Both are equally accurate. Gemma 4 31B is 3× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.46 | $0.15 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 0.8 sec | 2.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 2.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.58 | $0.21 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 0.8 sec | 1.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · German (150 documents)
Both are equally accurate. Gemma 4 31B is 2.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.52 | $0.18 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 0.7 sec | 2.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Spanish (150 documents)
Both are equally accurate. Gemma 4 31B is 2.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.51 | $0.18 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 0.8 sec | 1.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · French (150 documents)
Gemma 4 31B is more accurate by 0.4 pp. Gemma 4 31B is 2.8× cheaper.
| Metric | Gemini 3.5 Flash Lite | Gemma 4 31B |
|---|---|---|
| Correct fields | 99.56% | 100.00% |
| Error-free documents | 98.00% | 100.00% |
| Per 1,000 documents | $0.53 | $0.19 |
| To fix / 1,000 | 20 | 0 |
| Speed (median) | 0.8 sec | 2.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.