Gemma 4 31B vs GPT-6 Sol
Same documents, same prompt: where each model gets it right, what it really costs, and how fast it responds.
Verdict
GPT-6 Sol is more accurate; Gemma 4 31B is cheaper in 30 of 30 combinations.
More accurate
GPT-6 Sol
wins 14 of 30
Cheaper
Gemma 4 31B
cheaper in 30 of 30
Faster
Tie
2.9 sec vs 2.9 sec
Fewer fixes
GPT-6 Sol
13 vs 27 documents per 1,000
Across 30 task and document-language combinations (speed and fixes: median).
At a glance
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Sol | Difference | Gemma 4 31B | GPT-6 Sol | Difference |
| Invoice extraction | 97.64% | 99.94% | ▲ 2.3 | $0.17 | $9.65 | +5660 % |
| Email triage | 96.52% | 99.04% | ▲ 2.52 | $0.12 | $2.44 | +1918 % |
| Personal data detection | 99.90% | 99.71% | ▼ 0.19 | $0.08 | $2.09 | +2375 % |
| Contract clauses | 99.64% | 100.00% | ▲ 0.36 | $0.18 | $3.33 | +1750 % |
| Document summaries | 100.00% | 99.56% | ▼ 0.44 | $0.13 | $3.85 | +2916 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.15 | $2.36 | +1424 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Sol | Difference | Gemma 4 31B | GPT-6 Sol | Difference |
| Invoice extraction | 96.10% | 99.90% | ▲ 3.8 | $0.21 | $10.12 | +4677 % |
| Email triage | 97.11% | 98.96% | ▲ 1.85 | $0.16 | $3.19 | +1909 % |
| Personal data detection | 99.81% | 99.71% | ▼ 0.1 | $0.12 | $2.22 | +1769 % |
| Contract clauses | 99.95% | 100.00% | ▲ 0.05 | $0.25 | $4.62 | +1720 % |
| Document summaries | 100.00% | 100.00% | ±0 | $0.19 | $4.72 | +2395 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.21 | $3.95 | +1768 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Sol | Difference | Gemma 4 31B | GPT-6 Sol | Difference |
| Invoice extraction | 94.87% | 99.95% | ▲ 5.08 | $0.20 | $10.65 | +5253 % |
| Email triage | 95.11% | 98.22% | ▲ 3.11 | $0.15 | $2.73 | +1740 % |
| Personal data detection | 99.90% | 99.71% | ▼ 0.19 | $0.10 | $2.13 | +1987 % |
| Contract clauses | 94.41% | 100.00% | ▲ 5.59 | $0.22 | $4.36 | +1877 % |
| Document summaries | 100.00% | 100.00% | ±0 | $0.16 | $4.11 | +2423 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.18 | $3.23 | +1676 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Sol | Difference | Gemma 4 31B | GPT-6 Sol | Difference |
| Invoice extraction | 94.62% | 99.23% | ▲ 4.61 | $0.20 | $10.79 | +5235 % |
| Email triage | 97.11% | 98.44% | ▲ 1.33 | $0.15 | $2.77 | +1803 % |
| Personal data detection | 99.90% | 99.81% | ▼ 0.09 | $0.10 | $2.21 | +2040 % |
| Contract clauses | 94.41% | 100.00% | ▲ 5.59 | $0.24 | $4.48 | +1789 % |
| Document summaries | 99.67% | 97.89% | ▼ 1.78 | $0.16 | $5.02 | +3090 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.18 | $3.36 | +1752 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Sol | Difference | Gemma 4 31B | GPT-6 Sol | Difference |
| Invoice extraction | 93.54% | 99.90% | ▲ 6.36 | $0.22 | $11.00 | +4889 % |
| Email triage | 96.22% | 98.59% | ▲ 2.37 | $0.16 | $2.81 | +1662 % |
| Personal data detection | 99.81% | 99.71% | ▼ 0.1 | $0.12 | $2.23 | +1757 % |
| Contract clauses | 94.62% | 99.59% | ▲ 4.97 | $0.26 | $4.49 | +1658 % |
| Document summaries | 99.11% | 97.22% | ▼ 1.89 | $0.16 | $4.58 | +2688 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.19 | $3.43 | +1682 % |
Difference: GPT-6 Sol vs. Gemma 4 31B; accuracy in percentage points, price in percent. The better value is in bold. Click a task to see the details.
Invoice extraction · English (US) (150 documents)
GPT-6 Sol is more accurate by 2.3 pp. Gemma 4 31B is 57.6× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 97.64% | 99.94% |
| Error-free documents | 81.33% | 99.33% |
| Per 1,000 documents | $0.17 | $9.65 |
| To fix / 1,000 | 187 | 7 |
| Speed (median) | 8.9 sec | 4.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Czech (150 documents)
GPT-6 Sol is more accurate by 3.8 pp. Gemma 4 31B is 47.8× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 96.10% | 99.90% |
| Error-free documents | 66.67% | 98.67% |
| Per 1,000 documents | $0.21 | $10.12 |
| To fix / 1,000 | 333 | 13 |
| Speed (median) | 10.4 sec | 3.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · German (150 documents)
GPT-6 Sol is more accurate by 5.1 pp. Gemma 4 31B is 53.5× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 94.87% | 99.95% |
| Error-free documents | 64.00% | 99.33% |
| Per 1,000 documents | $0.20 | $10.65 |
| To fix / 1,000 | 360 | 7 |
| Speed (median) | 7.7 sec | 4.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Spanish (150 documents)
GPT-6 Sol is more accurate by 4.6 pp. Gemma 4 31B is 53.4× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 94.62% | 99.23% |
| Error-free documents | 50.67% | 90.00% |
| Per 1,000 documents | $0.20 | $10.79 |
| To fix / 1,000 | 493 | 100 |
| Speed (median) | 7.4 sec | 5.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · French (150 documents)
GPT-6 Sol is more accurate by 6.4 pp. Gemma 4 31B is 49.9× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 93.54% | 99.90% |
| Error-free documents | 50.67% | 98.67% |
| Per 1,000 documents | $0.22 | $11.00 |
| To fix / 1,000 | 493 | 13 |
| Speed (median) | 6.5 sec | 6.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · English (US) (150 documents)
GPT-6 Sol is more accurate by 2.5 pp. Gemma 4 31B is 20.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 96.52% | 99.04% |
| Error-free documents | 72.00% | 92.00% |
| Per 1,000 documents | $0.12 | $2.44 |
| To fix / 1,000 | 280 | 80 |
| Speed (median) | 2.2 sec | 2.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Czech (150 documents)
GPT-6 Sol is more accurate by 1.9 pp. Gemma 4 31B is 20.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 97.11% | 98.96% |
| Error-free documents | 74.67% | 90.67% |
| Per 1,000 documents | $0.16 | $3.19 |
| To fix / 1,000 | 253 | 93 |
| Speed (median) | 2.5 sec | 2.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · German (150 documents)
GPT-6 Sol is more accurate by 3.1 pp. Gemma 4 31B is 18.4× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 95.11% | 98.22% |
| Error-free documents | 62.00% | 84.00% |
| Per 1,000 documents | $0.15 | $2.73 |
| To fix / 1,000 | 380 | 160 |
| Speed (median) | 3.6 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Spanish (150 documents)
GPT-6 Sol is more accurate by 1.3 pp. Gemma 4 31B is 19× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 97.11% | 98.44% |
| Error-free documents | 74.00% | 86.00% |
| Per 1,000 documents | $0.15 | $2.77 |
| To fix / 1,000 | 260 | 140 |
| Speed (median) | 2.2 sec | 3.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · French (150 documents)
GPT-6 Sol is more accurate by 2.4 pp. Gemma 4 31B is 17.6× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 96.22% | 98.59% |
| Error-free documents | 68.67% | 87.33% |
| Per 1,000 documents | $0.16 | $2.81 |
| To fix / 1,000 | 313 | 127 |
| Speed (median) | 2.4 sec | 3.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 24.8× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.90% | 99.71% |
| Error-free documents | 99.33% | 98.00% |
| Per 1,000 documents | $0.08 | $2.09 |
| To fix / 1,000 | 7 | 20 |
| Speed (median) | 2.4 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Czech (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 18.7× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.81% | 99.71% |
| Error-free documents | 98.67% | 98.00% |
| Per 1,000 documents | $0.12 | $2.22 |
| To fix / 1,000 | 13 | 20 |
| Speed (median) | 2.9 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · German (150 documents)
Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 20.9× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.90% | 99.71% |
| Error-free documents | 99.33% | 98.00% |
| Per 1,000 documents | $0.10 | $2.13 |
| To fix / 1,000 | 7 | 20 |
| Speed (median) | 1.9 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Spanish (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 21.4× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.90% | 99.81% |
| Error-free documents | 99.33% | 98.67% |
| Per 1,000 documents | $0.10 | $2.21 |
| To fix / 1,000 | 7 | 13 |
| Speed (median) | 3.6 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · French (150 documents)
Gemma 4 31B is more accurate by 0.1 pp. Gemma 4 31B is 18.6× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.81% | 99.71% |
| Error-free documents | 98.67% | 98.00% |
| Per 1,000 documents | $0.12 | $2.23 |
| To fix / 1,000 | 13 | 20 |
| Speed (median) | 4.3 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · English (US) (150 documents)
GPT-6 Sol is more accurate by 0.4 pp. Gemma 4 31B is 18.5× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.64% | 100.00% |
| Error-free documents | 97.33% | 100.00% |
| Per 1,000 documents | $0.18 | $3.33 |
| To fix / 1,000 | 27 | 0 |
| Speed (median) | 2.5 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 18.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.95% | 100.00% |
| Error-free documents | 99.33% | 100.00% |
| Per 1,000 documents | $0.25 | $4.62 |
| To fix / 1,000 | 7 | 0 |
| Speed (median) | 6.0 sec | 2.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · German (150 documents)
GPT-6 Sol is more accurate by 5.6 pp. Gemma 4 31B is 19.8× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 94.41% | 100.00% |
| Error-free documents | 27.33% | 100.00% |
| Per 1,000 documents | $0.22 | $4.36 |
| To fix / 1,000 | 727 | 0 |
| Speed (median) | 2.6 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Spanish (150 documents)
GPT-6 Sol is more accurate by 5.6 pp. Gemma 4 31B is 18.9× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 94.41% | 100.00% |
| Error-free documents | 27.33% | 100.00% |
| Per 1,000 documents | $0.24 | $4.48 |
| To fix / 1,000 | 727 | 0 |
| Speed (median) | 5.0 sec | 2.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · French (150 documents)
GPT-6 Sol is more accurate by 5 pp. Gemma 4 31B is 17.6× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 94.62% | 99.59% |
| Error-free documents | 30.67% | 94.67% |
| Per 1,000 documents | $0.26 | $4.49 |
| To fix / 1,000 | 693 | 53 |
| Speed (median) | 5.3 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.4 pp. Gemma 4 31B is 30.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 99.56% |
| Error-free documents | 100.00% | 97.33% |
| Per 1,000 documents | $0.13 | $3.85 |
| To fix / 1,000 | 0 | 27 |
| Speed (median) | 2.2 sec | 8.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 24.9× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.19 | $4.72 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 3.4 sec | 7.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · German (150 documents)
Both are equally accurate. Gemma 4 31B is 25.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.16 | $4.11 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.9 sec | 7.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Spanish (150 documents)
Gemma 4 31B is more accurate by 1.8 pp. Gemma 4 31B is 31.9× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.67% | 97.89% |
| Error-free documents | 98.00% | 87.33% |
| Per 1,000 documents | $0.16 | $5.02 |
| To fix / 1,000 | 20 | 127 |
| Speed (median) | 2.6 sec | 7.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · French (150 documents)
Gemma 4 31B is more accurate by 1.9 pp. Gemma 4 31B is 27.9× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 99.11% | 97.22% |
| Error-free documents | 94.67% | 83.33% |
| Per 1,000 documents | $0.16 | $4.58 |
| To fix / 1,000 | 53 | 167 |
| Speed (median) | 3.2 sec | 6.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · English (US) (150 documents)
Both are equally accurate. Gemma 4 31B is 15.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.15 | $2.36 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.1 sec | 2.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 18.7× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.21 | $3.95 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 1.9 sec | 2.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · German (150 documents)
Both are equally accurate. Gemma 4 31B is 17.8× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.18 | $3.23 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.1 sec | 2.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Spanish (150 documents)
Both are equally accurate. Gemma 4 31B is 18.5× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.18 | $3.36 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 1.9 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · French (150 documents)
Both are equally accurate. Gemma 4 31B is 17.8× cheaper.
| Metric | Gemma 4 31B | GPT-6 Sol |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.19 | $3.43 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.1 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.