Gemma 4 31B vs GPT-6 Luna
Same documents, same prompt: where each model gets it right, what it really costs, and how fast it responds.
Verdict
GPT-6 Luna is more accurate; Gemma 4 31B is cheaper in 25 of 30 combinations.
More accurate
GPT-6 Luna
wins 13 of 30
Cheaper
Gemma 4 31B
cheaper in 25 of 30
Faster
Tie
2.9 sec vs 3.0 sec
Fewer fixes
Gemma 4 31B
27 vs 33 documents per 1,000
Across 30 task and document-language combinations (speed and fixes: median).
At a glance
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Luna | Difference | Gemma 4 31B | GPT-6 Luna | Difference |
| Invoice extraction | 97.64% | 99.70% | ▲ 2.06 | $0.17 | $0.51 | +203 % |
| Email triage | 96.52% | 97.48% | ▲ 0.96 | $0.12 | $0.13 | +12 % |
| Personal data detection | 99.90% | 99.62% | ▼ 0.28 | $0.08 | $0.11 | +25 % |
| Contract clauses | 99.64% | 99.79% | ▲ 0.15 | $0.18 | $0.20 | +12 % |
| Document summaries | 100.00% | 100.00% | ±0 | $0.13 | $0.16 | +25 % |
| Questions about a document | 100.00% | 99.11% | ▼ 0.89 | $0.15 | $0.15 | −1 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Luna | Difference | Gemma 4 31B | GPT-6 Luna | Difference |
| Invoice extraction | 96.10% | 99.28% | ▲ 3.18 | $0.21 | $0.51 | +138 % |
| Email triage | 97.11% | 97.93% | ▲ 0.82 | $0.16 | $0.18 | +12 % |
| Personal data detection | 99.81% | 99.62% | ▼ 0.19 | $0.12 | $0.12 | +1 % |
| Contract clauses | 99.95% | 100.00% | ▲ 0.05 | $0.25 | $0.27 | +6 % |
| Document summaries | 100.00% | 100.00% | ±0 | $0.19 | $0.21 | +12 % |
| Questions about a document | 100.00% | 100.00% | ±0 | $0.21 | $0.23 | +7 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Luna | Difference | Gemma 4 31B | GPT-6 Luna | Difference |
| Invoice extraction | 94.87% | 99.23% | ▲ 4.36 | $0.20 | $0.54 | +174 % |
| Email triage | 95.11% | 96.96% | ▲ 1.85 | $0.15 | $0.15 | +3 % |
| Personal data detection | 99.90% | 99.62% | ▼ 0.28 | $0.10 | $0.12 | +15 % |
| Contract clauses | 94.41% | 100.00% | ▲ 5.59 | $0.22 | $0.27 | +21 % |
| Document summaries | 100.00% | 100.00% | ±0 | $0.16 | $0.19 | +19 % |
| Questions about a document | 100.00% | 98.00% | ▼ 2 | $0.18 | $0.20 | +11 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Luna | Difference | Gemma 4 31B | GPT-6 Luna | Difference |
| Invoice extraction | 94.62% | 99.13% | ▲ 4.51 | $0.20 | $0.55 | +173 % |
| Email triage | 97.11% | 96.81% | ▼ 0.3 | $0.15 | $0.16 | +8 % |
| Personal data detection | 99.90% | 99.71% | ▼ 0.19 | $0.10 | $0.12 | +14 % |
| Contract clauses | 94.41% | 100.00% | ▲ 5.59 | $0.24 | $0.26 | +11 % |
| Document summaries | 99.67% | 97.00% | ▼ 2.67 | $0.16 | $0.19 | +20 % |
| Questions about a document | 100.00% | 99.67% | ▼ 0.33 | $0.18 | $0.19 | +7 % |
| Accuracy (correct fields) | Per 1,000 documents | |||||
|---|---|---|---|---|---|---|
| Task · documents | Gemma 4 31B | GPT-6 Luna | Difference | Gemma 4 31B | GPT-6 Luna | Difference |
| Invoice extraction | 93.54% | 99.08% | ▲ 5.54 | $0.22 | $0.56 | +153 % |
| Email triage | 96.22% | 97.26% | ▲ 1.04 | $0.16 | $0.16 | ±0 % |
| Personal data detection | 99.81% | 99.62% | ▼ 0.19 | $0.12 | $0.12 | +4 % |
| Contract clauses | 94.62% | 99.33% | ▲ 4.71 | $0.26 | $0.27 | +7 % |
| Document summaries | 99.11% | 98.78% | ▼ 0.33 | $0.16 | $0.20 | +20 % |
| Questions about a document | 100.00% | 99.56% | ▼ 0.44 | $0.19 | $0.20 | +6 % |
Difference: GPT-6 Luna vs. Gemma 4 31B; accuracy in percentage points, price in percent. The better value is in bold. Click a task to see the details.
Invoice extraction · English (US) (150 documents)
GPT-6 Luna is more accurate by 2.1 pp. Gemma 4 31B is 3× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 97.64% | 99.70% |
| Error-free documents | 81.33% | 96.67% |
| Per 1,000 documents | $0.17 | $0.51 |
| To fix / 1,000 | 187 | 33 |
| Speed (median) | 8.9 sec | 3.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Czech (150 documents)
GPT-6 Luna is more accurate by 3.2 pp. Gemma 4 31B is 2.4× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 96.10% | 99.28% |
| Error-free documents | 66.67% | 93.33% |
| Per 1,000 documents | $0.21 | $0.51 |
| To fix / 1,000 | 333 | 67 |
| Speed (median) | 10.4 sec | 3.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · German (150 documents)
GPT-6 Luna is more accurate by 4.4 pp. Gemma 4 31B is 2.7× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 94.87% | 99.23% |
| Error-free documents | 64.00% | 91.33% |
| Per 1,000 documents | $0.20 | $0.54 |
| To fix / 1,000 | 360 | 87 |
| Speed (median) | 7.7 sec | 3.7 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · Spanish (150 documents)
GPT-6 Luna is more accurate by 4.5 pp. Gemma 4 31B is 2.7× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 94.62% | 99.13% |
| Error-free documents | 50.67% | 88.67% |
| Per 1,000 documents | $0.20 | $0.55 |
| To fix / 1,000 | 493 | 113 |
| Speed (median) | 7.4 sec | 3.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Invoice extraction · French (150 documents)
GPT-6 Luna is more accurate by 5.5 pp. Gemma 4 31B is 2.5× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 93.54% | 99.08% |
| Error-free documents | 50.67% | 88.00% |
| Per 1,000 documents | $0.22 | $0.56 |
| To fix / 1,000 | 493 | 120 |
| Speed (median) | 6.5 sec | 3.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · English (US) (150 documents)
GPT-6 Luna is more accurate by 1 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 96.52% | 97.48% |
| Error-free documents | 72.00% | 77.33% |
| Per 1,000 documents | $0.12 | $0.13 |
| To fix / 1,000 | 280 | 227 |
| Speed (median) | 2.2 sec | 2.5 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Czech (150 documents)
GPT-6 Luna is more accurate by 0.8 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 97.11% | 97.93% |
| Error-free documents | 74.67% | 81.33% |
| Per 1,000 documents | $0.16 | $0.18 |
| To fix / 1,000 | 253 | 187 |
| Speed (median) | 2.5 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · German (150 documents)
GPT-6 Luna is more accurate by 1.9 pp.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 95.11% | 96.96% |
| Error-free documents | 62.00% | 72.67% |
| Per 1,000 documents | $0.15 | $0.15 |
| To fix / 1,000 | 380 | 273 |
| Speed (median) | 3.6 sec | 2.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · Spanish (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 97.11% | 96.81% |
| Error-free documents | 74.00% | 72.00% |
| Per 1,000 documents | $0.15 | $0.16 |
| To fix / 1,000 | 260 | 280 |
| Speed (median) | 2.2 sec | 3.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Email triage · French (150 documents)
GPT-6 Luna is more accurate by 1 pp.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 96.22% | 97.26% |
| Error-free documents | 68.67% | 76.00% |
| Per 1,000 documents | $0.16 | $0.16 |
| To fix / 1,000 | 313 | 240 |
| Speed (median) | 2.4 sec | 2.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 1.3× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.90% | 99.62% |
| Error-free documents | 99.33% | 97.33% |
| Per 1,000 documents | $0.08 | $0.11 |
| To fix / 1,000 | 7 | 27 |
| Speed (median) | 2.4 sec | 1.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Czech (150 documents)
Gemma 4 31B is more accurate by 0.2 pp.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.81% | 99.62% |
| Error-free documents | 98.67% | 97.33% |
| Per 1,000 documents | $0.12 | $0.12 |
| To fix / 1,000 | 13 | 27 |
| Speed (median) | 2.9 sec | 1.8 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · German (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.90% | 99.62% |
| Error-free documents | 99.33% | 97.33% |
| Per 1,000 documents | $0.10 | $0.12 |
| To fix / 1,000 | 7 | 27 |
| Speed (median) | 1.9 sec | 2.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · Spanish (150 documents)
Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.90% | 99.71% |
| Error-free documents | 99.33% | 98.00% |
| Per 1,000 documents | $0.10 | $0.12 |
| To fix / 1,000 | 7 | 20 |
| Speed (median) | 3.6 sec | 2.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Personal data detection · French (150 documents)
Gemma 4 31B is more accurate by 0.2 pp.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.81% | 99.62% |
| Error-free documents | 98.67% | 97.33% |
| Per 1,000 documents | $0.12 | $0.12 |
| To fix / 1,000 | 13 | 27 |
| Speed (median) | 4.3 sec | 2.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · English (US) (150 documents)
GPT-6 Luna is more accurate by 0.2 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.64% | 99.79% |
| Error-free documents | 97.33% | 98.67% |
| Per 1,000 documents | $0.18 | $0.20 |
| To fix / 1,000 | 27 | 13 |
| Speed (median) | 2.5 sec | 2.7 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.95% | 100.00% |
| Error-free documents | 99.33% | 100.00% |
| Per 1,000 documents | $0.25 | $0.27 |
| To fix / 1,000 | 7 | 0 |
| Speed (median) | 6.0 sec | 2.6 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · German (150 documents)
GPT-6 Luna is more accurate by 5.6 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 94.41% | 100.00% |
| Error-free documents | 27.33% | 100.00% |
| Per 1,000 documents | $0.22 | $0.27 |
| To fix / 1,000 | 727 | 0 |
| Speed (median) | 2.6 sec | 3.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · Spanish (150 documents)
GPT-6 Luna is more accurate by 5.6 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 94.41% | 100.00% |
| Error-free documents | 27.33% | 100.00% |
| Per 1,000 documents | $0.24 | $0.26 |
| To fix / 1,000 | 727 | 0 |
| Speed (median) | 5.0 sec | 3.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Contract clauses · French (150 documents)
GPT-6 Luna is more accurate by 4.7 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 94.62% | 99.33% |
| Error-free documents | 30.67% | 91.33% |
| Per 1,000 documents | $0.26 | $0.27 |
| To fix / 1,000 | 693 | 87 |
| Speed (median) | 5.3 sec | 3.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · English (US) (150 documents)
Both are equally accurate. Gemma 4 31B is 1.3× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.13 | $0.16 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.2 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.19 | $0.21 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 3.4 sec | 3.4 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · German (150 documents)
Both are equally accurate. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.16 | $0.19 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 2.9 sec | 3.3 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · Spanish (150 documents)
Gemma 4 31B is more accurate by 2.7 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.67% | 97.00% |
| Error-free documents | 98.00% | 82.00% |
| Per 1,000 documents | $0.16 | $0.19 |
| To fix / 1,000 | 20 | 180 |
| Speed (median) | 2.6 sec | 3.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Document summaries · French (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 1.2× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 99.11% | 98.78% |
| Error-free documents | 94.67% | 92.67% |
| Per 1,000 documents | $0.16 | $0.20 |
| To fix / 1,000 | 53 | 73 |
| Speed (median) | 3.2 sec | 3.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · English (US) (150 documents)
Gemma 4 31B is more accurate by 0.9 pp.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 99.11% |
| Error-free documents | 100.00% | 96.00% |
| Per 1,000 documents | $0.15 | $0.15 |
| To fix / 1,000 | 0 | 40 |
| Speed (median) | 2.1 sec | 3.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Czech (150 documents)
Both are equally accurate. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 100.00% |
| Error-free documents | 100.00% | 100.00% |
| Per 1,000 documents | $0.21 | $0.23 |
| To fix / 1,000 | 0 | 0 |
| Speed (median) | 1.9 sec | 3.1 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · German (150 documents)
Gemma 4 31B is more accurate by 2 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 98.00% |
| Error-free documents | 100.00% | 91.33% |
| Per 1,000 documents | $0.18 | $0.20 |
| To fix / 1,000 | 0 | 87 |
| Speed (median) | 2.1 sec | 3.2 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · Spanish (150 documents)
Gemma 4 31B is more accurate by 0.3 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 99.67% |
| Error-free documents | 100.00% | 98.67% |
| Per 1,000 documents | $0.18 | $0.19 |
| To fix / 1,000 | 0 | 13 |
| Speed (median) | 1.9 sec | 2.9 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.
Questions about a document · French (150 documents)
Gemma 4 31B is more accurate by 0.4 pp. Gemma 4 31B is 1.1× cheaper.
| Metric | Gemma 4 31B | GPT-6 Luna |
|---|---|---|
| Correct fields | 100.00% | 99.56% |
| Error-free documents | 100.00% | 98.00% |
| Per 1,000 documents | $0.19 | $0.20 |
| To fix / 1,000 | 0 | 20 |
| Speed (median) | 2.1 sec | 3.0 sec |
| Valid answers | 100.00% | 100.00% |
Free with login
Accuracy field by field and the documents where the models differ.
Just your email, no password.