Tyler Robertson
Text (the model received exactly this text)
Ticket #29494 · Damaged delivery Customer: Tyler Robertson Email: tyler.robertson@inbox.example Phone: 692-555-0029 The customer reports that the goods arrived damaged and asks for a replacement. The replacement ships no later than January 15, 2026. Copy sent to office@johnson-bell.example.
Verdict
Cheapest with no errors: Mistral Small 4 at $0.06 per 1,000 documents.
- Claude Sonnet 5.5 is also error-free, but 47.8× more expensive.
Prompt given to every model
Find all personal data of natural persons in the text below (for anonymisation) and return only JSON with these keys, each a list of strings (an empty list when there is none): person_names full names of people, each person once, as written in the text email_addresses email addresses of people phone_numbers phone numbers of people, as written postal_addresses home or delivery addresses of people, as written (street, city, ZIP) birth_dates dates of birth (YYYY-MM-DD) national_ids national identification numbers (e.g. SSN, birth numbers) bank_accounts bank account numbers of people, as written Rules: - Only data of natural persons. Company names, company registration or tax numbers (EIN, IČO, VAT), generic company addresses such as info@ or support@, order and ticket numbers are not personal data: leave them out. - Dates that are not dates of birth (meetings, deliveries, start dates) are not personal data. - Do not invent or complete values; copy them from the text.
How we score
Exact match per field after normalizing dates, amounts, and whitespace. Missing diacritics count as an error. Cost is the actual token spend of this run, converted at the rate shown in the footer.
What each model extracted
| Field | Correct value | Mistral Small 47 / 7 fields | Gemma 4 31B7 / 7 fields | GPT-6 Luna7 / 7 fields | Claude Haiku 5.57 / 7 fields | GLM-5.3 Flash7 / 7 fields | Qwen3.8 Flash7 / 7 fields | DeepSeek V4.1 Flash7 / 7 fields | Gemini 3.5 Flash Lite7 / 7 fields | Mistral Medium 3.57 / 7 fields | GPT-6 Sol7 / 7 fields | Gemini 3.8 Flash7 / 7 fields | Mistral Large 47 / 7 fields | Claude Sonnet 57 / 7 fields | Claude Sonnet 5.57 / 7 fields |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Names | Tyler Robertson | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) | Tyler Robertson (correct) |
| Email addresses | tyler.robertson@inbox.example | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) | tyler.robertson@inbox.example (correct) |
| Phone numbers | 692-555-0029 | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) | 692-555-0029 (correct) |
| Addresses | — | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) |
| Dates of birth | — | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) |
| National IDs | — | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) |
| Bank accounts | — | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) |
| Tokens | input / output | 324 / 71 | 347 / 71 | 439 / 85 | 1,039 / 97 | 314 / 267 | 644 / 349 | 569 / 248 | 335 / 104 | 324 / 74 | 439 / 58 | 335 / 331 | 522 / 917 | 1,037 / 97 | 1,039 / 97 |
| including hidden reasoning | billed as output | 0 | 0 | 25 | 0 | 207 | 251 | 184 | 0 | 0 | 0 | 271 | 872 | 0 | 0 |
| including cached input | input billed at a discount | 203 | 3 | 0 | 0 | 0 | 512 | 364 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Cost of this document | this run | $0.0001 | $0.0001 | $0.0001 | $0.0002 | $0.0002 | $0.0002 | $0.0002 | $0.0004 | $0.0010 | $0.0015 | $0.0015 | $0.0023 | $0.0030 | $0.0030 |
| Per 1,000 documents | for documents of this type | $0.06 | $0.08 | $0.09 | $0.15 | $0.18 | $0.19 | $0.21 | $0.36 | $1.04 | $1.46 | $1.49 | $2.27 | $3.04 | $3.05 |
| Latency | this run | 0.9 s | 1.4 s | 2.2 s | 1.6 s | 3.6 s | 6.9 s | 2.4 s | 0.7 s | 0.9 s | 2 s | 1.5 s | 74.2 s | 2.4 s | 1.9 s |