Anthropic
Claude Sonnet 5
How Claude Sonnet 5 by Anthropic handles real business documents: accuracy, true cost, speed and where it goes wrong.
List price: $2.00 per 1M input tokens, $10.00 per 1M output tokens · anthropic/claude-sonnet-5
Results
| Task · documents | Rank | Correct fields | Error-free documents | Per 1,000 documents | To fix / 1,000 | Speed (median) | Bar |
|---|---|---|---|---|---|---|---|
| Contract clauses · English (US) | 1/10 | 100.0% | 100% | $5.04 | 0 | 3.4 sec | meets the bar |
| Contract clauses · Czech | 1/10 | 100.0% | 100% | $6.05 | 0 | 3.6 sec | meets the bar |
| Contract clauses · German | 4/10 | 98.2% | 87% | $7.11 | 133 | 4.3 sec | meets the bar |
| Email triage · English (US) | 3/10 | 98.2% | 84% | $4.01 | 160 | 3.8 sec | meets the bar |
| Email triage · Czech | 4/10 | 98.2% | 83% | $4.91 | 167 | 4.7 sec | meets the bar |
| Email triage · German | 3/10 | 97.9% | 81% | $5.11 | 187 | 4.3 sec | meets the bar |
| Invoice extraction · English (US) | 2/10 | 99.8% | 99% | $8.84 | 13 | 3.2 sec | meets the bar |
| Invoice extraction · Czech | 1/10 | 100.0% | 100% | $9.41 | 0 | 3.6 sec | meets the bar |
| Invoice extraction · German | 5/10 | 99.0% | 87% | $10.42 | 133 | 3.8 sec | meets the bar |
| Invoice extraction · Spanish | 8/10 | 93.6% | 77% | $11.89 | 230 | 4.0 sec | below the bar |
| Invoice extraction · French | 6/10 | 98.7% | 96% | $11.02 | 41 | 4.4 sec | meets the bar |
| Personal data detection · English (US) | 2/10 | 99.7% | 98% | $3.50 | 20 | 2.5 sec | meets the bar |
| Personal data detection · Czech | 7/10 | 98.9% | 92% | $4.06 | 80 | 3.2 sec | meets the bar |
| Personal data detection · German | 4/10 | 99.7% | 98% | $4.40 | 20 | 3.7 sec | meets the bar |
Where it goes wrong
The fields it gets wrong most often and the kinds of errors, from all documents of the benchmark.
Contract clauses · English (US)
No errors on this task.
Contract clauses · Czech
No errors on this task.
Contract clauses · German
Weakest fields
- Provider ID87.3%
- Client ID89.3%
Kinds of errors (number of fields)
Email triage · English (US)
Weakest fields
- Sentiment86.7%
- Priority97.3%
Kinds of errors (number of fields)
Email triage · Czech
Weakest fields
- Sentiment87.3%
- Priority96.0%
Kinds of errors (number of fields)
Email triage · German
Weakest fields
- Sentiment84.7%
- Priority96.0%
Kinds of errors (number of fields)
Invoice extraction · English (US)
Weakest fields
- Net by tax rate98.7%
- Tax by rate99.3%
Kinds of errors (number of fields)
Invoice extraction · Czech
No errors on this task.
Invoice extraction · German
Weakest fields
- Net by tax rate89.3%
- Bank account98.0%
- Invoice date99.3%
Kinds of errors (number of fields)
Invoice extraction · Spanish
Weakest fields
- Net by tax rate79.0%
- Supplier94.0%
- Payment reference94.0%
- Supplier ID (EIN, IČO…)95.0%
Kinds of errors (number of fields)
Invoice extraction · French
Weakest fields
- Net by tax rate96.9%
- Payment reference97.9%
- Bank account97.9%
- Supplier99.0%
Kinds of errors (number of fields)
Personal data detection · English (US)
Weakest fields
- Addresses98.0%
Kinds of errors (number of fields)
Personal data detection · Czech
Weakest fields
- Names96.7%
- Addresses97.3%
- Dates of birth98.0%
Kinds of errors (number of fields)
Personal data detection · German
Weakest fields
- Names98.7%
- Dates of birth99.3%
Kinds of errors (number of fields)
Language tax
How many more tokens this model needs for the same text than in English.
- Czech 1.52×
- German 1.88×
- French 1.53×
- Spanish 1.50×
- Polish 1.76×
Compliance
- In the EU yes
- No retention (ZDR) yes
- DPA yes
- Open weights no
EU: Amazon Bedrock eu-west-1, Google Vertex europe. ZDR: Bedrock, Vertex.
Use it now
The exact request of the benchmark (same prompt, JSON schema, temperature 0), ready to call Claude Sonnet 5 via OpenRouter.
Contract clauses · English (US)
import json
import os
import httpx
PROMPT = "Extract the key terms of the contract below and return only JSON with these keys:\n\nprovider_name the party that delivers (provider, seller, landlord, licensor, contractor),\n exactly as written, including the legal form\nprovider_id that party's company ID (EIN), if stated\nclient_name the party that pays (client, buyer, tenant, licensee, customer)\nclient_id that party's company ID (EIN), if stated\ncontract_type one of: services, supply, lease, license, maintenance\nprice the agreed price excluding VAT / sales tax (number)\ncurrency ISO 4217 code (USD, EUR, CZK, …)\nprice_period one of: one_time, monthly, yearly\neffective_date the date the contract takes effect (YYYY-MM-DD)\nend_date the date a fixed term ends (YYYY-MM-DD); null if the term is indefinite\nnotice_period_months the termination notice period in months (number); null if there is none\npenalty_pct_per_day the late-payment penalty in percent of the overdue amount per day\n (e.g. 0.05 for 0.05 %); null if there is none\ngoverning_law ISO 3166 country code of the governing law; US states as US-XX (e.g. US-NY)\n\nRules:\n- Resolve relative dates (\"on the date of signature\", \"the first day of the following month\")\n against the signing date.\n- The signing date is not the effective date unless the contract says so.\n- Deposits, advance payments and fixed penalties for other breaches are not the price or the\n late-payment penalty.\n- If a value is not in the contract, return null. Do not guess.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"provider_name\":{\"type\":[\"string\",\"null\"]},\"provider_id\":{\"type\":[\"string\",\"null\"]},\"client_name\":{\"type\":[\"string\",\"null\"]},\"client_id\":{\"type\":[\"string\",\"null\"]},\"contract_type\":{\"type\":[\"string\",\"null\"],\"enum\":[\"services\",\"supply\",\"lease\",\"license\",\"maintenance\",null]},\"price\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"price_period\":{\"type\":[\"string\",\"null\"],\"enum\":[\"one_time\",\"monthly\",\"yearly\",null]},\"effective_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"end_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"notice_period_months\":{\"type\":[\"integer\",\"null\"]},\"penalty_pct_per_day\":{\"type\":[\"number\",\"null\"]},\"governing_law\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"provider_name\",\"provider_id\",\"client_name\",\"client_id\",\"contract_type\",\"price\",\"currency\",\"price_period\",\"effective_date\",\"end_date\",\"notice_period_months\",\"penalty_pct_per_day\",\"governing_law\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "anthropic/claude-sonnet-5",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "contracts", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Email triage · English (US)
import json
import os
import httpx
PROMPT = "You triage the support inbox. Read the email below and return only JSON with these keys:\n\ncategory one of: billing (invoices, payments, charges), delivery (where is the order, shipping,\n address change), complaint (damaged, wrong or defective goods, warranty claims),\n technical (login, app or system errors, integrations), cancellation (cancel an order,\n contract or subscription), sales (quotes, prices, interest before buying)\npriority one of: urgent, high, normal, low (rules below)\nsentiment one of: positive, neutral, negative (the customer's tone)\ncustomer_name full name of the customer who wrote the request\norder_number the customer's order number, if mentioned\ninvoice_number invoice number, if mentioned\namount the amount of money the request is about (number)\ncurrency ISO 4217 code of that amount (USD, EUR, CZK, …)\ndeadline the date by which the customer wants a resolution or answer (YYYY-MM-DD)\n\nPriority rules:\n- urgent: the customer cannot work right now, or the deadline is at most 1 day after the email date\n- high: the deadline is at most 7 days after the email date, or the customer threatens to cancel\n or escalate (lawyer, ending the business)\n- low: the customer says explicitly that it is not urgent\n- normal: everything else\n\nRules:\n- A thread: judge the newest message; order numbers, invoice numbers and amounts may come from the\n quoted earlier messages. A forwarded email: the customer is the original sender, not the\n colleague who forwarded it.\n- Resolve relative deadlines (\"by Friday\", \"by tomorrow\", \"by the end of the month\") against the\n date of the message that states them.\n- Support ticket numbers, tax IDs and phone numbers are not order or invoice numbers.\n- If a value is not in the email, return null. Do not guess.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"category\":{\"type\":[\"string\",\"null\"],\"enum\":[\"billing\",\"delivery\",\"complaint\",\"technical\",\"cancellation\",\"sales\",null]},\"priority\":{\"type\":[\"string\",\"null\"],\"enum\":[\"urgent\",\"high\",\"normal\",\"low\",null]},\"sentiment\":{\"type\":[\"string\",\"null\"],\"enum\":[\"positive\",\"neutral\",\"negative\",null]},\"customer_name\":{\"type\":[\"string\",\"null\"]},\"order_number\":{\"type\":[\"string\",\"null\"]},\"invoice_number\":{\"type\":[\"string\",\"null\"]},\"amount\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"deadline\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"}},\"required\":[\"category\",\"priority\",\"sentiment\",\"customer_name\",\"order_number\",\"invoice_number\",\"amount\",\"currency\",\"deadline\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "anthropic/claude-sonnet-5",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "emails", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Invoice extraction · English (US)
import json
import os
import httpx
PROMPT = "Extract the following data from the attached document and return only JSON with these keys:\n\nsupplier_name supplier's name (exactly as printed, including the legal form)\nsupplier_id supplier's tax ID (EIN), if printed\ninvoice_number document number\npayment_reference payment reference to quote with the payment\nissue_date invoice date (YYYY-MM-DD)\ndue_date due date (YYYY-MM-DD)\nnet_by_rate amount per sales tax rate, e.g. {\"8.25\": 1845.00}; non-taxable amounts under \"0\"\nvat_by_rate sales tax per rate, e.g. {\"8.25\": 152.21}\ntotal total due (number, negative for credit notes)\ncurrency ISO 4217 currency code (USD, GBP, EUR, …)\nbank_account supplier's bank details as \"routing-number account-number\", or IBAN\n\nRules:\n- Return the supplier's data only, never the customer's.\n- If a value is not on the document, return null. Do not guess.\n- If the document has no tax breakdown, return net_by_rate and vat_by_rate as null.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"supplier_name\":{\"type\":[\"string\",\"null\"]},\"supplier_id\":{\"type\":[\"string\",\"null\"]},\"invoice_number\":{\"type\":[\"string\",\"null\"]},\"payment_reference\":{\"type\":[\"string\",\"null\"]},\"issue_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"due_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"net_by_rate\":{\"type\":[\"object\",\"null\"],\"additionalProperties\":{\"type\":\"number\"}},\"vat_by_rate\":{\"type\":[\"object\",\"null\"],\"additionalProperties\":{\"type\":\"number\"}},\"total\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"bank_account\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"supplier_name\",\"supplier_id\",\"invoice_number\",\"payment_reference\",\"issue_date\",\"due_date\",\"net_by_rate\",\"vat_by_rate\",\"total\",\"currency\",\"bank_account\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "anthropic/claude-sonnet-5",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "invoices", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Personal data detection · English (US)
import json
import os
import httpx
PROMPT = "Find all personal data of natural persons in the text below (for anonymisation) and return only\nJSON with these keys, each a list of strings (an empty list when there is none):\n\nperson_names full names of people, each person once, as written in the text\nemail_addresses email addresses of people\nphone_numbers phone numbers of people, as written\npostal_addresses home or delivery addresses of people, as written (street, city, ZIP)\nbirth_dates dates of birth (YYYY-MM-DD)\nnational_ids national identification numbers (e.g. SSN, birth numbers)\nbank_accounts bank account numbers of people, as written\n\nRules:\n- Only data of natural persons. Company names, company registration or tax numbers (EIN, IČO,\n VAT), generic company addresses such as info@ or support@, order and ticket numbers are not\n personal data: leave them out.\n- Dates that are not dates of birth (meetings, deliveries, start dates) are not personal data.\n- Do not invent or complete values; copy them from the text.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"person_names\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"email_addresses\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"phone_numbers\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"postal_addresses\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"birth_dates\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"national_ids\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"bank_accounts\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}}},\"required\":[\"person_names\",\"email_addresses\",\"phone_numbers\",\"postal_addresses\",\"birth_dates\",\"national_ids\",\"bank_accounts\"],\"additionalProperties\":false}")
def run(document_text: str) -> dict:
"""Text of the document in, the extracted fields as a dict out."""
response = httpx.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
json={
"model": "anthropic/claude-sonnet-5",
"temperature": 0,
"messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
# For a scan or photo, send [{"type": "text", "text": PROMPT},
# {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
"response_format": {
"type": "json_schema",
"json_schema": {"name": "pii", "strict": False, "schema": SCHEMA},
},
},
timeout=120,
)
response.raise_for_status()
return json.loads(response.json()["choices"][0]["message"]["content"])
Badge
Embed the current result on your site; the badge updates with every run.
<a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/contracts-en.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/contracts-cs.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/contracts-de.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/emails-en.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/emails-cs.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/emails-de.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/invoices-en.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/invoices-cs.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/invoices-de.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/invoices-es.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/invoices-fr.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/pii-en.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/pii-cs.svg" alt="Claude Sonnet 5 on ModelVerdict"></a><a href="https://modelverdict.ai/en/models/claude-sonnet-5/"><img src="https://modelverdict.ai/badge/claude-sonnet-5/pii-de.svg" alt="Claude Sonnet 5 on ModelVerdict"></a>Compare with another model
- Claude Sonnet 5 vs DeepSeek V4.1 Flash
- Claude Sonnet 5 vs Gemini 3.5 Flash Lite
- Claude Sonnet 5 vs Gemini 3.8 Flash
- Claude Sonnet 5 vs Gemma 4 31B
- Claude Sonnet 5 vs GPT-6 Luna
- Claude Sonnet 5 vs GPT-6 Sol
- Claude Sonnet 5 vs Mistral Medium 3.5
- Claude Sonnet 5 vs Mistral Small 4
- Claude Sonnet 5 vs Qwen3.8 Flash