ModelVerdict

SLA: As of what date is the …

English (US)Document6 fields

Document en-p0009 · layout sla · 6 fields · public part of the dataset · Same invoice in Czech → · Report an error in this example

Document and questions (the model received exactly this text)

Myers and Sons
EXHIBIT B TO THE MASTER SERVICES AGREEMENT – SERVICE LEVEL AGREEMENT

1. Scope and Term
1.1 This Service Level Agreement (the "SLA") defines the operating and support levels for the cloud application that Myers and Sons (the "Provider") operates for the Customer.
1.2 The monthly fee for the services under this SLA is $4,990, excluding applicable taxes.
1.3 This SLA is effective as of July 26, 2025 and replaces the version dated August 30, 2023.

2. Service Availability
2.1 The Provider guarantees a monthly availability of the production environment of 99.8%. The target availability of the test environment is 97% and does not give rise to service credits.
2.2 Availability is measured by the Provider's monitoring system at 5-minute intervals. The Provider sends the Customer a monthly availability report by the 5th business day of the following month.
2.3 Scheduled maintenance does not count as downtime. The Provider will notify the Customer of scheduled maintenance at least 3 days in advance, and maintenance will not exceed 4 hours per calendar month.
2.4 Maintenance is normally performed on weekends during night hours.

3. Support and Response Times
3.1 The Customer reports incidents through the service portal or by email to help@myers.example. Priority 1 incidents are accepted 24 hours a day, 7 days a week.
3.2 For Priority 1 incidents (critical failure, the service is completely unavailable), the Provider responds within 2 hours of the report and restores the service or provides a workaround within 12 hours.
3.3 For Priority 2 incidents (limited functionality with no workaround), the Provider responds within 4 hours of the report during business hours.
3.4 Priority 3 and lower requests are handled no later than the next business day.

4. Service Credits
4.1 If the monthly availability of the production environment falls below the guaranteed level, the Customer is entitled to a service credit of 15% of the monthly fee for the affected month.
4.2 Service credits for any calendar month shall not exceed 50% of the monthly fee in total. The Customer must claim a credit in writing within 30 days after the end of the month.
4.3 Billing questions are handled by the Provider's sales team at accounts@myers.example.

---

Questions:
1. As of what date is the document effective? (date)
2. How far in advance must the Provider announce scheduled maintenance? (number of days)
3. What is the response time for Priority 1 incidents? (number of hours)
4. What is the monthly fee for the services, excluding taxes? (amount in USD)
5. For how long does the Provider keep data backups? (number of days)
6. What monthly availability of the production environment does the Provider guarantee? (percent)

Verdict

Cheapest with no errors: GPT-6 Luna at $0.15 per 1,000 documents.

  • Mistral Large 4 is also error-free, but 57.9× more expensive.

Prompt given to every model

You answer questions about a business document. Below the document there are six numbered
questions. Answer each of them using only the text of the document and return only JSON with the
keys q1 to q6 (q1 = the answer to question 1, and so on).

Rules:
- Each question states in parentheses the unit or format of the answer. Answer in that unit.
- Counts (of days, months, hours) and percentages as a number without the unit and without the %
  sign, e.g. "14" or "0.05".
- Amounts as a number with a decimal point, no thousands separators and no currency symbol,
  e.g. "1500.00".
- Dates as YYYY-MM-DD.
- Time ranges in 24-hour format HH:MM-HH:MM, e.g. "08:00-17:00".
- Email addresses exactly as they appear in the document.
- If the document does not contain the answer to a question, return null. Do not guess and do not
  add information that is not in the document.

How we score

Exact match per field after normalizing dates, amounts, and whitespace. Missing diacritics count as an error. Cost is the actual token spend of this run, converted at the rate shown in the footer.

Free with login

Every model's output for this document, field by field. The first three examples of each task are open.

Just your email, no password.

Log in with email

We use your email only to log you in (and for the digest if you tick it). No spam.