SLA: For how long does the Provider …
Document and questions (the model received exactly this text)
Hoffman, Spence and Villarreal EXHIBIT B TO THE MASTER SERVICES AGREEMENT – SERVICE LEVEL AGREEMENT 1. Scope and Term 1.1 This Service Level Agreement (the "SLA") defines the operating and support levels for the cloud application that Hoffman, Spence and Villarreal (the "Provider") operates for the Customer. 1.2 The monthly fee for the services under this SLA is $1,850, excluding applicable taxes. 1.3 This SLA is effective as of July 28, 2026 and replaces the version dated November 27, 2024. 2. Service Availability 2.1 The Provider guarantees a monthly availability of the production environment of 99.5%. The target availability of the test environment is 95% and does not give rise to service credits. 2.2 Availability is measured by the Provider's monitoring system at 5-minute intervals. The Provider sends the Customer a monthly availability report by the 5th business day of the following month. 2.3 Scheduled maintenance does not count as downtime. The Provider will notify the Customer of scheduled maintenance at least 14 days in advance, and maintenance will not exceed 4 hours per calendar month. 2.4 Maintenance is normally performed on weekends during night hours. 3. Support and Response Times 3.1 The Customer reports incidents through the service portal or by email to support@hoffman-spence.example. Priority 1 incidents are accepted 24 hours a day, 7 days a week. 3.2 For Priority 1 incidents (critical failure, the service is completely unavailable), the Provider responds within 1 hour of the report and restores the service or provides a workaround within 12 hours. 3.3 For Priority 2 incidents (limited functionality with no workaround), the Provider responds within 8 hours of the report during business hours. 3.4 Priority 3 and lower requests are handled no later than the next business day. 4. Service Credits 4.1 If the monthly availability of the production environment falls below the guaranteed level, the Customer is entitled to a service credit of 5% of the monthly fee for the affected month. 4.2 Service credits for any calendar month shall not exceed 50% of the monthly fee in total. The Customer must claim a credit in writing within 30 days after the end of the month. 4.3 Billing questions are handled by the Provider's sales team at accounts@hoffman-spence.example. --- Questions: 1. For how long does the Provider keep data backups? (number of days) 2. What is the response time for Priority 2 incidents? (number of hours) 3. What monthly availability of the production environment does the Provider guarantee? (percent) 4. What is the monthly fee for the services, excluding taxes? (amount in USD) 5. To what email address does the Customer report incidents? (email address) 6. What is the response time for Priority 1 incidents? (number of hours)
Verdict
Cheapest with no errors: Mistral Small 4 at $0.14 per 1,000 documents.
- Mistral Large 4 is also error-free, but 89.1× more expensive.
Prompt given to every model
You answer questions about a business document. Below the document there are six numbered questions. Answer each of them using only the text of the document and return only JSON with the keys q1 to q6 (q1 = the answer to question 1, and so on). Rules: - Each question states in parentheses the unit or format of the answer. Answer in that unit. - Counts (of days, months, hours) and percentages as a number without the unit and without the % sign, e.g. "14" or "0.05". - Amounts as a number with a decimal point, no thousands separators and no currency symbol, e.g. "1500.00". - Dates as YYYY-MM-DD. - Time ranges in 24-hour format HH:MM-HH:MM, e.g. "08:00-17:00". - Email addresses exactly as they appear in the document. - If the document does not contain the answer to a question, return null. Do not guess and do not add information that is not in the document.
How we score
Exact match per field after normalizing dates, amounts, and whitespace. Missing diacritics count as an error. Cost is the actual token spend of this run, converted at the rate shown in the footer.
Free with login
Every model's output for this document, field by field. The first three examples of each task are open.
Just your email, no password.