SLA: What service credit, as a share …
Documento y preguntas (el modelo recibió exactamente este texto)
Hopkins-Mendoza EXHIBIT B TO THE MASTER SERVICES AGREEMENT – SERVICE LEVEL AGREEMENT 1. Scope and Term 1.1 This Service Level Agreement (the "SLA") defines the operating and support levels for the cloud application that Hopkins-Mendoza (the "Provider") operates for the Customer. 1.2 The monthly fee for the services under this SLA is $2,400, excluding applicable taxes. 1.3 This SLA is effective as of February 19, 2026 and replaces the version dated June 7, 2025. 2. Service Availability 2.1 The Provider guarantees a monthly availability of the production environment of 99.7%. The target availability of the test environment is 97% and does not give rise to service credits. 2.2 Availability is measured by the Provider's monitoring system at 5-minute intervals. The Provider sends the Customer a monthly availability report by the 5th business day of the following month. 2.3 Scheduled maintenance does not count as downtime. The Provider will notify the Customer of scheduled maintenance at least 10 days in advance, and maintenance will not exceed 4 hours per calendar month. 2.4 Maintenance is normally performed on weekends during night hours. 3. Support and Response Times 3.1 The Customer reports incidents through the service portal or by email to support@hopkins-mendoza.example. Priority 1 incidents are accepted 24 hours a day, 7 days a week. 3.2 For Priority 1 incidents (critical failure, the service is completely unavailable), the Provider responds within 4 hours of the report and restores the service or provides a workaround within 6 hours. 3.3 For Priority 2 incidents (limited functionality with no workaround), the Provider responds within 8 hours of the report during business hours. 3.4 Priority 3 and lower requests are handled no later than the next business day. 4. Service Credits 4.1 If the monthly availability of the production environment falls below the guaranteed level, the Customer is entitled to a service credit of 15% of the monthly fee for the affected month. 4.2 Service credits for any calendar month shall not exceed 50% of the monthly fee in total. The Customer must claim a credit in writing within 30 days after the end of the month. 4.3 Billing questions are handled by the Provider's sales team at billing@hopkins-mendoza.example. --- Questions: 1. What service credit, as a share of the monthly fee, is the Customer entitled to if the guaranteed availability is not met? (percent) 2. How far in advance must the Provider announce scheduled maintenance? (number of days) 3. To what email address does the Customer report incidents? (email address) 4. What is the response time for Priority 1 incidents? (number of hours) 5. For how long does the Provider keep data backups? (number of days) 6. What is the response time for Priority 2 incidents? (number of hours)
Veredicto
El más barato sin errores: Mistral Small 4, a 0,12 € por cada 1000 documentos.
- Mistral Large 4 tampoco comete errores, pero es 41,7× más caro.
Prompt enviado a todos los modelos
You answer questions about a business document. Below the document there are six numbered questions. Answer each of them using only the text of the document and return only JSON with the keys q1 to q6 (q1 = the answer to question 1, and so on). Rules: - Each question states in parentheses the unit or format of the answer. Answer in that unit. - Counts (of days, months, hours) and percentages as a number without the unit and without the % sign, e.g. "14" or "0.05". - Amounts as a number with a decimal point, no thousands separators and no currency symbol, e.g. "1500.00". - Dates as YYYY-MM-DD. - Time ranges in 24-hour format HH:MM-HH:MM, e.g. "08:00-17:00". - Email addresses exactly as they appear in the document. - If the document does not contain the answer to a question, return null. Do not guess and do not add information that is not in the document.
Cómo puntuamos
Coincidencia exacta por campo tras normalizar fechas, importes y espacios. La falta de diacríticos cuenta como error. El coste es el gasto real en tokens de esta ejecución, convertido al tipo de cambio indicado en el pie de página.
Lo que extrajo cada modelo
| Campo | Valor correcto | Mistral Small 46 / 6 campos | Gemma 4 31B6 / 6 campos | GPT-6 Luna6 / 6 campos | GLM-5.3 Flash6 / 6 campos | Qwen3.8 Flash6 / 6 campos | Claude Haiku 5.56 / 6 campos | DeepSeek V4.1 Flash6 / 6 campos | Gemini 3.5 Flash Lite6 / 6 campos | Mistral Medium 3.56 / 6 campos | GPT-6 Sol6 / 6 campos | Gemini 3.8 Flash6 / 6 campos | Claude Sonnet 56 / 6 campos | Claude Sonnet 5.56 / 6 campos | Mistral Large 46 / 6 campos |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pregunta 1 | 15 | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) | 15 (correcto) |
| Pregunta 2 | 10 | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) | 10 (correcto) |
| Pregunta 3 | support@hopkins-mendoza.example | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) | support@hopkins-mendoza.example (correcto) |
| Pregunta 4 | 4 | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) | 4 (correcto) |
| Pregunta 5 | — | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) | — (correcto) |
| Pregunta 6 | 8 | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) | 8 (correcto) |
| Tokens | entrada / salida | 890 / 51 | 924 / 52 | 919 / 184 | 853 / 250 | 1121 / 265 | 1671 / 245 | 1020 / 191 | 912 / 66 | 890 / 51 | 919 / 44 | 912 / 592 | 1669 / 53 | 1671 / 53 | 978 / 2377 |
| de ellos, razonamiento oculto | se factura como salida | 0 | 0 | 138 | 199 | 198 | 188 | 152 | 0 | 0 | 0 | 551 | 0 | 0 | 2208 |
| de ellos, desde caché | entrada con descuento | 214 | 256 | 0 | 0 | 256 | 0 | 270 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Coste de este documento | esta ejecución | 0,0001 € | 0,0001 € | 0,0002 € | 0,0002 € | 0,0002 € | 0,0003 € | 0,0003 € | 0,0004 € | 0,0016 € | 0,0021 € | 0,0027 € | 0,0036 € | 0,0036 € | 0,0052 € |
| Por cada 1000 documentos | para el mismo tipo de documento | 0,12 € | 0,14 € | 0,17 € | 0,23 € | 0,24 € | 0,27 € | 0,27 € | 0,40 € | 1,58 € | 2,10 € | 2,67 € | 3,56 € | 3,56 € | 5,18 € |
| Latencia | esta ejecución | 0,7 s | 3,2 s | 4,4 s | 2,7 s | 5,4 s | 2,3 s | 4,6 s | 0,8 s | 0,7 s | 2,2 s | 3 s | 1,7 s | 2,5 s | 27,6 s |