Company policy: How many business days in advance …
Document and questions (the model received exactly this text)
Johnson Inc EMPLOYEE POLICY ON BUSINESS TRAVEL AND REMOTE WORK 1. General Provisions 1.1 This policy sets out the rules for business travel, expense reimbursement and remote work for all employees of Johnson Inc (the "Company"). 1.2 This policy is effective as of January 8, 2025 and supersedes the policy dated November 21, 2023. 1.3 Any expense above $2,500 must be approved in advance by the head of the department. Expenses above $10,000 also require the approval of the Chief Executive Officer. 2. Business Travel 2.1 All business trips must be approved in advance by the employee's direct manager. Employees may request a travel advance of up to 80% of the estimated trip cost. 2.2 The per diem for domestic business trips is $40 per calendar day for trips lasting 5 to 12 hours and $64 per calendar day for trips lasting more than 12 hours. 2.3 Hotel rates may not exceed $225 per night, or $375 per night in New York City. More expensive lodging requires the written consent of the head of the department. 2.4 Employees who use a personal vehicle for business travel are reimbursed at $0.65 per mile. Parking fees and tolls are reimbursed at actual cost against receipts. 3. Expense Reports and Reimbursement 3.1 Employees must submit an expense report with all receipts no later than 10 business days after returning from the trip. 3.2 Approved expenses are reimbursed within 14 days of approval, usually with the next regular payroll. 3.3 Questions about expense reports go to the accounting team at accounting@johnson.example; questions about the interpretation of this policy go to Human Resources at hr@johnson.example. 4. Remote Work 4.1 With the consent of their manager, employees may work remotely for up to 2 days per week. 4.2 Remote work days must be entered in the attendance system at least 5 business days in advance. 4.3 Employees are responsible for the security of Company equipment and data and must connect only through the Company VPN. Any costs of working from home are governed by a separate agreement. 4.4 On remote work days, employees must be reachable by phone and email during regular working hours. --- Questions: 1. How many business days in advance must remote work be entered in the attendance system? (number of business days) 2. What is the limit for a single taxi ride on a business trip? (amount in USD) 3. What is the per diem per calendar day for a domestic business trip lasting 5 to 12 hours? (amount in USD) 4. For at most how many days per week may an employee work remotely? (number of days) 5. What is the per diem per calendar day for a domestic business trip lasting more than 12 hours? (amount in USD) 6. What is the maximum hotel rate per night outside New York City? (amount in USD)
Verdict
Cheapest with no errors: Mistral Small 4 at $0.14 per 1,000 documents.
- Claude Sonnet 5 is also error-free, but 28.5× more expensive.
Prompt given to every model
You answer questions about a business document. Below the document there are six numbered questions. Answer each of them using only the text of the document and return only JSON with the keys q1 to q6 (q1 = the answer to question 1, and so on). Rules: - Each question states in parentheses the unit or format of the answer. Answer in that unit. - Counts (of days, months, hours) and percentages as a number without the unit and without the % sign, e.g. "14" or "0.05". - Amounts as a number with a decimal point, no thousands separators and no currency symbol, e.g. "1500.00". - Dates as YYYY-MM-DD. - Time ranges in 24-hour format HH:MM-HH:MM, e.g. "08:00-17:00". - Email addresses exactly as they appear in the document. - If the document does not contain the answer to a question, return null. Do not guess and do not add information that is not in the document.
How we score
Exact match per field after normalizing dates, amounts, and whitespace. Missing diacritics count as an error. Cost is the actual token spend of this run, converted at the rate shown in the footer.
Free with login
Every model's output for this document, field by field. The first three examples of each task are open.
Just your email, no password.