Heather Butler, April Cuevas, Terry Ramos
Texte du document (le modèle a reçu exactement ce texte)
From: Terry Ramos <terry.ramos@barton.example> Date: July 31, 2025 Subject: ERP Upgrade – budget and dates Hi all, Here is my proposal for the next phase of ERP Upgrade. The scope has grown since the original brief, so I suggest setting aside $5,500 for the fall marketing campaign. For comparison, last year the "staff training" item cost us $6,500. I would keep the interim report on October 7, 2025. The vendor quotes are attached, and I will fill in the detailed budget based on your comments. Please let me know what you think by the end of the week. Best regards, Terry Ramos ---------- From: Christopher Guerra <christopher.guerra@barton.example> Date: August 1, 2025 Subject: Re: ERP Upgrade – budget and dates Hi, I can't approve $5,500; we don't have that much in the project budget. I also think October 7, 2025 is unrealistic, because the numbers from accounting are still missing. Before we confirm anything, I need to see a more detailed budget. Let's discuss it at the next team meeting; please bring the numbers and a comparison of the quotes. Christopher Guerra ---------- From: Christopher Guerra <christopher.guerra@barton.example> Date: August 6, 2025 Subject: Re: ERP Upgrade – budget and dates Hi everyone, Thanks for today's meeting and for preparing the numbers. Here is a summary of what we finally agreed on: - Marketing campaign: $3,500 - Staff training: $11,500 - New date for the interim report: October 27, 2025 This means the proposal of $5,500 is off the table, and the original date of October 7, 2025 no longer applies. Action items: - Heather Butler: prepare the interim report for senior management, due October 27, 2025 - April Cuevas: order the equipment and arrange installation, ongoing - Terry Ramos: launch the campaign and prepare the web content, ongoing If I missed anything, please let me know. Best, Christopher Guerra
Verdict
Le moins cher sans erreur : Gemma 4 31B, à 0,12 € pour 1 000 documents.
- Mistral Large 4 ne fait aucune erreur non plus, mais coûte 56,4× plus cher.
Prompt soumis à chaque modèle
Summarize the document below (meeting minutes, a project status report or an email thread) in at
most 80 words.
Rules:
- Include every final decision: the approved amounts, the deadlines that apply and the people
responsible for the action items (full names).
- Do not mention rejected proposals or their amounts, and do not mention deadlines that were
changed (the original dates); state only the deadlines that apply.
- Copy amounts and dates exactly as in the document; do not round them.
- Write in the language of the document, factually and in a formal register, as running text
without headings.
- Return only JSON in the form {"summary": "…"}.
Notre notation
Correspondance exacte par champ, après normalisation des dates, des montants et des espaces. Les diacritiques manquants comptent comme une erreur. Le coût correspond à la consommation réelle de tokens de cette exécution, convertie au taux indiqué en bas de page.
Ce que chaque modèle a extrait
| Champ | Valeur correcte | Gemma 4 31B6 / 6 champs | GPT-6 Luna6 / 6 champs | Mistral Small 46 / 6 champs | Claude Haiku 5.56 / 6 champs | Qwen3.8 Flash6 / 6 champs | DeepSeek V4.1 Flash6 / 6 champs | Gemini 3.5 Flash Lite5 / 6 champs | GLM-5.3 Flash6 / 6 champs | Mistral Medium 3.56 / 6 champs | GPT-6 Sol6 / 6 champs | Claude Sonnet 5.56 / 6 champs | Claude Sonnet 56 / 6 champs | Gemini 3.8 Flash6 / 6 champs | Mistral Large 46 / 6 champs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Montants | 3500.0; 11500.0 | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) | 3500; 11500 (correct) |
| Échéances | 2025-10-27 | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) | 2025-10-27 (correct) |
| Responsables | Heather Butler; April Cuevas; Terry Ramos | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) | Heather Butler; April Cuevas; Terry Ramos (correct) |
| Propositions rejetées (ne doivent pas figurer)Montants de propositions que le document rejette. Ils ne doivent pas figurer dans le résumé ; ils induiraient le lecteur en erreur. | 5500.0 | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) |
| Échéances reportées (ne doivent pas figurer)Dates que le document repousse ensuite. Le résumé ne doit donner que la nouvelle date. | 2025-10-07 | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) | — (correct) |
| Longueur (mots max.)Le résumé ne doit pas dépasser la limite de mots de la consigne. | 80 | 49 / 80 (correct) | 50 / 80 (correct) | 53 / 80 (correct) | 71 / 80 (correct) | 61 / 80 (correct) | 57 / 80 (correct) | 81 / 80valeur erronée | 59 / 80 (correct) | 50 / 80 (correct) | 54 / 80 (correct) | 59 / 80 (correct) | 74 / 80 (correct) | 60 / 80 (correct) | 63 / 80 (correct) |
| Tokens | entrée / sortie | 677 / 78 | 624 / 164 | 653 / 84 | 1 099 / 453 | 776 / 703 | 682 / 548 | 665 / 123 | 615 / 1 253 | 653 / 81 | 624 / 178 | 1 099 / 125 | 1 097 / 156 | 665 / 1 052 | 647 / 3 192 |
| dont raisonnement caché | facturé comme sortie | 0 | 83 | 0 | 300 | 608 | 462 | 0 | 1 101 | 0 | 94 | 0 | 0 | 957 | 2 969 |
| dont en cache | entrée facturée à prix réduit | 3 | 0 | 0 | 0 | 256 | 90 | 0 | 0 | 69 | 0 | 0 | 0 | 0 | 0 |
| Coût de ce document | cette exécution | 0,0001 € | 0,0001 € | 0,0001 € | 0,0003 € | 0,0004 € | 0,0005 € | 0,0005 € | 0,0007 € | 0,0015 € | 0,0028 € | 0,0032 € | 0,0035 € | 0,0041 € | 0,0065 € |
| Pour 1 000 documents | pour ce même type de document | 0,12 € | 0,13 € | 0,14 € | 0,31 € | 0,38 € | 0,45 € | 0,47 € | 0,66 € | 1,46 € | 2,79 € | 3,17 € | 3,45 € | 4,09 € | 6,54 € |
| Latence | cette exécution | 4,4 s | 2,8 s | 1,1 s | 2,6 s | 10,5 s | 6,5 s | 0,9 s | 8 s | 0,9 s | 4,9 s | 1,8 s | 2,8 s | 4,2 s | 39,7 s |