ModelVerdict

SAS Marie et Associés · FAC25070013

FrenchScan13 fields

Document fr-p0034 · layout fr-L06 · 13 fields · public part of the dataset · Same invoice in English (US) → · Report an error in this example

SAS Marie et Associés · FAC25070013

Verdict

Cheapest without an error: DeepSeek V4.1 Flash at $0.57 per 1,000 documents.

  • GPT-6 Sol is also error-free but 28.4× more expensive.
  • Gemma 4 31B is 2.6× cheaper, but: wrong value (Supplier ID (EIN, IČO…)).
  • Mistral Small 4 is 1.1× cheaper, but: wrong value (VAT ID).

Prompt given to every model

Extrayez du document joint les informations suivantes et renvoyez uniquement du JSON avec ces clés :

supplier_name      Nom de l'émetteur de la facture (exactement comme sur le document, accents et forme juridique compris)
supplier_id        Numéro SIRET de l'émetteur (14 chiffres ; ni le numéro de TVA intracommunautaire ni le seul numéro RCS)
vat_id             Numéro de TVA intracommunautaire de l'émetteur
invoice_number     Numéro du document
payment_reference  Référence à rappeler lors du paiement
issue_date         Date d'émission (YYYY-MM-DD)
tax_point_date     Date de la livraison ou de la prestation de services (YYYY-MM-DD)
due_date           Date d'échéance / « à payer avant le » (YYYY-MM-DD)
net_by_rate        Montant HT par taux de TVA, p. ex. {"20": 18450.00, "5.5": 1200.00}
vat_by_rate        Montant de la TVA par taux, p. ex. {"20": 3690.00}
total              Montant total TTC / net à payer (nombre, négatif pour un avoir)
currency           Devise selon ISO 4217 (EUR, GBP, …)
bank_account       IBAN de l'émetteur

Règles :
- Ne renvoyez aucune information concernant le client, uniquement celles de l'émetteur.
- Si une information ne figure pas sur le document, renvoyez null. N'inventez rien.
- Si le document ne comporte pas de ventilation de la TVA (micro-entreprise, demande d'acompte), renvoyez net_by_rate et vat_by_rate à null.
- Montants sous forme de nombres avec un point décimal, sans séparateur de milliers et sans devise.

How we score

Exact match per field after normalising dates, amounts and whitespace. Missing diacritics count as an error. Cost is the real token spend of this run, converted at the rate shown in the footer.

What each model extracted

FieldCorrect valueGemma 4 31B12 / 13 fieldsMistral Small 411 / 13 fieldsDeepSeek V4.1 Flash13 / 13 fieldsGPT-6 Luna13 / 13 fieldsGemini 3.5 Flash Lite13 / 13 fieldsQwen3.8 Flash13 / 13 fieldsGemini 3.8 Flash13 / 13 fieldsMistral Medium 3.513 / 13 fieldsClaude Sonnet 513 / 13 fieldsGPT-6 Sol13 / 13 fields
SupplierSAS Marie et AssociésSAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)SAS Marie et Associés (correct)
Supplier ID (EIN, IČO…)49996968100117 49969100117wrong value49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)49996968100117 (correct)
VAT IDFR92499969681FR92499969681 (correct) FR92499996968wrong valueFR92499969681 (correct)FR92499969681 (correct)FR92499969681 (correct)FR92499969681 (correct)FR92499969681 (correct)FR92499969681 (correct)FR92499969681 (correct)FR92499969681 (correct)
Invoice numberFAC25070013FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)FAC25070013 (correct)
Payment referenceCL67627CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)CL67627 (correct)
Invoice date2025-07-252025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)
Tax point2025-07-252025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)2025-07-25 (correct)
Due date2025-08-012025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)2025-08-01 (correct)
Net by tax rate20 %: 8188.20, 5.5 %: 160.0520 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)20 %: 8188.20, 5.5 %: 160.05 (correct)
Tax by rate20 %: 1637.64, 5.5 %: 8.8020 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)20 %: 1637.64, 5.5 %: 8.80 (correct)
Total9994.699994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)9994.69 (correct)
CurrencyEUREUR (correct)EUR (correct)EUR (correct)EUR (correct)EUR (correct)EUR (correct)EUR (correct)EUR (correct)EUR (correct)EUR (correct)
Bank accountFR7610107021930003867556010FR76 1010 7021 9300 0386 7556 010 (correct) FR761010702193003867556010wrong valueFR7610107021930003867556010 (correct)FR7610107021930003867556010 (correct)FR7610107021930003867556010 (correct)FR76 1010 7021 9300 0386 7556 010 (correct)FR7610107021930003867556010 (correct)FR7610107021930003867556010 (correct)FR7610107021930003867556010 (correct)FR7610107021930003867556010 (correct)
Tokensinput / output717 / 2962,610 / 2401,771 / 4795,170 / 3121,529 / 2923,291 / 2,5761,529 / 1,0232,610 / 2395,387 / 2605,170 / 323
of which hidden reasoningbilled as output0028210802,33173300175
of which from cacheinput billed at a discount4400640006400000
Cost of this documentthis run$0.0002$0.0005$0.0006$0.0008$0.0012$0.0016$0.0050$0.0057$0.0134$0.0162
Per 1,000 documentsfor the same kind of document$0.22$0.54$0.57$0.80$1.19$1.62$4.98$5.71$13.37$16.15
Latencythis run10.9 s3 s7.6 s4.3 s1.7 s36.3 s4.7 s3.3 s5.5 s7.2 s