tfpdev
07 / 09Part 2 · Extraction

Pulling Data Out

Turning emails into customer records. Simply asking worked well. A strict fill-in form made the AI worse. Asking it to show its proof fixed that, but proof can't tell when a real number lands in the wrong box.

01Step 1

Try it

Pick an email from the company's inbox, or paste one. The AI fills in a customer record three different ways. You can see the right answers next to them.

Three ways of asking, same email
The email 

# Acme Tools: renewal price From: Acme Tools To: Vladislav Boerdonos Date: 21 January 2026 Subject: Brand Sprint quote is higher than last year Vladislav, The new Brand Sprint quote is €9,500. Last year we paid €8,500 for the same kind of engagement. I'm not arguing about the work. I want to know why the number moved, and whether the 2025 price still applies to us. We were happy with the last sprint. This is a question about the quote, not about whether to work with you again. Acme Tools From: Vladislav Boerdonos To: Acme Tools Date: 22 January 2026 Subject: Re: Brand Sprint quote is higher than last year Hello, The €8,500 is the 2025 price, and it's what you paid for the Brand Sprint you already did. That agreement isn't being re-priced. Nothing on the old invoice changes. The €9,500 is the 2026 price. 2026 prices apply to new agreements from 1 January 2026. A new Brand Sprint, quoted now, is a new agreement, so it sits on the 2026 card. 2025 prices still apply to 2025 agreements until those agreements renew. A finished fixed-price project doesn't renew. It ended at the price you paid. The quote is for a 2026 Brand Sprint: logo, colour, type, a 30-page brand book, and one simple logo animation, over 4 weeks. That is the engagement the €9,500 buys. I'm not able to put the 2025 number on a 2026 agreement. If you want to go ahead, it's €9,500, excl. VAT. If you want to wait, the quote doesn't keep the old price warm for you. Vladislav Boerdonos Account lead, Northwind Studio From: Acme Tools To: Vladislav Boerdonos Date: 22 January 2026 Subject: Re: Brand Sprint quote is higher than last year Thanks. That's a cleaner answer than I expected. We'll decide internally whether the new sprint goes ahead at €9,500. No need to hold anything. Acme Tools

What each version filled in
FieldJust askA strict formA form with proofRight answer
Company   Acme Tools
Who wrote   null
Their email   null
Our person   Vladislav Boerdonos
About   price
Plan   Brand Sprint
Price (€)   9500
Last email   2026-01-22

Green is right, pink is wrong, yellow is right but written in a way the CRM won't accept. A flag means the proof didn't check out, so a person should look. Click an answer in the last column to see the proof.

02Step 2

How often each one got it wrong

Every email was run three times through each version. Each box counts the mistakes in one field. Click it to see what the AI wrote and what was right.

Mistakes, of 54Just askA strict formA form with proof
Company
Who wrote
Their email
Our person
About
Plan
Price (€)
Last email
Fully right50/5415/5447/54
Who wrote · A strict formThe same fields as a form the AI must fill in, with fixed choices.
  1. Acme Tools: renewal price
    it wrote
    Acme Tools (contact name not given)
    right
    null
  2. Acme Tools: renewal price
    it wrote
    Acme Tools (contact name not given)
    right
    null
  3. Acme Tools: renewal price
    it wrote
    Acme Tools
    right
    null
  4. Atlas HR: rush request
    it wrote
    Atlas HR
    right
    null
  5. Atlas HR: rush request
    it wrote
    Atlas HR
    right
    null
  6. Atlas HR: rush request
    it wrote
    Atlas HR
    right
    null
  7. Becker Freight: old files
    it wrote
    Becker Freight
    right
    null
  8. Becker Freight: old files
    it wrote
    Becker Freight
    right
    null
  9. Becker Freight: old files
    it wrote
    Maksim Ryzjikov
    right
    null
  10. Brightlane: question about cancelling
    it wrote
    Brightlane
    right
    null
  11. Brightlane: question about cancelling
    it wrote
    Brightlane
    right
    null
  12. Brightlane: question about cancelling
    it wrote
    Brightlane
    right
    null
  13. Brightlane: rush agreed in February
    it wrote
    Vladislav Boerdonos
    right
    null
  14. Brightlane: rush agreed in February
    it wrote
    Vladislav Boerdonos
    right
    null
  15. Brightlane: rush agreed in February
    it wrote
    Vladislav Boerdonos
    right
    null
  16. Halden Health: can you do the fixes?
    it wrote
    Halden Health
    right
    null
  17. Halden Health: can you do the fixes?
    it wrote
    Halden Health
    right
    null
  18. Halden Health: can you do the fixes?
    it wrote
    Halden Health
    right
    null
  19. Kestrel Analytics: invoice question
    it wrote
    Kestrel Analytics finance
    right
    null
  20. Kestrel Analytics: invoice question
    it wrote
    Kestrel Analytics finance
    right
    null
  21. Kestrel Analytics: invoice question
    it wrote
    Kestrel Analytics finance
    right
    null
  22. Kestrel Analytics: urgent line
    it wrote
    Kestrel Analytics
    right
    null
  23. Kestrel Analytics: urgent line
    it wrote
    Kestrel Analytics
    right
    null
  24. Kestrel Analytics: urgent line
    it wrote
    UNKNOWN
    right
    null
  25. Open Data Collective: non-profit discount
    it wrote
    Open Data Collective
    right
    null
  26. Open Data Collective: non-profit discount
    it wrote
    Open Data Collective
    right
    null
  27. Open Data Collective: non-profit discount
    it wrote
    Open Data Collective
    right
    null
  28. Prospect: mobile app design?
    it wrote
    <UNKNOWN>
    right
    null
  29. Quill Notes: naming add-on
    it wrote
    Quill Notes
    right
    null
  30. Quill Notes: naming add-on
    it wrote
    Quill Notes
    right
    null
  31. Quill Notes: naming add-on
    it wrote
    Quill Notes
    right
    null
  32. Tidepool Payments: Figma editor access
    it wrote
    Tidepool Payments
    right
    null
  33. Verdant Energy: accessibility
    it wrote
    Verdant Energy
    right
    null
  34. Verdant Energy: accessibility
    it wrote
    Verdant Energy
    right
    null
  35. Verdant Energy: accessibility
    it wrote
    Verdant Energy
    right
    null
  1. Build 1Just askDescribe the fields and ask for the answer as text.428/432 fields right
  2. Build 2A strict formThe same fields as a form the AI must fill in, with fixed choices.384/432
  3. Build 3A form with proofEvery answer comes with the exact words from the email that prove it, and code checks them.425/432
03Step 3

What actually mattered

Just asking worked well

Simply describing the fields and asking got 428 of 432 answers right. The AI never once sent back something the computer couldn't read.

The strict form backfired

As a fill-in form, the AI got far fewer records fully right: 15 of 54. When no person was named, it wrote the company's name in the “who wrote” box instead of leaving it empty.

Asking for proof fixed it

Once the AI had to show the words that prove each answer, the “who wrote” mistakes nearly vanished: 35 down to 3.

Proof can't spot a number in the wrong box

Every wrong answer came with real words from the email. The AI didn't make things up; it put a real number in the wrong place, like the price of a job the client already bought.

It also raises false alarms

About one right answer in 8 got flagged because its proof was weak. A person has to look at those.

04Step 4

The code

What the two approaches do, in one sentence. The code is one click away.

A strict fill-in form made the AI worse at leaving boxes empty. Asking for the exact words behind every answer fixed that, but it can't tell when a real number is in the wrong box.

Show the code anyway
BreaksMade it worse
Strict schema, same descriptions.
lib/extract.ts · a strict formts
contact: {  anyOf: [{ type: "string" }, { type: "null" }],  description: "Full name of the person writing on the    client's side. Null if no person is named.",},// tool: { name: "save_record", strict: true, … }
Types, allowed values, null allowed, the same description as the plain prompt. The model stopped leaving the contact empty and wrote the company's name into it.
HoldsHeld
A quote per field, checked in code.
lib/extract.ts · a form with proofts
contact: {  value: { anyOf: [{ type: "string" }, { type: "null" }] },  quote: "The exact words from the thread that show          this value. Null only if the value is null.",},export function prove(field, value, quote, thread) {  if (value == null) return "null";  if (!quote) return "no-quote";  if (!flat(thread).includes(flat(quote))) return "not-in-thread";  return inQuote(field, value, quote) ? "verified" : "value-not-in-quote";}
Every field comes with the words that show it, and code checks them. It fixed the contact. It doesn't know whether a real number is in the right box.
05Step 5

Still curious?

A few more questions people ask. Open what you need.

Who decided the right answers?

I did, by reading each email. They're shown next to every answer above, so you can disagree. I changed one after the run, where two answers were both fair.

Why would a form make it worse?

I can't say for certain. In the form, the AI treated the company name at the top of the email as the person. Written as text, it didn't. Test this on your own emails before assuming a form is safer.

How would you catch a number in the wrong box?

Simple business rules written as code: a client moving to a smaller plan can't already be on it, a price for something already bought isn't a new quote. I haven't tested these here.

Is 18 emails enough?

Enough to see a pattern, not to promise a percentage. Every answer is from one run, stored as it came out, and you can read them all above.

What about PDFs and scanned paper?

These emails are clean text. Scans have to be turned into text first, which adds its own mistakes. Then checking the proof matters even more.