Accuracy benchmark

We emailed 561 PDFs to five business owners.

A roofer, a plumber, a restaurant group, a creative agency and an importer. Each owner has an email inbox connected to DocStreamAI and a QuickBooks company of their own. Every PDF landed in the owner's inbox the way supplier mail really does, as phone photos, faxes and credits with the minus sign in the wrong place. DocStreamAI picked each email up from that inbox, worked out what every attachment was, read the fields and posted the result to the owner's QuickBooks. Then we checked every step against an answer key.

Supplier email
Owner's inbox
DocStreamAI
QuickBooks

PDFs emailed: 561PDFs emailed
emails to 5 owners: 493emails to 5 owners
vendors: 64vendors
deliberate traps: 18deliberate traps

Pick a company and see its report

97.4%

Benchmark report

fields read correctly on text PDFs

Text PDFs, scored September 30, 2026. Rows: fields read correctly.

Delivery

How 493 emails got from the inbox to QuickBooks

Every email went to one business owner's inbox. DocStreamAI picked it up there, classified each attachment, extracted the fields and posted the record to that owner's QuickBooks company. File names and subject lines never say "statement", "duplicate" or "two bills", so the only way to get these right is to read the page.

77% of the emails are the everyday case, one document attached. The rest are the shapes that go wrong.

382emails
One document per email
Right result: One record in the books.
25emails
Several invoices on one email
Right result: One bill for each attachment.
20emails
A receipt and an invoice together
Right result: An expense for the receipt and a bill for the invoice.
6emails
Several bills inside one PDF
Right result: Each invoice becomes its own bill. A long invoice stays one bill.
25emails
Vendor statements of account
Right result: No new bills. Seven of these emails also carry one new invoice, and only that invoice becomes a bill.
20emails
Emails about money with nothing attached
Right result: Nothing created.
10emails
The same invoice sent twice
Right result: Flagged as a duplicate, never posted twice.
5emails
"Did we ever pay this?"
Right result: No second bill.

Results · inside DocStreamAI

What DocStreamAI read, text PDFs

Measured inside DocStreamAI from each company's inbox, and checked against the answer key generated with each document. These are text PDFs, the kind billing software makes. Scanned PDFs have their own section below.

273 text PDFs sent, the kind billing software makes. 264 reached extraction and were scored against their answer keys.

97.4%3342 of 3432
Extraction
Values read correctly off the page: vendor, dates, totals, tax, line items.
Every key field right at once* on 92.0% of documents (243 of 264)
98.9%261 of 264
Detection
Each document recognised as what it is: invoice, receipt or credit memo.
98.2%268 of 273
Intake
Documents that made it from the email to a record DocStreamAI read.
Vendor statements kept out of the books: 17 of 17
98.6%206 of 209
Vendor
Read after the vendor was in QuickBooks, and found it. Full breakdown in the vendor section below.
Not measured yet
Categorization
The expense account chosen matches the one the company's chart of accounts intends.

Extraction, field by field

Vendor name100.0%264 of 264
Document number100.0%264 of 264
Document date97.0%256 of 264
Due date97.3%257 of 264
Currency100.0%264 of 264
Subtotal98.1%259 of 264
Tax99.2%262 of 264
Total98.5%260 of 264
Number of lines97.7%258 of 264
Line descriptionsLower severity93.6%247 of 264
Line quantitiesLower severity93.2%246 of 264
Unit pricesLower severity93.6%247 of 264
Line amounts97.7%258 of 264

* Line descriptions, quantities and unit prices count as lower severity and are left out of this figure. QuickBooks records each line by its amount, so a wrong quantity or unit price does not change what lands in the books. Counting them too: 84.1% (222 of 264).

The five companies

Pick a business. Open its books.

Each one is invented, with its own QuickBooks company, its own email inbox and user, and a chart of accounts built for its trade. Open a company to see its full chart. Submission was set to automatic, so a document went straight to QuickBooks unless one of DocStreamAI's checks held it for review.

Sacramento, CA

Roofing & exterior contracting

Kestovane Ridge Roofing LLC

96.7%fields read correctly

110 documents72 account chart

Bills: 55 bills
Receipts: 40 receipts
Credits: 10 credits
Statements: 5 statements

42 scanned, photographed or faxed · 11 layouts · 12 vendors

Open the full report

Reno, NV

HVAC & plumbing services

Tolvanrie Heating & Plumbing Inc.

97.0%fields read correctly

112 documents68 account chart

Bills: 57 bills
Receipts: 40 receipts
Credits: 10 credits
Statements: 5 statements

43 scanned, photographed or faxed · 11 layouts · 11 vendors

Open the full report

San Francisco, CA

Restaurant group

Fennick & Thorlow Restaurant Group LLC

96.4%fields read correctly

110 documents75 account chart

Bills: 55 bills
Receipts: 40 receipts
Credits: 10 credits
Statements: 5 statements

47 scanned, photographed or faxed · 10 layouts · 12 vendors

Open the full report

Chicago, IL

Marketing & creative agency

Wendlemarsh Creative Studio LLC

98.5%fields read correctly

109 documents69 account chart

Bills: 54 bills
Receipts: 40 receipts
Credits: 10 credits
Statements: 5 statements

42 scanned, photographed or faxed · 9 layouts · 14 vendors

Open the full report

Portland, OR

E-commerce & import

Amblequist Row Home Goods Inc.

98.5%fields read correctly

109 documents71 account chart

Bills: 54 bills
Receipts: 40 receipts
Credits: 10 credits
Statements: 5 statements

45 scanned, photographed or faxed · 11 layouts · 14 vendors

Open the full report

Image quality

One invoice, five ways it shows up

219 of the documents carry no text layer at all. Scans with speckle and skew, photos taken at an angle, faxes crushed to black and white. The picture is all there is to read.

331
Clean PDF
Straight from the vendor's system
142
Office scan
Speckle, skew, grey paper
50
Phone photo
At an angle, shadow across it
14
Fax
One bit black and white
13
Photocopy
Dark band down one edge

Deliberate difficulty

18 traps, set on purpose

Most of them lead to a confident, plausible, wrong number. Nothing looks broken, which is exactly why they cost money. Most damaging first.

Subtotal1,284.00
Sales tax99.51
TOTAL1,383.51
Paid by card-1,383.51
BALANCE DUE0.00
The total is not the amount owed
The largest number on the page, the last number, and the number beside the word Total are all the same wrong number. Read it and you pay a bill that was already settled.
20 documents, modelled on a real customer document
Shingles, 12 bdl418.20
Drip edge, 10 ft46.90
Rewards year to date41,820.00
Amount due465.10
A bigger, unrelated number in the money column
It is frequently the largest figure on the document, often a hundred times the actual total, so anything that reaches for the biggest number returns it.
15 documents, modelled on a real customer document
Skelvane Ridge Building Products
The Skelvane Ridge Building Products Co
SKELVANE RIDGE BLDG PRODUCTS
one supplier, three vendor records
The same vendor, spelled several ways
Each spelling can create a separate vendor record, splitting one supplier's history across several records and breaking every report built on it.
15 documents, modelled on a real customer document
Returned material56.82-
Restock fee4.00
The minus sign after the number
Stop reading at the last digit and you get a positive 56.82. The sign is silently dropped, so a credit posts as a charge.
10 documents, modelled on a real customer document
Credit applied(56.82)
Invoice 20417512.40
Negatives in parentheses
Drop the parentheses and the sign flips. It is the same failure as above, arriving by a different route.
10 documents, modelled on a real customer document
Lines, tax included216.00
Includes tax16.46
Total216.00
Tax already inside the line amounts
Add the tax on top, as most documents require, and the bill is overstated by the tax amount.
10 documents, modelled on a real customer document
INV 4471 03/14640.00
INV 4478 03/14640.00
same vendor, same PO, two real bills
Two real invoices that look like one
Matching on vendor and amount collapses them into one, and a legitimate second bill is never paid.
10 documents, modelled on a real customer document
INV 2093 03/021,118.40
INV 2093 03/021,118.40
arrived again four days later
The same invoice delivered twice
Treat them as two documents and the same bill gets paid twice.
10 documents, modelled on a real customer document
€1.240,50£86.20C$312.00MX$4,150.00
Not in dollars
Read the number without the currency and the amount is right but the money is wrong.
10 documents
p.1 Pellworth FastenersINV 5521
p.2 Dunmere Sheet MetalINV 0907
one file, two vendors
Two separate invoices in one file
Read as a single document, two vendors' money merges into one bill and the second supplier never gets recorded.
5 documents, modelled on a real customer document
Charges480.00
Payment received-400.00
Credit memo-80.00
Balance0.00
Charges, then payments, then nothing owed
Post it as a payable and you have created a bill for money that is not owed.
5 documents, modelled on a real customer document
Invoice date04/09/26
Due date0/00/00
A date field that prints 0/00/00
The field is not empty, so every check for a missing value passes it straight through. Only an attempt to parse it reveals it is junk.
5 documents, modelled on a real customer document
TermsNet 30
Due date
Payment terms, but no due date
The due date has to be calculated from the terms and the invoice date, or a perfectly good bill gets held back for review.
5 documents, modelled on a real customer document
Qty PriceExt
12 8.5096.90
12 × 8.50 is 102.00, the vendor billed 96.90
Quantity times price does not equal the line total
Recalculate from quantity and unit price and you silently restate the invoice to a number the vendor never charged.
5 documents, modelled on a real customer document
Terms
Due date
No due date at all
This one should be held for a human. A bill with no due date is often a quote or an order confirmation that was misread as an invoice.
5 documents, modelled on a real customer document
Issued04/01/26
Due04/01/26
monthly plan, paid on issue
Issue date and due date are the same
It is not an overdue bill, and treating it as one puts a recurring charge into a chasing workflow.
5 documents, modelled on a real customer document
Freight.00
Fuel surcharge12.00
Amounts printed as .00
Strict number parsing rejects it, and a lenient parser can attach it to the wrong field.
5 documents, modelled on a real customer document
Romaine, cs 6141.00
short 2 cs, credit to follow
The driver wrote on it
The printed total is no longer what is owed, and only the handwriting knows the real number.
1 document, modelled on a real customer document

Results · wrong fields

When a field was wrong, did it reach the books?

Submission was automatic, so an invoice went to QuickBooks unless something kept it out. Here is every invoice read with a key field wrong, by how serious the mistake was.

40 invoices were read with at least one key field wrong, text and scanned together. 22 of them reached QuickBooks.

What was wrongInvoicesKept outPosted

Total, tax or subtotal wrong3invoicesKept out 2 · Posted 11
Document date wrong12invoicesKept out 5 · Posted 77
Due date wrong, document date right4invoicesKept out 1 · Posted 33
Vendor name or document number wrong1invoicesKept out 1 · Posted 00
Lower severity only: a description, quantity or unit price20invoicesKept out 9 · Posted 1111

Kept out of the booksPosted to QuickBooks

What "kept out" covers: 9 were held for review by one of DocStreamAI's checks. 5 were read as recurring charges with no matching expense and left unposted. 4 were held only because QuickBooks was rate limiting this run, which we count here but do not claim as a check working.

Results · vendors

Did each document find its vendor?

A vendor only needs to be set up once. After that, every bill, receipt and credit from them should land on the same vendor in QuickBooks, whatever way the name is written.

How this was tested

The test used 59 vendors across the five companies. When a document comes from a vendor that is not in the company's QuickBooks yet, DocStreamAI adds the vendor at the moment that document is posted. From then on, every document from the same vendor should find the vendor that is already there and use it, even when the name is written differently. This measure checks exactly that.

98.8%320 of 324

Documents read after their vendor was already in QuickBooks that found that vendor. Text and scanned PDFs, every document type.

Vendor name written differentlythe spelling trap100.0%11 of 11
Text PDFs98.6%206 of 209
Scanned PDFs99.1%114 of 115
Invoices100.0%159 of 159
Receipts97.5%118 of 121
Credit memos97.7%43 of 44

Roofing: 65 of 65
HVAC & plumbing: 77 of 80
Restaurant: 69 of 69
Creative agency: 63 of 63
E-commerce: 46 of 47

320found their vendor

These documents were read after their vendor was in QuickBooks, and DocStreamAI matched them to it. That includes all 11 documents where the vendor's name was written a different way, such as with "The" in front or "Co" at the end.

4missed

These were read after their vendor was in QuickBooks and did not use it. 3 HVAC & plumbing receipts tried to add the vendor again (1 of them posted). One E-commerce credit memo matched a different vendor record, and was posted.

5left out: two documents at almost the same moment

Picture two emails from the same new vendor arriving a few seconds apart. DocStreamAI starts posting the first one, and posting is the step that adds the vendor to QuickBooks. While that is still happening, it reads the second one and looks for the vendor. The vendor is a few seconds away from existing, so there is nothing to find yet.

Each of these 5 was read within one minute of its vendor being added. The vendor's name played no part, so we leave them out of the score in both directions: they do not count as found and they do not count as missed. We list them here so the gap stays visible.

78left out: the vendor was not in QuickBooks yet

When these were read, no document from their vendor had been posted, so the vendor did not exist in QuickBooks and there was nothing to find. Treating the vendor as new was the right call.

Most are receipts. A receipt waits to post until its payment card is matched in QuickBooks, and most receipts in this run were still waiting, so their vendors had not been added yet. The rest were invoices and credit memos held for review or waiting on another step.

Results · scanned PDFs

Scanned PDFs, on their own

A scan, phone photo, fax or photocopy saved as a PDF is a picture of a page. We report these separately because one part of handling them is still being built.

179 scanned PDFs sent: office scans, phone photos, faxes and photocopies saved as PDF. 143 reached extraction and were scored.

Once a scan is read

96.3%1790 of 1859
Fields read correctly
90.2%129 of 143
Every key field right*
100.0%143 of 143
Type detected

Text PDFs, for comparison: 97.4% of fields read correctly, and every key field right* on 92.0% of documents.

Getting a scan to that point

The only attachment on its email

143 of 143

Picked up and read every time.

Sent with other attachments

0 of 36

Not picked up yet. The other attachments on the same email were read as normal.

Scanned vendor statements kept out

0 of 8

Not recognised as statements yet. 6 of the 8 were posted as bills.

Planned for a future update: picking up scanned PDFs that arrive alongside other attachments, and recognising a scanned vendor statement for what it is.

All 452 documents together, text and scanned: reached extraction 90.9% (411 of 452), type detected 99.3% (404 of 407), fields read correctly 97.0% (5132 of 5291), every key field right* 91.4% (372 of 407). Vendor statements kept out of the books: 17 of 25.

Results · in QuickBooks

What landed in QuickBooks

The second place accuracy counts: the record in QuickBooks matches the document and what DocStreamAI read. Measured separately, per record type.

Not measured yet
Bills
Created from invoices.
Not measured in this run.
Not measured yet
Vendor credits
Created from credit memos.
Not measured in this run.
Not measured yet
Expenses
Created from receipts.
Waiting on the card match. A receipt posts once its payment card is matched to an account in QuickBooks, and that is not set up in these companies yet.

What this test does and does not prove

The documents are synthetic. The businesses, vendors and amounts are invented, and no customer's paperwork appears on this page. The difficult shapes are modelled on formats we have seen arrive from real suppliers.

So every answer is known. Because we generated each document, we know exactly what it says. Every figure is checked against that answer key, never against a second human reading.

The first 30 documents are left out. They went out in a trial run before the five companies had their final names, so they are addressed to a company that no longer exists.

33 more are left out because we drew them wrong. 31 files meant as invoices came out as account statements, with a balance forward, payments and an aging table, and one two-bill file says "receipt" on its face. Scoring them against the wrong answer key would measure our test kit, not DocStreamAI.

Text and scanned PDFs are reported apart. The headline figures are text PDFs. Scanned PDFs have their own section, with the part still being built stated plainly, and the figures for everything together are printed under it.

Your own documents are the real test. A benchmark on our own set is evidence. Whether it holds up on your vendors' paperwork is exactly what a trial is for.

We emailed 561 PDFs to five business owners.

How 493 emails got from the inbox to QuickBooks

What DocStreamAI read, text PDFs

Extraction

Detection

Intake

Vendor

Categorization

Pick a business. Open its books.

Roofing & exterior contracting

HVAC & plumbing services

Restaurant group

Marketing & creative agency

E-commerce & import

One invoice, five ways it shows up

18 traps, set on purpose

The total is not the amount owed

A bigger, unrelated number in the money column

The same vendor, spelled several ways

The minus sign after the number

Negatives in parentheses

Tax already inside the line amounts

Two real invoices that look like one

The same invoice delivered twice

Not in dollars

Two separate invoices in one file

Charges, then payments, then nothing owed

A date field that prints 0/00/00

Payment terms, but no due date

Quantity times price does not equal the line total

No due date at all

Issue date and due date are the same

Amounts printed as .00

The driver wrote on it

When a field was wrong, did it reach the books?

Did each document find its vendor?

Scanned PDFs, on their own

What landed in QuickBooks

Bills

Vendor credits

Expenses

What this test does and does not prove

Send it your own paperwork.