AI Data Extraction: Benchmarking GPT-4o vs. Veryfi

GPT-4o vs Veryfi for extracting financial data from irregular scans into Excel with Python: accuracy and cost per extraction across document types.

Benchmark · 12 Feb 2025

Overview

Benchmarked across multiple document types. Identified cost-per-extraction and accuracy tradeoffs between both approaches.

This project addresses the challenge of "dark data" trapped in irregular, low-quality scans and PDFs. By building a dual-engine extraction system, I compared OpenAI’s GPT-4o (a general-purpose Multimodal LLM) and Veryfi (a specialized financial OCR API).

The goal was to evaluate which system better handles real-world "noise" like faded text, line-item multipliers, and complex document layouts for downstream financial analysis.

Demo (Video)

<video controls preload="metadata" poster="/wizard/projects/pdf-excel-ai/pdf-excel-ai-preview.png" style="width:100%; height: auto; aspect-ratio: 16 / 9; border-radius:8px;"

Your browser does not support the vsideo tag.

Tech Stack

• Language: Python 3.11+
• AI/OCR: OpenAI API (GPT-4o), Veryfi SDK
• Data Orchestration: instructor (Pydantic-based extraction), pandas
• Preprocessing: pdf2image, Poppler
• Storage and Analysis: Microsoft Excel (openpyxl)

Python Scripts Explanation

<video controls preload="metadata" poster="/wizard/projects/pdf-excel-ai/python-script-ss.png" style="width:100%; height: auto; aspect-ratio: 16 / 9; border-radius:8px;"

Your browser does not support the vsideo tag.

OpenAI GPT-4o Integration

The OpenAI script utilizes the instructor library to enforce a strict Pydantic schema. Because GPT-4o is a reasoning model, the script uses specialized "System Prompts" to handle mathematical extraction.

• Process: Converts PDF to JPEG -> Base64 Encoding -> Vision-to-JSON extraction.
• The Logic: It doesn't just "read" text; it interprets it. When it sees "Item A 39.9x2", it calculates the unit price and quantity even if the receipt layout is cluttered.

Veryfi API Integration

The Veryfi script uses a dedicated financial OCR engine designed specifically for receipts and invoices.

• Process: Direct PDF/Image upload via the veryfi Python SDK.
• The Logic: It uses deterministic OCR rules to identify tax, subtotal, and vendor information instantly. It is pre-trained on millions of financial documents, making it extremely fast at identifying standard fields without complex prompting.

Key Learnings & Conclusion

• Reasoning vs. Recognition: GPT-4o showed superior "reasoning" for non-standard multipliers, whereas Veryfi showed superior "recognition" for standard tax and total fields.
• Accuracy vs. Cost:
  • OpenAI GPT-4o is the more economical choice with slightly lower raw OCR precision but higher adaptability.
  • Veryfi is highly accurate and purpose-built for finance, but comes at a significantly higher operational cost per document.

← All memories