Expense Receipt OCR

Mindee's Receipt OCR API utilizes advanced deep-learning to automatically, accurately, and instantaneously extract essential details from your receipts. In under a second, the API transforms your PDFs or photos of receipts into structured data, ready for integration into your applications.

Mindee's Receipt OCR API intelligently extracts a range of crucial data points, including:

Core Receipt Details:

• Receipt Identification:
  • Receipt Number
• Purchase Information:
  • Purchase Date
  • Purchase Time
• Supplier Information:
  • Supplier Name
  • Supplier Address
  • Supplier Phone Number
  • Supplier Company Registrations
• Financial Breakdown:
  • Total Amount
  • Total Net (excluding taxes)
  • Total Tax
  • Breakdown of Taxes
  • Tip & Gratuity
  • Line Items
• Additional Information:
  • Expense Locale, Purchase Category, Purchase Subcategory ... and much more!

Global Reach with Enhanced Support

Mindee's Receipt OCR API is designed with a global perspective. Our intelligent algorithms can process receipts from various countries. While we strive for optimal accuracy across all regions, we offer officially enhanced support for certain regions. This means receipts originating from these countries may benefit from fine-tuned accuracy and better handling of region-specific formats and data.

Our Officially Supported Regions:

North America

• Canada and United States

Europe

• Austria, Belgium, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Hungary, Iceland, Ireland, Italy, Latvia, Luxembourg, Monaco, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland and United Kingdom

📘

Key Takeaway

You can still use the Mindee Receipt OCR API to process receipts from countries not listed above. While the level of specific support might vary, our core extraction capabilities remain robust, and you may still obtain valuable region-specific data and common receipt details.

Set up the API

📘

To begin using the Mindee Invoice OCR API, your first step is to create your API key.

1. You'll need a receipt. You can use the sample receipt provided below.

2. To access the Invoice API, navigate to the APIs Store and click on the Receipt card
3. Sample code in popular languages and for command-line usage can be found in the API Reference section, accessible via the left navigation menu under Documentation

from mindee import Client, PredictResponse, product

# Init a new client
mindee_client = Client(api_key="my-api-key-here")

# Load a file from disk
input_doc = mindee_client.source_from_path("/path/to/the/file.ext")

# Load a file from disk and parse it.
# The endpoint name must be specified since it cannot be determined from the class.
result: PredictResponse = mindee_client.parse(product.ReceiptV5, input_doc)

# Print a summary of the API result
print(result.document)

# Print the document-level summary
# print(result.document.inference.prediction)

const mindee = require("mindee");
// for TS or modules:
// import * as mindee from "mindee";

// Init a new client
const mindeeClient = new mindee.Client({ apiKey: "my-api-key-here" });

// Load a file from disk
const inputSource = mindeeClient.docFromPath("/path/to/the/file.ext");

// Parse the file
const apiResponse = mindeeClient.parse(
  mindee.product.ReceiptV5,
  inputSource
);

// Handle the response Promise
apiResponse.then((resp) => {
  // print a string summary
  console.log(resp.document.toString());
});

using Mindee;
using Mindee.Input;
using Mindee.Product.Receipt;

string apiKey = "my-api-key-here";
string filePath = "/path/to/the/file.ext";

// Construct a new client
MindeeClient mindeeClient = new MindeeClient(apiKey);

// Load an input source as a path string
// Other input types can be used, as mentioned in the docs
var inputSource = new LocalInputSource(filePath);

// Call the API and parse the input
var response = await mindeeClient
    .ParseAsync<ReceiptV5>(inputSource);

// Print a summary of all the predictions
System.Console.WriteLine(response.Document.ToString());

// Print only the document-level predictions
// System.Console.WriteLine(response.Document.Inference.Prediction.ToString());

#
# Install the Ruby client library by running:
# gem install mindee
#

require 'mindee'

# Init a new client
mindee_client = Mindee::Client.new(api_key: 'my-api-key')

# Load a file from disk
input_source = mindee_client.source_from_path('/path/to/the/file.ext')

# Parse the file
result = mindee_client.parse(
  input_source,
  Mindee::Product::Receipt::ReceiptV5,
  enqueue: true
)

# Print a full summary of the parsed data in RST format
puts result.document

# Print the document-level parsed data
# puts result.document.inference.prediction

import com.mindee.MindeeClient;
import com.mindee.input.LocalInputSource;
import com.mindee.parsing.common.PredictResponse;
import com.mindee.product.receipt.ReceiptV5;
import java.io.File;
import java.io.IOException;

public class SimpleMindeeClient {

  public static void main(String[] args) throws IOException {
    String apiKey = "my-api-key-here";
    String filePath = "/path/to/the/file.ext";

    // Init a new client
    MindeeClient mindeeClient = new MindeeClient(apiKey);

    // Load a file from disk
    LocalInputSource inputSource = new LocalInputSource(filePath);

    // Parse the file
    PredictResponse<ReceiptV5> response = mindeeClient.parse(
        ReceiptV5.class,
        inputSource
    );

    // Print a summary of the response
    System.out.println(response.toString());

    // Print a summary of the predictions
//  System.out.println(response.getDocument().toString());

    // Print the document-level predictions
//    System.out.println(response.getDocument().getInference().getPrediction().toString());

    // Print the page-level predictions
//    response.getDocument().getInference().getPages().forEach(
//        page -> System.out.println(page.toString())
//    );
  }

}

<form onsubmit="mindeeSubmit(event)" >
    <input type="file" id="my-file-input" name="file" />
    <input type="submit" />
</form>

<script type="text/javascript">
    const mindeeSubmit = (evt) => {
        evt.preventDefault()
        let myFileInput = document.getElementById('my-file-input');
        let myFile = myFileInput.files[0]
        if (!myFile) { return }
        let data = new FormData();
        data.append("document", myFile, myFile.name);

        let xhr = new XMLHttpRequest();

        xhr.addEventListener("readystatechange", function () {
            if (this.readyState === 4) {
                console.log(this.responseText);
            }
        });

        xhr.open("POST", "https://api.mindee.net/v1/products/mindee/expense_receipts/v5/predict");
        xhr.setRequestHeader("Authorization", "Token my-api-key-here");
        xhr.send(data);
    }
</script>

curl -X POST \\
  https://api.mindee.net/v1/products/mindee/expense_receipts/v5/predict \\
  -H 'Authorization: Token my-api-key-here' \\
  -H 'content-type: multipart/form-data' \\
  -F document=@/path/to/your/file.png

<?php

use Mindee\Client;
use Mindee\Product\Receipt\ReceiptV5;

// Init a new client
$mindeeClient = new Client("my-api-key-here");

// Load a file from disk
$inputSource = $mindeeClient->sourceFromPath("/path/to/the/file.ext");

// Parse the file
$apiResponse = $mindeeClient->parse(ReceiptV5::class, $inputSource);

echo $apiResponse->document;

• Replace my-api-key-here with your new API key, or use the "select an API key" feature and it will be filled automatically.
• Copy and paste the sample code of your desired choice in your application, code environment or terminal.
• Replace /path/to/my/file with the path to your receipt.

❗️

Remember to replace your API key!

4. Run your code. You will receive a JSON response with the receipt details.

API Response

Here is the full JSON response you get when you call the API:

{
  "api_request": {
    "error": {},
    "resources": [
      "document"
    ],
    "status": "success",
    "status_code": 201,
    "url": "https://api.mindee.net/v1/products/mindee/expense_receipts/v5/predict"
  },
  "document": {
    "id": "cc875b84-e83e-4799-aabe-4b2c18f44b27",
    "name": "sample_receipt.jpg",
    "n_pages": 1,
    "is_rotation_applied": true,
    "inference": {
      "started_at": "2021-05-06T16:37:28",
      "finished_at": "2021-05-06T16:37:29",
      "processing_time": 1.121,
      "pages": [
        {
          "id": 0,
          "orientation": {"value": 0},
          "prediction": { .. },
          "extras": {}
        }
      ],
      "prediction": { .. },
      "extras": {}
    }
  }
}

The extracted receipt data (predictions) can be accessed in two locations:

• Document-level predictions (document > inference > prediction): Contains the consolidated receipt data across all pages. For multi-page receipt images or PDFs, this prediction combines data extracted from every page into a single, unified receipt object.
• Page-level predictions (document > inference > pages[ ] > prediction): Contains predictions specific to each individual page. For single-image receipts, you will have only one element in this array. For multi-page receipts (like long till rolls), each page will have a corresponding element with its own extracted data.

Most extracted fields contain the following properties

• value: The extracted information as text or numeric data.
• confidence: A score (between 0 and 1) indicating the reliability of the extracted information.
• polygon: Coordinates indicating the exact position of the extracted data within the document image.
• page_id: Identifier of the page from which the data was extracted (particularly useful for multi-page documents).

Please note:

1. These fields are optional and might occasionally be empty or null, depending on the document's content or extraction reliability.
2. The structure described here applies to most standard fields, but it may vary slightly for more complex data objects.

Receipt Data Fields Summary

Detailed Field Information

Category and Subcategory

These fields provide information about the nature of the purchase:

• category: (String or null) The primary category of the expense. Possible values are: toll, food, parking, transport, accommodation, gasoline, telecom, miscellaneous.
• subcategory: (String or null) A more specific subcategory of the expense. The possible values depend on the selected category:
  • If category is food: Possible values are restaurant, shopping.
  • If category is transport: Possible values are plane, taxi, train.

Line Items

The line_items field is an array of objects, where each object represents a single item listed on the receipt. Each line item object contains the following information:

• description: (String or null) A textual description of the purchased item.
• quantity: (Number or null) The quantity of the item purchased.
• total_amount: (Number or null) The total amount for this line item, usually including taxes.
• unit_price: (Number or null) The price of a single unit of the item.

Orientation

The orientation field provides information about the orientation of the document:

• degrees: (Number or null) The number of degrees the document might be rotated. Possible values are 0, 90, or 270.

📘

Note:

Currently, the API does not support the processing of documents with a 180° rotation.

Support for 180° rotations is planned for an upcoming release.

Supplier Company Registrations

The supplier_company_registrations field is an array designed to capture various official registration numbers associated with the supplier. Each element in this array is an object containing:

• value: (String) The actual company registration number as extracted from the document.
• type: (String) A standardized code indicating the type of the registration number. The following values are possible:

Currencies

The currency sub-field within locale will contain one of the following ISO 4217 currency codes:

EUR, USD, GBP, CHF, CAD, AED, SGD, MAD, BRL, CNY, SEK, HKD, PLN, RUB, MXN, JPY, AUD, XPF, INR, MYR, ARS, KRW, IDR, PHP, TRY, DKK, RON, TWD, THB, XOF, ZAR, TND, NOK, DZD, QAR, SAR, CZK, HUF, VND, COP, CLP, ILS, OMR, XAF and OTHER.

Taxes

The taxes field is an array of tax objects, where each object provides details about a specific tax identified on the receipt. Each tax object includes the following information:

• rate: (Number or null) The tax rate applied. This value is typically expressed as a percentage (e.g., 5 represents 5%).
• base: (Number or null) The taxable amount to which the tax rate is applied.
• value: (Number or null) The calculated amount of the tax.
• code: (String or null) A code or identifier for the type of tax. Possible values include: TVA, VAT, MWST, IVA, BTW, UST, GST, QST, PST, HST, RST, STATE TAX, CITY TAX, SALES TAX, COUNTY TAX, LOCAL TAX, FLIGHT TAX, ACCOMMODATION TAX, and others.

Questions?
  Join our Slack