Programming for Everyone

Posted: **Sun Nov 03, 2024 8:51 am**

Approach for the Task:

#### Steps to Complete the Task:

1. **Split PDF Files into Documents**:
- **Analyze Structure**: Set up software to recognize where documents would end and begin within multi-page PDFs by using OCR tools that identify where one document ends and another begins. This is normally done via keywords or attributes signaling the start of a new document.
- **Create Separate Files**: After the boundaries are identified, the original PDF will be divided into more than one file, each with one document inside it, up to a maximum of 3 pages.

2. **Text Recognition and Attribute Extraction:**
- **Data Extraction by Using OCR:** Raw images of pages will be processed using OCR to recognize the document title, number, and date, and convert them into text. The ideal tools for banking document processing are ABBYY FineReader and Tesseract OCR.
- **Parse Attributes**: Based on provided data formats, devise a system to extract from the text titles, numbers, and dates using for dates regular expressions and keywords for document titles.

3. **Save Files and Attributes**:
- **Save PDF Files**: Save into one searchable PDF each document.
- **Create Registry**: Create one Excel file listing name and attributes of each document: filename, attributes-title, number, date.

4. **Upload for Verification**:
* **Prepare Files to Upload**: Upload prepared PDFs and an Excel registry on the platform for verification by operators. This can be automated using an API, which is provided by the platform, or it can also be done by writing scripts for bulk uploads.

#### Tools and Technologies:
* **OCR Platform**: ABBYY FineReader for OCR and recognition of text.
- **Programming Language**: Python, which will help in the creation of the automation for splitting, parsing, and saving attributes.
- **Libraries**: PyMuPDF, pdfplumber, or PyPDF2 for processing PDFs, Pandas for creating the Excel registry, and regular expressions for extracting attributes.
- **Server Environment**: Creation of a server environment for OCR processing and splitting of documents, or using third-party cloud services for OCR.

#### Sample Algorithm:
1. The system reads the PDF and detects the document boundaries.
2. Make OCR of each section.
3. Scrape attributes and persist data in a data structure.
4. Save each one of the documents as PDFs.
5. Provide the Excel register.
6. Upload the documents and registry on the platform.

#### Note:
It is agreed that during the development, the format of the Excel registry should be agreed upon, including all the attributes necessary for its processing, with the aim of guaranteeing the correct recognition of information.

Posted: **Sun Nov 03, 2024 8:53 am**

Here’s a sample code for processing PDF files using OCR to create documents, store attributes, and export to Excel. We'll use the `PyMuPDF`, `pytesseract` for OCR, and `pandas` for Excel handling. Let's assume we have a PDF file that needs to be split into separate documents, recognize text, and save attributes to Excel.

### Description

This code performs the following functions:

1. **Splits the PDF into individual documents**.
2. **Applies OCR to recognize text** on each page.
3. **Stores attributes** (title, number, date) in an Excel format.

### Installation of Libraries

First, install the required libraries:

```bash
pip install PyMuPDF pytesseract pandas openpyxl
```

You also need to install Tesseract OCR. Download it from the [official site](https://github.com/tesseract-ocr/tesseract) and add it to your PATH.

### Code

```python

Code: Select all

import fitz  # PyMuPDF
import pytesseract
import pandas as pd
from PIL import Image

# Configure the path to Tesseract
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'

# Function to recognize text in an image
def ocr_image(image):
    text = pytesseract.image_to_string(image, lang='eng')
    return text

# Function to split PDF into documents
def split_pdf_to_documents(pdf_path):
    doc = fitz.open(pdf_path)
    documents = []
    for page in doc:
        # Save each page as an image
        pix = page.get_pixmap()
        img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
        documents.append(img)
    doc.close()
    return documents

# Function to extract attributes from text
def extract_attributes(text):
    lines = text.split('\n')
    title, number, date = '', '', ''
    for line in lines:
        if 'Title:' in line:
            title = line.split('Title:')[-1].strip()
        elif 'Number:' in line:
            number = line.split('Number:')[-1].strip()
        elif 'Date:' in line:
            date = line.split('Date:')[-1].strip()
    return title, number, date

# Main processing function
def process_pdf(pdf_path, output_excel):
    documents = split_pdf_to_documents(pdf_path)
    data = []
    for img in documents:
        text = ocr_image(img)
        title, number, date = extract_attributes(text)
        data.append({'Title': title, 'Number': number, 'Date': date})

    # Save results to Excel
    df = pd.DataFrame(data)
    df.to_excel(output_excel, index=False)

# Run the program
pdf_path = 'input.pdf'  # Specify the path to your PDF
output_excel = 'output.xlsx'
process_pdf(pdf_path, output_excel)
print("Processing completed. Results saved in", output_excel)

```

### Code Explanation

- **`ocr_image(image)`**: Uses Tesseract to recognize text from an image.
- **`split_pdf_to_documents(pdf_path)`**: Opens the PDF file and saves each page as an image.
- **`extract_attributes(text)`**: Extracts the title, number, and date from the recognized text. You can customize the extraction template for specific documents.
- **`process_pdf(pdf_path, output_excel)`**: The main function that integrates all steps: splitting the PDF, performing OCR, and saving results to Excel.

### Usage

1. Place the PDF file to be processed in the same directory as the code or specify the full path to the file.
2. Run the code. The results will be saved in an Excel file containing attributes of each document.

This code is a basic template and can be refined according to the specifics of your documents and requirements.

Programming for Everyone

Unlocking the Secrets of Successful Project Management

Unlocking the Secrets of Successful Project Management

Re: Unlocking the Secrets of Successful Project Management