World-class Document Processing Pipeline with Ground X

This application demonstrates how to build a Document Processing Pipeline that processes complex documents with tables, figures, and dense text using GroundX's state-of-the-art parsing technology. Users can upload documents and receive comprehensive insights including extracted text, semantic analysis, key insights, and interactive AI-powered document queries.

We use:

• Ground X for SOTA document processing and X-Ray analysis
• Streamlit for the UI
• Ollama for serving LLM locally

---

Setup and Installation

Ensure you have Python 3.8.1 or later installed on your system.

Install dependencies:

uv sync

Copy .env.example to .env and configure the following environment variables:

GROUNDX_API_KEY=your_groundx_api_key_here

# Install Ollama from https://ollama.ai/
# Pull the required model
ollama pull phi3:mini
# Start Ollama service
ollama serve

Run the Streamlit app:

streamlit run app.py

Project Structure

groundX-doc-pipeline/
├── app.py                          # Main Streamlit application (uses groundx_utils.py)
├── groundx_utils.py                # Utility functions for Ground X operations
├── .env                            # Environment variables (create from .env.example)
├── file/                           # Folder containing files for running evaluation
└── README.md                       # This file

📁 Evaluation Tools:
├── evaluation_geval.py             # GEval framework evaluation
└── run_evaluation_cli.py           # CLI evaluation runner

Usage

1. Upload a document using the sidebar (supports PDF, PNG, JPG, JPEG, DOCX)
2. Wait for the document to be processed by Ground X
3. Explore the X-Ray analysis results in different tabs:
   • JSON Output: Raw analysis data
   • Narrative Summary: Extracted narratives
   • File Summary: Document overview
   • Suggested Text: AI-suggested content
   • Extracted Text: Raw text extraction
   • Keywords: Document keywords
4. Use the chat interface to ask questions about your document

Features

The app implements a world-class document processing workflow:

• Ground X Bucket Management: Automatic bucket creation and document organization
• Document Ingestion: Support for PDF, Word docs, images, and more
• X-Ray Analysis: Rich structured data with summaries, page chunks, keywords, and metadata
• Context Engineering: Intelligent context preparation for LLM queries
• AI Chat Interface: Interactive Q&A powered by local LLM

---

📬 Stay Updated with Our Newsletter!

Get a FREE Data Science eBook 📖 with 150+ essential lessons in Data Science when you subscribe to our newsletter! Stay in the loop with the latest tutorials, insights, and exclusive resources. Subscribe now!

---

Contribution