If you read one line: pip install docling, run DocumentConverter().convert(path).document, export to Markdown or JSON, then hand the result to HybridChunker — that's the whole pipeline, and the code below is copy-pasteable.

Docling is IBM Research's open-source document converter, now stewarded by the LF AI & Data Foundation under the docling-project GitHub org. It turns PDFs, DOCX, PPTX, XLSX, HTML, images, and a long tail of other formats into a single unified DoclingDocument object — layout-aware, table-structured, and ready to export or chunk. If you're wiring a folder of contracts, reports, or manuals into an agent's retrieval layer, this is the fastest legitimate path from raw file to clean chunks. For the broader parser landscape, see our comparison of Docling, Unstructured, and LlamaParse and the wider PDF-parsing roundup.

Install and convert your first PDF#

pip install docling

Conversion is three lines:

from docling.document_converter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("contracts/msa_q3.pdf")  # path, URL, or stream
doc = result.document

print(doc.export_to_markdown())

convert() accepts a local path, a URL, or an in-memory DocumentStream (for binary uploads in a web app). The returned result.document is a DoclingDocument — a structured tree of text, tables, headings, and figures with layout metadata attached, not a flat string.

Why layout-aware parsing matters (the non-obvious part)#

Naive extractors like pdfminer or PyPDF read a PDF's content stream in the order text-drawing operators appear — which is often not the order a human reads the page. A two-column page gets its columns interleaved line-by-line. A table, which has no native "table" object in the PDF spec, dissolves into a scatter of positioned text runs that naive extraction just concatenates left-to-right, top-to-bottom — numbers from three different columns end up glued into one nonsense string.

Docling avoids this by running a layout model first: it detects regions (paragraph, table, header, caption, footer) via object detection, establishes real reading order from those regions, and only then extracts text per-region. For tables specifically, it runs a second model — TableFormer — that reconstructs actual row/column structure before anything is exported, so a table becomes a genuine grid object, not word soup. This is the single biggest reason layout-aware parsing beats naive extraction for retrieval quality: the failure is invisible in a chunk preview and only shows up when your agent confidently cites the wrong number from a scrambled table.

Extracting tables and controlling structure recognition#

Table extraction is on by default, but you can tune accuracy vs. speed:

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions, TableFormerMode
from docling.document_converter import DocumentConverter, PdfFormatOption

pipeline_options = PdfPipelineOptions()
pipeline_options.do_table_structure = True
pipeline_options.table_structure_options.mode = TableFormerMode.ACCURATE  # or .FAST

converter = DocumentConverter(
    format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}
)
result = converter.convert("finance/q3_report.pdf")

for i, table in enumerate(result.document.tables):
    df = table.export_to_dataframe(doc=result.document)
    print(f"Table {i}: {df.shape}")
    df.to_csv(f"table_{i}.csv")

ACCURATE mode (the default) gives you better structure on messy or merged-cell tables; FAST trades some accuracy for throughput on large batches. table.export_to_html(doc=result.document) is also available if you want an HTML table to embed directly in a chunk.

Exporting to Markdown and JSON#

Two export calls cover most downstream needs:

import json

# Markdown — readable, good for LLM context windows
with open("q3_report.md", "w") as f:
    f.write(doc.export_to_markdown())

# JSON — lossless serialization of the full DoclingDocument
with open("q3_report.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

Markdown is what you'll usually feed an LLM directly; JSON preserves every layout detail if you need to rebuild structure later or feed a custom chunker.

Chunking for RAG with HybridChunker#

Conversion alone doesn't give you RAG-ready chunks — you still need to split by token budget without breaking tables or headings mid-thought. That's HybridChunker, covered in more depth alongside general chunking strategy in our chunking guide:

from docling.chunking import HybridChunker
from docling_core.transforms.chunker.tokenizer.huggingface import HuggingFaceTokenizer
from transformers import AutoTokenizer

tokenizer = HuggingFaceTokenizer(
    tokenizer=AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2"),
    max_tokens=512,
)
chunker = HybridChunker(tokenizer=tokenizer)

chunks = list(chunker.chunk(dl_doc=doc))
for chunk in chunks:
    embed_text = chunker.contextualize(chunk=chunk)  # metadata-enriched text
    # embed(embed_text) and upsert into your vector store, alongside chunk.text

HybridChunker starts from Docling's hierarchical chunker (one chunk per structural element), then splits oversized chunks and merges undersized adjacent ones so each chunk fits your tokenizer's max_tokens — matched to your actual embedding model, not a guessed character count. Always embed chunker.contextualize(chunk), not raw chunk.text: it prepends headings and captions so the embedding carries context the bare chunk lost. For wide tables, set repeat_table_header=True (the default) so every chunk of a split table still carries its column headers.

OCR for scanned documents#

Scanned PDFs and photographed pages need OCR turned on explicitly:

from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.datamodel.base_models import InputFormat
from docling.document_converter import DocumentConverter, PdfFormatOption

pipeline_options = PdfPipelineOptions()
pipeline_options.do_ocr = True
pipeline_options.ocr_options.lang = ["en"]
pipeline_options.do_table_structure = True

converter = DocumentConverter(
    format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}
)
result = converter.convert("scans/old_contract.pdf")

EasyOCR is the default engine; swap pipeline_options.ocr_options = TesseractOcrOptions() (or TesseractCliOcrOptions(), or OcrMacOptions() on macOS) if you'd rather not pull in EasyOCR's dependency footprint. Docling auto-detects scanned pages, but for mixed documents — some born-digital pages, some scanned — leaving do_ocr=True on is the safe default; Docling only OCRs pages that need it.

Wiring it together#

The shape of a production ingestion job is: convert → export Markdown (for eyeballing) and JSON (for archival) → chunk with HybridChunker → embed contextualize() output → upsert with chunk.text as the stored payload. For 100+ documents, batch with DocumentConverter().convert_all(paths) rather than looping single convert() calls — it reuses loaded models across the batch instead of reloading per file. Start with the defaults; only reach for TableFormerMode.ACCURATE and OCR overrides once you've confirmed on a sample that the defaults are actually losing you table cells or scanned text.