RAG 색인을 위해 디지털 PDF를 텍스트 및 마크다운으로 대량 변환하고 스캔한 항목을 포착합니다.

Most "chat with your documents" projects die on step one: getting text out of the PDFs. A handful of files is easy. A few hundred reports, manuals and specs from different sources is where you find the edge cases: encrypted files, 400-page monsters, and scans that look like documents but contain only images.

This post walks through:

1. a DIY extraction script and its traps,
2. the same batch job with a hosted Actor and real output,
3. chunking the result for a search or RAG index, plus cost and limits.

The DIY version

With pypdf the basic extraction is a few lines:

import io, requests from pypdf import PdfReader def pdf_text(url): data = requests.get(url, timeout=60).content reader = PdfReader(io.BytesIO(data)) return "\n\n".join(page.extract_text() or "" for page in reader.pages) 

What bites you in production:

• Scanned PDFs return an empty string, silently. Your index now holds a "document" with no text, and nobody notices.
• Encrypted files raise errors or return nothing.
• Size and page limits. One huge file can stall a whole batch, so you need byte and page caps.
• Bookkeeping. You want page counts, title/author metadata and a status per file, not a pile of strings.

The hosted version

PDF to Text Markdown Extractor (Digital, No OCR) does this batch job with explicit statuses. Disclosure: I build this Actor at SteadyScrape. It downloads each direct PDF URL and extracts text locally. There's no OCR in this version, so scanned files are labelled image_only and incur no extraction charge.

Input matching the current schema:

{ "pdfUrls": [ "https://www.orimi.com/pdf-test.pdf", "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf", "https://www.rfc-editor.org/rfc/pdfrfc/rfc8259.txt.pdf" ], "format": "both", "maxFiles": 20, "includeMetadata": true } 

format is text, markdown or both. You can also upload files in the Apify Console instead of passing links.

From Python (token from an environment variable):

import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("steady-scrape/pdf-text-markdown").call(run_input={ "pdfUrls": [ "https://www.orimi.com/pdf-test.pdf", "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf", "https://www.rfc-editor.org/rfc/pdfrfc/rfc8259.txt.pdf", ], "format": "both", }) rows = list(client.dataset(run["defaultDatasetId"]).iterate_items()) 

Real output from a verified run of exactly those three files (text trimmed):

[ {"source": "https://www.orimi.com/pdf-test.pdf", "status": "ok", "pageCount": 1, "chars": 367, "title": "PDF Test Page", "text": "PDF Test File\n\nCongratulations, your computer is equipped with a PDF ...", "chargeEvent": "pdf-extracted"}, {"source": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf", "status": "image_only", "note": "No extractable text — PDF looks image-only/scanned. OCR is not included in v1. Not charged.", "text": null, "chargeEvent": null}, {"source": "https://www.rfc-editor.org/rfc/pdfrfc/rfc8259.txt.pdf", "status": "ok", "pageCount": 16, "chars": 23656, "text": "Internet Engineering Task Force (IETF) T. Bray, Ed.\nRequest for Comments: 8259 ...", "chargeEvent": "pdf-extracted"} ] 

That run charged two PDF extractions plus the run fee and the standard start event; the image-only file added no charge, and the row tells you why.

Chunking for a search or RAG index

Keep the ok rows, route image_only rows to an OCR queue (or a human), and split the Markdown on headings, with a size fallback:

import re def chunks(md, max_chars=2000): parts = re.split(r"\n(?=#{1,3} )", md) for part in parts: for i in range(0, len(part), max_chars): yield part[i:i + max_chars] index, needs_ocr = [], [] for r in rows: if r["status"] == "ok": for n, c in enumerate(chunks(r["markdown"] or r["text"])): index.append({"source": r["source"], "chunk": n, "text": c}) elif r["status"] == "image_only": needs_ocr.append(r["source"]) print(len(index), "chunks;", len(needs_ocr), "files need OCR") 

Embed index with whatever model you use and store source with each chunk so answers can cite the file.

The Markdown headings are heuristic: the PDF format doesn't reliably mark headings, so treat them as hints for splitting, not as a guaranteed document outline. Tables and multi-column layouts can come out in reading-order text rather than structured tables.

What it costs

From the Store pricing:

• $0.004 per successfully extracted digital PDF
• $0.003 once per run, only if at least one PDF was extracted
• Apify's standard $0.00005 Actor start event

200 successfully extracted digital PDFs in one run cost $0.80305 (200 × $0.004 + $0.003 + $0.00005). Scanned/image-only, empty, encrypted, oversized and failed files incur no extraction charge, and a run where no PDF is extracted doesn't pay the $0.003 run fee; Apify's $0.00005 start event still applies to every run.

Limits

• No OCR. Scanned documents are flagged, not read.
• Direct PDF links only. It doesn't crawl a website looking for PDFs.
• Per-run file caps and per-file page and size caps apply. Oversized files return too_large / too_many_pages (no extraction charge).
• Only process documents you're allowed to use.

Integrations

Run it via the Apify API, on a schedule, or from Make, Zapier or n8n through Apify's integrations. AI agents can call it through the Apify MCP server, which is handy as a "read this PDF" tool.

SteadyScrape (steady-scrape) builds small, pay-per-result data tools on Apify.