Serverless

7 posts with the tag “Serverless”

How to self-host the MinerU API on RunPod

Jun 8, 2026

Last Updated: 2026-06-08

If you’re calling the official MinerU API (mineru.net/api/v4) in production, you’ve probably hit one of three walls: the daily quota of 1,000 high-priority pages, the per-file 200 MB cap, or a compliance review asking why your documents leave your infrastructure for a third-party cloud. You can run the exact same MinerU engine yourself on RunPod Serverless, keep documents in your own bucket, drop the quota, and pay roughly $0.001 per page warm. The migration is close to drop-in: swap the client, keep your create-task / poll / download loop.

The cloud API is the right way to try MinerU. Once you’re parsing real volume, self-hosting changes the economics and the data path. Here’s why, what it costs, and how to move your code over.

Why self-host the MinerU API instead of using mineru.net?

Three reasons: cost at volume, data residency, and control. The cloud API meters a daily high-priority page quota and then deprioritizes you; a self-hosted endpoint has no quota. Self-hosted, your PDFs and their parsed output never leave your RunPod worker and your own S3 bucket. And you pin the model version, pick the GPU, and set your own concurrency.

The official MinerU API caps each file at 200 MB and gives each account a daily quota of 1,000 high-priority pages (see the published limits) before jobs drop to lower priority. That ceiling is fine for evaluation and light use. It becomes a planning problem the moment you’re ingesting thousands of pages a day on a deadline.

The data path matters too. On the SaaS, every document round-trips through mineru.net. Self-hosted, the worker pulls the file from a URL you control (or you send the bytes inline), parses on your GPU, and writes the result to your bucket. Nothing transits a vendor you have to put in a data-processing agreement.

The trade-off is honest: you run the infrastructure. There’s a cold-start cost, but FlashBoot blunts it: once a host has booted the worker, it restores from a process snapshot in ~7-8 s across scale-to-zero cycles, and only a brand-new host pays the full ~110 s (vLLM init plus loading the model into VRAM). The model is baked into the image, so nothing is downloaded at request time. If your traffic is a handful of pages a month, the cloud API’s free tier is simpler; cross into steady volume and the math flips.

What does self-hosting MinerU cost vs the cloud API?

Roughly $0.001 per page warm on a 24 GB RTX 4090, plus a ~$0.03 cold-start tax the first time a host boots the worker (≈110 s of GPU billing for vLLM init and loading the baked model into VRAM). FlashBoot then snapshots that state, so the same host restarts in ~7-8 s across scale-to-zero cycles. RunPod bills per second and scales to zero, so an idle endpoint costs nothing. The cloud API is free under its daily quota, then queues or meters you; self-hosting trades that ceiling for a steady, predictable per-page rate.

The real number depends on how well you amortize that first-boot tax. A thousand pages parsed across one warm window land near $0.001/page; a single short doc on a cold host lands closer to $0.005–$0.01 because you’re paying the one-time tax against very few pages. FlashBoot shrinks that in practice, since a host pays the full tax only once. I broke the full workload-shape math down in Serverless MinerU on RunPod: honest cost math.

One cost the marketing pages skip: the first-boot cold start (~110 s on a brand-new host) is billed GPU time. FlashBoot snapshots that boot, so the same host restarts in ~7-8 s afterward. The model is baked into the image, so there’s no model-download step at boot or request time. Budget for the first boot per host, not for every request.

How do you deploy your own MinerU endpoint on RunPod?

Deploy the open-source mineru-runpod template from the RunPod Hub, or fork it and point RunPod’s GitHub build at your fork. Create a Serverless Endpoint on a 24 GB GPU (ADA_24 / RTX 4090) with FlashBoot enabled. For full_zip_url parity with the cloud API, also set four BUCKET_* env vars pointing at an S3-compatible bucket.

The fastest path is the Hub listing: one click, fill in the deploy-time form, done. The full walkthrough (fork-and-build and bring-your-own-image included) is in the deploy guide.

The one piece worth getting right up front is object storage. The compat client returns results as a full_zip_url, which means the worker uploads the output archive to a bucket and hands back a presigned URL, exactly like the SaaS. That path needs four env vars on the endpoint:

BUCKET_ENDPOINT_URL=https://<account>.r2.cloudflarestorage.com
BUCKET_NAME=mineru-outputs
BUCKET_ACCESS_KEY_ID=<key>
BUCKET_SECRET_ACCESS_KEY=<secret>

Cloudflare R2 is a good pairing here because its egress is free, so downloading your own results costs nothing. Any S3-compatible store works (R2, Backblaze B2, MinIO, AWS S3). To get started you’ll need a RunPod account; sign up here (full disclosure: that’s a referral link). Add a few dollars of credit and it covers thousands of cold starts plus millions of warm pages.

How do you migrate your code off the MinerU API?

Install the mineru-client package and swap the official requests calls for MineruApiClient. It mirrors the cloud API’s create_task / get_task surface and returns the same response dicts, so your existing poll loop keeps working. Under the hood it requests archive_format="zip", so full_zip_url comes back as a real .zip like the SaaS does.

Install it (no version pin, so it tracks the repo):

pip install "mineru-client @ git+https://github.com/sergeyshmakov/mineru-runpod"

Here’s the before and after. The official API, polling by hand:

import requests, time

H = {"Authorization": f"Bearer {MINERU_TOKEN}"}
task_id = requests.post(
    "https://mineru.net/api/v4/extract/task",
    headers=H, json={"url": pdf_url, "model_version": "vlm"},
).json()["data"]["task_id"]

while True:
    data = requests.get(
        f"https://mineru.net/api/v4/extract/task/{task_id}", headers=H
    ).json()["data"]
    if data["state"] in ("done", "failed"):
        break
    time.sleep(2)
zip_url = data["full_zip_url"]   # then download + unzip yourself

Self-hosted, against your own endpoint:

from mineru_client import MineruApiClient

client = MineruApiClient(endpoint_id="<your-endpoint-id>", api_key="<runpod-key>")

task_id = client.create_task(pdf_url, model_version="vlm")["data"]["task_id"]
done = client.wait_for_task(task_id)        # polls to a terminal state
client.download_results(done, "./out")      # full_zip_url is a real .zip; unpacked for you

Same lifecycle, same {"code": 0, "data": {...}} response shape. The parameter names map across cleanly too: model_version to the worker’s backend, language to lang, enable_formula / enable_table straight through. The full field-by-field mapping is in Migrate from the MinerU API, and the Clients page covers the native client if you’d rather use the worker’s own (richer) request shape once you’ve moved.

One auth note: the self-hosted endpoint authenticates with your RunPod API key, also via Authorization: Bearer, so even that part of your code barely changes.

What doesn’t carry over from the cloud API?

Most of it carries; four things don’t. The compat client rejects callback (a RunPod webhook delivers a different, unsigned payload than MinerU’s signed {checksum, content} callback, so it raises instead of misleading you), and it doesn’t support extra_formats (docx/html/latex), the MinerU-HTML model, or multi-range page_ranges like "2,4-6". There’s no batch endpoint (RunPod’s queue already parallelizes individual jobs across workers, so one isn’t needed). And full_zip_url requires the BUCKET_* setup above.

The full list, so there are no surprises:

Cloud API feature	Self-hosted status
`create_task` / `get_task`, state machine, `full_zip_url`	Supported (with object storage configured)
`model_version: pipeline` / `vlm`	Supported (maps to the worker backends)
`model_version: MinerU-HTML`	Not supported, raises
`extra_formats` (docx / html / latex)	Not produced by this worker, raises
`page_ranges` multi-range (`"2,4-6"`)	One contiguous range per job, raises otherwise
`callback` + `seed`	Rejected (poll with `get_task` / `wait_for_task` instead)
Batch (`/extract/task/batch`)	Not offered; submit tasks individually and raise `workers_max` — RunPod’s queue parallelizes them across workers

Where it falls down: if your pipeline leans on webhook callbacks or exports to DOCX/LaTeX, the compat client isn’t a clean swap today. There’s no batch endpoint either, but you rarely need one: submit tasks individually and let RunPod fan them across workers_max workers (the queue parallelizes them). For everything else, the create / poll / download path behaves like the SaaS.

One more practical detail. The compat client is URL-only, mirroring the cloud API’s POST /extract/task, which doesn’t accept file uploads. For small local files you don’t need to host anything: use the native MineruClient with file_b64 instead, which sends the bytes inline (fine under RunPod’s ~20 MB request cap). I parsed a 522 KB scanned Russian invoice that way and got clean Cyrillic Markdown back, no bucket round-trip involved.

FAQ

Is the MinerU API free?

The official MinerU cloud API has a free daily quota of 1,000 high-priority pages, after which jobs run at lower priority. There’s no per-call charge inside the quota. Self-hosting removes the quota entirely and replaces it with per-second GPU billing (around $0.001 per page warm).

What are the MinerU API’s limits?

Each file is capped at 200 MB, with a daily quota of 1,000 high-priority pages plus rate limits of 50 files/minute and 5,000 files/day (see the published limits). Self-hosting on RunPod removes the daily quota; the per-file practical limit becomes your GPU’s memory and your endpoint’s job timeout rather than a fixed page count.

Can I self-host MinerU without RunPod?

Yes. MinerU is Apache-2.0 open source and runs anywhere with a CUDA GPU. RunPod Serverless is the path this template targets because it scales to zero, so a bursty workload doesn’t pay for an idle GPU. On a fixed VPS or your own hardware you’d run MinerU directly and skip the serverless wrapper.

Does the self-hosted output match the cloud API’s full_zip_url?

Yes, when the endpoint has object storage configured. The compat client requests archive_format="zip", so the worker uploads a .zip to your bucket and returns a presigned full_zip_url, the same container and field the SaaS returns. download_results fetches and unpacks it for you, and autodetects .tar.gz too if you change the format.

Is self-hosting actually cheaper than the MinerU API?

At steady volume, usually yes, because you stop being throttled by the daily quota and pay only for GPU seconds used. The crossover depends on cold-start amortization: dense traffic lands near $0.001/page, sparse one-doc-per-cold-start traffic closer to $0.005–$0.01. Below a few hundred pages a month, the cloud API’s free tier is the cheaper and simpler option.

Do my documents stay private when self-hosting?

Yes. The worker fetches each file from a URL you control or from bytes you send inline, parses on your own RunPod GPU, and writes output to your own S3 bucket. No document or result passes through a third-party parsing service, which is the usual blocker in a data-residency or compliance review.

Self-hosting MinerU isn’t about beating the cloud API on accuracy. The accuracy is identical: it’s the same model. It’s about removing the quota ceiling, keeping documents in your stack, and paying per-second instead of per-tier. If that’s the wall you’ve hit, fork the template, or deploy it from the RunPod Hub and grab a RunPod account to run the first parse.

Clause-aligned batching for large PDFs on MinerU + RunPod

Jun 3, 2026

Sergei Shmakov

Last Updated: 2026-06-03

ECMA-376 Part 1 (the Office Open XML File Formats — Fundamentals and Markup Language Reference) is 5,039 pages of dense, table-heavy, XML-schema-laden specification in a single 35 MB PDF. It is the document that defines .docx, .xlsx, and .pptx down to the attribute. If you want a machine-readable, clause-addressable version of it, you have to parse all 5,039 pages, and almost everything about that page count makes a naive approach fall over.

This is the story of parsing the whole thing through the mineru-runpod serverless worker. The headline result: 36 batches, 5,039 pages, 46,637 content blocks, 4,174 tables, full contiguous coverage, ~$1.15 of GPU time. The interesting part is not the total. It’s how you cut a 5,000-page document into pieces without breaking it, which it turns out is a decision about clause structure, not page numbers.

Why not just send the whole 5,000-page PDF?

The worker accepts a file_url and parses front-to-back, so technically you could send all 5,039 pages as one job. You shouldn’t, for four reasons that all get worse with size:

All-or-nothing failure. A single job that dies at page 4,800 (OOM, a transient GPU eviction, a timeout) costs you the entire run. At ~78 minutes of GPU work (more on that below), that’s an expensive coin flip.
No resumability. One job has no natural checkpoint. If it fails you start over.
The 20 MB response cap. MinerU’s output for a few hundred pages already blows past RunPod’s ~20 MB sync-response ceiling. For 5,000 pages it isn’t close: the extracted output here was 869 MB on disk.
Memory. Holding the layout model output for thousands of pages in one process is a needless VRAM/RAM risk when the work is embarrassingly sliceable.

Batching fixes all four, but only if you batch at the right boundaries.

Why cut at clause boundaries instead of every N pages?

The obvious split is “every 100 pages.” The problem: a standard isn’t a stream of interchangeable pages, it’s a tree of clauses. Clause §17.4 (Tables) might start three lines from the bottom of a page and run for 40 pages. If a batch boundary lands in the middle of it, you’ve torn a logical unit across two parse jobs, and every downstream step (clause extraction, cross-referencing, chunking for retrieval) has to stitch it back together.

So I don’t cut by page count. I cut by clause:

Build an outline from the PDF’s 4,600+ bookmarks, giving a clause → page index for the whole document.
Place batch boundaries only at clause starts, never mid-clause.
Treat the huge top-level clauses (§17 WordprocessingML, §18 SpreadsheetML, §19 PresentationML, §20/§21 DrawingML, §22 Shared MLs, and the annexes) as mandatory anchors, so a big reference section always begins a fresh batch.
Aim for ~100 pages per batch, allow up to ~200, and accept whatever the nearest clause boundary gives.

The result was 36 batches averaging 140 pages (smallest 66, largest 238). Every batch starts and ends on a clause edge, so no clause is ever split across the seam between two parse jobs.

(One calibration gotcha specific to this PDF: printed page 1 is PDF page index 9. There’s a 9-page front-matter offset you have to fold into the bookmark→page mapping or every boundary is off by nine.)

A useful consequence: because the worker slices the PDF server-side via start_page/end_page (see the API reference), you never pre-split the PDF. You upload it once and each batch job asks for its page range out of the same source file.

Did the batches actually cover the whole document?

Yes, and this is worth verifying mechanically rather than trusting. After the run, I checked each batch’s produced page span against its planned range and confirmed the batches tile the document with no gaps and no overlaps:

Metric	Value
Batches	36
Pages	5,039 (contiguous 0–5038, 0 gaps, 0 overlaps)
Content blocks	46,637
Tables	4,174
Code blocks	3,591
Pages/batch	mean 140, min 66 (b35), max 238 (b25)
Output downloaded	~465 MB (compressed tarballs)
Output on disk	869 MB (extracted)

The contiguity check is the one piece of validation I wouldn’t skip on a document this size: it’s the difference between “the run finished” and “the run is complete.”

How was the document transported in and out?

Two different transports, for two different size problems.

Input: R2 URL. At 35 MB the PDF is well over the 20 MB inline (file_b64) limit, so it can’t ride in the request body. I put it on Cloudflare R2 and passed a public URL as file_url. The worker downloads it (≤200 MB cap) and slices the requested pages itself. One upload, 36 jobs read from it.

Output: transport="s3". Per-batch output is large (the biggest batch produced a 32 MB tarball), so embedding results in the sync response was out. With transport="s3", the worker uploads each result .tar.gz back to R2 and returns a presigned URL the client downloads and extracts. The tarball carries everything: content_list.json (the flat, typed, page-indexed block list I treat as source of truth), the rendered markdown, middle.json, and a layout-overlay PDF.

The presigned URL has a 1-hour TTL, which has a real consequence for batching: you must download each batch’s result as its job finishes, not in a sweep at the end of a 78-minute run. By then the early URLs have expired.

What GPU and backend, and what did throughput look like?

Backend: vlm-auto-engine (MinerU 2.5 Pro, the MinerU2.5-Pro-2605-1.2B vision-language model) on a 24 GB AMPERE_24 (RTX A5000-class) RunPod serverless GPU. One parse per worker (MINERU_MAX_CONCURRENCY=1: vLLM’s KV cache isn’t safe to drive from concurrent parses on a 24 GB card). For how to pick a card, see Choosing a GPU.

Across the 35 timed batches, total GPU compute was 4,674.8 s (77.9 min) at an overall 1.04 pages/sec, with individual batches ranging 0.84–1.27 pp/s depending on table density. A few representative batches:

Batch	Clause	Pages	Worker time	pp/s
b00	§1 Scope (front matter)	176	145.0 s	1.21
b01	§17 WordprocessingML	100	102.8 s	0.97
b17	§18.17.7 (functions)	176	147.9 s	1.19
b25	§21.2 DrawingML – Charts	238	231.8 s	1.03
b32	Annex L Primer	102	80.2 s	1.27

Cost worked out to roughly $0.00023/page, ~$1.15 for the whole standard. Before committing to that, a 3-page smoke test (cents, ~110–130 s dominated by cold start) validated the entire pipeline end-to-end (URL fetch → parse → R2 upload → download → extract), the cheapest insurance you can buy on a big run.

The parallelism lesson: it’s RunPod-side, not in the worker

This is the part that cost the most confusion: a single batch is already parallelized inside the worker (the VLM batches many page-images through the GPU at once), but running multiple batches at once is a RunPod scaling decision, not something you trigger by submitting more jobs.

I learned this the hard way. The client submitted 3 batches concurrently, and RunPod ran exactly one while two sat in the queue. The endpoint was configured workersMax=1: one GPU worker, one batch at a time, no matter how many jobs you fire. Raising workersMax to 3 (and matching the client’s concurrency) is what actually delivered 3×: the remaining 31 batches then finished in 27.8 minutes wall-clock. The scaling guide covers how concurrency and workersMax interact.

The mental-model fix:

Inside one job: pages are parallelized on one GPU. Already maxed.
workersMax: how many separate GPUs run separate jobs at once. This is your throughput dial.

A related myth worth busting: MinerU’s pipeline logs mention a window_size=64. That is a GPU throughput batch (how many page-images stream through the model at a time to bound VRAM), not a context window. Pages are recognized independently regardless of it, so it has zero effect on content continuity across pages. Which is exactly why clause-aligned batch boundaries matter and the internal window size doesn’t: continuity is something you protect at the batch layer, not by tuning a throughput knob.

Which clauses produced the most structure?

Block and table counts track the content shape of the standard almost perfectly: the reference-material and function-catalog clauses dominate:

Batch	Clause	Blocks	Tables
b25	§21.2 DrawingML – Charts	2,829	378
b17	§18.17.7 (spreadsheet functions)	2,805	239
b26	§22 Shared MLs	2,509	—
b10	§17.17 Miscellaneous	—	238
b11	§18 SpreadsheetML	—	224

These are the dense element/attribute reference tables that make ECMA-376 what it is. They’re a good reminder to spot-check table fidelity on exactly these batches before trusting the output downstream.

The annex schema dumps look completely different

The most striking per-batch contrast is the annexes. Annex A (W3C XML Schema), Annex B (RELAX NG) and friends are long code listings, not prose with tables, and the numbers show it. Same ~150-page batch size, radically smaller output:

Batch	Annex	Tarball
b29	Annex B (RELAX NG)	1.15 MB
b30	B.3 PresentationML	1.75 MB
b27	Annex A (XML Schema)	2.06 MB
b28	A.3 PresentationML	2.23 MB

Compare that to the prose-and-table batches that ran 31–32 MB (b10, b11) for a similar page count: roughly a 15× size difference driven entirely by content type. MinerU classifies the schema listings as code, so they compress to almost nothing relative to a table-dense reference section.

How did resumability actually work?

The runner keeps a manifest.json keyed by batch, and writes each batch’s result atomically: extract into a temporary directory, then rename into place. A batch is only marked ok after its download, extraction, and rename all succeed. Two payoffs:

Pause/resume. Midway through, I paused the run to raise workersMax (you don’t want to change cluster settings while jobs are in flight). Stopping the client abandoned the in-flight jobs, but because their downloads hadn’t completed, the manifest never marked them done, so resuming re-ran them. Completed batches were skipped. No corruption, no duplicate downloads.
Crash recovery is free. The same mechanism means any crash resumes from the last completed batch.

For a 36-job run that you might interrupt, the resumable manifest is what turns “a long fragile script” into “a process you can walk away from.”

Where this falls down / what I’d change

Honest limitations:

1-hour presign expiry forces eager download. You cannot defer pulling results to the end of a long run; download each batch as it lands. My runner does this, but it’s a constraint to design around, not a free lunch.
Clause boundaries are only as good as the outline. The whole scheme leans on the PDF’s bookmark tree being accurate and complete. A document with missing or wrong bookmarks needs a fallback (TOC parsing, heading detection) before this works.
Table/code fidelity needs spot-checking. 4,174 tables and 3,591 code blocks is a lot of structure to trust blindly; the dense reference batches (b25, b17, b11) and the annex code dumps are where I’d sample-verify first.
One GPU is the ceiling. Throughput is fundamentally workersMax × per-GPU rate. There’s no in-job trick to go faster: you pay for more workers or you wait. And more workers means more cold starts, so wall-clock and cost don’t scale perfectly linearly.

What I’d change next time: drive client concurrency directly from the endpoint’s live workersMax so the two never drift, and prune the middle.json + layout PDF from batches where I only need content_list.json. They were roughly half the on-disk footprint.

FAQ

How long does it take to parse a 5,000-page PDF with MinerU?

About 78 minutes of single-GPU compute (~1 page/sec on a 24 GB RTX A5000-class card with the VLM backend), or ~28 minutes of wall-clock at 3× worker concurrency. Cost is roughly $1.15 total at ~$0.00023/page.

Why batch at clause boundaries instead of fixed page counts?

So no logical unit is split across two parse jobs. A clause can start mid-page and span dozens of pages; cutting by page count tears it in half and forces every downstream step to reassemble it. Cutting at clause starts keeps each clause whole within a batch.

How do you handle output larger than RunPod’s 20 MB response cap?

Use transport="s3": the worker uploads each result tarball to an S3-compatible bucket (Cloudflare R2 here) and returns a presigned URL you download. Per-batch output here reached 32 MB, far past the sync-response ceiling.

Does sending more concurrent jobs make a single endpoint faster?

No. Concurrency above the endpoint’s workersMax just fills the queue. Parallelism is the number of GPU workers RunPod runs, set by workersMax. Raise that to go wider.

Do I need to split the PDF before uploading?

No. Upload the full PDF once (or host it on R2 and pass file_url); each batch job requests its page range via start_page/end_page and the worker slices server-side.

How do I know the whole document was covered?

Verify mechanically: check each batch’s produced page span against its planned range and confirm the batches tile the document with zero gaps and zero overlaps. “The run finished” and “the run is complete” are not the same claim.

Where to next

The output is 36 batches of content_list.json with page-indexed, typed blocks. The next step is joining each block’s page back to the clause outline to emit a clause-addressable tree (one compact file per clause) that an agent or a retrieval index can navigate. The clause-aligned batching is what makes that join clean: every block already lives inside exactly one clause’s batch. Part 2 walks through that post-processing: cleaning the blocks, building the tree, cross-linking, and verifying the result against the source PDF.

ECMA-376 is freely available from Ecma International; it’s used here purely as a parsing benchmark. The parsed corpus is kept in a private repository for internal use, and this post shares only the parsing process and aggregate statistics, not the standard’s content.

If you want per-phase timings (fetch / parse / package) and throughput dashboards for a run like this, the worker can ship OpenTelemetry traces and metrics to any OTLP backend. See the observability guide.

If this saved you time, the easiest way to say thanks is signing up for RunPod through this link. Star the repo on GitHub for updates.

Disclosure: RunPod links in this post use a referral code that credits me at no cost to you. The post would read the same without it.

Fix RunPod's 'no resources to deploy your pod' error

Jun 3, 2026

Sergei Shmakov

Last Updated: 2026-06-03

If RunPod fails a deploy with this:

This machine does not have the resources to deploy your pod. Please try a different machine.

your Docker image is fine. This is a capacity error: RunPod’s scheduler tried to place your pod on a host that didn’t have a free GPU of the type you asked for, and bailed. It’s transient. The fix is to make RunPod try again on a different host, and how you trigger that retry depends on which RunPod product you’re using. On a Serverless endpoint wired to GitHub, push any commit to the watched branch. On a RunPod Hub template, cut a new GitHub Release. Those two triggers are not interchangeable, which is the part that trips people up.

I hit this constantly while maintaining the mineru-runpod template. The rest of this post is what the error actually means, why it’s not your fault, and the exact retry mechanic for each workflow.

What does “this machine does not have the resources to deploy your pod” mean on RunPod?

It means RunPod’s scheduler picked a physical host to run your pod, found that host couldn’t satisfy the requested GPU, RAM, or disk, and refused the placement. It’s a per-machine capacity miss, not a global outage and not a problem with your image. “Try a different machine” is literal advice: another host of the same GPU type may have room.

The message fires during the scheduling phase, before your container ever starts. That timing is the tell. A broken image fails differently: you’d see an image-pull error, a non-zero container exit, or a failed health check. This message means the scheduler never got that far. It looked at the GPU type you requested, compared it against free capacity on the candidate host, and found the fit impossible.

For most people the requested GPU is the binding constraint. Popular pools like the RTX 4090 get contended, especially in a busy region. When every host of that type in that data center is full, a fresh placement attempt fails until one frees up.

Is it a RunPod bug, or did I break something?

Neither. It’s a legitimate, expected response from RunPod’s scheduler reporting that the GPU pool you targeted had no free host at that moment. Your Dockerfile, your handler, and your config are all irrelevant to this error. GPU capacity on RunPod fluctuates minute to minute, so the same deploy that fails now often succeeds 30 seconds later on a different host.

The reason it feels like a bug is that it’s non-deterministic. The exact same config fails one minute and works the next, purely because the cluster’s free capacity moved. That’s also why retrying works: you’re not changing anything about your build, you’re just asking the scheduler to roll the dice again against a pool whose occupancy has shifted.

Where you actually meet this string matters. RunPod throws it whenever it spins up a real pod, which in this template’s world is two places: the Hub validator test pod that runs after a release, and a GPU Pod you launch directly. A Serverless worker that can’t find capacity usually surfaces it as workers stuck in a throttled or initializing state rather than this exact sentence, but the underlying cause and the recovery are the same: force a rebuild so RunPod reschedules.

How do I fix it on a RunPod Serverless endpoint?

Push any commit to the branch your endpoint watches. Per RunPod’s docs, “every git push to your specified branch results in an updated Endpoint”, so a no-op commit triggers a fresh build and redeploy. The new workers get scheduled again, almost always landing on a host with free capacity. You can also hit Rebuild in the RunPod console to do the same thing without a commit.

This is the path most people need. If you deployed your worker by connecting a GitHub repo, your endpoint redeploys on every push, so a trivial commit is the lowest-friction way to re-roll the scheduler:

git commit --allow-empty -m "chore: re-trigger RunPod build"
git push

The --allow-empty flag is the point: you don’t need a real change to force a rebuild, you just need a new commit on the watched branch. RunPod’s layer caching means the rebuild is fast after the first one, since only the layers that changed get rebuilt (and for an empty commit, none did).

If you’d rather not pollute history, the console’s manual Rebuild button is the cleaner equivalent. Either way you’re doing the same thing: asking RunPod to provision workers again, on hosts whose occupancy has moved since the last attempt.

How do I fix it when publishing a RunPod Hub template?

Cut a new GitHub Release. The Hub does not watch commits. Per RunPod’s publishing guide, “repository integration connects with GitHub repos using releases (not commits) for versioning”, so pushing to your branch does nothing on the Hub side. Only a new Release re-runs the build and the validator test pod, which is where this error shows up for template authors.

Here’s the trick that saves you: a Release is just a tag, and a tag can point at a commit you already have. You don’t need to change a single line of code to re-trigger the Hub. Tag the same HEAD you already shipped and publish a Release for it:

git tag v1.6.4          # same commit, new tag
git push origin v1.6.4
# then publish a GitHub Release for v1.6.4 in the UI or via gh

RunPod treats each tag as a distinct template version and re-runs the full pipeline: build the image, spin up the validator test pod from .runpod/tests.json, and index the listing (usually within an hour). If the previous Release failed only because the validator pod couldn’t get a GPU, the new Release gives it a fresh roll of the scheduler.

The catch worth stating: each retry adds a version to your Hub listing, even if two versions are byte-identical. That’s the cost of the release-driven model. It’s cosmetic, but if you retry five times you’ll have five versions, so don’t spin on it if the failure is actually persistent (see below).

Why is the retry different for Serverless vs the Hub?

Because the two products use different GitHub triggers. A Serverless endpoint rebuilds on every push to its watched branch, so a commit is your retry. The Hub builds only on a new Release tag, so a release is your retry. Pushing commits at a Hub listing does nothing; pushing releases at a Serverless endpoint isn’t how it watches for changes.

	Serverless endpoint	RunPod Hub template
Build trigger	Every push to the watched branch	New GitHub Release (tag)
Retry the deploy by	Empty commit, or Rebuild in console	New Release on the same commit
Who needs this	Anyone running a GitHub-connected worker	Template authors publishing to the Hub
Where the error appears	Workers stuck initializing / throttled	The validator test pod after a Release

Most readers are in the left column. You deployed a worker from a repo (or one-click from the Hub) and you’re iterating on it; a commit re-rolls it. The right column is for the smaller group of people authoring a Hub listing, where the validator test pod is the thing hitting the capacity wall. If you’re not publishing your own Hub template, you can ignore the release workflow entirely. For the full deploy walkthrough, the getting-started guide covers both paths.

What if re-triggering doesn’t fix it?

If the same GPU pool fails every time, the capacity miss is persistent, not transient, and rolling the scheduler won’t help. Switch to a higher-availability GPU pool, change region, or lower the resources you’re requesting. For the mineru-runpod Hub validator, that means editing the gpuTypeId in .runpod/tests.json to a pool that’s actually free, then cutting a new Release.

The template defaults its validator to "NVIDIA GeForce RTX 4090" because it has the best pool availability across RunPod’s regions. "NVIDIA RTX A5000" works too but tends to be scarcer. I’ve bounced the test pod’s GPU between the A40, the A5000, and the 4090 across releases, chasing whichever pool had capacity on a given day, and the 4090 wins most often.

Three levers when retries aren’t enough:

Change the GPU type to a less-contended pool. A 24 GB workload fits several pools; pick the one with capacity rather than the one you assumed.
Change the region if your endpoint or template pins one. Capacity is per data center, so a pool that’s full in one region can be wide open in another.
Reduce the request. Oversized container disk or volume sizes shrink the set of hosts that can fit your pod. Trim them if they’re padded.

For template authors, there’s also an escape hatch documented in the troubleshooting guide: if a release is urgent and the validator is the only blocker, rename .runpod/tests.json to .runpod/tests_.json so the Hub skips the test pod entirely. You lose all CI signal, so it’s a temporary unblock, not a default. For the GPU-pool math behind these choices, see Choosing a GPU.

FAQ

Is “this machine does not have the resources to deploy your pod” a RunPod outage?

No. It’s a per-host capacity miss, not a global outage. The scheduler tried one machine, found it full, and stopped. Other hosts of the same GPU type may have room, which is why a retry often succeeds within seconds even while RunPod is otherwise healthy.

Does pushing a commit fix the error on a RunPod Hub template?

No. The Hub builds only on new GitHub Releases, not commits. Pushing to your branch leaves the Hub listing untouched. You have to publish a new Release (a new tag, which can point at the same commit) to re-run the Hub build and its validator test pod.

How do I re-trigger a RunPod Serverless build without changing code?

Push an empty commit with git commit --allow-empty to the watched branch, or click Rebuild in the RunPod console. Both force a fresh build and redeploy, so workers get scheduled again on hosts whose free capacity has shifted since the last attempt.

Can I create a GitHub Release without a new commit?

Yes. A Release is a tag, and a tag can point at any existing commit. Tag your current HEAD and publish a Release for it. RunPod treats every tag as a new version and re-runs the build, so this re-triggers the Hub without any code change.

Why does the same deploy fail once and work the next time?

GPU capacity on RunPod fluctuates minute to minute. The same config hits a full host on one attempt and a free host on the next, with nothing about your image changing. That non-determinism is exactly why retrying is the first thing to try.

How do I stop hitting the capacity error repeatedly?

Stop targeting a contended pool. Switch gpuTypeId to a higher-availability GPU (RTX 4090 pools are usually the most available), change region, or reduce requested disk and volume sizes so more hosts can fit your pod.

Where to next

The error is annoying but harmless once you know it’s capacity and not your build. For a Serverless worker, a commit re-rolls it; for a Hub template, a Release does. If it persists, it’s a pool-availability problem, and the fix lives in your GPU choice, not your Dockerfile. The full set of Hub build failures (this one, the CUDA floor mismatch, and the 30-minute build timeout) is catalogued in the troubleshooting guide.

If this saved you a debugging session, star the repo on GitHub for updates, or open an issue if you hit a build failure that isn’t covered here.

How RunPod FlashBoot Actually Works (4-Request Test)

May 26, 2026

Sergei Shmakov

Last Updated: 2026-05-26

If you’re shipping vLLM or any heavy ML model on RunPod Serverless, you’ve probably looked at FlashBoot, ticked the checkbox, and then watched your cold starts still take 60-120 seconds. RunPod’s marketing says “1-second cold starts.” Their docs describe FlashBoot as “pre-loading container images.” Neither of those matches what most ML workloads actually see.

I ran four cold-start tests on a deployed RunPod endpoint serving a vLLM-backed PDF parser. The wall-clock numbers ranged from 7 seconds to 7 minutes. The point of this post is to explain why — what FlashBoot actually does at the systems level, when it kicks in, and how to set up your worker so it kicks in more often.

What does FlashBoot actually do?

FlashBoot is a CRIU-style process snapshot mechanism. When a worker scales to zero, RunPod captures the full process state (Python interpreter, CUDA VRAM, subprocess tree) into a snapshot on the host’s local storage. When the worker scales back up on the same host, RunPod restores from that snapshot. The restored process resumes mid-stride: model still in VRAM, vLLM engine subprocess still alive, IPC pipes still connected.

The key qualifier that RunPod’s docs don’t mention: snapshots are per (host, image SHA), not per endpoint. If the next scale-from-zero lands on a different host, there’s no snapshot to restore from. The worker boots fresh and pays the full warmup cost. Once.

The TL;DR for an ML workload: set up an eager warmup at worker boot, then let FlashBoot do its thing. Each new host pays the warmup tax once. Subsequent scale-from-zeroes on that same host get the snapshot restore and finish a typical request in single-digit seconds.

Why do “cold” starts sometimes take 7 seconds and sometimes 110?

Because they’re hitting different parts of the per-host model. Four consecutive requests against the same endpoint, single-page parse on each, with a deliberate scale-to-zero between every one:

Request	Wall-clock	Host	Snapshot?	What the worker did
1	456 s	A (post-rebuild)	none	Image pull + fitness checks + warmup (101 s) + parse (5.6 s)
2	7.6 s	A (same as R1)	yes	Snapshot restore + parse (4.7 s)
3	122 s	B (different host)	none	Fitness checks + warmup (101.5 s) + parse (5.6 s)
4	7.4 s	B (same as R3)	yes	Snapshot restore + parse (4.6 s)

First hit on a fresh host pays ~110 s for the warmup. Every subsequent restore on that same host is ~7-8 s. A new host, when RunPod’s scheduler picks one, starts the cycle over.

The 456 s on Request 1 included a one-time image pull (the worker image is ~27 GB; this was the first time that physical host had ever seen it). Strip that off and you get ~110 s of actual boot work, which matches Request 3 exactly.

How can you tell if a request hit a snapshot restore?

By what’s missing from the worker logs. A FlashBoot-restored worker skips its boot sequence entirely — no fitness checks, no Python import logs, no vLLM engine initialization, no model load. The first log line is Jobs in queue: 1, immediately followed by your handler’s “starting job” entry.

Compare a fresh boot to a snapshot restore for the same request shape:

Fresh boot (Request 3):

04:45:45  Running 7 fitness check(s)...
04:45:46  All fitness checks passed. (1285.99ms)
04:45:46  [mineru-warmup] starting (backend=vlm-auto-engine ...)
04:45:51  Using vllm-async-engine as the inference engine for VLM.
04:46:23  Initializing a V1 LLM engine (v0.11.2) ...
04:46:47  Model loading took 2.1601 GiB memory and 18.41 seconds
04:47:14  torch.compile takes 22.81 s in total
04:47:17  init engine (profile, create kv cache, warmup model) took 30.66 seconds
04:47:18  get vllm-async-engine predictor cost: 87.26s
04:47:28  [mineru-warmup] done in 101.5s
04:47:28  Jobs in queue: 1
04:47:28  Started.
04:47:28  "starting job" {...}
04:47:34  "done" {...elapsed_seconds: 5.58...}

Snapshot restore (Request 4):

04:51:25  Jobs in queue: 1
04:51:25  Started.
04:51:25  "starting job" {...}
04:51:26  Using vllm-async-engine ...   (instant — engine handle restored from snapshot)
04:51:30  "done" {...elapsed_seconds: 4.58...}

No boot sequence. Three timestamps. The vLLM engine subprocess PID from the previous boot is reused — same EngineCore_DP0 pid=NNN from the snapshot. If you grep your own worker logs for the gap between Jobs in queue: 1 and the previous activity, you’ll see whether RunPod did a fresh boot or a snapshot restore.

What does the FlashBoot snapshot preserve?

Everything that lived in the worker process at snapshot time, mediated by CRIU semantics:

Python interpreter state. Module imports stay loaded. Globals (job counters, contextvars, signal handlers) keep their values. The MinerU engine registry returns the same handles it returned before the snapshot.
GPU VRAM. Model weights (~2.16 GiB for our VLM), vLLM’s KV cache (~8.17 GiB on a 24 GB card), and captured CUDA graphs (~0.3 GiB) all survive. The first request after restore parses with the same allocations it had before.
The subprocess tree. vLLM runs its engine in a child process for memory isolation. That subprocess gets captured along with the parent and restored with its IPC pipes intact. The engine PID persists.
torch.compile cache. The JIT-compiled Dynamo / Inductor output stays valid across restore. No 22-second recompile.

What doesn’t survive: snapshot lifetime is limited. RunPod doesn’t publish the eviction policy, but obvious triggers include image rebuild (new SHA invalidates the snapshot), and presumably long enough idle on a busy host that the snapshot storage gets pushed out.

What broke before this worked? The asyncio gotcha

The “eager warmup at boot” idea is obvious in principle: run one throwaway parse during worker startup so the model is loaded and warm before any user request arrives. The implementation has one trap.

vLLM’s AsyncLLMEngine binds its IPC primitives (transports, queues) to the asyncio event loop that initialized it. If you call asyncio.run(warmup()) followed by runpod.serverless.start(), your warmup creates loop A, runs the parse, then tears loop A down when asyncio.run returns. Then runpod.serverless.start() creates loop B for serving. When the first user request tries to talk to the vLLM engine through loop B, the engine handle is bound to the now-dead loop A. Result:

"error_type": "EngineDeadError",
"error_message": "EngineCore encountered an issue. See stack trace (above) for the root cause."

The engine subprocess itself is still alive. It’s only the parent process’s IPC reference that’s broken.

The fix is to keep the warmup and the serve loop on the same asyncio event loop. RunPod’s runpod.serverless.start() internally calls asyncio.run(JobScaler.run()), but JobScaler (in runpod.serverless.modules.rp_scale) is constructible directly and its run() is an awaitable coroutine. So you can compose:

import asyncio
from runpod.serverless.modules import rp_ping, rp_scale
from runpod.serverless.modules.rp_fitness import run_fitness_checks

config = {"handler": handler, "concurrency_modifier": _concurrency_modifier, "rp_args": {}}

async def _bootstrap():
    await run_fitness_checks()
    await warmup_async()          # <- engine binds to THIS loop
    rp_ping.Heartbeat().start_ping()
    await rp_scale.JobScaler(config).run()   # <- and stays here

asyncio.run(_bootstrap())

Now both phases share one event loop. The engine handle stays valid across the warmup → serve transition. FlashBoot captures a snapshot of a process where the loop, the engine, and the IPC are all alive together. On restore, they come back together too.

This does reach into runpod-python’s internals (the runpod.serverless.modules.* submodules aren’t part of the documented public API). Cheap to guard against drift: a unit test that asserts JobScaler exists with the expected constructor and an awaitable run() method. If RunPod refactors, CI catches it before production does.

When does the warmup pay off and when doesn’t it?

Per host, not per endpoint. The math depends on your traffic pattern.

Scenario	Likely outcome
`workers_min ≥ 1` (always-on worker)	Worker stays on its host. Every request is on a fully warm worker (~5 s parse). No cold starts at all.
High-frequency endpoint, workers scale up and down fast	Same hosts get re-selected. Most cold starts are happy-path restores (~7 s).
Quiet endpoint, infrequent requests, long idle gaps	RunPod’s scheduler may pick a different host. Some cold starts will be on new hosts (~110 s).
First request after a rebuild	Always cold path. Every endpoint’s first request after a fresh image pays ~5-7 min (image pull) + ~110 s (warmup). One-time per worker host.
`MINERU_SKIP_WARMUP=1` (warmup off)	Every cold start is ~110-130 s. No per-host amortization. Don’t do this in production.

The case that stings is “quiet endpoint with sporadic traffic” — a few requests an hour, 10-minute idle gaps, RunPod bouncing between hosts. Without warmup, every cold start would be ~110-130 s. With warmup, you get a mix: some 7-second restores, some 110-second fresh boots. The mix tilts toward fast as the endpoint warms up across more hosts and RunPod’s scheduler starts re-selecting them.

If your traffic is sustained enough that you can pin a worker (workers_min=1), you skip the entire question. You’re paying for the GPU 24/7 but never paying a cold start. For workloads with even modest cost sensitivity, the warmup + FlashBoot path is the better trade.

What this means if you’re shipping vLLM on RunPod

Three takeaways from the live measurements:

Always set up an eager warmup at worker boot. Loading the model on first request is silently worse than it sounds — you don’t just pay 110 s once per cold start, you pay it every time a host doesn’t have a snapshot, AND you forfeit the per-host amortization that makes the second-hit-on-a-host cheap. Without warmup, FlashBoot has nothing to snapshot.
Compose warmup and the serving loop under one asyncio.run(). If you asyncio.run() the warmup separately, the engine dies at the loop boundary. The fix is straightforward but the failure mode is opaque (EngineDeadError 75 ms into the first request) — easy to misdiagnose as a vLLM bug.
Don’t market your cold start as “X seconds” without acknowledging the per-host mix. A snapshot-restore cold start is genuinely 7-8 seconds. A new-host cold start is ~110 s. Both are big improvements over the no-warmup baseline (~110-130 s per request, every request). But your users will see the mix, and a too-clean claim makes the bad days look broken.

The whole investigation was on a 24 GB A5000 / RTX 4090 class GPU running MinerU’s 1.2B VLM via vLLM 0.11.2. The numbers will shift on larger models (more VRAM to snapshot, longer model load on cold path) but the mechanism applies the same way. If your cold start dominates wall-clock latency on a serverless GPU workload, set up boot-time warmup, watch the worker logs for the snapshot pattern, and tune your workers_min accordingly.

FAQ

Does FlashBoot snapshot the vLLM engine subprocess?

Yes. The vLLM engine runs as a child process for memory isolation, and FlashBoot’s CRIU-style mechanism captures the full process tree including subprocesses. The engine’s PID persists across snapshot/restore, and its IPC pipes back to the parent stay connected.

Why does my cold start take 60-120 seconds even with FlashBoot enabled?

Most likely your model is being loaded lazily on first request rather than at worker boot. FlashBoot only snapshots state that already exists in the worker process when it scales to zero. If your model loads on first request, the snapshot captures a worker without the model, and every cold start has to load the model again. Move the model load to worker boot (before runpod.serverless.start()) and FlashBoot will start carrying the warm state forward.

What’s the difference between FlashBoot and a network volume?

A network volume is shared file storage attached to your worker (e.g., for model weights you don’t want to bake into the Docker image). FlashBoot is process-state preservation — it captures the running Python process, including data already loaded from disk into VRAM. They solve different problems and can be used together: a network volume avoids re-downloading model files on image pull; FlashBoot avoids re-loading them into VRAM on cold start.

Does FlashBoot work for non-GPU workloads?

The mechanism (process snapshot via CRIU or equivalent) doesn’t depend on GPU memory specifically. CPU-bound workloads with significant cold-start cost (heavy library imports, large in-memory indices, JIT compilation) should benefit similarly. The framing in this post happens to use a GPU workload because that’s where the cold-start tax is most painful.

How do I know if my worker is hitting a snapshot restore vs a fresh boot?

Check the worker logs in the RunPod dashboard. A fresh boot shows fitness checks, framework init logs, and any warmup output. A snapshot restore is silent until the first Jobs in queue: 1 line, then jumps straight to your handler’s request-processing logs. The presence or absence of the boot sequence is the cleanest signal.

Is FlashBoot the same as RunPod’s “Active Workers” tier?

No. Active Workers are a billing tier where you pre-commit to a number of workers that are always on, billed at a discount in exchange for the 24/7 commitment. FlashBoot is a free runtime optimization that applies to flex (scale-to-zero) workers. The two can be combined: an Active Worker on the same endpoint can also benefit from FlashBoot when it cycles, though for a worker that never goes idle there’s nothing to snapshot.

Will FlashBoot survive a Docker image rebuild?

No. Each image gets its own SHA, and FlashBoot snapshots are scoped to (host, image SHA). When you push a new image, all existing snapshots are invalid. The first request after a rebuild on any host pays the full cold-start cost (image pull + warmup). Once each host has served the new image once, subsequent restores work normally.

What’s next

The runpod-mineru repo wraps all of this into one Docker image: MinerU 3.2.x + the MinerU2.5-Pro-2605-1.2B VLM, the JobScaler-bypass composition for warmup, structured logging, and the rest. Open source (GitHub), MIT-licensed, deploys from the RunPod Hub in two clicks.

If you want the deeper breakdown of which phases of a cold start cost what, the troubleshooting guide has the per-phase timing table from the same test runs. The scaling guide covers when to pair FlashBoot with workers_min ≥ 1 for fully predictable latency.

Disclosure: RunPod links in this post use a referral code that credits me at no cost to you. The post would read the same without it.

Ship MinerU on RunPod logs to Axiom via OpenTelemetry

May 26, 2026

Sergei Shmakov

Last Updated: 2026-05-26

If you’re running a serverless worker on RunPod, you can’t ssh in to read logs and you can’t run a sidecar agent — the worker scales to zero between jobs. You need to ship logs and metrics off the box during the request lifetime, on the worker’s own time. The mineru-runpod template includes an OpenTelemetry exporter for exactly this, and Axiom is the sink I picked for my own deployment.

This post is the exact env-var layout. If you use a different OTLP backend (Honeycomb, Grafana, Datadog, Jaeger, your own OpenTelemetry Collector), the observability guide covers the vendor-neutral setup; come back here only for the Axiom-specific values.

Setup time from a fresh Axiom account to logs flowing: ~10 minutes, dominated by waiting for the worker’s next cold start.

Why use Axiom for serverless worker observability?

Axiom ingests OTLP/HTTP directly, charges per event rather than per host, and has no agent to install — which matters when your workers scale to zero between jobs. For a mineru-runpod deployment processing a few hundred PDFs a day, the events fit inside Axiom’s free tier, and the metrics dataset is queryable via APL.

Three concrete reasons it fits this workload:

OTLP-native ingest. No collector to run inside the container, no daemonset, no Fluentbit config. The mineru-runpod worker calls Axiom’s regional edge endpoint (https://eu-central-1.aws.edge.axiom.co/v1/logs or the US variant) directly via the OpenTelemetry Python SDK. The serverless model rules out anything that needs a long-running sidecar, so “the exporter IS the integration” is the only model that works.
Per-event pricing instead of per-host. A worker serving 500 PDFs a day emits roughly 5,000 log records and 2,500 spans. That fits comfortably inside Axiom’s free 0.5 GB/month tier. Per-host pricing models (the Datadog and NewRelic shape) penalize the ephemeral-worker pattern.
APL reads like SPL. Axiom Processing Language sits between Splunk SPL and KQL ergonomically. Filter by attribute, group by backend, drill into a span: the queries you actually run during an incident are easy in APL. No tutorial needed if you’ve used either.

I have no affiliation with Axiom. They’re the backend I picked for my own deployment after looking at Honeycomb, Grafana Cloud, and self-hosted Jaeger.

How do I create Axiom datasets for OpenTelemetry?

Create two datasets in the Axiom UI: one Events dataset (holds both traces and logs) and one Metrics dataset. Those are the only two dataset types Axiom exposes in the UI; metrics are kept separate because they use a different storage format under the hood.

Sign in to Axiom and go to Datasets → New dataset.
Create mineru-events with Events type. This holds traces and logs together.
Create mineru-metrics with Metrics type.

Names are arbitrary — substitute whatever fits your naming convention. I prefix everything with the service name so multiple endpoints (staging, prod, experiments) don’t collide in queries.

How do I generate an Axiom API token for OTLP ingest?

In the Axiom UI, go to Settings → API tokens → Generate new token. Use an Advanced API token (prefix xaat-) and explicitly grant the Ingest scope on both datasets you just created. Forgetting the scope on the metrics dataset is one of the common failure modes — it surfaces as 403 Forbidden in the worker logs while events ingest works fine. Copy the resulting xaat- prefixed string and paste it into your RunPod endpoint config in the next section.

Treat the token like any production secret: paste it only into RunPod’s Environment Variables UI (which encrypts at rest), never check it into git, and rotate when employees with access leave.

If your Axiom workspace is in the EU region, the management API lives at https://api.eu.axiom.co (US is https://api.axiom.co). This is the host you query for token CRUD and REST queries, not the OTLP ingest URL — that’s a separate edge-deployment hostname documented in the env-var section below.

What environment variables ship RunPod worker telemetry to Axiom?

Set four environment variables on your RunPod endpoint. OTEL_EXPORTER_OTLP_HEADERS covers both traces and logs (they share the Events dataset); OTEL_EXPORTER_OTLP_METRICS_HEADERS overrides for metrics only because Axiom uses a different header for its metrics ingest.

Paste this into your endpoint’s Environment Variables section in the RunPod dashboard, substituting your token and dataset names:

OTEL_EXPORTER_OTLP_ENDPOINT=https://eu-central-1.aws.edge.axiom.co
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer xaat-YOUR-TOKEN,x-axiom-dataset=mineru-events
OTEL_EXPORTER_OTLP_METRICS_HEADERS=Authorization=Bearer xaat-YOUR-TOKEN,x-axiom-metrics-dataset=mineru-metrics
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

The endpoint URL is your Axiom edge deployment, NOT api.axiom.co / api.eu.axiom.co. This is the single biggest gotcha and the one that cost me hours when I first set this up. The api.* hosts are for management API (token creation, queries via REST). OTLP ingest goes to your workspace’s edge deployment hostname. As of writing, the two are:

US East 1: https://us-east-1.aws.edge.axiom.co

EU Central 1: https://eu-central-1.aws.edge.axiom.co

Use the one that matches the region you picked when creating the Axiom workspace. The current full list lives at Axiom’s edge deployments doc. If you send OTLP to api.{eu.}axiom.co, Axiom returns 400 mismatched region or 403 forbidden depending on path — the OTel SDK logs only the HTTP status code, so you’ll see Failed to export ... code: 400 (or 403) with no clue why. Axiom support flagged this for me after I’d been chasing 403s for an hour against the wrong host.

How the SDK routes each signal:

Traces + logs → use the generic OTEL_EXPORTER_OTLP_HEADERS, so spans and log records both land in the mineru-events dataset.
Metrics → OTEL_EXPORTER_OTLP_METRICS_HEADERS overrides for metrics only, with the distinct x-axiom-metrics-dataset header that Axiom’s metrics ingest requires.

The dataset names in the two *-dataset headers must exactly match the names you created in step 1, and your API token must have ingest scope on both. This is the #1 source of “everything is configured but nothing shows up” — mineru-events and mineru-metrics above are example names, not magic strings. If you named your datasets differently, update the headers to match. Mismatches surface as 404 Not Found (wrong dataset name) or 403 Forbidden (token lacks ingest scope on that dataset). Both fail silently from the caller’s side; the only signal is in the worker’s stdout, where the OTel SDK logs each retry with the HTTP status code.

Three details that trip people up:

Base URL, not the full path. Set the endpoint to the edge-deployment root, e.g. https://eu-central-1.aws.edge.axiom.co, NOT https://eu-central-1.aws.edge.axiom.co/v1/traces. The OpenTelemetry Python SDK appends /v1/traces, /v1/logs, /v1/metrics per signal automatically. If you set the full path, the SDK double-appends and Axiom returns 404.

Header case differs by signal. Events ingest uses x-axiom-dataset. Metrics ingest uses x-axiom-metrics-dataset (different header name, with -metrics- in it). Copying the events headers into OTEL_EXPORTER_OTLP_METRICS_HEADERS as-is sends metrics to the events dataset and Axiom’s metrics view stays empty.

Protocol must be protobuf for metrics. Axiom’s metrics ingest accepts only protobuf, not JSON. The http/protobuf value above is the default the SDK ships with; don’t override it to http/json thinking it’s the safer choice. JSON works for logs and traces but quietly drops metrics.

Save the variables, redeploy any active workers (or wait for the next cold start), and Axiom should see records within ~60 seconds of the first request that hits a warm worker. The metric reader flushes every 10 s; traces and logs flush every 500 ms.

How do I verify OpenTelemetry data is reaching Axiom?

Send one request to the worker, then check three views in the Axiom UI: Stream and Traces on the events dataset, and Metrics on the metrics dataset.

In the Axiom UI:

Stream view → set dataset = mineru-events → you should see JSON log records flowing as soon as the worker handles a request. Each carries service.name=mineru-runpod, runpod.endpoint_id=<your-endpoint>, and job_id for correlation.
Traces view → on the same mineru-events dataset → one trace per RunPod job. The root span is mineru.job. Its children are mineru.fetch_input, mineru.parse, and mineru.package. The mineru.warmup span shows up once per worker boot.
Metrics view → set dataset = mineru-metrics → after the first 10-second flush you should see mineru.jobs.total, mineru.job.duration, the GPU memory gauges, and the rest of the metric catalog.

If nothing arrives in any view, open the worker’s stdout (RunPod dashboard → Logs) and grep for Failed to export. The OTel SDK logs each retry with the HTTP status code:

Failed to export ... code: 404, reason: Not Found — the dataset name in one of the *-dataset headers doesn’t exist in your Axiom workspace. Rename the dataset or update the env var so they match.
Failed to export ... code: 403, reason: Forbidden — the dataset exists but your API token doesn’t have ingest scope on it. Open the token in Settings → API tokens, add Ingest scope on the dataset, and update the secret in RunPod.
Failed to export ... code: 403, reason: Forbidden — the #1 cause is using api.axiom.co or api.eu.axiom.co as the endpoint instead of the edge-deployment URL. Confirm by running curl -v POST https://api.eu.axiom.co/v1/traces -H "Authorization: Bearer xaat-..." -H "x-axiom-dataset: <yours>" -H "Content-Type: application/x-protobuf" --data-binary "" against your endpoint — if you get 403 there but 422 against https://eu-central-1.aws.edge.axiom.co/v1/traces (or the US edge variant), the URL is the issue. Other 403 causes: token genuinely lacks Ingest scope on the dataset.
Failed to export ... code: 400, reason: Bad Request — the dataset resolves and auth works, but the payload is being rejected. Causes: region mismatch (workspace is EU but you’re hitting us-east-1.aws.edge.axiom.co, or vice versa — Axiom returns mismatched region in the body), wrong header name on metrics (use x-axiom-metrics-dataset, not x-axiom-dataset), or the dataset was created of the wrong type for the signal.
Failed to export ... code: 401, reason: Unauthorized — the API token is wrong or expired. Generate a fresh token in Settings → API tokens and update the env var.
No Failed to export lines AND no [mineru-telemetry] init failed either — OTEL_EXPORTER_OTLP_ENDPOINT is empty or the worker hasn’t cold-started since you set it. RunPod env-var changes only take effect on the next cold start; warm workers keep the previous values.

Which APL queries help debug a mineru-runpod worker?

APL queries answer the three operational questions that come up most: which errors fired in the last hour, which parses are slowest, and how throughput breaks down by backend or endpoint. All three queries hit the events dataset, where both traces and logs land. The templates below are starting points — adjust attribute paths to match how your Axiom workspace unrolls OTLP resource and span attributes.

Recent errors, grouped by error type:

['mineru-events']
| where ['service.name'] == "mineru-runpod"
| where ['severity_text'] == "ERROR"
| where _time > ago(1h)
| summarize count() by tostring(['attributes.error_type'])
| sort by count_ desc

Slowest parses in the last hour:

['mineru-events']
| where ['name'] == "mineru.parse"
| where _time > ago(1h)
| project _time, duration = ['duration'], backend = ['attributes.mineru.backend'], input_format = ['attributes.mineru.input_format']
| sort by duration desc
| take 20

Throughput by endpoint over time:

['mineru-events']
| where ['name'] == "mineru.job"
| where _time > ago(24h)
| summarize jobs = count() by endpoint_id = tostring(['resource.runpod.endpoint_id']), bin(_time, 5m)
| render timechart

Attribute paths in APL depend on how Axiom unrolls OTLP records — ['resource.runpod.endpoint_id'] works in my workspace but yours may need ['runpod.endpoint_id'] directly. Run a quick | take 5 | project * against the dataset first to see the actual field names your workspace produces.

Does enabling OpenTelemetry slow down cold starts?

Enabling OTel adds roughly 200–500 ms to the first-ever cold start (one-time SDK init plus DNS resolution for the OTLP endpoint). Subsequent FlashBoot snapshot restores on the same host inherit the warm state, so the cost amortizes per host, not per request. For a parse that takes 5–30 seconds wall-clock, the overhead is invisible.

The numbers from my own deployment: on an RTX 4090 with vlm-auto-engine, the bare cold start is ~110 s (image pull + vLLM init + model load + warmup parse). OTel init adds 200–500 ms on top of that. On a FlashBoot-restored boot, the same overhead is 0 — the snapshot captures the initialized SDK along with the rest of the process state. See the FlashBoot mechanism investigation for how the snapshot path actually works.

If you’re cost-sensitive about cold starts and don’t need observability on every deployment, leave OTEL_EXPORTER_OTLP_ENDPOINT unset. The mineru-runpod worker skips the OTel SDK import entirely when that variable is empty — zero overhead, zero behavior change. Flip it on for the endpoints where you actually want the visibility (production, staging) and leave it off for experimentation runs.

Where does this setup fall down?

Three real limitations: Axiom’s free tier caps ingest at 0.5 GB/month and 30-day retention; the GPU gauges emit one time series per device label per metric and cardinality adds up; and OTLP/HTTP export adds modest latency on cold starts (200–500 ms). None of these are blockers for a small or mid-volume serverless deployment, but they’re the things to watch as volume grows.

Free-tier ceilings. Axiom’s free plan is generous (0.5 GB ingest/month, 30-day retention) but you can blow through it with a chatty worker. A debug-logging worker processing 1,000 PDFs/day at ~30 KB of logs per parse hits ~900 MB/month — past the cap. Either keep the log level at info (the default) or move to Axiom’s paid tier (currently $25/month for 5 GB ingest).

Metric cardinality. The GPU gauges (mineru.gpu.memory_used_bytes, mineru.gpu.utilization_percent) emit one time series per device label per gauge per worker. A multi-GPU worker times multiple worker instances times four GPU metrics multiplies fast. Axiom’s metrics pricing is per-event rather than per-series, so this is a “watch the bill” concern rather than a hard limit. If you scale to dozens of concurrent workers, drop the device label or sample less frequently.

Cold-start latency. OTel init adds 200–500 ms on first-ever boot. For most workloads this is dominated by the existing ~110 s of vLLM init, so it doesn’t matter. If you’re optimizing the cold start specifically (chatbot-style low-latency workloads, for example), benchmark with and without OTel before committing.

Logs are mirrored, not exclusive. The worker still writes stdout JSON to RunPod’s dashboard regardless of OTel. That’s deliberate: RunPod’s UI remains a working fallback when the OTel pipeline misbehaves. The cost is paying twice for log storage if you care about long retention in both places. Most teams don’t, and the duplication is the price of the dashboard fallback.

FAQ

Does Axiom support OpenTelemetry natively?

Yes. Axiom ingests OTLP/HTTP traces, logs, and metrics on /v1/traces, /v1/logs, /v1/metrics paths — but the base hostname is your region’s edge deployment, not api.axiom.co. The two as of writing are https://us-east-1.aws.edge.axiom.co and https://eu-central-1.aws.edge.axiom.co (full list at Axiom’s edge deployments doc). The OpenTelemetry Python SDK in the mineru-runpod worker speaks this directly with no Collector or agent in between. See Axiom’s OpenTelemetry docs for the full list of supported signals and headers.

Why aren’t my metrics showing up in Axiom?

Three common causes. First: OTEL_EXPORTER_OTLP_METRICS_HEADERS uses the wrong header key. Axiom needs x-axiom-metrics-dataset (lowercase, with -metrics-dataset), distinct from the x-axiom-dataset header used for logs and traces. Second: the API token doesn’t have ingest scope on the metrics dataset — common when you create the token with only the logs dataset selected, then later add the metrics one. Surfaces as 403 Forbidden in the worker stdout. Third: OTEL_EXPORTER_OTLP_PROTOCOL is set to http/json and Axiom’s metrics endpoint accepts only protobuf.

What’s the cheapest way to observe a RunPod serverless worker?

For ≤1,000 jobs/day with structured logs at info level, Axiom’s free tier (0.5 GB/month, 30-day retention) is the cheapest path with real query power. The OTel SDK is already in the mineru-runpod image; setup is four env vars on the endpoint. RunPod’s own log dashboard is free but lacks query language, metrics, and traces — fine for triage, not for SLO tracking.

Can I send traces and logs to the same Axiom dataset?

Yes — that’s the standard setup. Axiom exposes only two dataset types in the UI (Events and Metrics), and the Events type holds both traces and logs. The configuration above routes traces + logs to one events dataset via OTEL_EXPORTER_OTLP_HEADERS, and metrics to a separate metrics dataset via OTEL_EXPORTER_OTLP_METRICS_HEADERS. No per-signal traces override is needed.

How much does OTLP/HTTP export add to cold start time?

200–500 ms on the first-ever cold start of a worker on a host. That’s one-time SDK init and DNS resolution for the OTLP endpoint. Subsequent FlashBoot snapshot restores on the same host pay zero overhead — the initialized SDK is captured in the process snapshot. On a baseline 110 s cold start (vLLM + model load + warmup), the OTel cost is invisible.

Why isn’t `api.axiom.co` the OTLP endpoint?

Because Axiom splits two concerns onto two hostnames: api.{eu.}axiom.co is the management API (token CRUD, REST queries, dashboards), while OTLP ingest goes to the edge deployment hostname for your workspace’s region. The split isn’t obvious from the OpenTelemetry side because most other backends expose ingest on the same hostname as the management API. Axiom’s own OpenTelemetry guide and edge deployments doc document the edge URLs, but it’s easy to miss if you start from a generic OTel tutorial.

Does enabling OpenTelemetry require an OpenTelemetry Collector?

No. The mineru-runpod worker uses the OpenTelemetry Python SDK with the OTLP/HTTP exporter, talking directly to Axiom’s ingest endpoint. A Collector is useful when you want to fan out to multiple backends, apply sampling rules, or buffer locally — none of which apply to a single-sink serverless worker shipping into Axiom.

Same template, different backend

The four env vars above are the only Axiom-specific configuration in this whole setup. Swap the URL and headers and the mineru-runpod worker ships the same logs, traces, and metrics to anything that speaks OTLP/HTTP — Honeycomb, Grafana Cloud, Datadog’s OTLP intake, your own OpenTelemetry Collector. Most other backends don’t even need the metrics-headers override that Axiom requires — a single OTEL_EXPORTER_OTLP_HEADERS value covers all three signals. The observability guide covers the vendor-neutral env-var layout and the metric catalog. Different backends, same template.

If this saved you time, the easiest way to say thanks is signing up for RunPod through this link. Star the repo on GitHub for updates.

Disclosure: RunPod links in this post use a referral code that credits me at no cost to you. The post would read the same without it.

RunPod 20 MB Response Cap: Fix NoneType with Cloudflare R2

May 20, 2026

Sergei Shmakov

Last Updated: 2026-05-26

If your RunPod serverless worker logs say done but your client raises unexpected handler return type: <class 'NoneType'>, you’ve hit RunPod’s bidirectional 20 MB payload cap on /runsync. The handler succeeded. The gateway dropped the response on the way back because the payload was too large.

The fix is two steps. Set return: "s3" on the job, and configure four env vars on the endpoint pointing at a Cloudflare R2 bucket. The worker uploads the result to R2 and returns a small presigned URL. Your client downloads from R2 directly. No gateway cap in the path.

I hit this on an 82-page Cyrillic fiscal report (30 MB input, ~25 MB output with embedded images) running my open-source mineru-runpod template. Two retries via return: "inline" and return: "tarball_b64" failed the same way. R2 mode worked first try. The rest of this post is the symptom, the env-var recipe, the cost comparison vs S3, and a few gotchas worth knowing.

Why does my RunPod worker return NoneType after a successful parse?

The worker handler completed and returned a valid dict. RunPod’s runtime then tried to POST that result back to RunPod’s API via /job-done, and the API returned HTTP 400 because the payload exceeded ~20 MB. The result was discarded. The SDK saw no output, returned None to the client, and the client wrapper raised the NoneType error.

The worker logs make the chain explicit:

[mineru-worker] done: elapsed=91.77s phase_ms={'fetch_input': 972, 'mineru_parse': 90789, 'package': 66}
{"requestId": "sync-fdcd03cd-...", "message": "Failed to return job results. | 400, message='Bad Request',
 url='https://api.runpod.ai/v2/<endpoint>/job-done/<worker>/sync-fdcd03cd-...?gpu=NVIDIA+RTX+A5000&isStream=false'"}

The first line shows the handler finished cleanly: 82 pages parsed in 91.8 s on the worker (this test ran on A5000; on the current 4090 default the warm parse is 2–3× faster). The second line shows the gateway rejecting the result. The handler already returned and never knows the rejection happened. The SDK sees the discarded result and returns None to your code.

If you see this NoneType error on a small doc, the diagnosis is different (worker OOM, crash, timeout). On a multi-page parse that the worker logs as done, the answer is almost always the 20 MB cap.

What is RunPod’s /runsync response payload limit?

RunPod’s /runsync gateway caps payloads at roughly 20 MB in both directions. The request cap affects file_b64 inline uploads. The response cap affects what the worker can return. Both are independent of execution time and memory budget. A fast, successful parse can hit the response cap simply by producing a large output.

Direction	Limit	What triggers it
Request → gateway → worker	~20 MB	`file_b64` inline transport for large PDFs
Worker → gateway → client	~20 MB	Multi-page parse outputs with embedded images

The request cap is in RunPod’s docs and widely discussed. The response cap is mentioned only in passing. I found three open issues on the runpod-workers repos where other users hit the same symptom and didn’t realise what it was, so this post is partly to make that searchable.

Practical threshold for mineru-runpod: pure-text PDFs are fine for longer. Image-heavy PDFs with embedded raster output hit the response cap around 50–80 pages on inline or tarball_b64 transport.

Does `return: "tarball_b64"` get around the 20 MB cap?

No. return: "tarball_b64" gzips the output into a single .tar.gz before base64-encoding it. Gzip compresses the JSON and Markdown text well, but the page images inside the tarball are already raster bytes (PNG, JPEG) and barely compress further. Multi-page parses with embedded images keep the tarball over 20 MB.

I confirmed this on the same 82-page PDF. Same 400 from /job-done. Same NoneType in the client. Both inline and tarball_b64 route through the gateway response, so both inherit the cap. Only return: "s3" avoids it because the worker uploads out-of-band.

How do I configure Cloudflare R2 to bypass the RunPod response cap?

Set return: "s3" in the job input, then add four env vars on the RunPod endpoint pointing at a Cloudflare R2 bucket. The worker uploads the gzipped tarball directly to R2 and returns a small presigned URL (~1 h TTL). Your client downloads from R2.

The job input changes one field:

{
  "input": {
    "file_url": "https://example.com/big.pdf",
    "return": "s3"
  }
}

The four env vars go on the endpoint (not the template — they’re secrets):

Env var	Cloudflare R2 value
`BUCKET_ENDPOINT_URL`	`https://<account-id>.r2.cloudflarestorage.com`
`BUCKET_NAME`	your bucket name
`BUCKET_ACCESS_KEY_ID`	R2 API token access key
`BUCKET_SECRET_ACCESS_KEY`	R2 API token secret
`BUCKET_REGION` (optional)	`auto`

You generate the access key pair in the Cloudflare dashboard: R2 → Manage R2 API Tokens → Create API Token → Object Read & Write scoped to the bucket. The worker auto-restarts when you save endpoint env vars in RunPod. Test with one small doc before sending production traffic.

Why pick Cloudflare R2 over AWS S3 for RunPod output storage?

R2 has zero egress fees, a 10 GB free storage tier, 1M Class A ops and 10M Class B ops per month free, and is fully S3-compatible. AWS S3 charges egress at roughly $0.085/GB plus storage at $0.023/GB/month. For a RunPod pipeline doing dozens of GB of I/O per month, R2’s bill stays near zero while S3 lands in the $5–$15 range.

A back-of-envelope month for the workload I tested:

1,000 multi-page parses, average output 8 MB → 8 GB stored then deleted
1,000 worker→bucket uploads + 1,000 client→bucket downloads = 2,000 ops
Storage: free (under 10 GB). Egress: free (R2 doesn’t bill egress). Ops: free (well under 1M Class A).

Same workload on S3: ~$0.18 storage + ~$0.68 egress + per-request fees, maybe $1–$3 total. Cheap but R2’s $0 is cheaper.

S3 still makes sense if you’re already deep in AWS, if you need IAM-controlled access patterns, or if RunPod workers and your AWS region are colocated tightly enough that egress doesn’t apply. For everyone else and especially for solo / indie deploys, R2 is the right default. See R2 pricing for current rates.

What does the parse flow look like end-to-end with `return: "s3"`?

The worker fetches the input PDF, runs MinerU, gzips the outputs into a tarball, uploads to R2 via the configured BUCKET_* env vars, and returns a small JSON response with tarball_url, tarball_url_expires_in (3600 s), and bucket_key. Your client follows the URL and extracts the tarball locally. No payload ever crosses RunPod’s 20 MB-capped response path.

Concrete numbers from the 82-page test (on A5000; current default is 4090):

result = client.parse_document(
    file_url="https://pub-....r2.dev/report.pdf",
    backend="vlm-auto-engine",
    return_format="s3",
)
# result["tarball_url"]            -> presigned R2 URL, valid ~1 h
# result["tarball_url_expires_in"] -> 3600
# result["bucket_key"]             -> "report-<hash>.tar.gz"

client.save_s3_tarball(result, "./out/")
# downloads + extracts -> out/report.md, out/report_content_list_v2.json, out/images/, ...

End-to-end wall-clock: 211 s for an 82-page doc on a cold worker. Breakdown: ~112 s before MinerU started parsing (worker boot + warmup), ~92 s warm parsing (1.1 s/page on A5000), ~11 s gzip and upload to R2 (the package phase). The extracted output: 313 KB Markdown plus structured JSON plus per-page images. Roughly 3.5 minutes for a document that previously couldn’t return its output at all.

The cold-start portion is a separate concern from the response cap. The FlashBoot mechanism investigation covers why the ~112 s exists, how the boot-time warmup interacts with RunPod’s snapshot system, and when subsequent cold starts are much faster.

What should I watch out for with the R2 bridge?

Four things the docs don’t say loudly. The presigned URL TTL is 60 minutes. R2 doesn’t auto-clean uploaded objects. One bucket can serve input and output. The 20 MB cap applies to /run (async) too, not just /runsync.

Presigned URL TTL is 60 minutes. If your client is slow to download (e.g. a job-queue worker that picks up results minutes later), bump _S3_PRESIGN_TTL_SECONDS in the handler. Don’t rely on the default in long-tail flows.
R2 doesn’t auto-clean uploaded objects. Add an R2 lifecycle rule (e.g. delete after 7 days) so your output bucket doesn’t grow forever.
One R2 bucket can serve input and output. Upload your PDFs to R2 ahead of time, pass file_url pointing at the R2 public dev URL, and the worker writes outputs to the same bucket at the root. Add BUCKET_PREFIX env var if you want outputs in a subdirectory.
The 20 MB cap applies to /run (async) too. Same gateway, same limit. Switching to async polling doesn’t help.

FAQ

How do I get the R2 access key for `BUCKET_ACCESS_KEY_ID` and `BUCKET_SECRET_ACCESS_KEY`?

In the Cloudflare dashboard: R2 → Manage R2 API Tokens → Create API Token. Set permissions to “Object Read & Write” scoped to the specific bucket. Cloudflare shows the access key ID and secret access key once; copy both into your RunPod endpoint env vars immediately. The secret isn’t retrievable later.

Does the presigned URL expire?

Yes. The default TTL is 3600 seconds (one hour). If your downstream client picks up the response asynchronously (job queue, cron, etc.), download promptly or bump _S3_PRESIGN_TTL_SECONDS in the worker handler before redeploying.

Can I reuse the same R2 bucket for input and output?

Yes. The worker doesn’t care about the bucket layout. Upload your input PDFs to bucket/inputs/ and the worker writes outputs to bucket/<basename>-<hash>.tar.gz at the root. Add BUCKET_PREFIX env var if you want outputs pushed into a subdirectory.

What if I can’t set up R2? Is there a fallback?

Page chunking. Split the parse with start_page and end_page into segments small enough that each output tarball stays under 20 MB, then concatenate the .md files client-side. Slower (you may pay multiple cold starts if the worker scales to zero between calls) and you handle joining yourself, but no infra changes needed.

Is the 20 MB cap on `/run` too, or only `/runsync`?

Both. RunPod’s /run (async) and /runsync (synchronous) share the same gateway and the same payload limits. Switching to async doesn’t help the response-size problem. The cap is at the gateway layer, not the polling protocol.

Does using `return: "s3"` add to cold-start time?

No. The S3 upload happens at the end of the parse, not the beginning. The handler’s package phase grew from ~95 ms (in-memory tarball) to ~11 s (gzip + upload to R2) on an 82-page job, but cold-start is unchanged. The S3 mode adds a small constant to warm-job latency, not a multiplier.

How big can the R2-uploaded tarball be?

Effectively unlimited for mineru-runpod workloads. R2 supports multipart uploads up to 5 TB per object. You’ll hit the worker’s executionTimeoutMs long before you hit R2’s per-object limit.

Does R2 work for input PDFs too, or only output?

Both. The worker accepts file_url pointing at an R2 public dev URL (or a presigned R2 GET URL for private buckets) and fetches the input from R2. This avoids the inbound 20 MB cap on file_b64 for large PDFs. You can run an R2-in / R2-out setup with one bucket and avoid every payload-size limit RunPod has.

Where to next

If you’ve shipped a multi-page PDF pipeline on RunPod and you’re not using return: "s3", you’ll hit the gateway cap eventually. Set it up before you need it. The cost is ten minutes of env-var configuration and possibly zero dollars per month at indie volumes.

If you’re new to the template, the getting-started guide walks through the full deploy in about ten minutes. For the cold-start side of the picture (separate from the response cap covered here), see the FlashBoot mechanism investigation. For GPU sizing, Choosing a GPU covers when the default ADA_24 (RTX 4090) is enough and when to opt up.

If this saved you time, the easiest way to say thanks is signing up for RunPod through this link. Star the repo on GitHub for updates.

Disclosure: RunPod links in this post use a referral code that credits me at no cost to you. The post would read the same without it.

Serverless MinerU on RunPod: honest cost math (2026)

May 19, 2026

Sergei Shmakov

Last Updated: 2026-05-26

If you’re building a RAG pipeline, a document indexer, or any product that ingests PDFs at scale, you’ve probably hit the same wall I did. Hosted OCR APIs charge pennies per page that compound into thousands per million. CPU parsers are too slow for production volume. A permanent GPU pod is wasteful when traffic comes in bursts.

MinerU 2.5 is genuinely state-of-the-art for PDF → Markdown / structured JSON. Apache 2.0 license. The MinerU2.5-Pro-2604-1.2B model fits comfortably on a 24 GB GPU. RunPod Serverless scales to zero when nothing is calling. Wiring those two together is the obvious move.

Real numbers from my open-source mineru-runpod template, measured on a 24 GB RTX 4090 in May 2026: ~$0.001 per page for warm parses, plus a ~$0.03 fixed tax per cold start. The all-in per-page cost depends on how much work you do before the worker scales back to zero. Here’s the deploy, the response shape, and the workload patterns this template is the right fit for.

What does it actually cost to run MinerU on RunPod Serverless?

About $0.001 per page on an RTX 4090 once the worker is warm. Each scale-from-zero adds a ~$0.03 fixed tax: roughly 110 seconds of GPU billing for vLLM engine init plus model load. Per-page math depends entirely on amortization. Sparse traffic with one short request per cold start lands closer to $0.005–$0.01 per page.

Real workload-shape math using ADA_24 (RTX 4090, ~$1.10/hr Flex):

Workload shape	Per-page cost
1,000 pages amortized across one cold start	~$0.001
100 pages amortized across one cold start	~$0.0013
10 pages then idle out	~$0.004
One short doc per scale-from-zero (worst case)	~$0.007

Compared to alternatives:

Tool / setup	Per-page cost	Notes
Hosted OCR APIs (typical)	$0.001 – $0.01	vendor lock-in, rate limits, documents leave your stack
Permanent GPU pod (24 h on A5000)	$0.001 – $0.003	24 h of bills whether you use it or not
mineru-runpod, amortized	~$0.001 – $0.004	scales to zero; cold-start tax is real
Marker / Nougat on CPU	$0 cash, $$$ time	~30 s/page sequential (Marker docs)

The trick is RunPod’s per-second billing. No worker running, no bill. The catch is every scale-from-zero pays a real fixed cost.

How do I deploy MinerU to RunPod Serverless in ten minutes?

Fork the repo, point RunPod’s GitHub auto-build at your fork, create a Serverless Endpoint with ADA_24 (RTX 4090) and FlashBoot enabled, send a request via the included Python client. Total wall-clock from RunPod sign-up to first parse: roughly ten minutes, dominated by the image build (~5–10 min) plus the first cold start (~110 s).

1. Get a RunPod account

2. Fork the repo

gh repo fork sergeyshmakov/mineru-runpod --clone
cd mineru-runpod

The repo stays small. Dockerfile, handler.py, a worker/ package, a Python client (mineru_client), three GitHub Actions workflows, Hub metadata under .runpod/. MIT licensed, ~30 files.

3. Wire RunPod’s GitHub auto-build

In the RunPod dashboard:

Serverless → Templates → New → Import Git Repository
Point at your fork. Branch main, Dockerfile path Dockerfile.
RunPod clones, builds the image, stores it in its own registry, and gives you a template_id. The build runs ~5–10 minutes. Watch the log if you want.

4. Create the endpoint

Dashboard path:

Serverless → Endpoints → New
Template: the one you just created
GPU pool: ADA_24 (RTX 4090, 24 GB)
Workers min: 0, max: 3
Idle timeout: 10 seconds
FlashBoot: on
Save, grab the endpoint id

Or as code (reproducible across redeploys):

pip install -e .[deploy]
python deploy.py --template-id <tid>

deploy.py exposes every endpoint setting as a CLI flag.

5. Parse your first PDF

from mineru_client import MineruClient

client = MineruClient(
    endpoint_id="<your-endpoint-id>",
    api_key="<your-runpod-api-key>",
)
result = client.parse_document(
    file_url="https://example.com/report.pdf",
    end_page=4,  # smoke test on first 5 pages
)
client.save_tarball(result, "./out/doc")
# → ./out/doc/<basename>.md
# → ./out/doc/<basename>_content_list_v2.json
# → ./out/doc/<basename>_middle.json
# → ./out/doc/images/*.png

First parse pays a cold start. Subsequent parses on the same warm worker run at ~1–6 s/page on the 4090, content density dependent. After 10 s of idle, the worker scales to zero.

What does the MinerU response actually contain?

Three structured outputs plus extracted images. <basename>.md is Markdown with LaTeX equations, HTML tables, and image references. <basename>_content_list_v2.json is a flat list of typed entries (text, equation, table, image, code) each tagged with page_idx. <basename>_middle.json carries the full layout with bounding boxes and reading order. Pick the transport via return: tarball_b64, inline, or s3.

For a document indexer or RAG pipeline, content_list_v2.json is the file you’ll spend the most time with. Group entries by level: "title" boundaries for section-based chunking. Embed each chunk and store page_idx for citation back to the source.

The Markdown is for human-readable display. middle.json has bounding boxes per span when you need page coordinates for hover-to-source UI.

Transport options on the request: tarball_b64 (default) for outputs under ~20 MB, inline if you want the markdown directly in the JSON response, s3 for anything that would exceed RunPod’s response cap. See the R2 bridge post for the s3 setup.

When does mineru-runpod fit your workload, and when doesn’t it?

Good fit: batch ingest jobs, bursty traffic (50 docs in a minute, then quiet), background pipelines, OCR-API replacement. Poor fit: interactive single-document apps (cold starts make users think it’s broken), sparse traffic (one job per cold start dominates the bill), strict latency SLOs without provisioning workers_min ≥ 1.

I run this template in production for a document indexer. Six months of operation, here’s the honest fit picture:

Good fit:

Batch ingest. Drop 500 PDFs into a queue. One cold start amortizes across the whole batch at ~$0.001 per page.
Bursty traffic. A user uploads 50 documents in a minute. One cold start, 49 warm parses.
Background pipelines. Nightly cron processes yesterday’s intake. Cold start cost is rounding error against a multi-hour batch.
OCR-API replacement. Comparable per-page cost without shipping documents to a third party.

Poor fit:

Interactive single-document parsing. Your user uploads one PDF and waits two minutes for the cold start. They’ll think it’s broken.
Sparse traffic (one job every 20–60 min). Almost every request is a cold start. The ~$0.03 cold-start tax dominates. Rent a permanent low-tier GPU pod and skip serverless instead.
Strict latency SLOs. Cold-start latency is partly outside your control. Provisioning workers_min ≥ 1 eliminates cold starts but you pay for the warm worker around the clock.

The repo’s defaults (workers_min=0, idle_timeout=10s) are tuned for batch-with-bursts. The dashboard’s scaling settings are where you tune for other patterns.

What’s the real cold-start cost on RunPod Serverless?

Roughly 110 seconds before MinerU starts parsing your first request after a scale-from-zero. The composition: ~3 s fitness checks, ~20 s vLLM engine config, ~20 s model load, ~25 s torch.compile, ~5 s CUDA graph capture, ~5 s of actual parse. Billed at ~$1.10/hr on the 4090 default, that’s roughly $0.03 per cold start.

The per-phase breakdown is documented in the troubleshooting guide if you want to see where the time goes. The boot-time warmup in this template loads MinerU’s model and JIT-compiles vLLM kernels before the worker accepts requests. When RunPod’s FlashBoot snapshot is available on a subsequent scale-from-zero, the wall-clock drops to ~7–8 seconds because the snapshot captured a warm process. When the snapshot isn’t available (new host, image rebuild), warmup re-runs and you pay the full ~110 s again.

The FlashBoot mechanism investigation covers when the fast path applies, with measured numbers across multiple consecutive cold starts.

What should I watch out for before going to production?

Three production gotchas the marketing won’t mention. The 20 MB response cap silently drops large outputs (symptom: NoneType after a successful parse — covered by the R2 bridge). execution_timeout defaults to 900 s and won’t cover full books. file_b64 inline payloads cap around 10 MB on the way in. None of these crash the worker; they manifest as confusing client-side errors.

20 MB response cap. RunPod’s /runsync gateway drops responses over ~20 MB. Multi-page parses with embedded images hit this around 50–80 pages. Worker logs done; client gets NoneType. Fix: return: "s3" + Cloudflare R2, walked through in the R2 bridge post.
Long-job timeout. Repo defaults execution_timeout=900s (good for ~150–300 pages on 4090). A 5,000-page book is 80–500 minutes depending on content density. Bump execution_timeout for long jobs; the endpoint upper limit is 24 hours.
Inline payload cap on the way in. file_b64 requests cap around 10 MB. For bigger files, pass file_url and let the worker fetch from your storage. R2 public dev URLs work well.
Cold-start economics. “Pennies per page” depends on amortization. Track average pages per cold start in your logs. If it’s under 30, bump idle_timeout or run workers_min=1.

Where to next

The repo ships with:

Typed Python client (MineruClient)
deploy.py / destroy.py for endpoint lifecycle automation
Reference adapter pattern for wrapping MinerU output into domain models
96 unit tests, CI on every PR
Commitlint + semantic-release for automated CHANGELOG / GitHub Releases

For the deeper context that didn’t fit:

How RunPod FlashBoot actually works — four-request investigation into the cold-start mechanism and the per-host snapshot caveat.
The R2 bridge for the 20 MB response cap — fix for NoneType on multi-page outputs.
Choosing a GPU — when 24 GB is enough, when to opt up to 48 GB.

If this saved you time, the easiest way to say thanks is signing up for RunPod through this link. Star the repo on GitHub for updates.

FAQ

How does mineru-runpod compare to hosted PDF APIs?

Per-page cost is in the same ballpark ($0.001–$0.004) when amortizing cold starts across reasonable batches. The differences are control and lock-in. You deploy your own RunPod endpoint, pick your GPU and concurrency, run whichever MinerU version you want, and never send documents to a third party. The trade-off is operating a serverless template instead of consuming a managed API.

Can MinerU 2.5 handle non-English PDFs?

Yes. The vlm-auto-engine default backend handles English and Chinese well per the model card. For other scripts (Cyrillic, Arabic, Devanagari, Japanese, Korean), the pipeline backend uses PaddleOCR with script-family models, covering 109 languages. Empirically the Pro VLM also handles Cyrillic correctly even though lang is ignored on the VLM path. Switch backends per-request via the backend field.

What’s the difference between `vlm-auto-engine`, `pipeline`, and `hybrid-auto-engine`?

vlm-auto-engine uses MinerU’s 1.2B VLM via vLLM. Fastest on English / Chinese, ~1–6 s/page warm. pipeline uses PaddleOCR plus dedicated layout / formula / table models. Slower (~3–5 s/page) but more memory-predictable (4 GB minimum VRAM) and covers 109 languages. hybrid-auto-engine routes each page through either backend based on content. Highest quality on mixed-content docs; needs 48 GB on dense layouts.

Does the per-page cost include the cold-start tax?

No. The ~$0.001 per page is warm-worker math. Each scale-from-zero adds a roughly $0.03 fixed cost on the 4090 default. Your effective per-page cost is (0.001 × pages) + (0.03 × cold_starts) / pages. For 100 pages across one cold start, that’s $0.0013 per page. For 10 pages, it’s $0.004.

Can I use mineru-runpod with my own MinerU model?

Yes. Fork the repo and update the Dockerfile’s huggingface_hub.snapshot_download call to point at your model. Rebuild and redeploy. The handler is model-agnostic; MinerU’s aio_do_parse resolves whatever model is in HF_HOME at runtime.

What GPU does the template default to?

ADA_24 (RTX 4090, 24 GB). Switched from AMPERE_24 (A5000) on 2026-05-26 after measuring per-page cost. The 4090 is 2–4× faster per page than the A5000 and cheaper per page despite the higher hourly rate. See Choosing a GPU for the full math and when to opt up to 48 GB.

How do I keep my RunPod endpoint warm to avoid cold starts?

Set workers_min=1 on the endpoint. You pay for the always-on worker around the clock (~$0.000306/s on the 4090 default, so ~$26/day or ~$800/month). Worth it if your traffic is steady enough that the warm worker stays busy, or if your latency SLO can’t tolerate the cold-start window. For bursty traffic, workers_min=0 with FlashBoot enabled is usually cheaper.

Disclosure: RunPod links in this post use a referral code that credits me at no cost to you. The post would read the same without it.

Serverless

Why self-host the MinerU API instead of using mineru.net?

What does self-hosting MinerU cost vs the cloud API?

How do you deploy your own MinerU endpoint on RunPod?

How do you migrate your code off the MinerU API?

What doesn’t carry over from the cloud API?

FAQ

Is the MinerU API free?

What are the MinerU API’s limits?

Can I self-host MinerU without RunPod?

Does the self-hosted output match the cloud API’s full_zip_url?

Is self-hosting actually cheaper than the MinerU API?

Do my documents stay private when self-hosting?

Why not just send the whole 5,000-page PDF?

Why cut at clause boundaries instead of every N pages?

Did the batches actually cover the whole document?

How was the document transported in and out?

What GPU and backend, and what did throughput look like?

The parallelism lesson: it’s RunPod-side, not in the worker

Which clauses produced the most structure?

The annex schema dumps look completely different

How did resumability actually work?

Where this falls down / what I’d change

FAQ

How long does it take to parse a 5,000-page PDF with MinerU?

Why batch at clause boundaries instead of fixed page counts?

How do you handle output larger than RunPod’s 20 MB response cap?

Does sending more concurrent jobs make a single endpoint faster?

Do I need to split the PDF before uploading?

How do I know the whole document was covered?

Where to next

What does “this machine does not have the resources to deploy your pod” mean on RunPod?

Is it a RunPod bug, or did I break something?

How do I fix it on a RunPod Serverless endpoint?

How do I fix it when publishing a RunPod Hub template?

Why is the retry different for Serverless vs the Hub?

What if re-triggering doesn’t fix it?

FAQ

Is “this machine does not have the resources to deploy your pod” a RunPod outage?

Does pushing a commit fix the error on a RunPod Hub template?

How do I re-trigger a RunPod Serverless build without changing code?

Can I create a GitHub Release without a new commit?

Why does the same deploy fail once and work the next time?

How do I stop hitting the capacity error repeatedly?

Where to next

What does FlashBoot actually do?

Why do “cold” starts sometimes take 7 seconds and sometimes 110?

How can you tell if a request hit a snapshot restore?

What does the FlashBoot snapshot preserve?

What broke before this worked? The asyncio gotcha

When does the warmup pay off and when doesn’t it?

What this means if you’re shipping vLLM on RunPod

FAQ

Does FlashBoot snapshot the vLLM engine subprocess?

Why does my cold start take 60-120 seconds even with FlashBoot enabled?

What’s the difference between FlashBoot and a network volume?

Does FlashBoot work for non-GPU workloads?

How do I know if my worker is hitting a snapshot restore vs a fresh boot?

Is FlashBoot the same as RunPod’s “Active Workers” tier?

Will FlashBoot survive a Docker image rebuild?

What’s next

Why use Axiom for serverless worker observability?

How do I create Axiom datasets for OpenTelemetry?

How do I generate an Axiom API token for OTLP ingest?

What environment variables ship RunPod worker telemetry to Axiom?

How do I verify OpenTelemetry data is reaching Axiom?

Which APL queries help debug a mineru-runpod worker?

Does enabling OpenTelemetry slow down cold starts?

Where does this setup fall down?

FAQ

Does Axiom support OpenTelemetry natively?

Why aren’t my metrics showing up in Axiom?

What’s the cheapest way to observe a RunPod serverless worker?

Can I send traces and logs to the same Axiom dataset?

How much does OTLP/HTTP export add to cold start time?

Why isn’t api.axiom.co the OTLP endpoint?

Does enabling OpenTelemetry require an OpenTelemetry Collector?

Same template, different backend

Why does my RunPod worker return NoneType after a successful parse?

What is RunPod’s /runsync response payload limit?

Why isn’t `api.axiom.co` the OTLP endpoint?

Does `return: "tarball_b64"` get around the 20 MB cap?

What does the parse flow look like end-to-end with `return: "s3"`?

How do I get the R2 access key for `BUCKET_ACCESS_KEY_ID` and `BUCKET_SECRET_ACCESS_KEY`?

Is the 20 MB cap on `/run` too, or only `/runsync`?

Does using `return: "s3"` add to cold-start time?

What’s the difference between `vlm-auto-engine`, `pipeline`, and `hybrid-auto-engine`?