Modal Serverless GPU Guide (2026): Python-First Deployment

Modal runs Python functions on GPUs without a Dockerfile, a cluster, or an instance to keep alive. You annotate a function, Modal builds the image, and a container starts when the function is called and stops when it goes idle. Billing is per second of container uptime.

How it works

The current API is built around modal.App. The older Stub class it replaced still appears in a lot of tutorials, and modal.Session was never part of the public API at all.

import modal

app = modal.App("example-inference")
image = modal.Image.debian_slim().uv_pip_install("transformers[torch]")

@app.function(gpu="h100", image=image)
def generate(prompt: str) -> str:
    from transformers import pipeline
    pipe = pipeline("text-generation", model="gpt2", device="cuda")
    return pipe(prompt)[0]["generated_text"]

Run it with modal run file.py. The gpu= parameter takes the accelerator name ("h100", "a100-80gb", "l40s", "t4"), and the image is built once and cached, not rebuilt per call.

The cold start is billable

This is the part that surprises people. Modal bills container uptime, not request duration, and the model load happens inside that window. A 20-second load on an H100 costs about $0.022 every time a container starts from zero. Under bursty traffic that repeatedly scales to zero, cold starts can become a real line item.

Two levers: keep warm containers pinned (they bill at the same per-second rate, so this is a cost decision, not a discount), and load weights into the image at build time instead of at call time so the container starts with them already on disk.

What it costs against renting a pod

Modal charges roughly double a rented instance for the same silicon: $3.95/h for an H100 against $1.99/h on RunPod Community Cloud. CPU and memory bill on top of the GPU rate. Full per-GPU figures are in the Modal pricing guide.

The maths only favours serverless at low duty cycle. A model serving 90 minutes of real traffic a day costs about $5.93/day on Modal; the same H100 rented around the clock costs $47.76/day. Past roughly 12 hours of daily GPU-busy time, the rented pod wins. Scale-to-zero is worth paying for when your GPU is idle most of the day and worth nothing when it isn’t.

Serverless GPU alternatives

These providers offer genuine scale-to-zero rather than instance rental:

Entry rates are verified provider floors as of July 2026 and refer to different GPUs, so treat them as a price band. AWS is absent deliberately: Lambda has no GPU support and SageMaker Serverless Inference is CPU-only. See the serverless GPU comparison for cold-start and concurrency detail.

FAQ

Does Modal publish its prices?

Yes, in full. modal.com/pricing lists a per-second and per-hour rate for every GPU it offers, from B300 at $7.10/h down to T4 at $0.59/h, plus separate CPU and memory rates. There is no quote-only tier for standard GPU access.

Do I need a Dockerfile?

No. You describe the image in Python with modal.Image, chaining calls like .debian_slim(), .uv_pip_install(...) and .apt_install(...). Modal builds and caches it. You can point at an existing registry image if you already have one, but the common path skips Docker entirely, and that is the main reason teams pick Modal over RunPod Serverless.

How much control do I have over the GPU?

You choose the accelerator and the count, and you get a normal CUDA environment inside the container. You do not choose the host, the region at rack level, or the neighbours. For workloads needing pinned hardware, a specific data centre, or an interconnect topology for multi-node training, rent instances instead — Modal is built for inference and short jobs rather than long distributed training runs.

Is there a free tier?

The Starter plan has no base fee and refreshes $30 of credit monthly, roughly 7.5 hours of H100 time or 50 hours on a T4. Credit does not carry over between months.