Congratulations, you've built an app on Databricks Apps, and it's gone from a couple of users to hundreds or thousands across your organization. Your users want it to respond fast and stay up. And your boss may not love having to pay for XLarge compute across a stack of instances.
How do you make an app that runs fast, stays up and handles growing demand? The intuitive answer is more compute, but that only helps once you know where requests spend their time and whether the app is built to scale: its framework, async code and ability to run on several instances, so the platform can easily scale it up or out.
In this guide, we walk through how to scale Databricks Apps across four stages: design; build; test and deploy; and operate (Figure 1).
The focus of this guide is primarily application performance and resiliency at scale; it does not cover deployment automation or performance optimization and scaling of other Databricks services that may interact with the app.
Figure 1: Four stages of scaling a Databricks App.
Good scaling decisions start before any code is written: agree what users need from the app, then decide where the work will run and where the data will live.
Start with targets: peak users and latency
Before choosing an architecture, it helps to pin down how much load the app has to handle and what a good experience looks like. We suggest agreeing two main targets:
- Performance: how quickly must key actions respond? We recommend end-to-end latency targets based on a percentile, such as p95, which captures the slower experiences that an average will not show. Include UI rendering, and treat cold starts separately. For streaming AI responses, time to first token matters too.
- Scalability: how many users are active at peak, how often do they use it and how much growth should the app support? Busy periods such as month end belong in this estimate because that's the load the app has to handle within its latency and error-rate targets.
Alongside these, decide which work must survive a restart and check that critical dependencies can support that. This shapes how you split state and how many instances you run.
Next, sketch a few user journeys, such as opening a dashboard or submitting a question. For each action, note the services it calls, roughly how many calls it makes and which depend on each other, how fast it should finish and how stale cached data can be. These assumptions shape the architecture and the first test targets. Once a prototype runs, you can measure the real calls and revise them.
Suppose a retailer gives its 2,000 store managers an app to check stock, backed by Lakebase. During the busiest hour of Black Friday, 200 of them are using it at once. If each makes two stock checks per minute on average, that is 400 checks per minute. If each check executes three SQL statements against Lakebase, expect 1,200 statements per minute, or 20 per second on average during that hour.
This estimates query rate, not concurrency. Test bursts and growth, validate against live usage, and budget connections separately (see "Budget the connections to Lakebase").
Design: plan the architecture
Keep app compute for serving the app
App compute is there to serve the UI and coordinate calls, not to do the heavy lifting. We recommend sending each kind of work to the service built for it (Figure 2). For long-running calls, run the work in the background, for example as a job, and let the UI poll for status or receive streamed updates, rather than holding one request open until it finishes.
If the app needs to run code it doesn't control, such as code written by users or generated by an agent, don't run it in the app. Run it in Databricks Sandbox, an isolated environment that can start from a custom image, so its CPU and memory use stays off the instances serving users.
Figure 2: Where each kind of work runs and where the state lives.
Plan for scale and choose a framework
When more people use your app, you can add capacity in two ways: scale up (vertical scaling) by moving to a larger compute size, which gives each instance more CPU and memory, or scale out (horizontal scaling) by adding instances to share the request load. With heavy work running outside the app, what usually limits it is how many requests it can handle at once.
For example, in the Black Friday case, each stock check is light, but checks from 200 store managers can arrive together in bursts. More instances spread those requests, and a larger instance helps if the app can use the extra cores, typically by running more workers. An app that keeps a large reference dataset or model in memory hits a different limit. A larger instance gives it the memory it needs; adding instances doesn't help, because each instance loads its own copy. You can start on Medium and let the load test later in this guide show which you need.
The framework is harder to change later, so it's worth choosing now. Databricks Apps runs apps built on Python or Node.js and you aren't limited to the frameworks below. For most new apps, we recommend AppKit, the TypeScript SDK that databricks apps init and the Databricks agent skills use by default. Caching, telemetry, retry logic and error handling come built in, giving you a starting point for several practices in this guide. AppKit runs one Node.js process per instance by default, so the simplest way to scale is by adding instances.
If you already have an app, or your back end needs Python:
- JavaScript front end with an API back end (FastAPI, Flask or Express): keep UI state in the browser and make server-side state recoverable across workers and instances. Python back ends can run several workers per instance, for example with Uvicorn or Gunicorn. Node.js back ends run one process per instance by default, so scale them by adding instances.
- Streamlit or Gradio frameworks: good for data and AI tools built quickly in Python. Streamlit runs as a single process, and both depend on instance-local session handling, so test session continuity during restarts and scaling. For apps that must scale across instances with seamless recovery, prefer one of the approaches above as these frameworks are not recommended.
Whichever you choose, if it keeps session state in server memory, it's worth testing what happens when a session moves to another instance before you scale out.
Plan for application state that survives restarts and rescaling
Most Databricks Apps keep some state in memory, and that's fine as long as you know which parts you can afford to lose. A useful first step is to list what each session holds and what must survive a restart, then split it into soft and hard state:
| What it is |
Soft state |
Hard state |
| What it is |
Temporary working data, kept for speed or context |
The source of truth, kept durably |
| Examples |
In-memory context, caches, work in progress |
Business records, checkpoints, saved preferences |
| Where it lives |
Usually instance memory or local disk, tied to one instance |
Lakebase or Unity Catalog tables, available to every instance |
| If it's lost |
Rebuilt from hard state when possible |
Must survive restarts and failures, within your recovery needs |
How that split works depends on the kind of app:
- AI agent: keep conversation context, tool outputs and plans in soft state while the loop runs, and save a checkpoint after each meaningful step.
- Workflow app: stream or poll in-progress status to the UI and save each completed step.
- Dashboard: keep filter values, reference lists and small datasets in memory, and save user preferences and saved filters as hard state.
Figure 3: When a session moves, another instance can rebuild it from hard state.
Databricks Apps keeps each user on the same instance on a best-effort basis (session affinity) and rebalances sessions when needed, so any per-user cache your app keeps locally usually stays available. Restarts, failures and scaling changes can still move a session, so another instance must be able to rebuild it from hard state (Figure 3).
Build: write code that scales and survives restarts
With targets and an architecture in place, the next step is code that holds up under load, including when a session moves or an instance stops.
Instrument the app before you need it
We recommend turning on telemetry from the start, so the data is there before you need it. App telemetry writes logs, traces and metrics to Unity Catalog tables through OpenTelemetry. Once it's enabled for an app, logs arrive automatically, but traces and metrics need instrumentation.
You should create custom spans around SQL, database, model and API calls to show where the time goes. Record caught exceptions on the span as well, so they show up in your traces. Trace and request IDs then let you follow one user's requests across instances. For example, a span around a SQL call:
Use async code and cut wasted calls
Most apps spend their time waiting on another service: a SQL query, a model call, an external API. With blocking calls, each wait holds an execution slot for the whole call. A handful of slow calls can exhaust that capacity. That's why async handlers matter for I/O-heavy work. When handlers await model and database calls, one worker can keep serving other requests while it waits, as long as those calls are non-blocking. Your traces and system metrics will show when this applies: if most of a request's time is spent in SQL, model or API spans, and latency rises with concurrent users while CPU stays low, requests are likely queuing behind blocking calls.
Even with async, one request can still be slow if it runs its queries one after another. Start independent queries together (for example, with asyncio.gather or Promise.all), so the request takes close to as long as its slowest query. It's also worth capping how many run in parallel, so one request can't flood the warehouse or database. Often it is better to combine queries than to run N+1 queries, where the app fetches a list and then runs one more query per item: a join or an IN clause does the same work in one or two.
Cache data per user by default
A cache speeds up an app, but it can also leak data: an app-level cache doesn't enforce Unity Catalog permissions, so a shared cache can show one user another user's data. Scope cache keys by the user's verified identity and the query parameters, and by session where needed; never key on an identity the client sends without verifying it. Share cached data only when it's identical for everyone. For shared entries many users read at once, let one request refresh an expired entry while the others wait, so they don't all hit the warehouse together. AppKit's cache, for example, deduplicates concurrent cache misses within each instance.
On app container restarts or session moves
Putting the soft-state and hard-state design into practice comes down to a few recommendations:
- Persist anything that must survive a restart or deployment. Memory alone is fine for disposable caches, but not for work users cannot afford to lose.
- Checkpoint wherever you wouldn't want to repeat the work, and make steps idempotent so a retry is safe.
- Make sessions portable, so any instance can rebuild them from hard state (Figure 3). Reload only what you need, and cache it to avoid repeated reads.
- Browser apps get session affinity automatically. For stateful API clients, send a stable
X-Routing-Key header and key it on whatever owns the soft state. Pick a key specific enough to spread load, such as a conversation or user ID rather than a tenant ID. The client sets the header, so use it only for routing, never to decide who the user is.
- Expect database connections to drop too. They drop when the database restarts or fails over. Reconnect with bounded backoff and jitter, so instances don't all reconnect at once and cause a connection storm. Before retrying a write, check the database to confirm it didn't already go through.
- Keep the startup light. Every restart, deployment and new instance pays for loading models or datasets, warming caches and opening pools before it can serve traffic. Avoid large installs and blocking external calls at startup, and load heavy resources only when needed.
Shut down gracefully
When you redeploy an app, Databricks sends SIGTERM, then SIGKILL if the app hasn't exited within 15 seconds. That window is for final checkpoints. Important work should already be saved during normal operation.
As an example, with FastAPI, Uvicorn finishes open requests before running the lifespan cleanup. save_in_progress_sessions() stands in for your own function that saves unfinished work:
This assumes an async pool (for example, psycopg's AsyncConnectionPool). The Apps-to-Lakebase tutorial uses a synchronous pool, which you close without await.
Set Uvicorn's --timeout-graceful-shutdown below 15 seconds (for example, 10) so a long-running request can't eat the checkpoint time, and test shutdown with your own workload. AppKit's Lakebase plugin closes its pools on shutdown; any custom checkpointing still has to fit in the same window.
Test and deploy: load test and size for the peak
Figure 4: System metrics during a stepped load test: latency jumps at the last step while CPU and memory stay below their limits.
Before launch, a load test will help you turn your estimates into decisions closer to reality: the instance size, the workers per instance and the instance count.
How to load test Databricks Apps
We recommend load testing before production, using real user journeys at your expected peak. A tool such as Locust reports requests per second, latency percentiles and the failure rate at each step, and the test passes or fails on the latency and error-rate targets you agreed for the key actions. While the test runs, the app's built-in System metrics show requests per minute, latency, and CPU and memory against their limits, so you can see which limit the app reaches first (Figure 4).
Ramp up concurrent users in steps until throughput stops increasing or latency and error rates break your targets. It's worth trying several instance sizes and worker counts, because a bigger setup isn't always faster. We also suggest testing recovery in a test environment: stop and start the app, not just redeploy or change the instance count during a run, then check whether users can continue, how long recovery takes and whether any work was lost. For agent apps, the load-testing guide walks through testing once with a mocked LLM to measure the app's own throughput, then with the real model for end-to-end latency.
The load generator itself can be the bottleneck: it can run out of CPU or connections first, and authentication errors or rate-limit responses can look like slowdowns. What counts is sustained successful requests within your latency target, not a brief spike.
The load test should also show whether the services behind the app keep up. If the warehouse starts queuing or a model endpoint throttles, scale that service, not the app.
Pick the instance size
Databricks Apps offers three sizes: Medium (the default, up to 2 vCPUs and 6 GB), Large (up to 4 vCPUs, 12 GB) and XLarge (up to 12 vCPUs, 48 GB). The right one depends on which limit your app hits first, so start on Medium and let the load test show you:
| What the load test shows |
Likely cause |
What to do |
| Slow requests with low app CPU |
The app is waiting on something behind it, or blocking calls are holding workers |
Keep the size. Fix the query, warehouse, connection pool or blocking calls first, and check per-process CPU too: one maxed-out core can look like low CPU overall |
| CPU near the limit, server runs several workers (for example, FastAPI with Uvicorn) |
The app can use more cores |
Move to Large |
| CPU near the limit, single-process app (Streamlit, or AppKit by default) |
Profiling shows the work bound to one core |
Add instances. A larger size only helps if the app can use extra cores, for example, through worker threads or native libraries |
| Memory near the limit |
Large datasets or models held in memory |
Move up for memory: Large doubles Medium, XLarge gives 48 GB in one instance |
| Recovery test: the app goes down when its only instance restarts |
No spare instance to take over |
Add instances, not a larger size |
How many workers should each instance run?
Workers are server processes within each instance. For Python servers that support this model, you set the count at startup (for example, uvicorn --workers 2), and it applies per instance.
A single worker can leave the CPU idle, but extra workers also compete for CPU and memory. In the benchmark from our load-testing guide, Medium did better with two workers than with four, and Large delivered roughly 2.2x Medium's throughput. Treat the documented starting points, two workers on Medium or eight on Large, as a first guess: the benchmark mocked the LLM and was CPU-bound, and a real app that mostly waits on model calls may get more out of extra workers. Choose the count from your own load test, with the real model.
How many instances should the app run?
With horizontal scaling, you set the instance count yourself. The default limit is five instances per app, and your Databricks representative can raise it if load testing shows you need more. Size the count for expected peaks and growth. The app keeps serving while the count changes, and compute cost scales linearly with the instance count. If you're converting an existing app, try it on a duplicate first.
For higher availability, we recommend at least two instances, with enough capacity to handle peak load after losing one: if the peak needs two instances, run three. Extra instances protect against an instance failing, not against problems in the services behind the app.
Budget the connections to Lakebase
For a Databricks App that connects to Lakebase with OAuth, we recommend starting with an app-side connection pool on the database's standard (non-PgBouncer) connection string. It reuses connections and fetches fresh OAuth credentials as it opens new ones, as the Apps-to-Lakebase tutorial shows for Python. Set the pool timeout to leave enough time for the query itself before the request deadline.
Those connections add up, so budget them before adding instances or workers. With one pool per worker, the upper bound is instances × workers per instance × maximum pool size: five instances with eight workers and a pool maximum of ten can open up to 400 connections. Add other clients and any connections the app opens on users' behalf, then compare the total with the database's documented limit, leaving headroom for deployments, when old and new versions run side by side.
On-behalf-of user (OBO) queries need identity capacity. This means each user who queries Lakebase as themselves needs their own Postgres role, and Lakebase allows 500 roles per branch. For larger user bases, decide early which queries really need per-user access. Adding instances doesn't change this limit. Where group-level permissions are enough, a Postgres role for a Databricks group lets its members connect with their own OAuth tokens, so one role covers the whole group.
Roll out without downtime and roll back safely
For horizontally scaled apps, Databricks updates one instance first and the rest after it succeeds, so the app keeps serving. It's still worth checking after each deployment that the app is healthy and sessions recover. Keeping the code in Git means you can roll back by redeploying a known good tag or commit. During a rollout, old and new versions run side by side, so keep database and checkpoint formats readable by both, and don't drop fields the previous version needs until you're sure you won't roll back.
Operate: monitor, diagnose and fix from data
After launch, the work becomes a loop: find the slowest step, fix it and measure again. That's why we keep telemetry on in production. On horizontally scaled apps, the Live view in the Logs tab shows one instance at a time; with telemetry enabled, App Telemetry Logs shows all of them.
When something is slow, trace the request end to end (Figure 5).
Figure 5: Where a request spends its time and the main bottlenecks.
Start from the symptom
These are the symptoms we see most often, and where they usually point:
| Symptom |
Where to look |
Likely cause |
What to do |
| Slow first request after an idle period or a deployment |
Insights tab System metrics and traces in app telemetry |
Cold start: app startup, or a service waking from idle |
Check app startup time and whether Lakebase or the SQL warehouse stops when idle, and keep startup light |
| Slow requests with low app CPU |
Insights tab System metrics and traces in app telemetry |
The app is waiting on something behind it, or blocking calls are holding workers |
|
| Check traces for query, warehouse or connection-pool delays before adding app compute |
|
|
|
For single-process apps, consider that low overall CPU can still mean one core is maxed out
|
| App CPU or memory near the limit | Insights tab System metrics | The app itself is the bottleneck, or it holds too much in memory | Profile CPU-heavy work, move large state out of memory, then see the guidance in "Pick the instance size" |
| Timeouts waiting for a database connection | App logs and Lakebase metrics | Pool contention, long transactions or database connection limits | Inspect connection usage and transaction duration, then adjust pools or database capacity within the connection budget |
| Users lose work or see errors during deployments or rescaling | Insights tab System metrics | Soft state that can't be rebuilt, or a shutdown that doesn't finish within 15 seconds | Checkpoint important work, make sessions rebuildable on any instance, and test shutdown with your real requests and streams |
Set up health alerts
We recommend planning alerting as part of setup: decide who is notified when the app goes down, and feed app_status.state and compute_status.state from the Apps API into your monitoring tool. App status alone doesn't prove users can work, so also check a representative user journey, with its latency and error rates. Allow for brief state changes during deployments when you set thresholds.
Summary
Scaling Databricks Apps means sizing the whole request path, not just the app server: set latency and peak-load targets, send heavy work to the service built for it, keep state recoverable, and let load tests and telemetry decide where capacity goes. Add compute when the measurements show you need it, not as a first reaction.
To go further, read how to build production-ready data and AI apps with Databricks Apps and Lakebase. For more details, see Best practices for Databricks Apps and Logging and Monitoring for Databricks Apps in the documentation. For architecture patterns by app type, such as internal tools, AI copilots and agentic workflows, watch Building Better Apps and Stateful Apps for Thousands of Users from Data + AI Summit 2026.
FAQs
How do I scale a Databricks App?
Scale it in four stages: design, build, test and deploy, and operate. Set latency and peak-load targets first, keep app compute for serving the UI, and write async code with recoverable state. Then let load tests and telemetry decide whether to scale up to a larger instance or out to more instances.
Should I scale up or scale out my Databricks App?
It depends on which limit the app hits first. Scale out by adding instances when requests arrive in bursts or a single-process app is bound to one core. Scale up to Large or XLarge when the app needs more memory, or when it runs several workers that can use extra cores.
What instance sizes does Databricks Apps offer?
Databricks Apps offers Medium (the default, up to 2 vCPUs and 6 GB), Large (up to 4 vCPUs and 12 GB) and XLarge (up to 12 vCPUs and 48 GB). Start on Medium and let a load test show which size your app needs.
How do I keep user sessions working when an app restarts or scales?
Split session state into soft state, which can be rebuilt, and hard state, which is stored durably in Lakebase or Unity Catalog tables. Databricks Apps keeps each user on the same instance on a best-effort basis, but restarts and scaling can still move a session, so any instance must be able to rebuild it from hard state.
How many Lakebase connections can a scaled app open?
With one pool per worker, the upper bound is instances × workers per instance × maximum pool size. For example, five instances with eight workers and a pool maximum of ten can open up to 400 connections. Compare that total with the database's documented limit, and leave headroom for deployments.
Ready to take your app from a few users to thousands? See Best practices for Databricks Apps and horizontal scaling for Databricks Apps in the documentation.
"Explore Databricks Apps" →
Instagram Engineering
Shopify Engineering