Dataverse API Limits Taking Down Your Portal: Protecting Interactive Users from Batch Jobs

Why does a nightly Dataverse job take the portal down during the day?

Because the job and the portal are competing for something they should not share. Dataverse applies service protection limits to each authenticated identity: the number of requests, the combined execution time and the number of concurrent requests that one user or application user sends, counted on each web server over a sliding window of a few minutes. Microsoft states that other users are not affected when one identity exceeds its limits. The trouble starts when the job is not really another user.

Microsoft documents that portal applications typically send requests from anonymous visitors through a service principal account, so a Power Pages site reaches Dataverse under an application identity of its own and every visitor shares that identity's limits. If a sync, an integration or a support script was configured with the same app registration or the same integration account, it spends the portal's budget, and the portal starts returning errors while the job looks healthy. The same happens to a custom website or middleware that reads Dataverse through one integration identity. A "nightly" job that overruns into the morning, or runs at night for one country and at midday for another, turns this into a daytime outage.

If the identities are already separate and the portal still slows while the job runs, the cause is shared resources rather than the per identity limits: synchronous plugins and flows fired by the job, locks on the same rows, and asynchronous work queued behind the batch. That is a different fix, covered in the plugins section below.

Which Dataverse limits apply to batch jobs and portals?

Two published mechanisms apply, plus the shared resources that neither one isolates, and each fails differently. Microsoft revises the values and the enforcement details, so confirm the current figures in the Dataverse service protection API limits and the Power Platform requests limits and allocations documentation before you plan capacity around them. The arithmetic for a one-off load is worked through in our Dataverse bulk import strategy and is not repeated here.

What should you do first when a batch job is taking the portal down?

Restore users first and diagnose second, but keep the evidence while you do it. The job can resume later from where it stopped if it was built to; the portal outage is costing you now.

• Pause the job rather than killing it mid-write where you can: flip its kill switch or configuration flag, turn off its schedule, cancel the running flow runs, or scale the worker count to zero. Note the time and the last record processed.
• Capture the failing responses from the portal side before they age out: the HTTP status, the Retry-After header value and the error code in the body. Power Pages diagnostic logging or Application Insights, if it is already enabled, is where these show up.
• Identify the calling identity of both the job and the portal: the application ID of the Power Pages site and the application user or account the job authenticates with. If they match, you have the cause.
• Wait out the Retry-After interval and confirm the portal recovers as a real signed-in and anonymous visitor, not as an administrator.
• Do not restart the job at full speed. Restart it later under its own identity, at lower concurrency, outside measured portal peaks, and watch the portal while it runs.
• Write down what happened with times. It is the input to the durable fix and to any customer communication.

How do you confirm throttling is the cause and not something else?

Match the symptom to the evidence. The service protection error codes below are as Microsoft documents them at the time of writing; the Web API returns the hexadecimal value. Reading individual payloads from flows, including connector limits that return 429 without a Dataverse code, is covered in our Power Automate support guide.

Why should every workload run under its own service identity?

Because the identity is the unit Dataverse measures, and it is also the unit you can see in logs, security roles and usage reports. One application user per workload means the nightly sync, the portal, the ERP integration and the reporting extract each get their own service protection window, their own least privilege security role and their own line in the usage data. When one misbehaves, the others keep working and you can tell which one it was.

Microsoft states that application users get the same service protection limits as other users, so this is isolation, not a way to raise capacity. Splitting one job across several identities to outrun the limits still lands on the same environment, and daily request entitlements for application users come from one tenant pool, so the heavy job still has to be shaped properly. Keep the portal's identity reserved for the portal, and never reuse its app registration or secret for anything else. Where external systems need Dataverse data, putting them behind a controlled interface rather than handing out integration credentials is the subject of our guide to exposing Dynamics 365 data to external consumers.

• One Microsoft Entra app registration and Dataverse application user per workload, named for the job, not for a person.
• A security role written for that workload, not System Administrator.
• Secrets or certificates in a vault, with an expiry date someone owns.
• Power Automate flows for different workloads on different connections, because flows sharing one connection share one identity.

Why is a global rate limiter better than per-thread retries?

A job with twenty threads that each catch 429 and retry on their own keeps nineteen threads sending requests while one waits. The platform sees continued demand from a throttled identity and, as Microsoft documents, extends the Retry-After duration, so the job spends longer throttled and the environment stays loaded. Per-thread retry is correct for a single caller and wrong for a pool.

Put one limiter in front of all workers of a job, whether they are threads in one process, instances of an Azure Function or worker flows. A token bucket works well: workers take a token before each request, the bucket refills at a rate you set, and a 429 received by any worker empties the bucket and pauses every worker until the Retry-After interval has passed. Start below the limits and increase gradually until the job meets its window without throttling, which is the approach Microsoft recommends for sustained throughput. The Dataverse ServiceClient in the .NET SDK retries on Retry-After by itself and exposes a recommended degree of parallelism, so use it and size the pool from it rather than guessing. For flows, the equivalent design is covered in our guide to Power Automate 429 throttling at scale.

• One limiter per job, shared by every worker, stored where all workers can read it if they run in separate processes.
• On 429, pause all workers for Retry-After, then resume below the previous rate.
• Cap concurrency deliberately; concurrent request errors are returned immediately.
• Log every 429 with its code and Retry-After value, and alert on the count rather than on job failure.

How do change tracking and scheduling keep batch work away from interactive users?

The cheapest request is the one the job no longer sends. A nightly sync that reads and rewrites every row costs the same whether ten records changed or ten thousand. Enable change tracking on the tables the job reads and request only the rows created, updated or deleted since the stored delta token, or use a modified-on watermark with an overlap and idempotent upserts where change tracking is not available. Skip updates where the incoming values match what is stored, because an update that changes nothing still fires plugins, flows and auditing.

Then schedule from evidence, not from the word "nightly". Pull the portal's request volume by hour from Application Insights or your web analytics, including every country it serves, and set the batch window where interactive traffic is genuinely lowest. Give the job a hard stop time and a resume point, so an overrun pauses instead of running into the morning peak. For heavy writes, use the bulk messages CreateMultiple, UpdateMultiple and UpsertMultiple where the table and your plugins support them, with modest batch sizes, because Microsoft notes that larger batches move you from request limits to execution time limits. The load mechanics are set out in the bulk import guide.

Do plugins triggered by a bulk update count against service protection limits?

Not as separate requests, but their time does. Microsoft documents that data operations performed by plugins and custom workflow activities run in the sandbox and do not use the public API endpoints, so they do not count towards the service protection request limits. The extra computation they add is charged to the execution time of the request that triggered them. A bulk update that fires three synchronous plugins per row therefore reaches the execution time limit far sooner than the same update on a table with no logic, and it holds database transactions open longer while it does, which is what users on the same rows feel.

Plugin operations do count towards Power Platform request entitlements when they create, update, assign or share records, so a chatty plugin multiplies the daily consumption of the job that triggers it. Sandbox plugins also have an execution timeout, documented as two minutes at the time of writing, and a plugin that loops over related records during a bulk run is the usual one to hit it.

• Filter plugin steps on the attributes that matter, so updates to unrelated columns do not fire them.
• Move work that does not need to block the transaction to asynchronous plugin steps, or out of the pipeline into a queue processed by Azure Functions.
• Avoid plugins that query or update many related rows per triggering row; aggregate once per batch instead.
• For a sanctioned migration or bulk correction, use the documented options to bypass custom business logic, which require a specific privilege on the calling identity, and record what was bypassed.
• If you move writes to UpdateMultiple, check how existing plugins are registered, because Microsoft documents specific behaviour for plugins on single and multiple operation messages.

How do you monitor Dataverse API consumption before users hit the limits?

Throttling shows up as slowness long before it shows up as an outage, so measure consumption per identity and alert on trends. None of these sources is complete on its own.

• Web API response headers. Microsoft documents x-ms-ratelimit-burst-remaining-xrm-requests and x-ms-ratelimit-time-remaining-xrm-requests for tracking remaining request and execution time budget. Microsoft also says they are intended for debugging and should not control how many requests you send, because they are per server connection. Log them from the job for diagnosis.
• Your own job telemetry: requests sent, 429 count by error code, Retry-After values and duration against the window. This is the most reliable signal you control.
• Power Platform admin center: the Dataverse analytics and API call statistics for the environment, and the downloadable Power Platform requests reports under licensing and capacity. Microsoft marks parts of this reporting as preview with limited coverage, so check what the current report includes before relying on it.
• Application Insights: export Dataverse telemetry to Application Insights where your licensing supports it, and enable Application Insights on the Power Pages site, so portal failures and job activity can be lined up on one timeline.
• Alerts on the rate of 429s and on portal response time, owned by someone who can pause the job.

Is Dataverse still the right platform when batch and portal traffic collide?

Usually yes. A collision between a batch job and a portal is almost always a design problem, a shared identity, a full table rewrite or heavy synchronous logic, and the fixes on this page resolve it without changing platform. It becomes a platform question when the volume the business needs to push through an external facing application is the product itself, and the licensing and limits shape keeps fighting it.

We recommend the right solution - whether that's Microsoft Dynamics 365, Power Platform, or a custom-built CRM. Some businesses need the Microsoft ecosystem. Others need full control without licensing. We deliver both. Where that is the situation, a custom-built CRM and portal on React, Node.js, PostgreSQL or .NET, sized and hosted for your own traffic, is the alternative. The trade offs between Power Pages and a custom front end are set out in our customer portal comparison.

How does Solzet help protect interactive users from batch jobs?

We start with the stabilisation alongside your team: pausing the job safely, capturing the evidence and confirming the portal has recovered. Then we map every identity calling the environment, what each workload sends and when, and which plugins and flows its writes trigger, and we deliver the durable design: separate application users, a shared rate limiter, delta processing, a measured batch window, lighter plugin logic and monitoring with alerts.

Senior consultants and full-stack developers deliver this remotely from Yerevan, Armenia, with 8+ years of Dynamics 365 Customer Engagement and Power Platform work, directly or white-label for Microsoft partners. The build side sits in our Power Platform development service, and where you want an independent review of identities, plugins, integrations and capacity across the whole environment, that is our Dynamics 365 health check and technical audit.