BigQuery: Exceeded rate limits — too many concurrent DML statements / table update operations

Too many UPDATE/DELETE/MERGE statements (or table metadata updates) hit the same table at once.

Exact message you may see Exceeded rate limits: too many table update operations for this tableExceeded rate limits: too many concurrent DML statementsBigQuery rateLimitExceeded DMLJob exceeded rate limits: Your table exceeded quota for imports or query appends per tableexceeded quota for table update operations
Seen on: BigQuery Google Cloud SQL

Meaning

BigQuery queues DML per table (up to 20 concurrent, with limits per day). Batch changes into fewer larger statements, use MERGE, load via temporary tables, prefer append-only designs with streaming/partition replace, and retry with backoff for rateLimitExceeded.

Common causes

  • Many small DML statements
  • Concurrent writers to one table
  • Row-by-row updates
  • Table metadata updates too frequent
  • Quota for table operations

⚡ Quick fix

  1. Batch into a single MERGE/INSERT SELECT
  2. Serialise writers or use a queue
  3. Retry with exponential backoff

Detailed fix by platform

SQL

  1. MERGE proj.ds.target T USING proj.ds.staging S ON T.id = S.id WHEN MATCHED THEN UPDATE SET v = S.v WHEN NOT MATCHED THEN INSERT (id, v) VALUES (S.id, S.v);

How to diagnose

  1. DML — Many small?
  2. Writers — Concurrent?
  3. Backoff — Used?

🔧 Still not fixed?

Many errors look alike. If the steps above didn’t solve it, one of these is probably what you’re facing:

Where this error sits

Origin: Platform & cloud. Errors that are often confused with or caused by this one (click to open):

429 — Cloud Storage 429: rate of change requests to a single object exceeded (1 write per second) (Google Cloud / Firebase)429429 RESOURCE_EXHAUSTED: Quota exceeded for quota metric — Google API: 429 RESOURCE_EXHAUSTED — Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute per user' (Google Cloud / Firebase)429 RESOURCE_EXHAUSTE…BigQuery: Access Denied: Project: User does not have bigquery.jobs.create permission — BigQuery: Access Denied: Project my-project: User does not have bigquery.jobs.create permission in project my-project (Google Cloud / Firebase)BigQuery: Access Deni…BigQuery: Not found: Dataset was not found in location — BigQuery: Not found: Dataset my-project:analytics was not found in location EU (Google Cloud / Firebase)BigQuery: Not found: …BigQuery: Quota exceeded: Your project exceeded quota for free query bytes scanned — BigQuery: Quota exceeded: Your project exceeded quota for free query bytes scanned. Consider enabling billing (Google Cloud / Firebase)BigQuery: Quota excee…BigQuery: Unrecognized name — BigQuery: Syntax error / Unrecognized name: customer_id at [3:8] (Did you mean customerId?) (Google Cloud / Firebase)BigQuery: Unrecognize…Compute Engine: Quota exceeded (regional limit) — Compute Engine: Quota 'N2_CPUS' exceeded. Limit: 24.0 in region europe-west1 / QUOTA_EXCEEDED (Google Cloud / Firebase)Compute Engine: Quota…Dataproc — Dataproc cluster creation failed: quota, network or initialization action error (Google Cloud / Firebase)DataprocDML limits

Explore the map of this error Full error map →

🧠 Still stuck? Analyze your error

Paste the full message, response headers or stack trace — we'll detect the platform and point to the most likely cause.

💬 What worked for others

No tips yet — did something else fix this for you?

Share what worked for you

Tips are checked by an editor before they appear. Links are removed.