SiliconFlow API Rate Limits and 429 Retry Strategies

LLM API rate limits may go unnoticed during testing because traffic is small and predictable. Production traffic is different. Multiple users can send requests at the same time, long prompts can consume token capacity quickly, and uncontrolled retries can turn a brief traffic spike into repeated 429 errors. On SiliconFlow, stable API integration requires understanding RPM, TPM, model-specific limits, concurrency, and temporary service errors before they affect users.

Why Do LLM APIs Have Different Rate Limits?

Request count alone does not represent the processing requirements of an LLM workload.

A short classification request may contain 200 tokens and return one label. A repository analysis request may include thousands of input tokens and generate a detailed response. Both count as one request, but their token consumption, processing time, and infrastructure requirements differ significantly.

Multiple rate limits address different traffic patterns:

• Short, high-frequency requests are more likely to reach request limits.

• Long prompts and large context windows can reach token limits with relatively few requests.

• Long outputs increase token consumption and keep requests active for longer.

• Traffic bursts may exceed a limit even when average usage looks acceptable.

• Bulk workloads sent through real-time endpoints can consume online capacity even when they do not require an immediate response.

SiliconFlow uses several rate-limit metrics, including RPM, RPH, RPD, TPM, TPD, IPM, and IPD. Language-model applications mainly need to monitor RPM and TPM, although certain models or accounts may also have hourly or daily limits. A request can be limited when any applicable metric reaches its maximum.

RPM vs. TPM: Understanding SiliconFlow API Limits

RPM and TPM measure different dimensions of API usage.

What Does RPM Measure?

RPM limits how many requests an account can send to a model within one minute.

An application can reach its RPM limit even when every request contains only a small prompt. This often occurs with autocomplete tools, chat applications, agent loops, or workflows that divide one user action into several model calls.

For example, one agent task may require separate requests for planning, tool selection, result validation, and final response generation. The user submits one task, but the application consumes four or more requests.

What Does TPM Measure?

TPM limits how many tokens an application can process within one minute.

Long system prompts, conversation history, retrieved documents, source code, reasoning content, and large output limits can increase token usage. Applications should monitor observed prompt_tokens, completion_tokens, and total_tokens instead of estimating capacity from request count alone.

Consider two applications that each send ten requests per minute:

• Application A uses 300 total tokens per request.

• Application B uses 8,000 total tokens per request.

Both consume 10 RPM. Application A processes approximately 3,000 tokens, while Application B processes approximately 80,000. Application B may reach its TPM limit even though its request volume remains low.

How SiliconFlow Applies Rate Limits

SiliconFlow rate limits are defined at the user-account level rather than separately for each API key. Creating several API keys under one account does not create independent capacity.

Limits are also configured separately by model. Reaching the limit for one model does not normally prevent the account from calling another model with available capacity. Free models may have fixed limits, while paid-model limits can vary by account tier, model category, and model size. Check the current model details before configuring production traffic.

Why Does Your Application Receive 429 Errors?

An HTTP 429 response means that an applicable rate limit has been reached. It does not always mean that the application sent too many individual requests.

Too Many Requests

A burst of small calls may exceed RPM. Agent applications are particularly vulnerable because one user action can trigger several API requests.

Too Many Tokens

A few long requests may exceed TPM. Common causes include repeated system prompts, full conversation histories, large retrieval results, long code files, and excessive output limits.

Multiple Services Share the Same Limit

Several applications, backend services, workers, or API keys may use the same account and model. Each service may appear to have low traffic while their combined usage reaches the account-level limit.

Traffic Arrives in Bursts

An application may remain below its average hourly capacity but still send too many requests or tokens within a shorter period. Queue depth and per-minute usage are therefore more useful for diagnosing 429 errors than daily averages alone.

Automatic Retries Increase the Load

Immediate retries send more traffic into a system that is already limited. When many workers retry together, they can create a retry storm that extends the original problem.

Model-Specific Limits Apply

Some models may have additional hourly or daily restrictions. Do not assume that the limits shown for one model apply to every other model available through the API.

A Dedicated Instance Is Configured Incorrectly

Dedicated instances typically do not use the standard rate limits applied to general API access. A 429 response in this setup may indicate that the application is calling the wrong model ID or using an API key that does not match the dedicated instance.

The first diagnostic step is to record the HTTP status and complete response body. SiliconFlow 429 responses include a message that can help identify whether RPM, TPM, RPD, TPD, IPM, or IPD caused the failure.

How to Distinguish 429, 503, and 504 Errors

Not every temporary API failure is a rate-limit error. SiliconFlow’s error handling reference lists the status codes the platform returns.

Repeated 504 errors on long non-streaming requests may not be resolved by sending the same request again. For chat workloads, consider streaming the response, reducing the requested output, or reviewing the client timeout. SiliconFlow documentation also recommends streaming for applicable chat and text-to-speech requests when 503 or 504 errors occur.

How to Handle LLM API Rate Limits Correctly

A production application should retry only temporary failures and should never retry indefinitely.

Use Exponential Backoff

Exponential backoff increases the delay after each failed attempt. A retry sequence might wait approximately one second, two seconds, four seconds, and eight seconds.

Set both a maximum delay and a maximum number of attempts. Without these boundaries, an interactive request may continue long after it is useful to the user.

Add Jitter

Jitter adds a small random variation to the retry delay. This prevents multiple application instances from retrying at exactly the same time.

Without jitter, requests that fail together may also retry together, causing another traffic spike.

Use a Request Queue

A queue absorbs temporary bursts and releases requests at a controlled rate. Interactive requests can receive priority, while background jobs wait for available capacity.

A production rate limiter may use a token-bucket or leaky-bucket design to control how quickly new requests are released. This is different from limiting only the number of requests currently in progress.

Set a Maximum Retry Count

Three to five attempts may be reasonable for some interactive applications, but the correct number depends on the request timeout, user experience, and business importance.

After the final attempt, return a clear error, save the job for later processing, or route it to an approved fallback model.

Do Not Repeat External Side Effects

Agent workflows may call payment systems, databases, file operations, or other external tools. Retrying the model request should not automatically repeat an action that already succeeded.

Use idempotency keys, operation IDs, or another deduplication mechanism to protect downstream actions.

Use One Retry Layer

Some SDKs, including the OpenAI Python library used above, include automatic retries. Adding another retry loop at the application level can multiply the total number of attempts.

The following Python example disables SDK-level retries and implements one bounded retry policy with exponential backoff and jitter, and honors the Retry-After header when the server provides one:

import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from typing import Optional

from openai import APIConnectionError, APIStatusError, OpenAI

client = OpenAI(
    api_key=os.environ["SILICONFLOW_API_KEY"],
    base_url="https://api.siliconflow.com/v1",
    max_retries=0,
    timeout=60.0,
)


def parse_retry_after(error: APIStatusError) -> Optional[float]:
    """Return Retry-After in seconds when the response provides it."""
    value = error.response.headers.get("retry-after")
    if not value:
        return None
    try:
        return max(0.0, float(value))
    except ValueError:
        pass
    try:
        retry_time = parsedate_to_datetime(value)
        if retry_time.tzinfo is None:
            retry_time = retry_time.replace(tzinfo=timezone.utc)
        now = datetime.now(timezone.utc)
        return max(0.0, (retry_time - now).total_seconds())
    except (TypeError, ValueError, OverflowError):
        return None


def create_chat_completion(
    messages: list[dict[str, str]],
    max_attempts: int = 5,
):
    model = os.environ["SILICONFLOW_MODEL"]
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(
                model=model,
                messages=messages,
                max_tokens=800,
            )
        except APIStatusError as error:
            if error.status_code not in {429, 503, 504}:
                raise
            if attempt == max_attempts - 1:
                raise
            retry_after = parse_retry_after(error)
            backoff = min(30.0, 2.0 ** attempt)
            delay = retry_after if retry_after is not None else backoff
        except APIConnectionError:
            if attempt == max_attempts - 1:
                raise
            delay = min(30.0, 2.0 ** attempt)

        jitter = random.uniform(0.0, min(1.0, delay * 0.25))
        time.sleep(delay + jitter)

    raise RuntimeError("The request failed after all retry attempts.")

import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from typing import Optional

from openai import APIConnectionError, APIStatusError, OpenAI

client = OpenAI(
    api_key=os.environ["SILICONFLOW_API_KEY"],
    base_url="https://api.siliconflow.com/v1",
    max_retries=0,
    timeout=60.0,
)


def parse_retry_after(error: APIStatusError) -> Optional[float]:
    """Return Retry-After in seconds when the response provides it."""
    value = error.response.headers.get("retry-after")
    if not value:
        return None
    try:
        return max(0.0, float(value))
    except ValueError:
        pass
    try:
        retry_time = parsedate_to_datetime(value)
        if retry_time.tzinfo is None:
            retry_time = retry_time.replace(tzinfo=timezone.utc)
        now = datetime.now(timezone.utc)
        return max(0.0, (retry_time - now).total_seconds())
    except (TypeError, ValueError, OverflowError):
        return None


def create_chat_completion(
    messages: list[dict[str, str]],
    max_attempts: int = 5,
):
    model = os.environ["SILICONFLOW_MODEL"]
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(
                model=model,
                messages=messages,
                max_tokens=800,
            )
        except APIStatusError as error:
            if error.status_code not in {429, 503, 504}:
                raise
            if attempt == max_attempts - 1:
                raise
            retry_after = parse_retry_after(error)
            backoff = min(30.0, 2.0 ** attempt)
            delay = retry_after if retry_after is not None else backoff
        except APIConnectionError:
            if attempt == max_attempts - 1:
                raise
            delay = min(30.0, 2.0 ** attempt)

        jitter = random.uniform(0.0, min(1.0, delay * 0.25))
        time.sleep(delay + jitter)

    raise RuntimeError("The request failed after all retry attempts.")

import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from typing import Optional

from openai import APIConnectionError, APIStatusError, OpenAI

client = OpenAI(
    api_key=os.environ["SILICONFLOW_API_KEY"],
    base_url="https://api.siliconflow.com/v1",
    max_retries=0,
    timeout=60.0,
)


def parse_retry_after(error: APIStatusError) -> Optional[float]:
    """Return Retry-After in seconds when the response provides it."""
    value = error.response.headers.get("retry-after")
    if not value:
        return None
    try:
        return max(0.0, float(value))
    except ValueError:
        pass
    try:
        retry_time = parsedate_to_datetime(value)
        if retry_time.tzinfo is None:
            retry_time = retry_time.replace(tzinfo=timezone.utc)
        now = datetime.now(timezone.utc)
        return max(0.0, (retry_time - now).total_seconds())
    except (TypeError, ValueError, OverflowError):
        return None


def create_chat_completion(
    messages: list[dict[str, str]],
    max_attempts: int = 5,
):
    model = os.environ["SILICONFLOW_MODEL"]
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(
                model=model,
                messages=messages,
                max_tokens=800,
            )
        except APIStatusError as error:
            if error.status_code not in {429, 503, 504}:
                raise
            if attempt == max_attempts - 1:
                raise
            retry_after = parse_retry_after(error)
            backoff = min(30.0, 2.0 ** attempt)
            delay = retry_after if retry_after is not None else backoff
        except APIConnectionError:
            if attempt == max_attempts - 1:
                raise
            delay = min(30.0, 2.0 ** attempt)

        jitter = random.uniform(0.0, min(1.0, delay * 0.25))
        time.sleep(delay + jitter)

    raise RuntimeError("The request failed after all retry attempts.")

SiliconFlow supports OpenAI-compatible calls through the https://api.siliconflow.com/v1 base URL. Developers can add retry, queueing, monitoring, and model-routing logic while keeping a familiar SDK workflow.

How to Calculate a Safe API Concurrency Limit

RPM and TPM measure throughput. Concurrency measures how many requests are active at the same time. The values are related, but they are not interchangeable.

Use observed production data to estimate a starting point:

RPM-based request rate = RPM ÷ 60
TPM-based request rate = TPM ÷ observed total tokens per request ÷ 60
Safe request rate = min(RPM-based rate, TPM-based rate) × safety factor
Estimated active concurrency = safe request rate × P95 request latency

RPM-based request rate = RPM ÷ 60
TPM-based request rate = TPM ÷ observed total tokens per request ÷ 60
Safe request rate = min(RPM-based rate, TPM-based rate) × safety factor
Estimated active concurrency = safe request rate × P95 request latency

RPM-based request rate = RPM ÷ 60
TPM-based request rate = TPM ÷ observed total tokens per request ÷ 60
Safe request rate = min(RPM-based rate, TPM-based rate) × safety factor
Estimated active concurrency = safe request rate × P95 request latency

Suppose a model has:

• 1,200 RPM

• 60,000 TPM

• 1,200 observed total tokens per request

• A P95 response time of four seconds

• An 80% safety factor

The RPM-based rate is:

1,200 ÷ 60 = 20 requests per second

1,200 ÷ 60 = 20 requests per second

1,200 ÷ 60 = 20 requests per second

The TPM-based rate is:

60,000 ÷ 1,200 ÷ 60 = 0.83 requests per second

60,000 ÷ 1,200 ÷ 60 = 0.83 requests per second

60,000 ÷ 1,200 ÷ 60 = 0.83 requests per second

TPM is the tighter constraint. After applying the safety factor:

0.83 × 0.8 = 0.67 requests per second

0.83 × 0.8 = 0.67 requests per second

0.83 × 0.8 = 0.67 requests per second

The estimated active concurrency is:

0.67 × 4 = 2.68 active requests

0.67 × 4 = 2.68 active requests

0.67 × 4 = 2.68 active requests

The application could begin with a concurrency cap of two or three active requests and adjust it using production data.

This is a capacity-planning estimate, not a guaranteed API limit. Token usage and latency vary by prompt, model, output length, and traffic pattern. A concurrency cap also does not enforce RPM or TPM by itself. Use it with a queue-based, token-bucket, or leaky-bucket rate limiter that controls when new requests are released.

How to Design an Application Around Rate Limits

Rate-limit handling should be part of the application architecture rather than an exception added after launch.

Estimate Requests per User Action

Measure how many model calls one user action creates. Agentic workflows may generate several API requests even when the interface shows only one user message.

Monitor Token Distribution

Average token usage can hide unusually large prompts. Track median, P95, and maximum values for prompt, completion, and total tokens.

Control Traffic by Model

Because models have separate limits, heavily used models may need their own queues, concurrency caps, and monitoring. One global limit can underuse available capacity or allow one workload to dominate the application.

Set Prompt and Output Boundaries

Remove duplicated context, limit retrieval results, summarize older conversation history, and use realistic output limits. These changes reduce TPM pressure and inference cost at the same time.

Route Workloads to Suitable Models

SiliconFlow provides access to a broad range of language and multimodal models through a unified API. Applications can route classification, extraction, reasoning, coding, and multimodal workloads according to quality, latency, cost, and capacity requirements.

Model routing should reflect task requirements rather than serve as a way to avoid platform limits. Each route still requires quality, latency, token, and failure monitoring.

Move Offline Jobs to the Batch API

Evaluations, document processing, classification, and large extraction jobs do not always need real-time responses.

Where available, the SiliconFlow Batch API may be suitable for eligible offline workloads. Confirm the currently supported models, endpoints, processing behavior, and applicable limits before implementation.

Request Higher RPM or TPM Limits

Prompt optimization, queues, and retry controls can manage temporary traffic spikes, but they cannot resolve a permanent capacity mismatch.

When normal production traffic consistently approaches the available RPM or TPM limit, open the Model Plaza, select Higher Limits at the bottom of the left sidebar, and submit the SiliconFlow RPM and TPM Increase Request form.

Include realistic traffic estimates, expected token consumption, workload patterns, and production requirements. Submitting the form starts an evaluation and does not guarantee that every requested limit will be approved.

Evaluate Dedicated Capacity

Applications with sustained high throughput or a need for more predictable capacity can also evaluate a Dedicated Instance.

Dedicated instances typically do not use the standard account Rate Limits applied to general model access. They should still be monitored for instance throughput, latency, timeouts, queue depth, and limits elsewhere in the application stack.

FAQs About LLM API Rate Limits

Q1: What Does TPM Mean in an LLM API?

TPM means tokens per minute. It measures how many tokens an application can process within a minute. Long prompts, conversation history, retrieved documents, reasoning content, and long outputs can reach TPM before the application reaches its RPM limit.

Q2: Why Am I Getting a 429 Error With Low Request Volume?

A few long-context requests may consume a high number of tokens. Burst traffic, shared account usage, automatic retries, or model-specific hourly and daily limits can also produce a 429 response even when total request volume appears low.

Q3: Should I Retry Every Failed LLM API Request?

No. Retry temporary failures such as 429 errors, selected 503 or 504 errors, connection failures, and timeouts with bounded backoff. A 400 error usually indicates invalid parameters or an incorrect model ID, while 401 points to an invalid API key, 403 indicates insufficient permission, and 404 may indicate an incorrect endpoint or resource path. These issues should be corrected before the request is submitted again.

Q4: Does Increasing RPM Also Increase TPM?

Not automatically. RPM and TPM measure different constraints. Available limits can vary by model, account tier, model size, and deployment type. Check the current model details before configuring production capacity.

Q5: How Can I Reduce LLM API Rate-Limit Errors?

Shorten unnecessary context, control request release rates, limit concurrency, use bounded retries with jitter, and monitor token usage. Move eligible offline jobs to the Batch API and request higher limits when sustained production demand approaches the available RPM or TPM.