Rate limiting best practices

Limiting request by user agent

You can restrict the number of requests that are allowed for a specific user agent. The following rule example allows mobile app users to make up to 100 requests every 10 minutes. You might also create a separate rule limiting the rate for desktop browsers.

Allowing specific IP addresses or ASNs

Control access by including or excluding certain IP addresses or Autonomous System Numbers (ASNs) from a rate limiting rule.

The following rate limiting rule example allows up to 10 requests per minute from the same IP address and make a GET request to the /status path, provided the IP address is not included in the IP list entitled partner_ips.

Limiting requests by referrer

You can limit requests that originate from referrer pages, such as third-party advertisements or external websites. This use case helps to reduce the risk of indirect denial-of-service (DDoS) attacks and helps you to manage request quotas.

This example rule requires Advanced Rate Limiting.

Rule 1: Initial protection threshold

Rule 1 allows up to four failed login attempts per minute. When the limit is exceeded, the system triggers a Managed Challenge. This configuration helps legitimate users recover from occasional login errors while discouraging automated bots.

Rule 2: Intermediate protection threshold

If legitimate users pass the challenge when reaching rule 1's rate limit, Rule 2 applies additional protection for clients that continue to make failed login attempts. It allows up to 10 failed attempts over 10 minutes before triggering another Managed Challenge.

Rule 3: Strict protection threshold

Rule 3 enforces a stronger penalty to clients exceeding the rule 2 threshold by blocking an IP address for one day after 20 failed login attempts within an hour. This rule provides a final defense against persistent attack attempts.

All these example rules require a business plan or above.

These three rules have a counting expression separate from the rule expression (also known as mitigation expression). When you configure a separate counting expression, the matching criteria is used when an action is triggered. In the counting expression, you can include conditions based on the HTTP response status code and HTTP response headers to integrate the rate limiting with your backend logic.

You can also decide to have two different expressions: a counting expression and a rule/mitigation expression — to define:

1. The requests used to compute the rate.
2. The requests actually acted upon.

For example, Rule 3 example computes the rate considering POST requests to /login that returned a 401 or 403 HTTP status code. However, when the rate limit is exceeded, CIS blocks every request to the example.com host generated by the same IP.

Preventing content scraping by using query string

In this example, clients perform operations (such as price lookups or adding items to a basket) on an e-commerce website through query string parameters. For example, a typical request that is sent by a client might be similar to the following:

GET https://store.com/merchant?action=lookup_price&product_id=215
Cookie: session_id=12345

Your security team might want to consider setting up a limit on the number of times a client can lookup prices to prevent bots — which might have eluded CIS Bot Management — from scraping the store's entire catalog.

These two rate limiting rules match requests performing a selected action (look up price, in this example) and use IP as the counting characteristic. Similarly, to the /login example, the two rules help reduce false positives in case of persistent (but legitimate) visitors.

You can limit the lookup of a specific product_id by using a query string parameter. By adding a query parameter as a counting characteristic, the rate is calculated across all requests, regardless of the client.

The following example limits the number of lookups for each product_id to 50 requests in 10 seconds.

This example rule requires Advanced Rate Limiting.

You might follow the same pattern of rate limiting rules to protect applications handling reservations and bookings.

Preventing content scraping by using request body

Consider an application that handles the operation and its parameters through the request body in JSON format. For example, the lookup_price operation might look like the following:

POST https://api.store.com/merchant
Cookie: session_id=12345

Body:
{
  "action": "lookup_price",
  "product_id": 215
}

In this scenario, you can create a following rule to limit the number of actions from individual sessions:

This example rule requires Advanced Rate Limiting and payload inspection.

You can also limit the number of lookups of each product_id regardless of the client making the requests by deploying a rule like the following:

This example rule requires Advanced Rate Limiting and payload inspection.

If the request body is not JSON format, you can use the http.request.body.raw field and regular expressions (along with the matches operator) to achieve the same goal.

Limiting requests from bots

You can use rate limiting to control automated traffic from bots. A common approach is to monitor requests that return a high number of 403 or 404 response status codes, which often indicate automated scraping activity.

In this situation, you might configure a rule similar to the following:

This example rule requires a Business plan or above.

To control the rate of actions performed by automated sources, consider use rate limiting rules together with Bot Management. With Bot Management, you can use the bot score as part of the matching criteria to apply the rule only to automated or likely automated traffic. For example, you can use a maximum score (or threshold) of 30 for likely automated traffic and 10 for automated traffic.

You can enhance protection by combining rate limiting with Bot Management. With Bot Management, you can use the bot score as part of the matching criteria to apply the rule only to automated or likely automated traffic.

For example:

• A bot score lesser than 30 represents likely automated traffic.
• A bot score lesser than 10 represents confirmed automated traffic.

Protecting resources

Even GET requests can strain the application or consume bandwidth when used for large data downloads, such as files or images.

For instance, consider the following endpoint:

GET https://api.store.com/files/<FILE_ID>
Header: x-api-key=9375

To prevent abuse while allowing legitimate downloads, you can define a rule that limits file requests without writing separate rules for each file.

This example rule requires Advanced Rate Limiting.

This rule limits downloads to 10 requests per 10 minutes per file under https://api.store.com/files/*. By using Path as the counting characteristic, you can avoid creating new rules for every new <FILE_ID>. The rate is calculated per file, regardless of client IP or session ID.

You can further strengthen protection by combining Path with a client identifier such as x-api-key or IP. This approach allows you to restrict the number of downloads a specific client can make for a given file.

This example rule requires Advanced Rate Limiting.

Limiting the number of operations

To limit the rate of actions, create the following rule:

This example rule requires Advanced Rate Limiting and payload inspection.

Limiting the total amount of query complexity

The complexity of handling a GraphQL request can vary significantly. Because the API uses a single endpoint, it can be difficult to determine the complexity of each request before it is processed.

To prevent resource exhaustion on the origin server, limit the total request complexity per client over time, rather than limiting the number of requests. CIS Rate Limiting enables you to create rules that track complexity over time and block requests that exceed a defined complexity budget.

This method requires the origin server to assign a complexity score to each request and include that score in the HTTP response header. The rate-limiting mechanism then uses the score information to update the complexity budget for that specific client.

The following example defines a total complexity budget of 1,000 per hour:

This example rule requires Advanced Rate Limiting and payload inspection.

When the origin server processes a request, it adds a score HTTP header to the response with a value which indicates how much work the origin has performed to handle it. For example, 100. In the next hour, the same client can perform requests up to an additional budget of 900. As soon as this budget is exceeded, later requests are blocked until the timeout expires.

Limiting any individual query’s complexity

API Shield customers can use GraphQL malicious query protection to protect their GraphQL APIs. This feature scans incoming GraphQL traffic for queries that might overload the origin server and cause a denial-of-service condition.

You can create rules to limit the depth and size of incoming GraphQL queries. These rules help block suspicious or excessively complex queries before they impact performance.