periodlimit

[!TIP] This document is machine-translated by Google. If you find grammatical and semantic errors, and the document description is not clear, please PR

Whether in a single service or in a microservice, the API interface provided by the developer for the front end has an upper limit of access. When the frequency of access or the amount of concurrency exceeds its tolerance, we must consider current limit to ensure the interface. Availability or degraded availability. That is, the interface also needs to be installed with a fuse to prevent the system from being paralyzed due to excessive pressure on the system by unexpected requests.

This article will introduce periodlimit.

Usage

  1. const (
  2.     seconds = 1
  3.     total   = 100
  4.     quota   = 5
  5. )
  6. // New limiter
  7. l := NewPeriodLimit(seconds, quota, redis.NewRedis(s.Addr(), redis.NodeType), "periodlimit")
  9. // take source
  10. code, err := l.Take("first")
  11. if err != nil {
  12.     logx.Error(err)
  13.     return true
  14. }
  16. // switch val => process request
  17. switch code {
  18.     case limit.OverQuota:
  19.         logx.Errorf("OverQuota key: %v", key)
  20.         return false
  21.     case limit.Allowed:
  22.         logx.Infof("AllowedQuota key: %v", key)
  23.         return true
  24.     case limit.HitQuota:
  25.         logx.Errorf("HitQuota key: %v", key)
  26.         // todo: maybe we need to let users know they hit the quota
  27.         return false
  28.     default:
  29.         logx.Errorf("DefaultQuota key: %v", key)
  30.         // unknown response, we just let the sms go
  31.         return true
  32. }

periodlimit

go-zero adopts a sliding window counting method to calculate the number of accesses to the same resource within a period of time. If it exceeds the specified limit, access is denied. Of course, if you are accessing different resources within a period of time, the amount of access to each resource does not exceed the limit. In this case, a large number of requests are allowed to come in.

In a distributed system, there are multiple microservices to provide services. So when instantaneous traffic accesses the same resource at the same time, how to make the counter count normally in the distributed system? At the same time, when computing resources are accessed, multiple calculations may be involved. How to ensure the atomicity of calculations?

  • go-zero counts resource visits with the help of incrby of redis
  • Use lua script to do the whole window calculation to ensure the atomicity of calculation

Let’s take a look at several key attributes controlled by lua script:

argument mean
key[1] Logo for access to resources
ARGV[1] limit => the total number of requests, if it exceeds the rate limit. Can be set to QPS
ARGV[2] window size => sliding window, use ttl to simulate the effect of sliding 
  1. -- to be compatible with aliyun redis, 
  2. -- we cannot use `local key = KEYS[1]` to reuse thekey
  3. local limit = tonumber(ARGV[1])
  4. local window = tonumber(ARGV[2])
  5. -- incrbt key 1 => key visis++
  6. local current = redis.call("INCRBY", KEYS[1], 1)
  7. -- If it is the first visit, set the expiration time => TTL = window size
  8. -- Because it only limits the number of visits for a period
  9. if current == 1 then
  10.     redis.call("expire", KEYS[1], window)
  11.     return 1
  12. elseif current < limit then
  13.     return 1
  14. elseif current == limit then
  15.     return 2
  16. else
  17.     return 0
  18. end

As for the above return code, return it to the caller. The caller decides to request subsequent operations:

return code tag call code mean
0 OverQuota 3 over limit
1 Allowed 1 in limit
2 HitQuota 2 hit limit

The following picture describes the process of request entry and the subsequent situation when the request triggers limit: image.png image.png

Subsequent processing

If a large batch of requests comes in at a certain point in the service, the periodlimit reaches the limit threshold in a short period of time, and the set time range is far from reaching. The processing of subsequent requests becomes a problem.

It is not processed in periodlimit, but code is returned. The processing of subsequent requests is left to the developer.

  1. If it is not processed, it is simply to reject the request
  2. If these requests need to be processed, developers can use mq to buffer the requests to ease the pressure of the requests
  3. Use tokenlimit to allow temporary traffic impact

So in the next article, we will talk about tokenlimit

Summary

The periodlimit current limiting scheme in go-zero is based on redis counters. By calling redis lua script, it guarantees the atomicity of the counting process and guarantees that the counting is normal under distributed conditions. However, this scheme has disadvantages because it needs to record all behavior records within the time window. If this amount is particularly large, memory consumption will become very serious.

Reference