sentinel流控規則校驗之原始碼分析

努力工作的小碼農發表於2021-02-05

前言：

　　上節給大家把sentinel流控整個執行大致過了，但涉及到最核心的流控演算法還沒有講，先提前說明一下 sentinel用的流控演算法是令牌桶演算法，參考了Guava的RateLimiter，有讀過RateLimiter原始碼再理解sentinel限流演算法會更容易，本節依然以原始碼為主給大家撥開sentinel流控演算法的原理

接著上節沒有講到的FlowSlot來看，先來看對應流控規則配置

FlwSlot

/***********************************************FlowSlot***********************************************/
public class FlowSlot extends AbstractLinkedProcessorSlot<DefaultNode> {

    private final FlowRuleChecker checker;

    public FlowSlot() {
        this(new FlowRuleChecker());
    }

    FlowSlot(FlowRuleChecker checker) {
        AssertUtil.notNull(checker, "flow checker should not be null");
        this.checker = checker;
    }

    @Override
    public void entry(Context context, ResourceWrapper resourceWrapper, DefaultNode node, int count,
                      boolean prioritized, Object... args) throws Throwable {
        checkFlow(resourceWrapper, context, node, count, prioritized);

        fireEntry(context, resourceWrapper, node, count, prioritized, args);
    }

    void checkFlow(ResourceWrapper resource, Context context, DefaultNode node, int count, boolean prioritized)
        throws BlockException {
        checker.checkFlow(ruleProvider, resource, context, node, count, prioritized);
    }
}

public class FlowRuleChecker {

    public void checkFlow(Function<String, Collection<FlowRule>> ruleProvider, ResourceWrapper resource,
                          Context context, DefaultNode node, int count, boolean prioritized) throws BlockException {
        if (ruleProvider == null || resource == null) {
            return;
        }
        // 拿到當前資源對應的流控規則
        Collection<FlowRule> rules = ruleProvider.apply(resource.getName());
        if (rules != null) {
            for (FlowRule rule : rules) {
                //通行校驗
                if (!canPassCheck(rule, context, node, count, prioritized)) {
                    throw new FlowException(rule.getLimitApp(), rule);
                }
            }
        }
    }

    public boolean canPassCheck(/*@NonNull*/ FlowRule rule, Context context, DefaultNode node,
                                                    int acquireCount) {
        return canPassCheck(rule, context, node, acquireCount, false);
    }

    public boolean canPassCheck(/*@NonNull*/ FlowRule rule, Context context, DefaultNode node, int acquireCount,
                                                    boolean prioritized) {
        String limitApp = rule.getLimitApp();
        // 對應控制檯配置的來源
        if (limitApp == null) {
            return true;
        }
        if (rule.isClusterMode()) {
            // 叢集模式校驗
            return passClusterCheck(rule, context, node, acquireCount, prioritized);
        }
        // 本地應用校驗
        return passLocalCheck(rule, context, node, acquireCount, prioritized);
    }

    private static boolean passLocalCheck(FlowRule rule, Context context, DefaultNode node, int acquireCount,
                                          boolean prioritized) {
        // 針對配置的流控模式拿到對應的node                                  
        Node selectedNode = selectNodeByRequesterAndStrategy(rule, context, node);
        if (selectedNode == null) {
            return true;
        }
        // 針對配置的流控效果來作校驗
        return rule.getRater().canPass(selectedNode, acquireCount, prioritized);
    }
    
    static Node selectNodeByRequesterAndStrategy(/*@NonNull*/ FlowRule rule, Context context, DefaultNode node) {
        String limitApp = rule.getLimitApp();
        int strategy = rule.getStrategy();
        String origin = context.getOrigin();

        if (limitApp.equals(origin) && filterOrigin(origin)) {
            if (strategy == RuleConstant.STRATEGY_DIRECT) {
                // 配置的來源和當前相同 流控模式為直接
                return context.getOriginNode();
            }

            return selectReferenceNode(rule, context, node);
        } else if (RuleConstant.LIMIT_APP_DEFAULT.equals(limitApp)) {
            if (strategy == RuleConstant.STRATEGY_DIRECT) {
                // 配置來源為default 流控模式為直接
                return node.getClusterNode();
            }

            return selectReferenceNode(rule, context, node);
        } else if (RuleConstant.LIMIT_APP_OTHER.equals(limitApp)
            && FlowRuleManager.isOtherOrigin(origin, rule.getResource())) {
            if (strategy == RuleConstant.STRATEGY_DIRECT) {
                // 配置來源為other 流控模式為直接
                return context.getOriginNode();
            }

            return selectReferenceNode(rule, context, node);
        }

        return null;
    }

    static Node selectReferenceNode(FlowRule rule, Context context, DefaultNode node) {
        // 關聯資源
        String refResource = rule.getRefResource();
        int strategy = rule.getStrategy();

        if (StringUtil.isEmpty(refResource)) {
            return null;
        }
        // 流控模式為關聯
        if (strategy == RuleConstant.STRATEGY_RELATE) {
            return ClusterBuilderSlot.getClusterNode(refResource);
        }
        // 流控模式為鏈路
        if (strategy == RuleConstant.STRATEGY_CHAIN) {
            if (!refResource.equals(context.getName())) {
                return null;
            }
            return node;
        }
        // No node.
        return null;
    }
}

流控型別只針對qps或執行緒數限制

流控模式分別有三種直接：originNode，關聯ClusterNode，鏈路EntranceNode，針對這幾個node的區別上節已經做過說明，不清楚的讀者開啟上節node樹形結構一看便知

流控效果對應四種，具體實現由TrafficShappingController的實現類完成

快速失敗(DefaultController)：如果是限制qps會丟擲異常，執行緒數返回false不通過
Warm Up(WarmUpController)：預熱，防止流量突然暴增，導致系統負載過重，有時候系統設定的最大負載是在理想狀態達到的，當系統長時間處於冷卻狀態需要通過一定時間的預熱才能達到最大負載，比如跟資料庫建立連線
排隊等待(RateLimiterController)：當前沒有足夠的令牌通過，會進行睡眠等待，直接能拿到足夠的令牌數
WarmUpRateLimiterController：第二種和第三種的結合

來看TrafficShappingController具體的實現類

DefaultController

public class DefaultController implements TrafficShapingController {

    private static final int DEFAULT_AVG_USED_TOKENS = 0;

    //閥值
    private double count;
    
    // 1=qps，0=ThreadNum
    private int grade;

    public DefaultController(double count, int grade) {
        this.count = count;
        this.grade = grade;
    }

    @Override
    public boolean canPass(Node node, int acquireCount) {
        return canPass(node, acquireCount, false);
    }

    @Override
    public boolean canPass(Node node, int acquireCount, boolean prioritized) {
        // 獲取當前node的ThreadNum或QPS
        int curCount = avgUsedTokens(node);
        if (curCount + acquireCount > count) { 
　　　　　　　 // 設定當前流量為優先順序和流控模式為QPS
            if (prioritized && grade == RuleConstant.FLOW_GRADE_QPS) {
                long currentTime;
                long waitInMs;
                currentTime = TimeUtil.currentTimeMillis();
                // 算出拿到當前令牌數的等待時間(ms)
                waitInMs = node.tryOccupyNext(currentTime, acquireCount, count);
                // OccupyTimeoutProperty.getOccupyTimeout = 500ms
                // 如果流量具有優先順序，會獲取未來的令牌數
                if (waitInMs < OccupyTimeoutProperty.getOccupyTimeout()) {
　　　　　　　　　　　　// 新增佔用未來的QPS，會呼叫OccupiableBucketLeapArray.addWaiting(long time, int acquireCount)
                    node.addWaitingRequest(currentTime + waitInMs, acquireCount);
                    node.addOccupiedPass(acquireCount);
                    sleep(waitInMs);

                    // PriorityWaitException indicates that the request will pass after waiting for {@link @waitInMs}.
                    throw new PriorityWaitException(waitInMs);
                }
            }
            // 控制的是執行緒數返回false
            return false;
        }
        return true;
    }

    private int avgUsedTokens(Node node) {
        if (node == null) {
            return DEFAULT_AVG_USED_TOKENS;
        }
        return grade == RuleConstant.FLOW_GRADE_THREAD ? node.curThreadNum() : (int)(node.passQps());
    }

    private void sleep(long timeMillis) {
        try {
            Thread.sleep(timeMillis);
        } catch (InterruptedException e) {
            // Ignore.
        }
    }
}

在上節講滑動視窗的時候，還有一個秒維度的視窗OccupiableBucketLeapArray沒有講解，它同樣繼承LeapArray，但它還有額外的概念未來佔用，在DefaultController中當前令牌數不夠並且流量具有優先順序，那麼會提前獲取未來的令牌，因為閥值固定每秒能獲取的令牌數也固定，既然佔用了未來的令牌數，那等到時間到了這個未來時間點，當前可獲取的令牌數=閥值—之前佔用的令牌

OccupiableBucketLeapArray有個FutureBucketLeapArray就是來儲存佔用未來的視窗資料

public class OccupiableBucketLeapArray extends LeapArray<MetricBucket> {

    private final FutureBucketLeapArray borrowArray;

    public OccupiableBucketLeapArray(int sampleCount, int intervalInMs) {
        // This class is the original "CombinedBucketArray".
        super(sampleCount, intervalInMs);
        this.borrowArray = new FutureBucketLeapArray(sampleCount, intervalInMs);
    }

    @Override
    // LeapArray.currentWindow(long timeMillis)新增新視窗時會呼叫
    public MetricBucket newEmptyBucket(long time) {
        MetricBucket newBucket = new MetricBucket();
        // 已被之前佔用，將之前佔用的資料新增到當前新的視窗中
        MetricBucket borrowBucket = borrowArray.getWindowValue(time);
        if (borrowBucket != null) {
            newBucket.reset(borrowBucket);
        }

        return newBucket;
    }

    @Override
    protected WindowWrap<MetricBucket> resetWindowTo(WindowWrap<MetricBucket> w, long time) {
        // Update the start time and reset value.
        w.resetTo(time);
        // 重置時當前視窗資料時，也需要考慮被之前佔用的情況
        MetricBucket borrowBucket = borrowArray.getWindowValue(time);
        if (borrowBucket != null) {
            w.value().reset();
            w.value().addPass((int)borrowBucket.pass());
        } else {
            w.value().reset();
        }

        return w;
    }

    @Override
    public long currentWaiting() {
        // 獲取當前時間被之前佔用的qps
        borrowArray.currentWindow();
        long currentWaiting = 0;
        List<MetricBucket> list = borrowArray.values();

        for (MetricBucket window : list) {
            currentWaiting += window.pass();
        }
        return currentWaiting;
    }

    @Override
    public void addWaiting(long time, int acquireCount) {
        // 新增到未來視窗
        WindowWrap<MetricBucket> window = borrowArray.currentWindow(time);
        window.value().add(MetricEvent.PASS, acquireCount);
    }
}

當新視窗新增或重置舊視窗資料都需要考慮之前佔用的情況，然後把之前佔用的視窗資料新增進去

RateLimiterController

跟Guava中SmoothBursty原理類似

public class RateLimiterController implements TrafficShapingController {

    // 最大等待時間
    private final int maxQueueingTimeMs;
    // 閥值
    private final double count;

    // 最新一次拿令牌的時間
    private final AtomicLong latestPassedTime = new AtomicLong(-1);

    public RateLimiterController(int timeOut, double count) {
        this.maxQueueingTimeMs = timeOut;
        this.count = count;
    }

    @Override
    public boolean canPass(Node node, int acquireCount) {
        return canPass(node, acquireCount, false);
    }

    @Override
    public boolean canPass(Node node, int acquireCount, boolean prioritized) {
        // Pass when acquire count is less or equal than 0.
        if (acquireCount <= 0) {
            return true;
        }
        // Reject when count is less or equal than 0.
        // Otherwise,the costTime will be max of long and waitTime will overflow in some cases.
        if (count <= 0) {
            return false;
        }

        long currentTime = TimeUtil.currentTimeMillis();
        // 計算獲取acquireCount需要多少毫秒
        long costTime = Math.round(1.0 * (acquireCount) / count * 1000);

        // 預期獲取令牌的時間
        long expectedTime = costTime + latestPassedTime.get();

        // 如果小於當前時間 則直接返回
        if (expectedTime <= currentTime) {
            // Contention may exist here, but it's okay.
            latestPassedTime.set(currentTime);
            return true;
        } else {
            // 否則進行等待
            // Calculate the time to wait.
            long waitTime = costTime + latestPassedTime.get() - TimeUtil.currentTimeMillis();
            if (waitTime > maxQueueingTimeMs) {
                return false;
            } else {
                // oldTime 為當前拿到令牌的時長 是更新後的值
                long oldTime = latestPassedTime.addAndGet(costTime);
                try {
                    waitTime = oldTime - TimeUtil.currentTimeMillis();
                    if (waitTime > maxQueueingTimeMs) {
                        latestPassedTime.addAndGet(-costTime);
                        return false;
                    }
                    // in race condition waitTime may <= 0
                    if (waitTime > 0) {
                        Thread.sleep(waitTime);
                    }
                    return true;
                } catch (InterruptedException e) {
                }
            }
        }
        return false;
    }

}

前面兩種流控策略都比較簡單，主要來看WarmUpController

WarmUpController

它參考了Guava中SmoothWarmingUp的設計，但具體策略不一樣，先來看它的原始碼，後續講解它們的不同

public class WarmUpController implements TrafficShapingController {

    // qps閾值
    protected double count;
    // 負載因子 用來控制預熱速度 預設為3
    private int coldFactor;
    // 進入冷卻狀態的警戒線
    protected int warningToken = 0;
    // 最大的令牌數
    private int maxToken;
    // 斜率
    protected double slope;

    // 當前令牌容量
    protected AtomicLong storedTokens = new AtomicLong(0);
    // 上一次新增令牌時間
    protected AtomicLong lastFilledTime = new AtomicLong(0);

    public WarmUpController(double count, int warmUpPeriodInSec, int coldFactor) {
        construct(count, warmUpPeriodInSec, coldFactor);
    }

    public WarmUpController(double count, int warmUpPeriodInSec) {
        construct(count, warmUpPeriodInSec, 3);
    }

    private void construct(double count, int warmUpPeriodInSec, int coldFactor) {

        if (coldFactor <= 1) {
            throw new IllegalArgumentException("Cold factor should be larger than 1");
        }

        this.count = count;

        this.coldFactor = coldFactor;

        // thresholdPermits = 0.5 * warmupPeriod / stableInterval.
        // warningToken = 100;
        warningToken = (int)(warmUpPeriodInSec * count) / (coldFactor - 1);
        // / maxPermits = thresholdPermits + 2 * warmupPeriod /
        // (stableInterval + coldInterval)
        // maxToken = 200
        maxToken = warningToken + (int)(2 * warmUpPeriodInSec * count / (1.0 + coldFactor));

        // slope
        // slope = (coldIntervalMicros - stableIntervalMicros) / (maxPermits- thresholdPermits);
        slope = (coldFactor - 1.0) / count / (maxToken - warningToken);

    }

    @Override
    public boolean canPass(Node node, int acquireCount) {
        return canPass(node, acquireCount, false);
    }

    @Override
    public boolean canPass(Node node, int acquireCount, boolean prioritized) {
        // 當前視窗通過的qps
        long passQps = (long) node.passQps();

        // 上一個視窗的qps
        long previousQps = (long) node.previousPassQps();
        // 更新當前令牌數
        syncToken(previousQps);

        // 開始計算它的斜率
        // 如果進入了警戒線，開始調整他的qps
        long restToken = storedTokens.get();
        if (restToken >= warningToken) {
            long aboveToken = restToken - warningToken;
            // 消耗的速度要比warning快，但是要比慢
            // current interval = restToken*slope+1/count 算出當前進入預熱狀態能得到最大的qps
            double warningQps = Math.nextUp(1.0 / (aboveToken * slope + 1.0 / count));
            if (passQps + acquireCount <= warningQps) {
                return true;
            }
        } else {
            if (passQps + acquireCount <= count) {
                // 這裡之所以用count來判斷而不是storedTokens來，是因為storedTokens是每次新增都是以count倍速來新增 count自然不會大過storedTokens
                return true;
            }
        }

        return false;
    }

    protected void syncToken(long passQps) {
        long currentTime = TimeUtil.currentTimeMillis();
        // 因為每一個秒維度的總視窗間隔為1s，減去當前時間的毫秒數 即可得到當前視窗的開始時間
        currentTime = currentTime - currentTime % 1000;
        long oldLastFillTime = lastFilledTime.get();
        if (currentTime <= oldLastFillTime) {
            // 說明還處於當前視窗
            return;
        }
        // 進入到這裡代表第一次進入新的視窗 需要新增當前令牌容量
        // 令牌數量的舊值
        long oldValue = storedTokens.get();
        // 算出新的令牌數 舊值+(根據上一個視窗的qps 算出從上次新增令牌的時間到當前時間需要新增的令牌數)
        long newValue = coolDownTokens(currentTime, passQps);

        if (storedTokens.compareAndSet(oldValue, newValue)) {
            // 減去上個視窗的qps，然後設定storedTokens最新值
            long currentValue = storedTokens.addAndGet(0 - passQps);
            if (currentValue < 0) {
                storedTokens.set(0L);
            }
            lastFilledTime.set(currentTime);
        }

    }

    private long coolDownTokens(long currentTime, long passQps) {
        long oldValue = storedTokens.get();
        long newValue = oldValue;

        // 新增令牌的判斷前提條件:
        // 當令牌的消耗程度遠遠低於警戒線的時候
        if (oldValue < warningToken) {
            // 當前令牌數小於警戒值 可正常新增令牌 這裡只是新增令牌 並不考慮新增令牌後大於warningToken情況，因為後面拿令牌還會重新判斷
            // currentTime - lastFilledTime.get() = 與上一次新增令牌間隔多少毫秒
            newValue = (long)(oldValue + (currentTime - lastFilledTime.get()) * count / 1000);
        } else if (oldValue > warningToken) {
            // 如果當前passQps > (count / coldFactor) 也就是說當前系統消耗令牌速度大於冷卻速度 則不需要繼續新增令牌
            if (passQps < (int)count / coldFactor) {
                // 如果消耗速度小於冷卻速度 則正常新增令牌
                newValue = (long)(oldValue + (currentTime - lastFilledTime.get()) * count / 1000);
            }
        }
        return Math.min(newValue, maxToken);
    }

}

大致流程:
先根據當前視窗時間間隔時間新增令牌數，正常情況是以count*(last interval second) 來新增令牌數,
如果新增令牌的時候當前令牌容量已經達到警戒值，需要根據當前視窗passqps來判斷當前系統消耗令牌的速率是否大於冷卻速率(也就是系統負載狀態不需要再進行預熱)，大於冷卻速率則不需要繼續新增令牌
新增完令牌後減去上一個視窗的qps得到最新的令牌數，再判斷最新令牌數是否到了警戒值，到了警戒值，通過slope算出目前視窗能得到最大的qps，passQps+acquireCount 不允許大於它

我們再簡單說說 Guava 的 SmoothWarmingUp 和 Sentinel 的 WarmupController 的區別

Guava在於控制獲取令牌的速度，根據時間推進增加令牌，通過當前令牌容量判斷獲取令牌下一個時間點，如當前令牌容量超過了閥值會進行預熱增加等待時長

Sentinel在於控制QPS，根據時間推進增加令牌，根據通過QPS減少令牌，如果QPS持續下降，當前storeTokens會持續增加，直到超過warningTokens閥值，越過閥值會根據進入預熱狀態後能提供最大的QPS來做限制

WarmUpRateLimiterController

看完RateLimiterController WarmUpController實現後，再來看這個就非常簡單了，看名稱就知道是它兩的結合

public class WarmUpRateLimiterController extends WarmUpController {

    private final int timeoutInMs;
    private final AtomicLong latestPassedTime = new AtomicLong(-1);

    public WarmUpRateLimiterController(double count, int warmUpPeriodSec, int timeOutMs, int coldFactor) {
        super(count, warmUpPeriodSec, coldFactor);
        this.timeoutInMs = timeOutMs;
    }

    @Override
    public boolean canPass(Node node, int acquireCount) {
        return canPass(node, acquireCount, false);
    }

    @Override
    public boolean canPass(Node node, int acquireCount, boolean prioritized) {
        long previousQps = (long) node.previousPassQps();
        syncToken(previousQps);

        long currentTime = TimeUtil.currentTimeMillis();

        long restToken = storedTokens.get();
        long costTime = 0;
        long expectedTime = 0;
        if (restToken >= warningToken) {
            // 在預熱範圍拿到的token
            long aboveToken = restToken - warningToken;

            // current interval = restToken*slope+1/count
            double warmingQps = Math.nextUp(1.0 / (aboveToken * slope + 1.0 / count));
            costTime = Math.round(1.0 * (acquireCount) / warmingQps * 1000);
        } else {
            costTime = Math.round(1.0 * (acquireCount) / count * 1000);
        }
        // 拿到令牌的時間
        expectedTime = costTime + latestPassedTime.get();

        if (expectedTime <= currentTime) {
            latestPassedTime.set(currentTime);
            return true;
        } else {
            long waitTime = costTime + latestPassedTime.get() - currentTime;
            if (waitTime > timeoutInMs) {
                return false;
            } else {
                long oldTime = latestPassedTime.addAndGet(costTime);
                try {
                    waitTime = oldTime - TimeUtil.currentTimeMillis();
                    if (waitTime > timeoutInMs) {
                        latestPassedTime.addAndGet(-costTime);
                        return false;
                    }
                    if (waitTime > 0) {
                        Thread.sleep(waitTime);
                    }
                    return true;
                } catch (InterruptedException e) {
                }
            }
        }
        return false;
    }
}

Sentinel Dashboard（基於1.8.1）流控規則持久化到Nacos——涉及部分Sentinel Dashboard原始碼改造
2020-12-15
持久化原始碼
微服務架構 | 5.4 Sentinel 流控、統計和熔斷的原始碼分析
2022-01-31
微服務架構原始碼
GCD原始碼分析之再解 sentinel
2016-09-25
GC原始碼
一個名叫Sentinel-Rules-SDK的元件，使得Sentinel的流控&熔斷規則的配置更加方便
2021-03-15
元件
drf 認證校驗及原始碼分析
2020-10-31
原始碼
springMVC:校驗框架:多規則校驗,巢狀校驗,分組校驗;ssm整合技術
2021-03-25
SpringMVC框架巢狀SSM
前端Vue中常用rules校驗規則
2019-11-14
前端Vue
校驗檔案的搜尋規則
2017-11-09
Sentinel基本使用與原始碼分析
2023-05-18
原始碼
element-ui自定義表單校驗規則及常用表單校驗
2020-09-29
UI
drf 許可權校驗設定與原始碼分析
2020-11-01
原始碼
Sentinel 實戰-規則持久化
2019-01-09
持久化
SpringBoot專案校驗規則優化
2019-02-12
Spring Boot優化
Sentinel Dashboard中修改規則同步到Nacos
2019-05-21
常用後端正則校驗
2017-01-21
後端
原始碼分析 — Activity的清單註冊校驗及動態注入
2018-03-20
原始碼
Java原始碼的折行規則（編碼規範）
2010-05-20
Java原始碼
Tars | 第4篇 Subset路由規則業務分析與原始碼探索
2021-09-10
路由原始碼
Vue Element-ui表單校驗規則，你掌握了哪些？
2021-07-09
VueUI
【勝通】mysql字符集與校驗規則的設定
2008-07-09
MySql
Guava 原始碼分析之 EventBus 原始碼分析
2018-08-01
Guava原始碼
管理規則和基於規則的轉換——流
2010-08-18
Android 原始碼分析之 AsyncTask 原始碼分析
2019-03-04
Android原始碼
【原始碼解析】AsyncTask的用法與規則
2019-11-14
原始碼
[Spring-Cloud-Alibaba] Sentinel 規則持久化
2019-07-22
SpringCloud持久化
Sentinel規則基於Apollo(阿波羅)持久化
2020-10-23
持久化
SOFA 原始碼分析— 事件匯流排
2019-02-15
原始碼事件
風控規則引擎（一）：Java 動態指令碼
2024-03-14
Java指令碼
主資料之編碼規則
2020-07-06
【勝通】mysql連線通道中的字符集和校驗規則
2008-07-09
MySql
浮動流元素排列規則
2020-11-09
9.Spring Cloud Alibaba Sentinel流控熔斷元件
2020-12-02
SpringCloud元件
Restful規範-APIView原始碼分析
2021-07-03
RESTAPIView原始碼
java安全編碼指南之:輸入校驗
2020-09-21
Java
原始碼分析之 HashMap
2019-03-04
原始碼HashMap
原始碼分析之AbstractQueuedSynchronizer
2017-03-28
原始碼
原始碼分析之ArrayList
2017-04-12
原始碼
【MySQL】資料庫字元校對規則
2014-05-11
MySql資料庫字元