springcloud3(六) 服務降級限流熔斷元件Resilience4j

Brian_Huang發表於2021-11-05
SpringGCCloud元件

程式碼地址：https://github.com/showkawa/springBoot_2017/tree/master/spb-demo/spb-gateway/src/test/java/com/kawa/spbgateway/circuitbreaker/resilience4j

公司的閘道器(基於Spring Cloud Gateway)上線有一段時間了，目前只有一個簡單的動態路由的功能，接下來的工作一部分會涉及到服務的保護和服務健壯性方面，也就是要加入限流，熔斷和降級等特性。此處找了下業界成熟的開源框架如下表的對比

 Sentinel（Alibaba開源）Hystrix(不再維護)resilience4j（Spring官方推薦）
隔離策略 訊號量隔離（併發控制） 執行緒池隔離/訊號量隔離 訊號量隔離
熔斷降級策略 基於慢呼叫比例、異常比例、異常數 基於異常比例 基於異常比例、響應時間
實時統計實現 滑動視窗（LeapArray） 滑動視窗（基於 RxJava） Ring Bit Buffer
動態規則配置 支援多種資料來源 支援多種資料來源 有限支援
擴充套件性 多個擴充套件點 外掛的形式 介面的形式
基於註解的支援 支援 支援 支援
限流 基於 QPS，支援基於呼叫關係的限流 有限的支援 Rate Limiter
流量整形 支援預熱模式與勻速排隊控制效果 不支援 簡單的 Rate Limiter 模式
系統自適應保護 支援 不支援 不支援
多語言支援 Java/Go/C++ Java Java
Service Mesh 支援 支援 Envoy/Istio 不支援 不支援
控制檯 提供開箱即用的控制檯，可配置規則、實時監控、機器發現等 簡單的監控檢視 不提供控制檯，可對接其它監控系統

對比來自：https://github.com/alibaba/Sentinel/wiki/Guideline:-%E4%BB%8E-Hystrix-%E8%BF%81%E7%A7%BB%E5%88%B0-Sentinel

最終基於公司的需求，準備引入Resilience4j元件, 所以這篇部落格是來梳理Resilience4j的元件的使用方式, 下一篇部落格寫結合Spring Cloud Gateway的實現自定義的服務限流保護策略

1. Resilience4j

Resilience4j官方guide: https://resilience4j.readme.io/docs

Resilience4j 常用的元件有5個 -> CircuitBreaker，Bulkhead，RateLimiter，Retry 和 TimeLimiter （Cache不推薦在生產環境使用，所以這篇部落格不做介紹 ）, 本篇部落格基於1.7.0的版本介紹

1.1 CircuitBreaker

斷路器是通過具有三個正常狀態的有限狀態機實現的：CLOSED、OPEN 和 HALF_OPEN 以及兩個特殊狀態 DISABLED 和 FORCED_OPEN。CircuitBreaker 使用滑動視窗來儲存和聚合呼叫的結果。您可以在基於計數的滑動視窗和基於時間的滑動視窗之間進行選擇。基於計數的滑動視窗聚合最後 N 次呼叫的結果。基於時間的滑動視窗聚合了最近 N 秒的呼叫結果。

1.1.1 CircuitBreakerConfig

CircuitBreakerConfig看名字大家也知道了它是做什麼的(好的編碼就是見文知意)，CircuitBreaker的配置類，在實際專案中除了全域性的配置，有些場景需要我們自定義一些CircuitBreaker的配置，這個時候就需要用到Circuitreakeronfig，Circuitreakeronfig全部屬性如下表

配置屬性 預設值 描述
failureRateThreshold 50 以百分比形式配置失敗率閾值。
當故障率等於或大於閾值時，斷路器轉換為斷開並開始短路呼叫。
slowCallRateThreshold 100 以百分比配置閾值。當呼叫持續時間大於 或等於閾值時，斷路器將呼叫視為慢速呼叫。當慢速呼叫的百分比等於或大於閾值時，斷路器轉換為斷開並開始短路呼叫。slowCallDurationThreshold
slowCallDurationThreshold 60000 [毫秒] 配置持續時間閾值，該數值的呼叫速度緩慢並增加呼叫的速度。
permittedNumberOfCalls
InHalfOpenState		 10 配置半開時允許的呼叫數量。
maxWaitDurationInHalfOpenState 0 [毫秒] 配置最大等待持續時間，控制斷路器在切換到開啟狀態之前可以保持在半開狀態的最長時間。
值 0 表示斷路器將在 HalfOpen 狀態無限等待，直到所有允許的呼叫都完成。
slidingWindowType COUNT_BASED 配置用於記錄CircuitBreaker關閉時呼叫結果的滑動視窗的型別。
滑動視窗可以是基於計數的，也可以是基於時間的。
如果滑動視窗為 COUNT_BASED，則記錄並彙總最後一次呼叫。 如果滑動視窗是 TIME_BASED，則記錄和聚合最後幾秒的呼叫。slidingWindowSize
slidingWindowSize
slidingWindowSize 100 配置用於記錄關閉時呼叫視窗的視窗大小。
minimumNumberOfCalls 100 配置在斷路器計算錯誤率或慢速呼叫率之前所需的最小呼叫數（每個滑動視窗週期）。
例如，如果minimumNumberOfCalls為10，則必須至少記錄10個呼叫，然後才能計算失敗率。
如果僅記錄了9個呼叫，則即使有9個呼叫都失敗，斷路器也不會轉換為開啟狀態。
waitDurationInOpenState 60000 [毫秒] 半從開啟轉換到開啟之前應等待的時間。
automaticTransition
FromOpenToHalfOpenEnabled		 FALSE 如果設定為 true，則意味著 CircuitBreaker 將自動從開啟狀態轉換為半開啟狀態，並且不需要呼叫來觸發轉換。建立一個執行緒來監視 CircuitBreakers 的所有例項，一旦 waitDurationInOpenState 通過，將它們轉換為 HALF_OPEN。然而，如果設定為 false，則僅在進行呼叫時才會轉換到 HALF_OPEN，即使在傳遞了 waitDurationInOpenState 之後也是如此。這裡的優點是沒有執行緒監視所有斷路器的狀態。
recordExceptions empty 記錄為失敗並因此增加失敗率的異常列表。
任何匹配或從列表之一繼承的異常都算作失敗，除非通過。 如果您指定異常列表，則所有其他異常都算作成功，除非它們被明確忽略。ignoreExceptions
ignoreExceptions empty 被忽略且既不計為失敗也不計為成功的異常列表。
即使異常是。recordExceptions
recordFailurePredicate throwable -> true
預設情況下，所有異常都記錄為失敗。		 一個自定義Predicate，用於評估是否應將異常記錄為失敗。
如果異常應算作失敗，則謂詞必須返回 true。如果異常
應算作成功，則謂詞必須返回 false，除非異常被 顯式忽略。ignoreExceptions
ignoreExceptions throwable -> false
預設情況下不會忽略任何異常。		 一個自定義Predicate，用於評估是否應忽略異常並且既不視為失敗也不成功。
如果應忽略異常，謂詞必須返回 true。
如果異常應算作失敗，則謂詞必須返回 false。

1.1.2 CircuitBreakerRegistry

 CircuitBreakerRegistry是CircuitBreaker的註冊器，其有一個唯一的實現類InMemoryCircuitBreakerRegistry，核心方法如下

// 根據name返回CircuitBreaker或返回預設的CircuitBreaker
// 下面的幾個過載的方法，也是一樣的邏輯，有就直接返回，沒有就建立後返回
public CircuitBreaker circuitBreaker(String name) 

public CircuitBreaker circuitBreaker(String name, io.vavr.collection.Map<String, String> tags) 

public CircuitBreaker circuitBreaker(String name, CircuitBreakerConfig config) {

public CircuitBreaker circuitBreaker(String name, CircuitBreakerConfig config, io.vavr.collection.Map<String, String> tags) 

public CircuitBreaker circuitBreaker(String name, String configName)

public CircuitBreaker circuitBreaker(String name, String configName,  io.vavr.collection.Map<String, String> tags) {

public CircuitBreaker circuitBreaker(String name, Supplier<CircuitBreakerConfig> circuitBreakerConfigSupplier) 

public CircuitBreaker circuitBreaker(String name, Supplier<CircuitBreakerConfig> circuitBreakerConfigSupplier, io.vavr.collection.Map<String, String> tags)

1.1.3 CircuitBreaker

現在到了我們的核心介面CircuitBreaker,下面的靜態方法有20多個，在這我就列幾個常用的方法，其它方法可以看原始碼註釋的描述

// 返回一個被CircuitBreaker包裝的 CheckedFunction0.
// CheckedFunction0 是由vavr封裝的類似java8中Supplier的函式
static <T> CheckedFunction0<T> decorateCheckedSupplier(CircuitBreaker circuitBreaker, CheckedFunction0<T> supplier) 

// 返回一個被CircuitBreaker包裝的 CheckedRunnable.
//CheckedRunnable 是由avr封裝的 Runnable 
static CheckedRunnable decorateCheckedRunnable(CircuitBreaker circuitBreaker,  CheckedRunnable runnable) 

// 返回一個被CircuitBreaker包裝的 Callable.
static <T> Callable<T> decorateCallable(CircuitBreaker circuitBreaker, Callable<T> callable) 

// 返回一個被CircuitBreaker包裝的 Supplier.
static <T> Supplier<T> decorateSupplier(CircuitBreaker circuitBreaker, Supplier<T> supplier) 

 // 返回一個可以retry的 Supplierstatic <T> Supplier<Try<T>> decorateTrySupplier(CircuitBreaker circuitBreaker,  Supplier<Try<T>> supplier)

// 返回一個被CircuitBreaker包裝的 Consumer.
static <T> Consumer<T> decorateConsumer(CircuitBreaker circuitBreaker, Consumer<T> consumer) 
    
// 返回一個被CircuitBreaker包裝的 CheckedConsumer.
// CheckedConsumer 是由avr封裝的CheckedConsumer
static <T> CheckedConsumer<T> decorateCheckedConsumer(CircuitBreaker circuitBreaker, CheckedConsumer<T> consumer) 

// 返回一個被CircuitBreaker包裝的 Runnable.
static Runnable decorateRunnable(CircuitBreaker circuitBreaker, Runnable runnable) 

// 返回一個被CircuitBreaker包裝的 Function.
static <T, R> Function<T, R> decorateFunction(CircuitBreaker circuitBreaker, Function<T, R> function)

// 返回一個被CircuitBreaker包裝的 CheckedFunction1.
// CheckedFunction1是由avr封裝的Function
static <T, R> CheckedFunction1<T, R> decorateCheckedFunction(CircuitBreaker circuitBreaker,  CheckedFunction1<T, R> function) 

// 返回一個被CircuitBreaker包裝的 Supplier<Future>.
static <T> Supplier<Future<T>> decorateFuture(CircuitBreaker circuitBreaker,   Supplier<Future<T>> supplier)

從上面列舉的常用方法看到有很多好像有重複的方法，CircuitBreaker有返回封裝Supplier, Consumer, Function, Runnable的方法，然後還有一個與之對應的返回封裝CheckedSupplier, CheckedConsumer, CheckedFunction, CheckedRunnable的方法。 為什麼有兩套實現呢？resilience4j，這個專案是基於Java 8開發的，但是java8受限於 Java 標準庫的通用性要求和二進位制檔案大小，Java 標準庫對函數語言程式設計的 API 支援相對比較有限。函式的宣告只提供了 Function 和 BiFunction 兩種，流上所支援的操作的數量也較少。基於這些原因，需要vavr 來更好得使用Java 8進行函式式開發。

簡單看下方法decorateCheckedSupplier(CircuitBreaker circuitBreaker, CheckedFunction0<T> supplier)

    static <T> CheckedFunction0<T> decorateCheckedSupplier(CircuitBreaker circuitBreaker,
        CheckedFunction0<T> supplier) {
        return () -> {
　　　　       // 申請執行函式方法supplier.apply()的許可
             // 具體邏輯在CircuiBreakerStateMachine中的CircuitBreakerState中實現　　　　
            circuitBreaker.acquirePermission();
            final long start = circuitBreaker.getCurrentTimestamp();
            try {
　　　　　　　　　　// 執行目標方法
                T result = supplier.apply();
                long duration = circuitBreaker.getCurrentTimestamp() - start;
                //目標方法執行完呼叫onResult(),check result最終呼叫onSuccess() 
                circuitBreaker.onResult(duration, circuitBreaker.getTimestampUnit(), result);
                return result;
            } catch (Exception exception) {
                // Do not handle java.lang.Error
                long duration = circuitBreaker.getCurrentTimestamp() - start;
                // 如果出現異常就呼叫onError(),執行onError策略的邏輯
                circuitBreaker.onError(duration, circuitBreaker.getTimestampUnit(), exception);
                throw exception;
            }
        };
    }

 大體流程如下圖

關於vavr的詳情可以檢視官網文件：https://docs.vavr.io/

CircuitBreaker唯一的實現類CircuitBreakerStateMachine

CircuitBreakerStateMachine是一個有線狀態的狀態機。斷路器管理後端系統的狀態。斷路器通過具有五種狀態的有限狀態機實現：CLOSED、OPEN、HALF_OPEN、DISABLED 和 FORCED_OPEN。 CircuitBreakerStateMachine可以做到這些狀態的轉換,比如下面的幾個方法

@Override
 public void transitionToDisabledState() {
     stateTransition(DISABLED, currentState -> new DisabledState());
 }

 @Override
 public void transitionToMetricsOnlyState() {
     stateTransition(METRICS_ONLY, currentState -> new MetricsOnlyState());
 }

 @Override
 public void transitionToForcedOpenState() {
     stateTransition(FORCED_OPEN,
         currentState -> new ForcedOpenState(currentState.attempts() + 1));
 }

 @Override
 public void transitionToClosedState() {
     stateTransition(CLOSED, currentState -> new ClosedState());
 }

 @Override
 public void transitionToOpenState() {
     stateTransition(OPEN,
         currentState -> new OpenState(currentState.attempts() + 1, currentState.getMetrics()));
 }

 @Override
 public void transitionToHalfOpenState() {
     stateTransition(HALF_OPEN, currentState -> new HalfOpenState(currentState.attempts()));
 }

這些狀態的流轉是通過釋出事件來完成的，可以看下面都是CircuitBreakerStateMachine的事件

private void publishResetEvent() {
     final CircuitBreakerOnResetEvent event = new CircuitBreakerOnResetEvent(name);
    publishEventIfPossible(event);
}

private void publishCallNotPermittedEvent() {
    final CircuitBreakerOnCallNotPermittedEvent event = new CircuitBreakerOnCallNotPermittedEvent(
        name);
    publishEventIfPossible(event);
}

private void publishSuccessEvent(final long duration, TimeUnit durationUnit) {
    final CircuitBreakerOnSuccessEvent event = new CircuitBreakerOnSuccessEvent(name,
        Duration.ofNanos(durationUnit.toNanos(duration)));
    publishEventIfPossible(event);
}

private void publishCircuitErrorEvent(final String name, final long duration,
    TimeUnit durationUnit, final Throwable throwable) {
    final CircuitBreakerOnErrorEvent event = new CircuitBreakerOnErrorEvent(name,
        Duration.ofNanos(durationUnit.toNanos(duration)), throwable);
    publishEventIfPossible(event);
}

private void publishCircuitIgnoredErrorEvent(String name, long duration, TimeUnit durationUnit,
    Throwable throwable) {
    final CircuitBreakerOnIgnoredErrorEvent event = new CircuitBreakerOnIgnoredErrorEvent(name,
        Duration.ofNanos(durationUnit.toNanos(duration)), throwable);
    publishEventIfPossible(event);
}

private void publishCircuitFailureRateExceededEvent(String name, float failureRate) {
    final CircuitBreakerOnFailureRateExceededEvent event = new CircuitBreakerOnFailureRateExceededEvent(name,
        failureRate);
    publishEventIfPossible(event);
}

private void publishCircuitSlowCallRateExceededEvent(String name, float slowCallRate) {
    final CircuitBreakerOnSlowCallRateExceededEvent event = new CircuitBreakerOnSlowCallRateExceededEvent(name,
        slowCallRate);
    publishEventIfPossible(event);
}

private void publishCircuitThresholdsExceededEvent(Result result, CircuitBreakerMetrics metrics) {
    if (Result.hasFailureRateExceededThreshold(result)) {
        publishCircuitFailureRateExceededEvent(getName(), metrics.getFailureRate());
    }
    if (Result.hasSlowCallRateExceededThreshold(result)) {
        publishCircuitSlowCallRateExceededEvent(getName(), metrics.getSlowCallRate());
    }
}

 1.1.4 CircuitBreaker Demo

引入測試元件spring-cloud-starter-contract-stub-runner

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-contract-stub-runner</artifactId>
    <scope>test</scope>
</dependency>

Resilience4jTestHelper測試輔助類

    /**
     * get the CircuitBreaker status and metrics
     *
     * @param prefixName
     * @param circuitBreaker
     * @return circuitBreaker state
     */
    public static String getCircuitBreakerStatus(String prefixName, CircuitBreaker circuitBreaker) {

        CircuitBreaker.Metrics metrics = circuitBreaker.getMetrics();
        float failureRate = metrics.getFailureRate();
        int failedCalls = metrics.getNumberOfFailedCalls();
        int successfulCalls = metrics.getNumberOfSuccessfulCalls();
        long notPermittedCalls = metrics.getNumberOfNotPermittedCalls();
        int bufferedCalls = metrics.getNumberOfBufferedCalls();
        float slowCallRate = metrics.getSlowCallRate();
        int slowCalls = metrics.getNumberOfSlowCalls();
        int slowFailedCalls = metrics.getNumberOfSlowFailedCalls();
        int slowSuccessfulCalls = metrics.getNumberOfSlowSuccessfulCalls();

        log.info(prefixName + " state=" + circuitBreaker.getState() + " , metrics[ failureRate=" + failureRate +
                ", failedCalls=" + failedCalls +
                ", successCalls=" + successfulCalls +
                ", notPermittedCalls=" + notPermittedCalls +
                ", bufferedCalls=" + bufferedCalls +
                ", \n\tslowCallRate=" + slowCallRate +
                ", slowCalls=" + slowCalls +
                ", slowFailedCalls=" + slowFailedCalls +
                ", slowSuccessfulCalls=" + slowSuccessfulCalls +
                " ]"
        );
        log.info(prefixName + " circuitBreaker tags:{}", circuitBreaker.getTags());
        return circuitBreaker.getState().name();
    }

    public static void circuitBreakerEventListener(CircuitBreaker circuitBreaker) {
        circuitBreaker.getEventPublisher()
                .onSuccess(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()))
                .onError(event -> {
                    log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName());
                    Throwable throwable = event.getThrowable();
                    if (throwable instanceof TimeoutException) {
                        // TODO record to slow call
                    }
                })
                .onIgnoredError(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()))
                .onReset(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()))
                .onStateTransition(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()))
                .onCallNotPermitted(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()))
                .onFailureRateExceeded(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()))
                .onSlowCallRateExceeded(event -> log.info("---------- CircuitBreakerEvent:{}  CircuitBreakerName:{}", event.getEventType(), event.getCircuitBreakerName()));
    }

Resilience4jTest測試類

    @Rule
    public WireMockRule wireMockRule = new WireMockRule(wireMockConfig().port(8080));

    private WebTestClient testClient;

    private CircuitBreakerRegistry circuitBreakerRegistry;private CircuitBreaker circuitBreaker;

    private CircuitBreaker circuitBreakerWithTags;

    private CircuitBreakerConfig circuitBreakerConfig;private String PATH_200 = "/api/pancake/v1/yee/query";

    private String PATH_400 = "/api/hk/card/v1/er/query";

    private String PATH_408 = "/api/pancake/v1/coin/query";

    private String PATH_500 = "/api/hk/card/v1/card/query";

    @Before
    public void setup() {
        HttpClient httpClient = HttpClient.create().wiretap(true);
        testClient = WebTestClient.bindToServer(new ReactorClientHttpConnector(httpClient))
                .baseUrl("http://localhost:8080")
                .responseTimeout(Duration.ofDays(1))
                .build();

        circuitBreakerRegistry = new InMemoryCircuitBreakerRegistry();
        circuitBreakerConfig = CircuitBreakerConfig
                .custom()
                .failureRateThreshold(70)
                .slowCallRateThreshold(90)
                .slowCallDurationThreshold(Duration.ofMillis(1000 * 1))
                .minimumNumberOfCalls(10)
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
                .slidingWindowSize(10)
                .build();
        circuitBreaker = circuitBreakerRegistry.circuitBreaker("resilience4jTest", circuitBreakerConfig);
        Resilience4jTestHelper.circuitBreakerEventListener(circuitBreaker);

        stubFor(post(urlMatching(PATH_200))
                .willReturn(okJson("{}")));

        stubFor(post(urlMatching(PATH_400))
                .willReturn(badRequest()));

        stubFor(post(urlMatching(PATH_408))
                .willReturn(okJson("{\"message\":\"time out\"}").withFixedDelay(1000 * 2)));

        stubFor(post(urlMatching(PATH_500))
                .willReturn(serverError()));
    }


    @Test
    public void When_Test_CircuitBreaker_Expect_Close() {
        AtomicInteger count = new AtomicInteger();
        for (int i = 0; i < 10; i++) {
            Resilience4jTestHelper.recordResponseToCircuitBreaker(circuitBreaker, testClient, PATH_200);
            Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> end call " + count.incrementAndGet(), circuitBreaker);
        }
        assertEquals(CircuitBreaker.State.CLOSED.name(), circuitBreaker.getState().name());
    }

    @Test
    public void When_CircuitBreaker_Expect_Open() {
        circuitBreakerWithTags = circuitBreakerRegistry.circuitBreaker("circuitBreakerWithTags", circuitBreakerConfig, HashMap.of("resilience4jTest", "When_CircuitBreaker_Expect_Open"));
        Resilience4jTestHelper.circuitBreakerEventListener(circuitBreakerWithTags);

        AtomicInteger count = new AtomicInteger();
        for (int i = 0; i < 10; i++) {
            Resilience4jTestHelper.recordResponseToCircuitBreaker(circuitBreakerWithTags, testClient, PATH_400);
            Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> end call " + count.incrementAndGet(), circuitBreakerWithTags);
        }
        assertEquals(CircuitBreaker.State.OPEN.name(), circuitBreakerWithTags.getState().name());
    }

    @Test
    public void When_Test_CircuitBreaker_Expect_SlowCall() throws Throwable {
        AtomicInteger count = new AtomicInteger();
        for (int i = 0; i < 10; i++) {
            circuitBreaker.executeCheckedSupplier(() -> {
                Resilience4jTestHelper.recordSlowCallResponseToCircuitBreaker(circuitBreaker, testClient, PATH_408);
                return null;
            });
            Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> end call " + count.incrementAndGet(), circuitBreaker);
        }
        assertEquals(CircuitBreaker.State.OPEN.name(), circuitBreaker.getState().name());
    }

    @Test
    public void When_CircuitBreaker_Expect_Fallback() {
        AtomicInteger count = new AtomicInteger();
        for (int i = 0; i < 20; i++) {
            String path = PATH_500;
            CheckedFunction0<String> response =
                    circuitBreaker.decorateCheckedSupplier(() -> Resilience4jTestHelper.responseToCircuitBreaker(circuitBreaker, testClient, path));
            Try<String> result = Try.of(response).map(val -> {
                Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> call success " + count.incrementAndGet(), circuitBreaker);
                return val;
            }).recover(CallNotPermittedException.class, throwable -> {
                Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> open CircuitBreaker " + count.incrementAndGet(), circuitBreaker);
                return "hit CallNotPermittedException";
            }).recover(throwable -> {
                Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> call fallback " + count.incrementAndGet(), circuitBreaker);
                return "hit fallback";
            });
            log.info(">>>>>>>>>> result:{}", result.get());
            if (count.get() > 10) {
                assertEquals("hit CallNotPermittedException", result.get());
            }
        }
    }

1.2 Bulkhead

Bulkhead提供了兩種隔板模式的實現，可用於限制併發執行的數量

1. 使用訊號量  SemaphoreBulkhead
2. 使用有界佇列和固定執行緒池  FixedThreadPoolBulkhead

其中執行緒池的方式屬於資源佔用型，在這個不做討論，如果感興趣可以去看看官方的樣例

1.2.1 BulkheadConfig 

BulkheadConfig是Bulkhead的配置類，使用BulkheadConfig配置類，自定義Blukhead配置。配置類BulkheadConfig有以下屬性

 

配置屬性 預設值 描述
maxConcurrentCalls 25 隔板允許的最大並行執行量
maxWaitDuration 0 嘗試進入飽和的Bulkhead時應阻塞執行緒的最長時間

1.2.2 BulkheadRegistry

和CircuitBreaker模組一樣，BulkheadRegistry提供了一個記憶體中的實現類InMemoryBulkheadRegistry，可以使用它來管理（建立和獲取）Bulkhead例項。

1.2.3 Bulkhead

Bulkhead介面的靜態方法和CircuitBreaker方法命名類似，如下下面的decorateCheckedSupplier方法

    static <T> CheckedFunction0<T> decorateCheckedSupplier(Bulkhead bulkhead,
        CheckedFunction0<T> supplier) {
        return () -> {
            bulkhead.acquirePermission();
            try {
                return supplier.apply();
            } finally {
                bulkhead.onComplete();
            }
        };
    }

Bulkhead的靜態方法，中主要靠bulkhead.acquirePermission()和bulkhead.tryAcquirePermission()申請執行許可權，靠bulkhead.onComplete()是釋放執行許可權，當然還有一個方法bulkhead.releasePermission() 也可以釋放執行許可權，兩者區別就是bulkhead.onComplete()多了一個觸發執行完成的事件publishBulkheadEvent(() -> new BulkheadOnCallFinishedEvent(name))。

如果我們不想用Bulkhead自帶的靜態方法也是可以的，比如我下面的demo, 僅僅使用bulkhead.tryAcquirePermission()和bulkhead.onComplete()，就可以模擬一個服務過載的場景

1.2.4 Bulkhead Demo

Resilience4jTestHelper測試輔助類

    public static String responseToBulkhead(Bulkhead bulkhead, WebTestClient testClient, String path) {
        WebTestClient.ResponseSpec responseSpec = testClient.post().uri(path).exchange();
        if (bulkhead.getMetrics().getAvailableConcurrentCalls() < 1) {
            throw BulkheadFullException.createBulkheadFullException(bulkhead);
        }
        try {
            responseSpec.expectStatus().is4xxClientError();
            throw new RuntimeException("<<<<< hit 4XX >>>>>");
        } catch (Throwable error) {
        }

        try {
            responseSpec.expectStatus().is5xxServerError();
            throw new RuntimeException("<<<<< hit 5XX >>>>>");
        } catch (Throwable error) {
        }
        responseSpec.expectStatus().is2xxSuccessful();
        return "hit 200";
    }

    /**
     * get the Bulkhead status and metrics
     * * @param prefixName
     *
     * @param bulkhead
     */
    public static void getBulkheadStatus(String prefixName, Bulkhead bulkhead) {
        Bulkhead.Metrics metrics = bulkhead.getMetrics();
        int availableCalls = metrics.getAvailableConcurrentCalls();
        int maxCalls = metrics.getMaxAllowedConcurrentCalls();
        log.info(prefixName + "bulkhead metrics[ availableCalls=" + availableCalls +
                ", maxCalls=" + maxCalls + " ],tags=" + bulkhead.getTags());
    }

    public static void bulkheadEventListener(Bulkhead bulkhead) {
        bulkhead.getEventPublisher()
                .onCallRejected(event -> log.info("---------- BulkheadEvent:{}  BulkheadName:{}", event.getEventType(), event.getBulkheadName()))
                .onCallFinished(event -> log.info("---------- BulkheadEvent:{}  BulkheadName:{}", event.getEventType(), event.getBulkheadName()));
    }


    static int[] container = new int[100];
    // 模擬一定概率的不釋放資源
    public static boolean releasePermission() {
        if (container[0] != 1) {
            for (int i = 0; i < 70; i++) {
                container[i] = 1;
            }
            for (int i = 70; i < 100; i++) {
                container[i] = 0;
            }
        }
        int index = (int) (Math.random() * 100);
        return container[index] == 1;
    }

Resilience4jTest測試類

private BulkheadRegistry bulkheadRegistry;private String PATH_200 = "/api/pancake/v1/yee/query";

    private String PATH_400 = "/api/hk/card/v1/er/query";

    private String PATH_408 = "/api/pancake/v1/coin/query";

    private String PATH_500 = "/api/hk/card/v1/card/query";


    @Before
    public void setup() {
        HttpClient httpClient = HttpClient.create().wiretap(true);
        testClient = WebTestClient.bindToServer(new ReactorClientHttpConnector(httpClient))
                .baseUrl("http://localhost:8080")
                .responseTimeout(Duration.ofDays(1))
                .build();

        bulkheadRegistry = new InMemoryBulkheadRegistry();

        stubFor(post(urlMatching(PATH_200))
                .willReturn(okJson("{}")));

        stubFor(post(urlMatching(PATH_400))
                .willReturn(badRequest()));

        stubFor(post(urlMatching(PATH_408))
                .willReturn(okJson("{\"message\":\"time out\"}").withFixedDelay(1000 * 2)));

        stubFor(post(urlMatching(PATH_500))
                .willReturn(serverError()));
    }


    @Test
    public void When_Test_CircuitBreaker_With_Bulkhead_Expect_Hit_BulkheadFullException() {
        AtomicInteger count = new AtomicInteger();
        ExecutorService executorService = Executors.newFixedThreadPool(50);
        Bulkhead bulkhead1 = bulkheadRegistry.bulkhead("bulkhead1",
                BulkheadConfig
                        .custom()
                        .maxConcurrentCalls(20)
                        .maxWaitDuration(Duration.ofMillis(100))
                        .build());

        Resilience4jTestHelper.bulkheadEventListener(bulkhead1);
        for (int i = 0; i < 100; i++) {
            if (bulkhead1.tryAcquirePermission()) {
                log.info(">>>>>>>>>> acquire permission {}", count.incrementAndGet());
                Future<String> futureStr = executorService.submit(() -> Resilience4jTestHelper.responseToBulkhead(bulkhead1, testClient, PATH_200));
                Try.of(futureStr::get).andThen(val -> log.info(">>>>>>>>>> success {}: {}", count.get(), val)).recover(throwable -> {
                    if (throwable instanceof ExecutionException) {
                        Throwable cause = (ExecutionException) throwable.getCause();
                        if (cause instanceof BulkheadFullException) {
                            log.info(">>>>>>>>>> BulkheadFullException {}: {}", count.get(), throwable.getMessage());
                        } else {
                            log.info(">>>>>>>>>> ExecutionException {}: {}", count.get(), throwable.getMessage());
                        }
                    }
                    return "hit ExecutionException";
                });
                if (releasePermission()) {
                    bulkhead1.onComplete();
                    log.info("---------- release permission");
                }
                Resilience4jTestHelper.getBulkheadStatus(")))))))))) ", bulkhead1);
            } else {
                log.info(">>>>>>>>>> tryAcquirePermission false {}", count.incrementAndGet());
                continue;
            }
        }
        executorService.shutdown();
    }

1.3 RateLimiter

Resilience4j提供了一個RateLimiter作為限速器，Ratelimiter限制了服務被呼叫的次數，每隔一段時間重置該次數，服務在超出等待時間之後返回異常或者fallback方法。跟CircuitBreaker的程式碼結構一樣，核心類有RateLimiterRegistry和其實現類InMemoryRateLimiterRegistry，RateLimiterConfig 還有RateLimiter

其中RateLimiterConfig的屬性如下表

配置屬性 預設值 描述
timeoutDuration 5 [s] 執行緒等待許可權的預設等待時間
limitRefreshPeriod 500 [ns] 限制重新整理的週期。在每個週期之後，速率限制器將其許可權計數設定回 limitForPeriod 值
limitForPeriod 50 一個限制重新整理期間可用的許可權數

所以如果你想限制某個方法的呼叫率不高於1000 req/s，可以做如下配置

RateLimiterConfig.custom()
     .timeoutDuration(Duration.ofMillis(1000*5))
     .limitRefreshPeriod(Duration.ofSeconds(1))
     .limitForPeriod(1000)
     .build());

1.3.1 RateLimiter Demo

Resilience4jTestHelper測試輔助類

    /**
     * get the RateLimiter status and metrics
     * * @param prefixName
     *
     * @param rateLimiter
     */
    public static void getRateLimiterStatus(String prefixName, RateLimiter rateLimiter) {
        RateLimiter.Metrics metrics = rateLimiter.getMetrics();
        int availablePermissions = metrics.getAvailablePermissions();
        int waitingThreads = metrics.getNumberOfWaitingThreads();
        log.info(prefixName + "rateLimiter metrics[ availablePermissions=" + availablePermissions +
                ", waitingThreads=" + waitingThreads + " ]"
        );
    }

    public static void rateLimiterEventListener(RateLimiter rateLimiter) {
        rateLimiter.getEventPublisher()
                .onSuccess(event -> log.info("---------- rateLimiter success:{}", event))
                .onFailure(event -> log.info("---------- rateLimiter failure:{}", event));
    }

    public static String responseToRateLimiter(RateLimiter rateLimiter,WebTestClient testClient, String path) {
        WebTestClient.ResponseSpec responseSpec = testClient.post().uri(path).exchange();
        try {
            responseSpec.expectStatus().is4xxClientError();
            rateLimiter.onError(new RuntimeException("<<<<< hit 4XX >>>>>"));
            throw new RuntimeException("<<<<< hit 4XX >>>>>");
        } catch (Throwable error) {
        }

        try {
            responseSpec.expectStatus().is5xxServerError();
            rateLimiter.onError(new RuntimeException("<<<<< hit 5XX >>>>>"));
            throw new RuntimeException("<<<<< hit 5XX >>>>>");
        } catch (Throwable error) {
        }
        responseSpec.expectStatus().is2xxSuccessful();
        rateLimiter.onSuccess();
        return "hit 200";
    }

Resilience4jTest測試類

    private RateLimiterRegistry rateLimiterRegistry;
    private RateLimiter rateLimiter;
    
        rateLimiterRegistry = new InMemoryRateLimiterRegistry();
        rateLimiter = rateLimiterRegistry.rateLimiter("resilience4jTest",
                RateLimiterConfig
                        .custom()
                        .timeoutDuration(Duration.ofMillis(100))
                        .limitRefreshPeriod(Duration.ofSeconds(1))
                        .limitForPeriod(20)
                        .build());
        Resilience4jTestHelper.rateLimiterEventListener(rateLimiter);
        
    @Test
    public void When_Test_CircuitBreaker_Expect_Hit_RateLimiter() throws Exception {
        AtomicInteger count = new AtomicInteger();
        ExecutorService executorService = Executors.newFixedThreadPool(50);
        String path = expectError() ? PATH_500 : PATH_200;
        for (int i = 0; i < 100; i++) {
            Future<String> futureStr = executorService.submit(() -> Resilience4jTestHelper.responseToRateLimiter(rateLimiter, testClient, path));
            try {
                Future<String> stringFuture = rateLimiter.executeCallable(() -> futureStr);
                Try.of(stringFuture::get).andThen(val -> {
                    log.info(">>>>>>>>>> success {}: {}", count.incrementAndGet(), val);
                }).recover(throwable -> {
                    log.info(">>>>>>>>>> exception {}: {}", count.incrementAndGet(), throwable.getMessage());
                    return "hit fallback";
                });
                Resilience4jTestHelper.getRateLimiterStatus(")))))))))) ", rateLimiter);
            } catch (RequestNotPermitted exception){
                assertEquals("RateLimiter 'resilience4jTest' does not permit further calls" , exception.getMessage());
            }
        }
        executorService.shutdown();
    }

1.4 Retry

Retry在服務呼叫返回失敗時提供了額外嘗試呼叫的功能，其中RetryConfig的屬性如下表

配置屬性 預設值 描述
maxAttempts 3 最大嘗試次數（包括首次呼叫作為第一次嘗試）
waitDuration 500 [ms] 兩次重試的時間間隔
intervalFunction numOfAttempts -> waitDuration 自定義的IntervalFunction，可以根據當前嘗試的次數動態的修改重試的時間間隔
intervalBiFunction (numOfAttempts, Either<throwable, result>) -> waitDuration 根據嘗試次數和結果或異常修改失敗後等待間隔的函式。與 intervalFunction 一起使用時會丟擲 IllegalStateException。
retryOnResultPredicate result -> false 自定義的Predicate，根據服務返回的結果判斷是否應該重試。如果需要重試Predicate應返回true，否則返回false
retryExceptionPredicate throwable -> true 自定義的Predicate，根據服務返回的異常判斷是否應該重試。如果需要重試Predicate應返回true，否則返回false
retryExceptions empty 異常列表，遇到列表中的異常或其子類則重試
注意：如果您使用 Checked Exceptions，則必須使用 CheckedSupplier
ignoreExceptions empty 異常列表，遇到列表中的異常或其子類則不重試。此引數支援子型別。
failAfterMaxRetries false 當重試達到配置的 maxAttempts 並且結果仍未通過 retryOnResultPredicate 時啟用或禁用丟擲 MaxRetriesExceededException 的布林值

1.4.1 Retry Demo

Resilience4jTestHelper測試輔助類

    /**
     * get the Retry status and metrics
     * * @param prefixName
     *
     * @param retry
     */
    public static void getRetryStatus(String prefixName, Retry retry) {

        Retry.Metrics metrics = retry.getMetrics();
        long successfulCallsWithRetryAttempt = metrics.getNumberOfSuccessfulCallsWithRetryAttempt();
        long successfulCallsWithoutRetryAttempt = metrics.getNumberOfSuccessfulCallsWithoutRetryAttempt();
        long failedCallsWithRetryAttempt = metrics.getNumberOfFailedCallsWithRetryAttempt();
        long failedCallsWithoutRetryAttempt = metrics.getNumberOfFailedCallsWithoutRetryAttempt();

        log.info(prefixName + " -> retry metrics[ successfulCallsWithRetry=" + successfulCallsWithRetryAttempt +
                ", successfulCallsWithoutRetry=" + successfulCallsWithoutRetryAttempt +
                ", failedCallsWithRetry=" + failedCallsWithRetryAttempt +
                ", failedCallsWithoutRetry=" + failedCallsWithoutRetryAttempt +
                " ]"
        );
    }

    public static void retryEventListener(Retry retry) {
        retry.getEventPublisher()
                .onSuccess(event -> log.info("))))))))))) retry service success:{}", event))
                .onError(event -> {
                    log.info("))))))))))) retry service failed:{}", event);
                    Throwable exception = event.getLastThrowable();
                    if (exception instanceof TimeoutException) {
                        // TODO
                    }
                })
                .onIgnoredError(event -> log.info("))))))))))) retry service failed and ignore:{}", event))
                .onRetry(event -> log.info("))))))))))) retry call service: {}", event.getNumberOfRetryAttempts()));

    }
    
    public static String responseToRetry(Retry retry, WebTestClient testClient, String path) {
        WebTestClient.ResponseSpec responseSpec = testClient.post().uri(path).exchange();
        try {
            responseSpec.expectStatus().is4xxClientError();
            return "HIT_ERROR_4XX";
        } catch (Throwable error) {
        }
        try {
            responseSpec.expectStatus().is5xxServerError();
            return "HIT_ERROR_5XX";
        } catch (Throwable error) {
        }
        responseSpec.expectStatus().is2xxSuccessful();
        return "HIT_200";
    }

Resilience4jTest測試類

    private RetryRegistry retryRegistry;
    private Retry retry;
    retryRegistry = new InMemoryRetryRegistry();
    retry = retryRegistry.retry("resilience4jTest",
            RetryConfig
                    .custom()
                    .maxAttempts(5)
                    .waitDuration(Duration.ofMillis(500))
                    .retryOnResult(val -> val.toString().contains("HIT_ERROR_"))
 //                     .retryExceptions(RuntimeException.class)
                    .build());

    Resilience4jTestHelper.retryEventListener(retry);
   @Test
    public void When_Test_CircuitBreaker_Expect_Retry() {
        AtomicInteger count = new AtomicInteger();
        for (int i = 0; i < 30; i++) {
            String path = expectError() ? PATH_200 : PATH_400;
            Callable<String> response = Retry.decorateCallable(retry, () -> Resilience4jTestHelper.responseToRetry(retry, testClient, path));
            Try.of(response::call).andThen(val -> log.info(">>>>>>>>>> result {}: {}", count.incrementAndGet(), val));
            Resilience4jTestHelper.getRetryStatus("))))))))))", retry);
        }
    }

1.5 TimeLimiter

TimeLImiter超時控制，和CircuitBreaker的slowCall相似，只是CircuitBreaker的slowCall觸發了超時只是將超時記錄在Metrics中不會丟擲異常，而TimeLimiter觸發了超時會直接丟擲異常。

而且TimeLimiter配置類很簡單

配置屬性 預設值 描述
timeoutDuration 5 [s] 超時時間，預設1s
cancelRunningFuture TRUE 當觸發超時時是否取消執行中的Future

1.5.1 TimeLimiter Demo

Resilience4jTestHelper測試輔助類

    public static void timeLimiterEventListener(TimeLimiter timeLimiter) {
        timeLimiter.getEventPublisher()
                .onSuccess(event -> log.info("---------- timeLimiter success:{}", event))
                .onError(event -> log.info("---------- timeLimiter error:{}", event))
                .onTimeout(event -> log.info("---------- rateLimiter timeout:{}", event));
    }

    public static String responseToTimeLimiter(TimeLimiter timeLimiter, CircuitBreaker circuitBreaker, WebTestClient testClient, String path) {
        WebTestClient.ResponseSpec responseSpec = testClient.post().uri(path).exchange();
        try {
            responseSpec.expectStatus().is4xxClientError();
            circuitBreaker.onError(0, TimeUnit.MILLISECONDS, new RuntimeException("<<<<< hit 4XX >>>>>"));
            timeLimiter.onError(new RuntimeException("<<<<< hit 4XX >>>>>"));
            throw new RuntimeException("<<<<< hit 4XX >>>>>");
        } catch (Throwable error) {
        }

        try {
            responseSpec.expectStatus().is5xxServerError();
            circuitBreaker.onError(0, TimeUnit.MILLISECONDS, new RuntimeException("<<<<< hit 5XX >>>>>"));
            timeLimiter.onError(new RuntimeException("<<<<< hit 5XX >>>>>"));
            throw new RuntimeException("<<<<< hit 5XX >>>>>");
        } catch (Throwable error) {
        }
        responseSpec.expectStatus().is2xxSuccessful();
        timeLimiter.onSuccess();
        return "hit 200";
    }

Resilience4jTest測試類

    private TimeLimiterRegistry timeLimiterRegistry;
    private TimeLimiter timeLimiter;
    
    timeLimiterRegistry = new InMemoryTimeLimiterRegistry();
    timeLimiter = timeLimiterRegistry.timeLimiter("resilience4jTest",
            TimeLimiterConfig
                    .custom()
                    .timeoutDuration(Duration.ofMillis(1000 * 1))
                    .cancelRunningFuture(true)
                    .build());
                    
    Resilience4jTestHelper.timeLimiterEventListener(timeLimiter);
    
    @Test
    public void When_Test_CircuitBreaker_Expect_Timeout() {
        AtomicInteger count = new AtomicInteger();
        ExecutorService executorService = Executors.newFixedThreadPool(10);
        for (int i = 0; i < 30; i++) {
            String path = expectError() ? PATH_408 : PATH_200;
            Future<String> futureStr =
                    executorService.submit(() -> Resilience4jTestHelper.responseToTimeLimiter(timeLimiter, circuitBreaker, testClient, path));
            Callable<String> stringCallable = timeLimiter.decorateFutureSupplier(() -> futureStr);
            Callable<String> response = circuitBreaker.decorateCallable(stringCallable);
            Try.of(response::call).andThen(val -> log.info(">>>>>>>>>> success {} {}", count.incrementAndGet(), val))
                .recover(CallNotPermittedException.class, throwable -> {
                Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> open CircuitBreaker " + count.incrementAndGet(), circuitBreaker);
                return "hit CircuitBreaker";
            }).recover(throwable -> {
                Resilience4jTestHelper.getCircuitBreakerStatus(">>>>>>>>>> call fallback " + count.incrementAndGet(), circuitBreaker);
                log.error(">>>>>>>>>> fallback:{}", throwable.getMessage());
                return "hit Fallback";
            });
        }
    }

到此Resilience4j元件的基本用法介紹完畢，上面的測試程式碼我沒有截圖測試的結果，附上程式碼地址各位看官可以在本地跑跑測試程式碼

程式碼地址：https://github.com/showkawa/springBoot_2017/tree/master/spb-demo/spb-gateway/src/test/java/com/kawa/spbgateway/circuitbreaker/resilience4j

相關文章