SpringCloud升級之路2020.0.x版-39. 改造 resilience4j 粘合 WebClient

乾貨滿滿張雜湊 發表於 2021-11-21
Spring

SpringCloud升級之路2020.0.x版-39. 改造 resilience4j 粘合 WebClient

本系列程式碼地址:https://github.com/JoJoTec/spring-cloud-parent

要想實現我們上一節中提到的:

  • 需要在重試以及斷路中加一些日誌,便於日後的優化
  • 需要定義重試的 Exception,並且與斷路器相結合,將非 2xx 的響應碼也封裝成特定的異常
  • 需要在斷路器相關的 Operator 中增加類似於 FeignClient 中的負載均衡的資料更新,使得負載均衡更加智慧

我們需要將 resilience4j 本身提供的粘合庫做一些改造,其實主要就是對 resilience4j 實現的 project reactor 的 Operator 進行改造。

關於斷路器的改造

首先,WebClient 的返回物件只可能是 ClientResponse 型別,所以我們這裡改造出來的 Operator 不必帶上形參,只需要針對 ClientResponse 即可,即:

public class ClientResponseCircuitBreakerOperator implements UnaryOperator<Publisher<ClientResponse>> {
    ...
}

在原有的斷路器邏輯中,我們需要加入針對 GET 方法以及之前定義的可以重試的路徑匹配配置可以重試的邏輯,這需要我們拿到原有請求的 URL 資訊。但是 ClientResponse 中並沒有暴露這些資訊的介面,其預設實現 DefaultClientResponse(我們只要沒有自己給 WebClient 加入特殊的改造邏輯,實現都是 DefaultClientResponse) 中的 request() 方法可以獲取請求 HttpRequest,其中包含 url 資訊。但是這個類還有方法都是 package-private 的,我們需要反射出來:

ClientResponseCircuitBreakerSubscriber

private static final Class<?> aClass;
private static final Method request;

static {
    try {
        aClass = Class.forName("org.springframework.web.reactive.function.client.DefaultClientResponse");
        request = ReflectionUtils.findMethod(aClass, "request");
        request.setAccessible(true);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

之後,在獲取到 ClientResponse 之後記錄斷路器的邏輯中,需要加入上面提到的關於重試的改造,以及負載均衡器的記錄:

ClientResponseCircuitBreakerSubscriber

protected void hookOnNext(ClientResponse clientResponse) {
    if (!isDisposed()) {
        if (singleProducer && successSignaled.compareAndSet(false, true)) {
            int rawStatusCode = clientResponse.rawStatusCode();
            HttpStatus httpStatus = HttpStatus.resolve(rawStatusCode);
            try {
                HttpRequest httpRequest = (HttpRequest) request.invoke(clientResponse);
                //判斷方法是否為 GET,以及是否在可重試路徑配置中,從而得出是否可以重試
                if (httpRequest.getMethod() != HttpMethod.GET && !webClientProperties.retryablePathsMatch(httpRequest.getURI().getPath())) {
                    //如果不能重試,則直接返回結果
                    circuitBreaker.onResult(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit(), clientResponse);
                } else {
                    if (httpStatus != null && httpStatus.is2xxSuccessful()) {
                        //如果成功,則直接返回結果
                        circuitBreaker.onResult(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit(), clientResponse);
                    } else {
                        /**
                         * 如果異常,參考 DefaultClientResponse 的程式碼進行異常封裝
                         * @see org.springframework.web.reactive.function.client.DefaultClientResponse#createException
                         */
                        Exception exception;
                        if (httpStatus != null) {
                            exception = WebClientResponseException.create(rawStatusCode, httpStatus.getReasonPhrase(), clientResponse.headers().asHttpHeaders(), EMPTY, null, null);
                        } else {
                            exception = new UnknownHttpStatusCodeException(rawStatusCode, clientResponse.headers().asHttpHeaders(), EMPTY, null, null);
                        }
                        circuitBreaker.onError(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit(), exception);
                        downstreamSubscriber.onError(exception);
                        return;
                    }
                }
            } catch (Exception e) {
                log.fatal("judge request method in circuit breaker error! the resilience4j feature would not be enabled: {}", e.getMessage(), e);
                circuitBreaker.onResult(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit(), clientResponse);
            }
        }
        eventWasEmitted.set(true);
        downstreamSubscriber.onNext(clientResponse);
    }
}

同樣的,在原有的完成,取消還有失敗的記錄邏輯中,也加上記錄負載均衡資料:

ClientResponseCircuitBreakerSubscriber

@Override
protected void hookOnComplete() {
    if (successSignaled.compareAndSet(false, true)) {
        serviceInstanceMetrics.recordServiceInstanceCalled(serviceInstance, true);
        circuitBreaker.onSuccess(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit());
    }

    downstreamSubscriber.onComplete();
}

@Override
public void hookOnCancel() {
    if (!successSignaled.get()) {
        serviceInstanceMetrics.recordServiceInstanceCalled(serviceInstance, true);
        if (eventWasEmitted.get()) {
            circuitBreaker.onSuccess(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit());
        } else {
            circuitBreaker.releasePermission();
        }
    }
}

@Override
protected void hookOnError(Throwable e) {
    serviceInstanceMetrics.recordServiceInstanceCalled(serviceInstance, false);
    circuitBreaker.onError(circuitBreaker.getCurrentTimestamp() - start, circuitBreaker.getTimestampUnit(), e);
    downstreamSubscriber.onError(e);
}

粘合 WebClient 與 resilience4j 的同時覆蓋重試邏輯

由於前面的斷路器中,我們針對可以重試的非 2XX 響應封裝成為 WebClientResponseException。所以在重試器中,我們需要加上針對這個異常的重試。

同時,需要將重試器放在負載均衡器之前,因為每次重試,都要從負載均衡器中獲取一個新的例項。同時,斷路器需要放在負載均衡器之後,因為只有在這個之後,才能獲取到本次呼叫的例項,我們的的斷路器是針對例項方法級別的:

WebClientDefaultConfiguration.java

@Bean
public WebClient getWebClient(
        ReactorLoadBalancerExchangeFilterFunction lbFunction,
        WebClientConfigurationProperties webClientConfigurationProperties,
        Environment environment,
        RetryRegistry retryRegistry,
        CircuitBreakerRegistry circuitBreakerRegistry,
        ServiceInstanceMetrics serviceInstanceMetrics
) {
    String name = environment.getProperty(WebClientNamedContextFactory.PROPERTY_NAME);
    Map<String, WebClientConfigurationProperties.WebClientProperties> configs = webClientConfigurationProperties.getConfigs();
    if (configs == null || configs.size() == 0) {
        throw new BeanCreationException("Failed to create webClient, please provide configurations under namespace: webclient.configs");
    }
    WebClientConfigurationProperties.WebClientProperties webClientProperties = configs.get(name);
    if (webClientProperties == null) {
        throw new BeanCreationException("Failed to create webClient, please provide configurations under namespace: webclient.configs." + name);
    }
    String serviceName = webClientProperties.getServiceName();
    //如果沒填寫微服務名稱,就使用配置 key 作為微服務名稱
    if (StringUtils.isBlank(serviceName)) {
        serviceName = name;
    }
    String baseUrl = webClientProperties.getBaseUrl();
    //如果沒填寫 baseUrl,就使用微服務名稱填充
    if (StringUtils.isBlank(baseUrl)) {
        baseUrl = "http://" + serviceName;
    }

    Retry retry = null;
    try {
        retry = retryRegistry.retry(serviceName, serviceName);
    } catch (ConfigurationNotFoundException e) {
        retry = retryRegistry.retry(serviceName);
    }
    //覆蓋其中的異常判斷
    retry = Retry.of(serviceName, RetryConfig.from(retry.getRetryConfig()).retryOnException(throwable -> {
        //WebClientResponseException 會重試,因為在這裡能 catch 的 WebClientResponseException 只對可以重試的請求封裝了 WebClientResponseException
        //參考 ClientResponseCircuitBreakerSubscriber 的程式碼
        if (throwable instanceof WebClientResponseException) {
            log.info("should retry on {}", throwable.toString());
            return true;
        }
        //斷路器異常重試,因為請求沒有發出去
        if (throwable instanceof CallNotPermittedException) {
            log.info("should retry on {}", throwable.toString());
            return true;
        }
        if (throwable instanceof WebClientRequestException) {
            WebClientRequestException webClientRequestException = (WebClientRequestException) throwable;
            HttpMethod method = webClientRequestException.getMethod();
            URI uri = webClientRequestException.getUri();
            //判斷是否為響應超時,響應超時代表請求已經發出去了,對於非 GET 並且沒有標註可以重試的請求則不能重試
            boolean isResponseTimeout = false;
            Throwable cause = throwable.getCause();
            //netty 的讀取超時一般是 ReadTimeoutException
            if (cause instanceof ReadTimeoutException) {
                log.info("Cause is a ReadTimeoutException which indicates it is a response time out");
                isResponseTimeout = true;
            } else {
                //對於其他一些框架,使用了 java 底層 nio 的一般是 SocketTimeoutException,message 為 read time out
                //還有一些其他異常,但是 message 都會有 read time out 欄位,所以通過 message 判斷
                String message = throwable.getMessage();
                if (StringUtils.isNotBlank(message) && StringUtils.containsIgnoreCase(message.replace(" ", ""), "readtimeout")) {
                    log.info("Throwable message contains readtimeout which indicates it is a response time out");
                    isResponseTimeout = true;
                }
            }
            //如果請求是 GET 或者標註了重試,則直接判斷可以重試
            if (method == HttpMethod.GET || webClientProperties.retryablePathsMatch(uri.getPath())) {
                log.info("should retry on {}-{}, {}", method, uri, throwable.toString());
                return true;
            } else {
                //否則,只針對請求還沒有發出去的異常進行重試
                if (isResponseTimeout) {
                    log.info("should not retry on {}-{}, {}", method, uri, throwable.toString());
                } else {
                    log.info("should retry on {}-{}, {}", method, uri, throwable.toString());
                    return true;
                }
            }
        }
        return false;
    }).build());


    HttpClient httpClient = HttpClient
            .create()
            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, (int) webClientProperties.getConnectTimeout().toMillis())
            .doOnConnected(connection ->
                    connection
                            .addHandlerLast(new ReadTimeoutHandler((int) webClientProperties.getResponseTimeout().toSeconds()))
                            .addHandlerLast(new WriteTimeoutHandler((int) webClientProperties.getResponseTimeout().toSeconds()))
            );

    Retry finalRetry = retry;
    String finalServiceName = serviceName;
    return WebClient.builder()
            .exchangeStrategies(ExchangeStrategies.builder()
            .codecs(configurer -> configurer
                    .defaultCodecs()
                    //最大 body 佔用 16m 記憶體
                    .maxInMemorySize(16 * 1024 * 1024))
            .build())
            .clientConnector(new ReactorClientHttpConnector(httpClient))
            //Retry在負載均衡前
            .filter((clientRequest, exchangeFunction) -> {
                return exchangeFunction
                        .exchange(clientRequest)
                        .transform(ClientResponseRetryOperator.of(finalRetry));
            })
            //負載均衡器,改寫url
            .filter(lbFunction)
            //例項級別的斷路器需要在負載均衡獲取真正地址之後
            .filter((clientRequest, exchangeFunction) -> {
                ServiceInstance serviceInstance = getServiceInstance(clientRequest);
                serviceInstanceMetrics.recordServiceInstanceCall(serviceInstance);
                CircuitBreaker circuitBreaker;
                //這時候的url是經過負載均衡器的,是例項的url
                //需要注意的一點是,使用非同步 client 的時候,最好不要帶路徑引數,否則這裡的斷路器效果不好
                //斷路器是每個例項每個路徑一個斷路器
                String instancId = clientRequest.url().getHost() + ":" + clientRequest.url().getPort() + clientRequest.url().getPath();
                try {
                    //使用例項id新建或者獲取現有的CircuitBreaker,使用serviceName獲取配置
                    circuitBreaker = circuitBreakerRegistry.circuitBreaker(instancId, finalServiceName);
                } catch (ConfigurationNotFoundException e) {
                    circuitBreaker = circuitBreakerRegistry.circuitBreaker(instancId);
                }
                log.info("webclient circuit breaker [{}-{}] status: {}, data: {}", finalServiceName, instancId, circuitBreaker.getState(), JSON.toJSONString(circuitBreaker.getMetrics()));
                return exchangeFunction.exchange(clientRequest).transform(ClientResponseCircuitBreakerOperator.of(circuitBreaker, serviceInstance, serviceInstanceMetrics, webClientProperties));
            }).baseUrl(baseUrl)
            .build();
}

private ServiceInstance getServiceInstance(ClientRequest clientRequest) {
    URI url = clientRequest.url();
    DefaultServiceInstance defaultServiceInstance = new DefaultServiceInstance();
    defaultServiceInstance.setHost(url.getHost());
    defaultServiceInstance.setPort(url.getPort());
    return defaultServiceInstance;
}

這樣,我們就實現了我們封裝的基於配置的 WebClient

微信搜尋“我的程式設計喵”關注公眾號,每日一刷,輕鬆提升技術,斬獲各種offer

SpringCloud升級之路2020.0.x版-39. 改造 resilience4j 粘合 WebClient