重新整理 .net core 周邊閱讀篇————AspNetCoreRateLimit 之規則[二]

敖毛毛發表於2021-10-08
NetCoreMIT

前言

本文和上文息息相關。

https://www.cnblogs.com/aoximin/p/15315102.html

是緊接著上文invoke來書寫的，那麼現在來逐行分析invoke到底幹了啥。

正文

invoke 是一個方法,那麼其一般符合一個套路。

  1. 引數檢查

  2. 引數轉換與檢查（可能有可能無）

  3. 核心處理

  4. 返回引數（包括返回void）

那麼就按照這個套路來看吧。

先看1和2 步驟吧。

// check if rate limiting is enabled
if (_options == null)
{
	await _next.Invoke(context);
	return;
}

// compute identity from request
var identity = await ResolveIdentityAsync(context);

// check white list
if (_processor.IsWhitelisted(identity))
{
	await _next.Invoke(context);
	return;
}

上面檢查是否配置為空，如果為空就將請求轉到下一個中介軟體。

// compute identity from request
var identity = await ResolveIdentityAsync(context);

這個屬於引數轉換。

檢視ResolveIdentityAsync：

public virtual async Task<ClientRequestIdentity> ResolveIdentityAsync(HttpContext httpContext)
{
	string clientIp = null;
	string clientId = null;

	if (_config.ClientResolvers?.Any() == true)
	{
		foreach (var resolver in _config.ClientResolvers)
		{
			clientId = await resolver.ResolveClientAsync(httpContext);

			if (!string.IsNullOrEmpty(clientId))
			{
				break;
			}
		}
	}

	if (_config.IpResolvers?.Any() == true)
	{
		foreach (var resolver in _config.IpResolvers)
		{
			clientIp = resolver.ResolveIp(httpContext);

			if (!string.IsNullOrEmpty(clientIp))
			{
				break;
			}
		}
	}

	return new ClientRequestIdentity
	{
		ClientIp = clientIp,
		Path = httpContext.Request.Path.ToString().ToLowerInvariant().TrimEnd('/'),
		HttpVerb = httpContext.Request.Method.ToLowerInvariant(),
		ClientId = clientId ?? "anon"
	};
}

這種一般先看返回值的,因為其在前方法中起作用的是返回值。

return new ClientRequestIdentity
{
	ClientIp = clientIp,
	Path = httpContext.Request.Path.ToString().ToLowerInvariant().TrimEnd('/'),
	HttpVerb = httpContext.Request.Method.ToLowerInvariant(),
	ClientId = clientId ?? "anon"
};

從這裡面可以得知，是通過context，獲取了ClientIp、Path、HttpVerb、clientId。

那麼前文說過，我們只看下ip部分，那麼看下這個ClientIp 是如何獲取的吧。

if (_config.IpResolvers?.Any() == true)
{
	foreach (var resolver in _config.IpResolvers)
	{
		clientIp = resolver.ResolveIp(httpContext);

		if (!string.IsNullOrEmpty(clientIp))
		{
			break;
		}
	}
}

前文提及過了。這裡再提及一遍。

這個_config 是IRateLimitConfiguration。

然後我們註冊了配置:

services.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>();

RateLimitConfiguration 中的IpResolvers:

 public IList<IIpResolveContributor> IpResolvers { get; } = new List<IIpResolveContributor>();

中介軟體初始化的時候:

 _config.RegisterResolvers();

呼叫了RateLimitConfiguration的RegisterResolvers:

public virtual void RegisterResolvers()
{
	string clientIdHeader = GetClientIdHeader();
	string realIpHeader = GetRealIp();

	if (clientIdHeader != null)
	{
		ClientResolvers.Add(new ClientHeaderResolveContributor(clientIdHeader));
	}

	// the contributors are resolved in the order of their collection index
	if (realIpHeader != null)
	{
		IpResolvers.Add(new IpHeaderResolveContributor(realIpHeader));
	}

	IpResolvers.Add(new IpConnectionResolveContributor());
}

這裡IpResolvers 就新增了一些ip的獲取方式,這個在上文中細講了，這裡就只說其功能。

那麼會到invoke中來，對於ip 限制來說，限制獲取了clientip、path、methodverb。

那麼invoke對clientip進行了檢查，檢視是是否在白名單中。

// check white list
if (_processor.IsWhitelisted(identity))
{
	await _next.Invoke(context);
	return;
}

IsWhitelisted 方法:

public virtual bool IsWhitelisted(ClientRequestIdentity requestIdentity)
{
	if (_options.ClientWhitelist != null && _options.ClientWhitelist.Contains(requestIdentity.ClientId))
	{
		return true;
	}

	if (_options.IpWhitelist != null && IpParser.ContainsIp(_options.IpWhitelist, requestIdentity.ClientIp))
	{
		return true;
	}

	if (_options.EndpointWhitelist != null && _options.EndpointWhitelist.Any())
	{
		string path = _options.EnableRegexRuleMatching ? $".+:{requestIdentity.Path}" : $"*:{requestIdentity.Path}";

		if (_options.EndpointWhitelist.Any(x => $"{requestIdentity.HttpVerb}:{requestIdentity.Path}".IsUrlMatch(x, _options.EnableRegexRuleMatching)) ||
				_options.EndpointWhitelist.Any(x => path.IsUrlMatch(x, _options.EnableRegexRuleMatching)))
			return true;
	}

	return false;
}

關注一下這個:

if (_options.IpWhitelist != null && IpParser.ContainsIp(_options.IpWhitelist, requestIdentity.ClientIp))

這裡是返回是否在白名單的，如果有興趣可以看下ContainsIp，裡面關於了ip6的處理，雖然現在ip6用的不多，但是可以看看，萬一真的有使用者用ip6呢。

接下來就看下核心處理邏輯:

var rules = await _processor.GetMatchingRulesAsync(identity, context.RequestAborted);

var rulesDict = new Dictionary<RateLimitRule, RateLimitCounter>();

foreach (var rule in rules)
{
	// increment counter
	var rateLimitCounter = await _processor.ProcessRequestAsync(identity, rule, context.RequestAborted);

	if (rule.Limit > 0)
	{
		// check if key expired
		if (rateLimitCounter.Timestamp + rule.PeriodTimespan.Value < DateTime.UtcNow)
		{
			continue;
		}

		// check if limit is reached
		if (rateLimitCounter.Count > rule.Limit)
		{
			//compute retry after value
			var retryAfter = rateLimitCounter.Timestamp.RetryAfterFrom(rule);

			// log blocked request
			LogBlockedRequest(context, identity, rateLimitCounter, rule);

			if (_options.RequestBlockedBehaviorAsync != null)
			{
				await _options.RequestBlockedBehaviorAsync(context, identity, rateLimitCounter, rule);
			}

			if (!rule.MonitorMode)
			{
				// break execution
				await ReturnQuotaExceededResponse(context, rule, retryAfter);

				return;
			}
		}
	}
	// if limit is zero or less, block the request.
	else
	{
		// log blocked request
		LogBlockedRequest(context, identity, rateLimitCounter, rule);

		if (_options.RequestBlockedBehaviorAsync != null)
		{
			await _options.RequestBlockedBehaviorAsync(context, identity, rateLimitCounter, rule);
		}

		if (!rule.MonitorMode)
		{
			// break execution (Int32 max used to represent infinity)
			await ReturnQuotaExceededResponse(context, rule, int.MaxValue.ToString(System.Globalization.CultureInfo.InvariantCulture));

			return;
		}
	}

	rulesDict.Add(rule, rateLimitCounter);
}

先看核心功能要用到的引數:

var rules = await _processor.GetMatchingRulesAsync(identity, context.RequestAborted);

var rulesDict = new Dictionary<RateLimitRule, RateLimitCounter>();

看下GetMatchingRulesAsync：

public async Task<IEnumerable<RateLimitRule>> GetMatchingRulesAsync(ClientRequestIdentity identity, CancellationToken cancellationToken = default)
{
	var policies = await _policyStore.GetAsync($"{_options.IpPolicyPrefix}", cancellationToken);

	var rules = new List<RateLimitRule>();

	if (policies?.IpRules?.Any() == true)
	{
		// search for rules with IP intervals containing client IP
		var matchPolicies = policies.IpRules.Where(r => IpParser.ContainsIp(r.Ip, identity.ClientIp));

		foreach (var item in matchPolicies)
		{
			rules.AddRange(item.Rules);
		}
	}

	return GetMatchingRules(identity, rules);
}

這個是先獲取該ip是否是我們特殊ip處理的規則，然後通過GetMatchingRules 判斷其是否符合規則。

GetMatchingRules 應該就是處理核心了。

protected virtual List<RateLimitRule> GetMatchingRules(ClientRequestIdentity identity, List<RateLimitRule> rules = null)
{
	var limits = new List<RateLimitRule>();

	if (rules?.Any() == true)
	{
		if (_options.EnableEndpointRateLimiting)
		{
			// search for rules with endpoints like "*" and "*:/matching_path"

			string path = _options.EnableRegexRuleMatching ? $".+:{identity.Path}" : $"*:{identity.Path}";

			var pathLimits = rules.Where(r => path.IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
			limits.AddRange(pathLimits);

			// search for rules with endpoints like "matching_verb:/matching_path"
			var verbLimits = rules.Where(r => $"{identity.HttpVerb}:{identity.Path}".IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
			limits.AddRange(verbLimits);
		}
		else
		{
			// ignore endpoint rules and search for global rules only
			var genericLimits = rules.Where(r => r.Endpoint == "*");
			limits.AddRange(genericLimits);
		}

		// get the most restrictive limit for each period 
		limits = limits.GroupBy(l => l.Period).Select(l => l.OrderBy(x => x.Limit)).Select(l => l.First()).ToList();
	}

	// search for matching general rules
	if (_options.GeneralRules != null)
	{
		var matchingGeneralLimits = new List<RateLimitRule>();

		if (_options.EnableEndpointRateLimiting)
		{
			// search for rules with endpoints like "*" and "*:/matching_path" in general rules
			var pathLimits = _options.GeneralRules.Where(r => $"*:{identity.Path}".IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
			matchingGeneralLimits.AddRange(pathLimits);

			// search for rules with endpoints like "matching_verb:/matching_path" in general rules
			var verbLimits = _options.GeneralRules.Where(r => $"{identity.HttpVerb}:{identity.Path}".IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
			matchingGeneralLimits.AddRange(verbLimits);
		}
		else
		{
			//ignore endpoint rules and search for global rules in general rules
			var genericLimits = _options.GeneralRules.Where(r => r.Endpoint == "*");
			matchingGeneralLimits.AddRange(genericLimits);
		}

		// get the most restrictive general limit for each period 
		var generalLimits = matchingGeneralLimits
			.GroupBy(l => l.Period)
			.Select(l => l.OrderBy(x => x.Limit).ThenBy(x => x.Endpoint))
			.Select(l => l.First())
			.ToList();

		foreach (var generalLimit in generalLimits)
		{
			// add general rule if no specific rule is declared for the specified period
			if (!limits.Exists(l => l.Period == generalLimit.Period))
			{
				limits.Add(generalLimit);
			}
		}
	}

	foreach (var item in limits)
	{
		if (!item.PeriodTimespan.HasValue)
		{
			// parse period text into time spans	
			item.PeriodTimespan = item.Period.ToTimeSpan();
		}
	}

	limits = limits.OrderBy(l => l.PeriodTimespan).ToList();

	if (_options.StackBlockedRequests)
	{
		limits.Reverse();
	}

	return limits;
}

這樣看起來程式碼挺多的，但是這種也說明水不水特別深，為什麼這麼說呢？因為這裡面基本沒有呼叫其他的方法，都是寫基礎邏輯處理。

因為有很多if，那麼就通過if來分段看。

var limits = new List<RateLimitRule>();

if (rules?.Any() == true)
{
	if (_options.EnableEndpointRateLimiting)
	{
		// search for rules with endpoints like "*" and "*:/matching_path"

		string path = _options.EnableRegexRuleMatching ? $".+:{identity.Path}" : $"*:{identity.Path}";

		var pathLimits = rules.Where(r => path.IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
		limits.AddRange(pathLimits);

		// search for rules with endpoints like "matching_verb:/matching_path"
		var verbLimits = rules.Where(r => $"{identity.HttpVerb}:{identity.Path}".IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
		limits.AddRange(verbLimits);
	}
	else
	{
		// ignore endpoint rules and search for global rules only
		var genericLimits = rules.Where(r => r.Endpoint == "*");
		limits.AddRange(genericLimits);
	}

	// get the most restrictive limit for each period 
	limits = limits.GroupBy(l => l.Period).Select(l => l.OrderBy(x => x.Limit)).Select(l => l.First()).ToList();
}

這個一段是對我們特殊ip規則的處理,然後發現裡面的邏輯其實是圍繞著_options.EnableEndpointRateLimiting展開的。

那麼從文件中EnableEndpointRateLimiting是什麼呢?

If EnableEndpointRateLimiting is set to false then the limits will apply globally and only the rules that have as endpoint * will apply. For example, if you set a limit of 5 calls per second, any HTTP call to any endpoint will count towards that limit.

If EnableEndpointRateLimiting is set to true, then the limits will apply for each endpoint as in {HTTP_Verb}{PATH}. For example if you set a limit of 5 calls per second for *:/api/values a client can call GET /api/values 5 times per second but also 5 times PUT /api/values.

這上面是說如果EnableEndpointRateLimiting 是false 的話，那麼限制只用於端點為"*"的情況。舉了一個例子：如果你設定了每秒訪問5次，那麼你訪問任何端點都會被計數。

如果EnableEndpointRateLimiting設定為true，那麼限制將適用於每個端點，如{HTTP_Verb}{PATH}。例如，如果你為*:/api/值設定了每秒5次呼叫的限制，客戶端可以每秒5次呼叫GET /api/值，也可以5次呼叫PUT /api/值。

說白了就是是否可以設定訪問特殊Endpoint的訪問限制。

有了上面的文件解釋，那麼看著程式碼只要按照這思路去看就行。

接下來看下一個if:

// search for matching general rules
if (_options.GeneralRules != null)
{
	var matchingGeneralLimits = new List<RateLimitRule>();

	if (_options.EnableEndpointRateLimiting)
	{
		// search for rules with endpoints like "*" and "*:/matching_path" in general rules
		var pathLimits = _options.GeneralRules.Where(r => $"*:{identity.Path}".IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
		matchingGeneralLimits.AddRange(pathLimits);

		// search for rules with endpoints like "matching_verb:/matching_path" in general rules
		var verbLimits = _options.GeneralRules.Where(r => $"{identity.HttpVerb}:{identity.Path}".IsUrlMatch(r.Endpoint, _options.EnableRegexRuleMatching));
		matchingGeneralLimits.AddRange(verbLimits);
	}
	else
	{
		//ignore endpoint rules and search for global rules in general rules
		var genericLimits = _options.GeneralRules.Where(r => r.Endpoint == "*");
		matchingGeneralLimits.AddRange(genericLimits);
	}

	// get the most restrictive general limit for each period 
	var generalLimits = matchingGeneralLimits
		.GroupBy(l => l.Period)
		.Select(l => l.OrderBy(x => x.Limit).ThenBy(x => x.Endpoint))
		.Select(l => l.First())
		.ToList();

	foreach (var generalLimit in generalLimits)
	{
		// add general rule if no specific rule is declared for the specified period
		if (!limits.Exists(l => l.Period == generalLimit.Period))
		{
			limits.Add(generalLimit);
		}
	}
}

同樣我們應該看:_options.GeneralRules。

GeneralRules 就是我們限定的規則，裡面同樣看的是Endpoint是否匹配。

然後看最後一段:

foreach (var item in limits)
{
	if (!item.PeriodTimespan.HasValue)
	{
		// parse period text into time spans	
		item.PeriodTimespan = item.Period.ToTimeSpan();
	}
}

limits = limits.OrderBy(l => l.PeriodTimespan).ToList();

if (_options.StackBlockedRequests)
{
	limits.Reverse();
}

return limits;

上面for 迴圈就是將我們的時間字串轉換為timespan（時間區間），然後從小到大排序一下。

接下來就看下_options.StackBlockedRequests，還是那個老套路看到配置檔案查文件。

If StackBlockedRequests is set to false, rejected calls are not added to the throttle counter. If a client makes 3 requests per second and you've set a limit of one call per second, other limits like per minute or per day counters will only record the first call, the one that wasn't blocked. If you want rejected requests to count towards the other limits, you'll have to set StackBlockedRequests to true.

我直接用自己的理解說哈，如果StackBlockedRequests 設定為false，如果被拒絕的請求將不會加入到計數中。如果一個客戶端每秒3次請求，你設定了每秒請求一次。那麼其他的限制像每分鐘和每天的計數器將只沒有被拒絕的記錄一次。

如果想拒絕的請求請求進行計數，那麼你應該設定StackBlockedRequests 為true。

這裡面就是說白了，就是拒絕的請求是否進行計數。

當然在這裡還沒有涉及到計數，StackBlockedRequests為true是將時間區間，從大到小排序了，這將成為後面的一個關鍵。

這裡可以進行一個大膽的猜測，StackBlockedRequests 為fale的情況下，limits 是根據PeriodTimespan 從小到大排序的，也就是說是秒 分 小時 天這樣排序的。

根據正常邏輯一般是秒先達到閥值，那麼可能計數邏輯就是進行for迴圈，然後如果到達了限制那麼就進行request block，很巧妙的一種設計。

這裡可能就有人問了，如果是分到達了限制，那麼秒不還是進行計數了嗎？

這是沒有關係的，因為分裡面包含了秒。這裡其實解決的是這樣的一個問題，比如我在1秒中內請求了60次，那麼有59次是失敗的，那麼如果請求算60次的話，那麼會達到每分鐘60次的現在，那麼這個使用者在一分鐘內無法請求，故而建議StackBlockedRequests 設定為false。

結

因為篇幅限制，下一節是關於如何計數的。

相關文章