Kubernetes:kube-apiserver 之鑑權

---

kubernetes:kube-apiserver 系列文章：

• Kubernetes:kube-apiserver 之 scheme(一)
• Kubernetes:kube-apiserver 之 scheme(二)
• Kubernetes:kube-apiserver 之啟動流程(一)
• Kubernetes:kube-apiserver 之啟動流程(二)
• Kubernetes:kube-apiserver 和 etcd 的互動
• Kubernetes:kube-apiserver 之認證

0. 前言

上一篇文章介紹了 kube-apiserver 的認證機制。這裡繼續往下走，介紹 kube-apiserver 的鑑權。kube-apiserver 處理認證和鑑權非常類似，建議閱讀鑑權機制前先看看 kube-apiserver 的 認證。

1. 鑑權 Authorization

1.1 Authorization handler

進入 DefaultBuildHandlerChain 看 Authorization handler 建立過程。

# kubernetes/vendor/k8s.io/apiserver/pkg/server/config.go
func DefaultBuildHandlerChain(apiHandler http.Handler, c *Config) http.Handler {
	handler := apiHandler

	handler = genericapifilters.WithAuthorization(handler, c.Authorization.Authorizer, c.Serializer)
	handler = filterlatency.TrackStarted(handler, c.TracerProvider, "authorization")

    handler = genericapifilters.WithAuthentication(handler, c.Authentication.Authenticator, failedHandler, c.Authentication.APIAudiences, c.Authentication.RequestHeaderConfig)
	handler = filterlatency.TrackStarted(handler, c.TracerProvider, "authentication")
}

這裡 handler chain 的 handler 處理順序是由下往上的，即處理完 authentication handler 在處理 authorization handler。

進入 genericapifilters.WithAuthorization 檢視鑑權 handler 的建立過程。

# kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go
func WithAuthorization(hhandler http.Handler, auth authorizer.Authorizer, s runtime.NegotiatedSerializer) http.Handler {
	return withAuthorization(hhandler, auth, s, recordAuthorizationMetrics)
}

func withAuthorization(handler http.Handler, a authorizer.Authorizer, s runtime.NegotiatedSerializer, metrics recordAuthorizationMetricsFunc) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
		ctx := req.Context()
		authorizationStart := time.Now()

        // 獲取 request 攜帶的使用者資訊
		attributes, err := GetAuthorizerAttributes(ctx)
		if err != nil {
			responsewriters.InternalError(w, req, err)
			return
		}

        // 對使用者資訊進行鑑權
		authorized, reason, err := a.Authorize(ctx, attributes)

		...
	})
}

鑑權過程包括兩部分。

一部分透過 GetAuthorizerAttributes 獲取 RESTful API 請求中攜帶的使用者資訊。

# kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go
func GetAuthorizerAttributes(ctx context.Context) (authorizer.Attributes, error) {
	attribs := authorizer.AttributesRecord{}

	user, ok := request.UserFrom(ctx)
	if ok {
		attribs.User = user
	}

	requestInfo, found := request.RequestInfoFrom(ctx)
	if !found {
		return nil, errors.New("no RequestInfo found in the context")
	}

	// Start with common attributes that apply to resource and non-resource requests
	attribs.ResourceRequest = requestInfo.IsResourceRequest
	attribs.Path = requestInfo.Path
	attribs.Verb = requestInfo.Verb

	attribs.APIGroup = requestInfo.APIGroup
	attribs.APIVersion = requestInfo.APIVersion
	attribs.Resource = requestInfo.Resource
	attribs.Subresource = requestInfo.Subresource
	attribs.Namespace = requestInfo.Namespace
	attribs.Name = requestInfo.Name

	return &attribs, nil
}

獲取到使用者資訊後透過 a.Authorize(ctx, attributes) 對使用者及其請求進行鑑權。其中 a 是實現了 Authorizer 鑑權器介面的例項。

type Authorizer interface {
	Authorize(ctx context.Context, a Attributes) (authorized Decision, reason string, err error)
}

檢視 a.Authorize(ctx, attributes) 實際是看 Config.Authorization.Authorizer 中的例項。

func DefaultBuildHandlerChain(apiHandler http.Handler, c *Config) http.Handler {
	handler = genericapifilters.WithAuthorization(handler, c.Authorization.Authorizer, c.Serializer)
}

1.2 Authorization authorizer

Config.Authorization.Authorizer 在 BuildGenericConfig 的 BuildAuthorizer 函式內建立。

# kubernetes/pkg/controlplane/apiserver/config.go
func BuildGenericConfig(
	s controlplaneapiserver.CompletedOptions,
	schemes []*runtime.Scheme,
	getOpenAPIDefinitions func(ref openapicommon.ReferenceCallback) map[string]openapicommon.OpenAPIDefinition,
) (
	genericConfig *genericapiserver.Config,
	versionedInformers clientgoinformers.SharedInformerFactory,
	storageFactory *serverstorage.DefaultStorageFactory,

	lastErr error,
) {
    genericConfig.Authorization.Authorizer, genericConfig.RuleResolver, err = BuildAuthorizer(s, genericConfig.EgressSelector, versionedInformers)
	if err != nil {
		lastErr = fmt.Errorf("invalid authorization config: %v", err)
		return
	}
}

進入 BuildAuthorizer 檢視 Authorizer 是怎麼建立的。

# kubernetes/pkg/controlplane/apiserver/config.go
func BuildAuthorizer(s controlplaneapiserver.CompletedOptions, EgressSelector *egressselector.EgressSelector, versionedInformers clientgoinformers.SharedInformerFactory) (authorizer.Authorizer, authorizer.RuleResolver, error) {
	authorizationConfig := s.Authorization.ToAuthorizationConfig(versionedInformers)

	if EgressSelector != nil {
		egressDialer, err := EgressSelector.Lookup(egressselector.ControlPlane.AsNetworkContext())
		if err != nil {
			return nil, nil, err
		}
		authorizationConfig.CustomDial = egressDialer
	}

	return authorizationConfig.New()
}

建立 Authorizer 分為兩塊。首先建立 authorizationConfig，接著透過 authorizationConfig.New() 例項化 authorizer。

# kubernetes/pkg/kubeapiserver/authorizer/config.go
func (config Config) New() (authorizer.Authorizer, authorizer.RuleResolver, error) {
	var (
		authorizers   []authorizer.Authorizer
		ruleResolvers []authorizer.RuleResolver
	)

	for _, authorizationMode := range config.AuthorizationModes {
		switch authorizationMode {
		case modes.ModeNode:
			...
		case modes.ModeAlwaysAllow:
			...
		case modes.ModeAlwaysDeny:
            ...
		case modes.ModeABAC:
			...
		case modes.ModeWebhook:
			...
		case modes.ModeRBAC:
			...
		default:
			return nil, nil, fmt.Errorf("unknown authorization mode %s specified", authorizationMode)
		}
	}

	return union.New(authorizers...), union.NewRuleResolvers(ruleResolvers...), nil
}

可以看到，authorizationConfig.New() 內根據 config.AuthorizationModes 確定需要建立的鑑權器型別。這裡有 modes.ModeNode，modes.ModeAlwaysAllow，modes.ModeAlwaysDeny，modes.ModeABAC，modes.ModeWebhook 和 modes.ModeRBAC 六種鑑權器。

config.AuthorizationModes 是在建立選項 NewOptions 中定義的，例項化過程如下：

func (o *BuiltInAuthorizationOptions) ToAuthorizationConfig(versionedInformerFactory versionedinformers.SharedInformerFactory) authorizer.Config {
	return authorizer.Config{
		AuthorizationModes:          o.Modes,
	}
}

# kubernetes/pkg/controlplane/apiserver/options/options.go
func NewOptions() *Options {
	s := Options{
		Authorization:           kubeoptions.NewBuiltInAuthorizationOptions()
	}

	return &s
}

# kubernetes/pkg/kubeapiserver/options/authorization.go
func NewBuiltInAuthorizationOptions() *BuiltInAuthorizationOptions {
	return &BuiltInAuthorizationOptions{
		Modes:                       []string{authzmodes.ModeAlwaysAllow},
		WebhookVersion:              "v1beta1",
		WebhookCacheAuthorizedTTL:   5 * time.Minute,
		WebhookCacheUnauthorizedTTL: 30 * time.Second,
		WebhookRetryBackoff:         genericoptions.DefaultAuthWebhookRetryBackoff(),
	}
}

這裡的 config.AuthorizationModes 為 authzmodes.ModeAlwaysAllow。那麼，將建立 alwaysAllowAuthorizer 鑑權器。

# kubernetes/pkg/kubeapiserver/authorizer/config.go
func (config Config) New() (authorizer.Authorizer, authorizer.RuleResolver, error) {
	var (
		authorizers   []authorizer.Authorizer
		ruleResolvers []authorizer.RuleResolver
	)

	for _, authorizationMode := range config.AuthorizationModes {
		switch authorizationMode {
		case modes.ModeAlwaysAllow:
			alwaysAllowAuthorizer := authorizerfactory.NewAlwaysAllowAuthorizer()
			authorizers = append(authorizers, alwaysAllowAuthorizer)
			ruleResolvers = append(ruleResolvers, alwaysAllowAuthorizer)
        }
    }

    return union.New(authorizers...), union.NewRuleResolvers(ruleResolvers...), nil
}

# kubernetes/vendor/k8s.io/apiserver/pkg/authorization/authorizerfactory/builtin.go
func NewAlwaysAllowAuthorizer() *alwaysAllowAuthorizer {
	return new(alwaysAllowAuthorizer)
}

type alwaysAllowAuthorizer struct{}

func (alwaysAllowAuthorizer) Authorize(ctx context.Context, a authorizer.Attributes) (authorized authorizer.Decision, reason string, err error) {
	return authorizer.DecisionAllow, "", nil
}

func (alwaysAllowAuthorizer) RulesFor(user user.Info, namespace string) ([]authorizer.ResourceRuleInfo, []authorizer.NonResourceRuleInfo, bool, error) {
	return []authorizer.ResourceRuleInfo{
			&authorizer.DefaultResourceRuleInfo{
				Verbs:     []string{"*"},
				APIGroups: []string{"*"},
				Resources: []string{"*"},
			},
		}, []authorizer.NonResourceRuleInfo{
			&authorizer.DefaultNonResourceRuleInfo{
				Verbs:           []string{"*"},
				NonResourceURLs: []string{"*"},
			},
		}, false, nil
}

alwaysAllowAuthorizer 鑑權器實現了 Authorizer 介面，其總是返回 authorizer.DecisionAllow。

每種鑑權器透過 union.New 加到鑑權器組中。

func New(authorizationHandlers ...authorizer.Authorizer) authorizer.Authorizer {
	return unionAuthzHandler(authorizationHandlers)
}

// Authorizes against a chain of authorizer.Authorizer objects and returns nil if successful and returns error if unsuccessful
func (authzHandler unionAuthzHandler) Authorize(ctx context.Context, a authorizer.Attributes) (authorizer.Decision, string, error) {
	var (
		errlist    []error
		reasonlist []string
	)

	for _, currAuthzHandler := range authzHandler {
		decision, reason, err := currAuthzHandler.Authorize(ctx, a)

		if err != nil {
			errlist = append(errlist, err)
		}
		if len(reason) != 0 {
			reasonlist = append(reasonlist, reason)
		}
		switch decision {
		case authorizer.DecisionAllow, authorizer.DecisionDeny:
			return decision, reason, err
		case authorizer.DecisionNoOpinion:
			// continue to the next authorizer
		}
	}

	return authorizer.DecisionNoOpinion, strings.Join(reasonlist, "\n"), utilerrors.NewAggregate(errlist)
}

前面 handler 的 a.Authorize(ctx, attributes) 實際執行的是鑑權器組 unionAuthzHandler 的 Authorize 方法。在 unionAuthzHandler.Authorize 內遍歷執行每種鑑權器的 Authorize 方法，如果有一種鑑權器鑑權透過，則返回鑑權成功。如果鑑權器返回 authorizer.DecisionNoOpinion 則執行下一個鑑權器。如果鑑權器鑑權失敗則返回鑑權失敗。

1.3 authorization rules

前面介紹 alwaysAllowAuthorizer 鑑權器的時候我們看到 alwaysAllowAuthorizer.RulesFor 方法，該方法內定義了使用者可以訪問的 RESTful API 資源和非 RESTful API 資源。如 RESTful API 資源定義了訪問資源的動作 Verbs，資源組APIGroups 和資源 Resources。

上例的 alwaysAllowAuthorizer 並沒有看出 RulesFor 的運用是因為 alwaysAllowAuthorizer 總是允許請求訪問 RESTful API 資源和非 RESTful API 資源。

我們以 rbacAuthorizer 鑑權器為例，看 RulesFor 是怎麼被用上的。

func (config Config) New() (authorizer.Authorizer, authorizer.RuleResolver, error) {
	for _, authorizationMode := range config.AuthorizationModes {
		// Keep cases in sync with constant list in k8s.io/kubernetes/pkg/kubeapiserver/authorizer/modes/modes.go.
		switch authorizationMode {
            case modes.ModeRBAC:
			rbacAuthorizer := rbac.New(
				&rbac.RoleGetter{Lister: config.VersionedInformerFactory.Rbac().V1().Roles().Lister()},
				&rbac.RoleBindingLister{Lister: config.VersionedInformerFactory.Rbac().V1().RoleBindings().Lister()},
				&rbac.ClusterRoleGetter{Lister: config.VersionedInformerFactory.Rbac().V1().ClusterRoles().Lister()},
				&rbac.ClusterRoleBindingLister{Lister: config.VersionedInformerFactory.Rbac().V1().ClusterRoleBindings().Lister()},
			)
			authorizers = append(authorizers, rbacAuthorizer)
			ruleResolvers = append(ruleResolvers, rbacAuthorizer)
        }
    }
}

func New(roles rbacregistryvalidation.RoleGetter, roleBindings rbacregistryvalidation.RoleBindingLister, clusterRoles rbacregistryvalidation.ClusterRoleGetter, clusterRoleBindings rbacregistryvalidation.ClusterRoleBindingLister) *RBACAuthorizer {
	authorizer := &RBACAuthorizer{
		authorizationRuleResolver: rbacregistryvalidation.NewDefaultRuleResolver(
			roles, roleBindings, clusterRoles, clusterRoleBindings,
		),
	}
	return authorizer
}

func (r *RBACAuthorizer) Authorize(ctx context.Context, requestAttributes authorizer.Attributes) (authorizer.Decision, string, error) {
	ruleCheckingVisitor := &authorizingVisitor{requestAttributes: requestAttributes}

	r.authorizationRuleResolver.VisitRulesFor(requestAttributes.GetUser(), requestAttributes.GetNamespace(), ruleCheckingVisitor.visit)
	if ruleCheckingVisitor.allowed {
		return authorizer.DecisionAllow, ruleCheckingVisitor.reason, nil
	}
}

可以看到，rbacAuthorizer 鑑權器的 Authorize 方法內的 r.authorizationRuleResolver.VisitRulesFor 結合使用者資訊和鑑權器的 rules 判斷鑑權是否透過。

2. 總結

透過本篇文章介紹了 kube-apiserver 中的 Authorization 鑑權流程，下一篇將繼續介紹 kube-apiserver 的 Adimission 准入流程。

---