Kubernetes Dashboard 与 LDAP 的集成
本文展示了如何使用 Apache 来集成 LDAP 进行 Kubernetes Dashboard 的身份认证,本文使用的系统为 Ubuntu 20、Kubernetes 1.18 版本,单节点集群,主节点 IP 地址为 192.168.15.200,LDAP 的 IP 地址为 192.168.15.10。
集群安装
主节点安装
安装所需软件包的列表。
$ apt-get update
$ apt-get install apt-transport-https wget gnupg ntpdate curl mlocate
安装 Docker 服务。
apt-get install docker.io
设置开机启动。
enable docker.service systemctl
编辑 Docker 服务配置文件。
updatedb
locate docker.service
vi /etc/systemd/system/multi-user.target.wants/docker.service
在启动的项(EXECSTART)末尾添加以下配置:
--exec-opt native.cgroupdriver=systemd
下面是配置之前的文件。
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
下面是配置后的文件。
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --exec-opt native.cgroupdriver=systemd
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
然后创建系统配置文件。
vi /etc/sysctl.d/k8s.conf
这是文件内容。
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
然后启用系统配置文件。
sysctl --system
然后编辑如下 modules 配置文件,在文件末尾添加以下配置:
vi /etc/modules-load.d/modules.conf
文件末尾添加下面的配置
overlay
br_netfilter
编辑 FSTAB 配置文件并禁用交换内存。
$ vi /etc/fstab
# 注释掉 swap 的配置
/dev/disk/by-uuid/a24f00e7-918a-4a05-b4c9-35bdef750fb4 / ext4 defaults 0 0
# /swap.img none swap sw 0 0
设置一个唯一的主机名。
set-hostname kubernetes-01.local hostnamectl
创建文件来配置所需的环境变量。
$ vi /etc/profile.d/kubernetes.sh
# 文件内容如下所示
export KUBECONFIG=/etc/kubernetes/admin.conf
# 然后重启节点
$ reboot
然后配置 Kubernetes 软件源:
国内用户可以自行配置阿里云的软件源
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add
添加官方的存储库
"deb http://apt.kubernetes.io/ kubernetes-xenial main" apt-add-repository
安装 Kubernetes 软件包
apt-get update
apt-get install kubeadm kubectl kubelet
提前下载所需的镜像:
$ kubeadm config images pull
[.18.6 ] Pulled k8s.gcr.io/kube-apiserver:v1
[.18.6 ] Pulled k8s.gcr.io/kube-controller-manager:v1
[.18.6 ] Pulled k8s.gcr.io/kube-scheduler:v1
[.18.6 ] Pulled k8s.gcr.io/kube-proxy:v1
[3.2 ] Pulled k8s.gcr.io/pause:
[3.4.3-0 ] Pulled k8s.gcr.io/etcd:
[1.6.7 ] Pulled k8s.gcr.io/coredns:
然后初始化 Kubernetes 集群:
$ kubeadm init --pod-network-cidr=10.244.0.0/16
# 命令输出为
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.100.9:6443 --token l7fezg.2nm80pvehn2f2bbz \
--discovery-token-ca-cert-hash sha256:6dfc8c80e8e125c6d4d79ec82ea64deb0dcfa0a4bda33e16a8a9fa93794e3aae
安装上面的提示执行下面的命令:
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
接着安装所需的网络插件:
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml
默认情况下,不允许 Kubernetes 主节点运行普通的 Pod,我们可以执行下面的命令移除污点来解决,因为我们这里的示例只是一个单节点的 Kubernetes 集群:
kubectl taint nodes --all node-role.kubernetes.io/master-
安装 Dashboard
安装所需的软件包:
$ apt-get update
$ apt-get install wget curl mlocate
下载要部署的 YAML 资源清单文件:
$ mkdir /downloads
$ cd /downloads
$ wget https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended.yaml
直接安装 Dashboard:
kubectl create -f recommended.yaml
安装完成后为 Dashboard 创建一个 ServiceAccount 帐号,并绑定 cluster-admin 集群角色:
$ kubectl create serviceaccount apache-proxy -n kube-system
$ kubectl create clusterrolebinding apache-proxy --clusterrole=cluster-admin --serviceaccount=kube-system:apache-proxy
然后获取 Apache 代理 ServiceAccount 对应的 Token 密钥:
$ kubectl get secrets --all-namespaces | grep -E "NAME|apache"
NAMESPACE NAME TYPE DATA AGE
kube-system apache-proxy-token-8gk9p kubernetes.io/service-account-token 3 39s
# 获取 TOKEN 值
$ kubectl describe -n kube-system secret apache-proxy-token-8gk9p
Name: apache-proxy-token-8gk9p
Namespace: kube-system
Labels:
Annotations: kubernetes.io/service-account.name: apache-proxy
kubernetes.io/service-account.uid: e07884b4-282c-4ef1-8662-e2a3eaf3c448
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6InhrMHFrdEdYQ1gxVlRqZjBpN1hxdEgwaTlRRENMWmhUdC1IMC13OTgyM28ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhcGFjaGUtcHJveHktdG9rZW4tOGdrOXAiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiYXBhY2hlLXByb3h5Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZTA3ODg0YjQtMjgyYy00ZWYxLTg2NjItZTJhM2VhZjNjNDQ4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmFwYWNoZS1wcm94eSJ9.qKzSvCYOqDfvJ1S9W4-nEzZScmGwjYsi3g5df542_aDyZRugbiWbFmjrBTB70tDwuGTeupbPmS3ptweyjbb_7K5VbSk-2zCazgNeiCZ_q7M4yXgPi6rNYNiBX9pQEBaBQCWgH8-VVFGe34xrLR88cv9YNLfH9ZzrdJf2jPEBhptvdVKr6Ljpbhz-4P-mr5_IRsru_72wsRRZZptL80ARp6PkPdrYIQZ3bMQNsq3GEWxMl8SRPVqEuvVXykxfEt1Hx5URwtiwh_MLEZ5ClCIuGsGs8fWQhCLm_l0SY6p9B2DmU-XhhT_HjuotDI3pm2p5pJb9WmO4dMLx_NsBnLSz_Q
上面的 token 值就是需要的数据。然后获取访问 Dashboard Service 的 ClusterIP 地址:
$ kubectl get Service -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.99.244.167 <none> 8000/TCP 20m
kubernetes-dashboard ClusterIP 10.107.55.24 <none> 443/TCP 20m
在我们的示例中,Dashboard 使用 TCP 端口 443 和 IP 地址 10.107.55.24,如果你本地不能访问到 ClusterIP,则可以将 Dashboard 更改为 NodePort 类型进行访问。
使用 Apache 进行身份验证
接下来安装 Apache 来对 Dashboard 进行身份验证。首先在节点上安装 Apache:
$ apt-get update
$ apt-get install apache2 apache2-utils openssl
启用所需的 Apache 模块:
a2enmod authnz_ldap
a2enmod ssl
a2enmod rewrite
a2enmod headers
a2enmod proxy
a2enmod proxy_http
编辑 Apache 配置文件:
vi /etc/apache2/apache2.conf
在文件末尾添加如下信息
AllowOverride All
然后使用 OpenSSL 命令创建私钥和证书:
mkdir /etc/apache2/certificate
cd /etc/apache2/certificate
openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out apache-certificate.crt -keyout apache.key
# 输入请求的信息
Generating a RSA private key
............++++
.......................................................++++
writing new private key to 'apache.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:ChengDu
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:TechExpert
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:192.168.15.200
Email Address []:
在 Common Name
选项上输入 IP 地址或者主机名,我们这里的示例 IP 地址就是 192.168.15.200
。
使用格式 PEM 将现有的 Kubernetes 代理证书及其密钥转换为单个文件:
$ cat /etc/kubernetes/pki/front-proxy-client.crt /etc/kubernetes/pki/front-proxy-client.key > /etc/apache2/certificate/front-proxy-client.pem
然后修改默认网站的 Apache 配置文件:
vi /etc/apache2/sites-enabled/000-default.conf
# 做如下配置
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R=301,L]
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
SSLEngine on
SSLCertificateFile /etc/apache2/certificate/apache-certificate.crt
SSLCertificateKeyFile /etc/apache2/certificate/apache.key
SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
SSLProxyCACertificateFile /etc/kubernetes/pki/ca.crt
SSLProxyMachineCertificateFile /etc/apache2/certificate/front-proxy-client.pem
RequestHeader set Authorization "Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6InhrMHFrdEdYQ1gxVlRqZjBpN1hxdEgwaTlRRENMWmhUdC1IMC13OTgyM28ifQ.eyJpc3MiOiJrdWJ cm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm hbWUiOiJhcGFjaGUtcHJveHktdG9rZW4tOGdrOXAiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiYXBhY2hlLXByb3h5Iiwia3ViZXJuZXRlcy5pby9zZ J2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZTA3ODg0YjQtMjgyYy00ZWYxLTg2NjItZTJhM2VhZjNjNDQ4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmFw WNoZS1wcm94eSJ9.qKzSvCYOqDfvJ1S9W4-nEzZScmGwjYsi3g5df542_aDyZRugbiWbFmjrBTB70tDwuGTeupbPmS3ptweyjbb_7K5VbSk-2zCazgNeiCZ_q7M4yXgPi6rNYNiBX9pQEBaBQCWgH8-VVF e34xrLR88cv9YNLfH9ZzrdJf2jPEBhptvdVKr6Ljpbhz-4P-mr5_IRsru_72wsRRZZptL80ARp6PkPdrYIQZ3bMQNsq3GEWxMl8SRPVqEuvVXykxfEt1Hx5URwtiwh_MLEZ5ClCIuGsGs8fWQhCLm_l0SY p9B2DmU-XhhT_HjuotDI3pm2p5pJb9WmO4dMLx_NsBnLSz_Q"
AuthType Basic
AuthName "Secure area - Authentication required"
AuthBasicAuthoritative Off
AuthBasicProvider ldap
AuthLDAPURL "ldap://192.168.15.10/CN=Users,DC=tech,DC=local?sAMAccountName?sub?(objectClass=*)"
AuthLDAPBindDN "bind@tech.local"
AuthLDAPBindPassword kamisama123..
AuthLDAPGroupAttribute member
Require ldap-group CN=KUBERNETES-ADMIN,CN=Users,DC=TECH,DC=LOCAL
ProxyPass https://10.107.55.24:443/
ProxyPassReverse https://10.107.55.24:443/
将名为 AuthLDAPURL
的配置项的 IP 地址更改为 LDAP 的 IP 地址,根据你自己的 LDAP 环境进行相应的配置,将名为 PROXYPASS
和 PROXYPASSREVERSE
的配置项的 IP 地址更改为 Dashboard 的 ClusterIP 地址。将名为 REQUESTHEADER
的配置项的 Token 值更改为以前创建的 Apache ServiceAccount 的 Token 值。
在示例中,我们使用自签名证书启用了 HTTPS。
SSLEngine on
SSLCertificateFile /etc/apache2/certificate/apache-certificate.crt
SSLCertificateKeyFile /etc/apache2/certificate/apache.key
此外我们还配置了 LDAP 身份验证的信息:
AuthType Basic
AuthName "Secure area - Authentication required"
AuthBasicAuthoritative Off
AuthBasicProvider ldap
AuthLDAPURL "ldap://192.168.15.10/CN=Users,DC=tech,DC=local?sAMAccountName?sub?(objectClass=*)"
AuthLDAPBindDN "bind@tech.local"
AuthLDAPBindPassword kamisama123..
AuthLDAPGroupAttribute member
Require ldap-group CN=KUBERNETES-ADMIN,CN=Users,DC=TECH,DC=LOCAL
Apache 将代理用户与 Dashboard 在集群中的 ClusterIP 之间进行 HTTPS 通信:
ProxyPass https://10.107.55.24:443/
ProxyPassReverse https://10.107.55.24:443/
Apache 将使用在 Kubernetes 服务器安装期间自动创建的证书和密钥对 Dashboard 执行相互 TLS 身份验证:
SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
SSLProxyCACertificateFile /etc/kubernetes/pki/ca.crt
SSLProxyMachineCertificateFile /etc/apache2/certificate/front-proxy-client.pem
此外 Apache 服务器还将向发送到 Dashboard 的所有数据包添加 Header 头信息,用来配置 Token 信息:
RequestHeader set Authorization "Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6InhrMHFrdEdYQ1gxVlRqZjBpN1hxdEgwaTlRRENMWmhUdC1IMC13OTgyM28ifQ.eyJpc3MiOiJrdWJ cm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm hbWUiOiJhcGFjaGUtcHJveHktdG9rZW4tOGdrOXAiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiYXBhY2hlLXByb3h5Iiwia3ViZXJuZXRlcy5pby9zZ J2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZTA3ODg0YjQtMjgyYy00ZWYxLTg2NjItZTJhM2VhZjNjNDQ4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmFw WNoZS1wcm94eSJ9.qKzSvCYOqDfvJ1S9W4-nEzZScmGwjYsi3g5df542_aDyZRugbiWbFmjrBTB70tDwuGTeupbPmS3ptweyjbb_7K5VbSk-2zCazgNeiCZ_q7M4yXgPi6rNYNiBX9pQEBaBQCWgH8-VVF e34xrLR88cv9YNLfH9ZzrdJf2jPEBhptvdVKr6Ljpbhz-4P-mr5_IRsru_72wsRRZZptL80ARp6PkPdrYIQZ3bMQNsq3GEWxMl8SRPVqEuvVXykxfEt1Hx5URwtiwh_MLEZ5ClCIuGsGs8fWQhCLm_l0SY p9B2DmU-XhhT_HjuotDI3pm2p5pJb9WmO4dMLx_NsBnLSz_Q"
此外 Apache 还会将 HTTP 用户重定向到 HTTPS :
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1
最后配置完成后,我们可以重新启动 Apache 服务:
service apache2 restart
配置完成后我们就可以在浏览器中访问 Apache 服务器的 IP 地址,我们这里在浏览器中输入https://192.168.15.200
即可,Apache 服务器会要求执行用户身份验证。
登录成功后,就可以正常使用 Kubernetes Dashboard 了:
Apache 代理将自动使用名为"AUTH HEADER"
的功能在 Kubernetes Dashboard 上执行身份验证。
到这里我们就成功完成了使用 Apache 作为 Kubernetes Dashboard 的代理配置。
K8S进阶训练营,点击下方图片了解详情