一、keadm安装
有多种安装方式,从Github下载和Docker Hub及源码编译,此处主要写前两种安装方式,第三种可参考官方文档。注意云端和边缘端都需要安装,且云端和边缘端使用的 kubeEdge 版本应该是相同的。
1、从Github下载安装
wget https://github.com/kubeedge/kubeedge/releases/download/v1.12.1/keadm-v1.12.1-linux-amd64.tar.gz
tar -zxvf keadm-v1.12.1-linux-amd64.tar.gz
cp keadm-v1.12.1-linux-amd64/keadm/keadm /usr/local/bin/keadm
2、利用dockerhub的KubeEdge 官方发布镜像下载安装。
docker run --rm kubeedge/installation-package:v1.12.1 cat /usr/local/bin/keadm > /usr/local/bin/keadm && chmod +x /usr/local/bin/keadm
3、验证是否安装成功
执行命令
keadm version
输出如下说明安装成功
version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.1-dirty", GitCommit:"919ad5378eaca3cb0c666c22a19db01261cbc9a6", GitTreeState:"dirty", BuildDate:"2022-11-02T08:29:56Z", GoVersion:"go1.17", Compiler:"gc", Platform:"linux/amd64"}
二、设置云端(KubeEdge主节点)
keadm init 提供了集成 cloudcore helm chart的解决方案,cloudcore 将以容器模式部署到云端节点。
执行如下init命令(ip地址根据实际情况填写)
keadm init --advertise-address="192.168.31.223" --profile version=v1.12.1 --kube-config=/root/.kube/config
命令输出如下:
Kubernetes version verification passed, KubeEdge installation will start...
CLOUDCORE started
=========CHART DETAILS=======
NAME: cloudcore
LAST DEPLOYED: Mon Jan 30 16:02:03 2023
NAMESPACE: kubeedge
STATUS: deployed
REVISION: 1
可以运行命令kubectl get all -n kubeedge 查看结果,确保cloudcore像下面一样启动成功
# kubectl get all -n kubeedge
NAME READY STATUS RESTARTS AGE
pod/cloudcore-5876c76687-w6669 1/1 Running 0 55s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cloudcore ClusterIP 172.16.39.45 <none> 10000/TCP,10001/TCP,10002/TCP,10003/TCP,10004/TCP 55s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/cloudcore 1/1 1 1 55s
NAME DESIRED CURRENT READY AGE
replicaset.apps/cloudcore-5876c76687 1 1 1 55s
三、设置边缘端(KubeEdge工作节点)
1、从云端获取Token
在云端运行 keadm gettoken 将返回令牌,这将在连接边缘节点时使用。
# keadm gettoken
27a37ef16159f7d3be8fae95d588b79b3adaaf92727b72659eb89758c66ffda2.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTAyMTYwNzd9.JBj8LLYWXwbbvHKffJBpPd5CyxqapRQYDIXtFZErgYE
2、边缘端节点加入
Keadm join 将安装 edgecore。它还提供了一个可以设置特定版本的标志。它将从 dockerhub 获取映像 kubeedge/install-package,并将二进制 edgecore 从容器复制到主机路径,然后将 edgecore 作为系统服务启动。
keadm join --cloudcore-ipport="192.168.31.233":10000 --token=27a37ef16159f7d3be8fae95d588b79b3adaaf92727b72659eb89758c66ffda2.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTAyMTYwNzd9.JBj8LLYWXwbbvHKffJBpPd5CyxqapRQYDIXtFZErgYE --kubeedge-version=v1.12.1
执行后输出如下:
I0130 14:10:50.991981 3246 join.go:100] 9. Run EdgeCore daemon
I0130 14:10:53.013548 3246 join.go:317]
I0130 14:10:53.013661 3246 join.go:318] KubeEdge edgecore is running, For logs visit: journalctl -u edgecore.service -xe
接着使用命令journalctl -u edgecore.service -xe 查看日志,发现有许多如下错误信息
Jan 30 14:30:49 edge edgecore[3660]: I0130 14:30:49.870215 3660 operation_generator.go:867] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/94b357f7-06ab-46c1-829f-bd9854c0689>
Jan 30 14:30:49 edge edgecore[3660]: E0130 14:30:49.871317 3660 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input
Jan 30 14:30:49 edge edgecore[3660]: W0130 14:30:49.871371 3660 driver-call.go:149] FlexVolume: driver call failed: executable: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/ud>
Jan 30 14:30:49 edge edgecore[3660]: E0130 14:30:49.871423 3660 plugins.go:746] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Er>
Jan 30 14:30:49 edge edgecore[3660]: I0130 14:30:49.872052 3660 operation_generator.go:867] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/94b357f7-06ab-46c1-829f-bd9854c0689>
Jan 30 14:30:49 edge edgecore[3660]: I0130 14:30:49.872156 3660 reconciler.go:196] "operationExecutor.UnmountVolume started for volume \"policysync\" (UniqueName: \"kubernetes.io/host-path/94b357f>
Jan 30 14:30:49 edge edgecore[3660]: E0130 14:30:49.872748 3660 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input
Jan 30 14:30:49 edge edgecore[3660]: W0130 14:30:49.872812 3660 driver-call.go:149] FlexVolume: driver call failed: executable: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/ud>
Jan 30 14:30:49 edge edgecore[3660]: E0130 14:30:49.872899 3660 plugins.go:746] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Er
由于edge节点没有部署kubelet, Calico pod调度到了边缘节点会创建失败。不影响KubeEdge的使用,可以先忽略这个问题。
考虑到边缘端很多情况存在不稳定通信,不适合运行 Calico 这种 CNI 组件,更多使用 EdgeMesh 进行云边通信和服务发现,因此解决方式也很简单,我们可以手动 Patch Pod 以防止非边缘节点调度至工作节点。
执行patch.sh脚本后使用命令journalctl -u edgecore.service -xe 查看日志停止输出错误,其中脚本内容如下:
#!/bin/bash
NoShedulePatchJson='{"spec":{"template":{"spec":{"affinity":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-role.kubernetes.io/edge","operator":"DoesNotExist"}]}]}}}}}}}'
ns="kube-system"
DaemonSets=("nodelocaldns" "kube-proxy" "calico-node")
length=${#DaemonSets[@]}
for((i=0;i<length;i++));
do
ds=${DaemonSets[$i]}
echo "Patching resources:DaemonSet/${ds}" in ns:"$ns",
kubectl -n $ns patch DaemonSet/${ds} --type merge --patch "$NoShedulePatchJson"
sleep 1
done
四、验证安装结果
边缘端在启动edgecore后,会与云端的cloudcore进行通信,K8s进而会将边缘端作为一个node纳入K8s的管控。
在云端节点执行kubectl get nodes 和kubectl get pods -n kube-system命令查看是否安装成功
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 3h22m v1.19.6
edge Ready agent,edge 21m v1.22.6-kubeedge-v1.12.1
# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-bc4f7c685-4c87x 1/1 Running 0 3h24m
calico-node-sn44f 1/1 Running 0 5m34s
coredns-6d56c8448f-82gpp 1/1 Running 0 3h26m
coredns-6d56c8448f-n9lcg 1/1 Running 0 3h26m
etcd-k8s-master 1/1 Running 0 3h26m
kube-apiserver-k8s-master 1/1 Running 0 3h26m
kube-controller-manager-k8s-master 1/1 Running 0 3h26m
kube-proxy-ghd6k 1/1 Running 0 5m34s
kube-scheduler-k8s-master 1/1 Running 0 3h26m
五、测试从云侧下发任务到边缘侧
KubeEdge提供了一个nginx模板,我们可以直接使用
kubectl apply -f https://github.com/kubeedge/kubeedge/raw/master/build/deployment.yaml
由于github下载文件不稳定,可以直接在本地创建了nginx.yaml文件,复制deployment.yaml内容进去
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
hostPort: 80
部署应用
# kubectl apply -f nginx.yaml
deployment.apps/nginx-deployment created
# kubectl get pod -A -owide | grep nginx
default nginx-deployment-77f96fbb65-d657k 1/1 Running 0 36s 172.17.0.3 edge <none> <none>
测试功能是否正常,进入边缘节点,curl nginx的IP:172.17.0.3
# curl 172.17.0.3
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
至此,部署KubeEdge成功完成。