因为etcd新版本和旧版本在使用的命令上有写不同,为了验证etcd的功能,我这里就先安装旧版本的etcd,在下一个章节的etcd界面可视化的时候会针对etcd新版本的使用,做进一步的说明。先安装etcd,每个节点上都同样的部署,这里就值写一个节点的部署方法,其他两个节点的部署方法一样:
安装节点
下载etcd文件,并且解压后方到指定的路径下。
[root@www.wulaoer.org ~]# wget https://github.com/etcd-io/etcd/releases/download/v3.3.10/etcd-v3.3.10-linux-amd64.tar.gz [root@www.wulaoer.org ~]# tar -zxf etcd-v3.3.10-linux-amd64.tar.gz [root@www.wulaoer.org ~]# mv etcd-v3.3.10-linux-amd64 /usr/local/etcd [root@www.wulaoer.org ~]# cd /usr/local/etcd/
配置etcd信息
这里配置etcd的配置信息,以及etcd的数据存储位置
[root@www.wulaoer.org etcd]# cp etcd* /usr/local/bin/ [root@www.wulaoer.org ~]# mkdir /usr/local/etcd/{data,config} [root@www.wulaoer.org ~]# cat <<EOF | tee /usr/local/etcd/config/etcd.conf > #节点名称 > ETCD_NAME=$(hostname -s) > #数据存放位置 > ETCD_DATA_DIR=/usr/local/etcd/data > EOF
设置etcd的开机自启动信息,使用systemctl启动etcd服务
[root@www.wulaoer.org ~]# cat <<EOF | tee /etc/systemd/system/etcd.service > [Unit] > Description=Etcd Server > Documention=https://github.com/coreos/etcd > After=network.target > [Service] > User=root > Type=notify > EnvironmentFile=/usr/local/etcd/config/etcd.conf > ExecStart=/usr/local/etcd/data > RestartSec=10s > LimitNOFILE=40000 > [Install] > WantedBy=multi-user.target > EOF [Unit] Description=Etcd Server Documention=https://github.com/coreos/etcd After=network.target [Service] User=root Type=notify EnvironmentFile=/usr/local/etcd/config/etcd.conf ExecStart=/usr/local/etcd/etcd RestartSec=10s LimitNOFILE=40000 [Install] WantedBy=multi-user.target
按照以上方法,在另外两个节点上安装,不需要修改任何地方,配置后,按照下面的方法进行重启。
启动etcd服务
使用systemctl启动etcd服务,没有报错。不过注意,如果配置好集群之后需要把数据删除,如果不删除会出现所有节点都是集群。
[root@www.wulaoer.org ~]# systemctl daemon-reload && systemctl enable etcd && systemctl start etcd [root@wulaoer etcd]# systemctl daemon-reload && systemctl enable etcd && systemctl start etcd Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /etc/systemd/system/etcd.service. [root@wolf etcd]# systemctl daemon-reload && systemctl enable etcd && systemctl start etcd Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /etc/systemd/system/etcd.service.
所有节点已经配置好,并且已经启动没有错误,如果启动出现问题可以根据下面的查看日志的方法,解决问题。
查看etcd日志
[root@www.wulaoer.org ~]# journalctl -u etcd
我们可以根据日志查看etcd启动失败的原因,然后解决问题。
配置静态集群
所有节点已经配置好etcd,并且启动了服务,下面开始配置etcd配置信息和启动etcd信息,节点www.wulaoer.org的配置。
[root@www.wulaoer.org ~]# vim /usr/local/etcd/config/etcd.conf #节点名称 ETCD_NAME=www.wulaoer.org #数据存放位置 ETCD_DATA_DIR=/usr/local/etcd/data #集群信息 ETCD_LISTEN_PEER_URLS="http://192.168.111.128:2380" ETCD_LISTEN_CLIENT_URLS="http://192.168.111.128:2379,http://127.0.0.1:2379" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.111.128:2380" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.111.128:2379" ETCD_INITIAL_CLUSTER="www.wulaoer.org=http://192.168.111.128:2380,wulaoer=http://192.168.111.129:2380,wolf=http://192.168.111.130:2380" #这里的www.wulaoer.org,wulaoer,wolf是节点名称需要和上面的匹配 ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" [root@www.wulaoer.org ~]# vim /etc/systemd/system/etcd.service [Unit] Description=Etcd Server Documentation=https://github.com/coreos/etcd After=network.target [Service] User=root Type=notify WorkingDirectory=/usr/local/etcd/ EnvironmentFile=-/usr/local/etcd/config/etcd.conf ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/etcd/etcd \ --name=\"${ETCD_NAME}\" \ --data-dir=\"${ETCD_DATA_DIR}\" \ --listen-peer-urls=\"${ETCD_LISTEN_PEER_URLS}\" \ --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" \ --initial-advertise-peer-urls=\"${ETCD_INITIAL_ADVERTISE_PEER_URLS}\" \ --advertise-client-urls=\"${ETCD_ADVERTISE_CLIENT_URLS}\" \ --initial-cluster=\"${ETCD_INITIAL_CLUSTER}\" \ --initial-cluster-token=\"${ETCD_INITIAL_CLUSTER_TOKEN}\" \ --initial-cluster-state=\"${ETCD_INITIAL_CLUSTER_STATE}\"" Restart=on-failure RestartSec=10s LimitNOFILE=40000 [Install] WantedBy=multi-user.target
节点wulaoer的配置信息和启动信息
[root@wulaoer ~]# cat /usr/local/etcd/config/etcd.conf #节点名称 ETCD_NAME=wulaoer #数据存放位置 ETCD_DATA_DIR=/usr/local/etcd/data #集群信息 ETCD_LISTEN_PEER_URLS="http://192.168.111.129:2380" ETCD_LISTEN_CLIENT_URLS="http://192.168.111.129:2379,http://127.0.0.1:2379" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.111.129:2380" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.111.129:2379" ETCD_INITIAL_CLUSTER="www.wulaoer.org=http://192.168.111.128:2380,wulaoer=http://192.168.111.129:2380,wolf=http://192.168.111.130:2380" #这里的www.wulaoer.org,wulaoer,wolf是节点名称需要和上面的匹配 ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" [root@wulaoer ~]# cat /etc/systemd/system/etcd.service [Unit] Description=Etcd Server Documentation=https://github.com/coreos/etcd After=network.target [Service] User=root Type=notify WorkingDirectory=/usr/local/etcd/ EnvironmentFile=-/usr/local/etcd/config/etcd.conf ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \ --name=\"${ETCD_NAME}\" \ --data-dir=\"${ETCD_DATA_DIR}\" \ --listen-peer-urls=\"${ETCD_LISTEN_PEER_URLS}\" \ --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" \ --initial-advertise-peer-urls=\"${ETCD_INITIAL_ADVERTISE_PEER_URLS}\" \ --advertise-client-urls=\"${ETCD_ADVERTISE_CLIENT_URLS}\" \ --initial-cluster=\"${ETCD_INITIAL_CLUSTER}\" \ --initial-cluster-token=\"${ETCD_INITIAL_CLUSTER_TOKEN}\" \ --initial-cluster-state=\"${ETCD_INITIAL_CLUSTER_STATE}\"" Restart=on-failure RestartSec=10s LimitNOFILE=40000 [Install] WantedBy=multi-user.target
节点wolf的配置信息和启动信息
[root@wolf ~]# cat /usr/local/etcd/config/etcd.conf #节点名称 ETCD_NAME=wolf #数据存放位置 ETCD_DATA_DIR=/usr/local/etcd/data #集群信息 ETCD_LISTEN_PEER_URLS="http://192.168.111.130:2380" ETCD_LISTEN_CLIENT_URLS="http://192.168.111.130:2379,http://127.0.0.1:2379" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.111.130:2380" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.111.130:2379" ETCD_INITIAL_CLUSTER="www.wulaoer.org=http://192.168.111.128:2380,wulaoer=http://192.168.111.129:2380,wolf=http://192.168.111.130:2380" #这里的www.wulaoer.org,wulaoer,wolf是节点名称需要和上面的匹配 ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" [root@wolf ~]# cat /etc/systemd/system/etcd.service [Unit] Description=Etcd Server Documentation=https://github.com/coreos/etcd After=network.target [Service] User=root Type=notify WorkingDirectory=/usr/local/etcd/ EnvironmentFile=-/usr/local/etcd/config/etcd.conf ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \ --name=\"${ETCD_NAME}\" \ --data-dir=\"${ETCD_DATA_DIR}\" \ --listen-peer-urls=\"${ETCD_LISTEN_PEER_URLS}\" \ --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" \ --initial-advertise-peer-urls=\"${ETCD_INITIAL_ADVERTISE_PEER_URLS}\" \ --advertise-client-urls=\"${ETCD_ADVERTISE_CLIENT_URLS}\" \ --initial-cluster=\"${ETCD_INITIAL_CLUSTER}\" \ --initial-cluster-token=\"${ETCD_INITIAL_CLUSTER_TOKEN}\" \ --initial-cluster-state=\"${ETCD_INITIAL_CLUSTER_STATE}\"" Restart=on-failure RestartSec=10s LimitNOFILE=40000 [Install] WantedBy=multi-user.target
所有节点信息配置之后需要启动一下所有节点,安装下面的顺序,先关闭防火墙在启动etcd服务
[root@www.wulaoer.org ~]# systemctl stop firewalld [root@www.wulaoer.org ~]# systemctl disable firewalld [root@www.wulaoer.org ~]# systemctl daemon-reload [root@www.wulaoer.org ~]# systemctl restart etcd.service [root@www.wulaoer.org ~]# systemctl enable etcd.service
启动之后,为了避免节点时间统一,所以我们需要做一个定时任务,统一时间
[root@www.wulaoer.org ~]# crontab -l 0 1 * * * /usr/sbin/ntpdate ntp.sjtu.edu.cn >> /var/log/ntpdate.log 2>&1 & [root@www.wulaoer.org ~]# systemctl reload crond.service
验证etcd集群
所有节点的etcd服务已经启动了,也做了时间同步,我们看一下我们的etcd集群是否创建成功,先检查一下etcd的状态。
[root@www.wulaoer.org ~]# etcdctl cluster-health member 25f2724acdf5b013 is healthy: got healthy result from http://192.168.111.128:2379 member d3ddf28b4a611036 is healthy: got healthy result from http://192.168.111.130:2379 member ebecc1c831d00be2 is healthy: got healthy result from http://192.168.111.129:2379 cluster is healthy
也可以这样
[root@www.wulaoer.org ~]# ETCDCTL_API=3 etcdctl --endpoints=http://192.168.111.128:2379,http://192.168.111.130:2379,http://192.168.111.129:2379 endpoint health http://192.168.111.130:2379 is healthy: successfully committed proposal: took = 1.307688ms http://192.168.111.129:2379 is healthy: successfully committed proposal: took = 2.050772ms http://192.168.111.128:2379 is healthy: successfully committed proposal: took = 2.464973ms
或者
[root@www.wulaoer.org ~]# etcdctl --endpoints=http://192.168.111.128:2379,http://192.168.111.130:2379,http://192.168.111.129:2379 cluster-health member 25f2724acdf5b013 is healthy: got healthy result from http://192.168.111.128:2379 member d3ddf28b4a611036 is healthy: got healthy result from http://192.168.111.130:2379 member ebecc1c831d00be2 is healthy: got healthy result from http://192.168.111.129:2379 cluster is healthy
查看一下集群中的leader选举是否已经产出
[root@www.wulaoer.org ~]# etcdctl member list 25f2724acdf5b013: name=www.wulaoer.org peerURLs=http://192.168.111.128:2380 clientURLs=http://192.168.111.128:2379 isLeader=true d3ddf28b4a611036: name=wolf peerURLs=http://192.168.111.130:2380 clientURLs=http://192.168.111.130:2379 isLeader=false ebecc1c831d00be2: name=wulaoer peerURLs=http://192.168.111.129:2380 clientURLs=http://192.168.111.129:2379 isLeader=false
第一个已经是leader了,我们做一个验证把www.wulaoer.org服务停掉,看一下集群是否重新选举leader。
[root@www.wulaoer.org ~]# systemctl stop etcd [root@www.wulaoer.org ~]# systemctl start etcd [root@www.wulaoer.org ~]# etcdctl member list 25f2724acdf5b013: name=www.wulaoer.org peerURLs=http://192.168.111.128:2380 clientURLs=http://192.168.111.128:2379 isLeader=false d3ddf28b4a611036: name=wolf peerURLs=http://192.168.111.130:2380 clientURLs=http://192.168.111.130:2379 isLeader=false ebecc1c831d00be2: name=wulaoer peerURLs=http://192.168.111.129:2380 clientURLs=http://192.168.111.129:2379 isLeader=true
停掉www.wulaoer.org节点的etcd服务之后,立刻有wulaoer节点的etcd担任了leader,服务管理其他服务的数据。在www.wulaoer.org停掉服务的过程中,在其他节点上检查服务的状态就会出现错误。
[root@wulaoer ~]# etcdctl cluster-health failed to check the health of member 25f2724acdf5b013 on http://192.168.111.128:2379: Get http://192.168.111.128:2379/health: dial tcp 192.168.111.128:2379: connect: connection refused member 25f2724acdf5b013 is unreachable: [http://192.168.111.128:2379] are all unreachable member d3ddf28b4a611036 is healthy: got healthy result from http://192.168.111.130:2379 member ebecc1c831d00be2 is healthy: got healthy result from http://192.168.111.129:2379 cluster is degraded
我们测试一下数据的安全问题,我现在www.wulaoer.org节点上创建一个键值,然后把www.wulaoer.org节点stop掉,在wulaoer节点上去获取创建的键值,看是否成功。
[root@www.wulaoer.org ~]# etcdctl set --ttl '0' wulaoer "www.wulaoer.org" www.wulaoer.org [root@wulaoer ~]# etcdctl get wulaoer www.wulaoer.org [root@wolf ~]# etcdctl get wulaoer www.wulaoer.org
集群中的数据存储不会丢失,这时我们可以在etcd的集群前加一个负载均衡,根据etcd的2379端口做一个自动检测,如果服务不可用自动从负载均衡上去掉,例如nginx的反向代理就可以,或者lvs以及proxy。
您可以选择一种方式赞助本站
支付宝扫一扫赞助
微信钱包扫描赞助
赏