基于Docker的Etcd集群搭建
cfssl的使用
下载cfssl可执行文件
到https://pkg.cfssl.org下载相应系统的cfssl与cfssljson
生成根证书
创建ca配置文件
ca-config.json
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"server": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
},
"client": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
},
"peer": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
创建CA证书签名请求
ca-csr.json
{
"CN": "EECHAIN CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "CA",
"O": "eechain.com",
"ST": "BeiJing",
"OU": "etcd-cluster"
}
]
}
生成ca根证书
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
生成server端证书
创建server端配置文件
server.json
{
"CN": "server",
"hosts": [
"localhost",
"0.0.0.0",
"127.0.0.1",
"192.16.32.89",
"192.16.32.87"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "CN",
"ST": "CA",
"L": "BeiJing"
}
]
}
生成服务端证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server.json | cfssljson -bare server
生成节点证书
创建各个节点配置文件
member[i].json
{
"CN": "member1",
"hosts": [
"192.16.32.89",
"127.0.0.1",
"etcd.ee-chain.com1",
"member1"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}
生成节点证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer member1.json | cfssljson -bare member1
生成客户端证书
创建客户端配置文件
client.json
{
"CN": "client",
"hosts": [""],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "CN",
"ST": "CA",
"L": "BeiJing"
}
]
}
生成客户端证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client
各个etcd节点的启动
对各个节点，改掉THIS_NAME与THIS_IP，然后启动
REGISTRY=quay.io/coreos/etcd
ETCD_VERSION=latest
TOKEN=my-etcd-token
CLUSTER_STATE=new
CLUSTER=etcd-node-1=https://192.16.32.89:2380,etcd-node-2=https://192.16.32.87:2380
THIS_NAME=etcd-node-1
THIS_IP=192.16.32.89
func run(){
docker run -d -p 2379:2379 -p 2380:2380 \
--volume=${DATA_DIR}:/etcd-data \
--volume=/opt/app/ebcgateway/etcd/cfssl:/etc/ssl/etcd \
--name etcd ${REGISTRY}:${ETCD_VERSION} \
/usr/local/bin/etcd \
--data-dir=/etcd-data \
--name ${THIS_NAME} \
--initial-advertise-peer-urls https://${THIS_IP}:2380 \
--listen-peer-urls https://0.0.0.0:2380 \
--advertise-client-urls https://${THIS_IP}:2379 \
--listen-client-urls https://0.0.0.0:2379 \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} \
--initial-cluster ${CLUSTER} \
--cert-file=/etc/ssl/etcd/server.pem \
--key-file=/etc/ssl/etcd/server-key.pem \
--client-cert-auth \
--trusted-ca-file=/etc/ssl/etcd/ca.pem \
--peer-client-cert-auth \
--peer-trusted-ca-file=/etc/ssl/etcd/ca.pem \
--peer-cert-file=/etc/ssl/etcd/{{PEER_NAME}}.pem \
--peer-key-file=/etc/ssl/etcd/{{PEER_NAME}}-key.pem
}
各个参数含义
* 客户端到服务器通信：*
--cert-file = <path>：用于与etcd的SSL/TLS连接的证书。设置此选项时，advertise-client-urls可以使用HTTPS模式。
--key-file = <path>：证书的密钥。必须未加密。
--client-cert-auth：设置此选项后，etcd将检查所有来自受信任CA签署的客户端证书的传入HTTPS请求，因此不提供有效客户端证书的请求将失败。
--trusted-ca-file = <path>：受信任的证书颁发机构。
* 对等（服务器到服务器/集群）通信：*
对等选项的工作方式与客户端到服务器选项的工作方式相同：
--peer-cert-file = <path>：用于对等体之间SSL / TLS连接的证书。这将用于侦听对等体地址以及向其他对等体发送请求。
--peer-key-file = <path>：证书的密钥。必须未加密。
--peer-client-cert-auth：设置时，etcd将检查来自集群的所有传入对等请求，以获得由提供的CA签名的有效客户端证书。
--peer-trusted-ca-file = <path>：受信任的证书颁发机构。
测试
在其中一个节点进入容器.
docker exec -it etcd /bin/sh执行以下命令列出集群成员
ETCDCTL_API=3 etcdctl --cacert /etc/ssl/etcd/ca.pem --cert /etc/ssl/etcd/client.pem --key /etc/ssl/etcd/client-key.pem --endpoints=https://47.96.235.92:2379 member list执行以下命令进行健康检查
etcdctl \ --ca-file=/etc/ssl/etcd/ca.pem \ --cert-file=/etc/ssl/etcd/server.pem \ --key-file=/etc/ssl/etcd/server-key.pem \ --endpoints=https://47.96.235.92:2379,https://192.16.32.89:2379\ cluster-health执行以下命令进行存取检查
ETCDCTL_API=3 etcdctl --cacert /etc/ssl/etcd/ca.pem --cert /etc/ssl/etcd/client.pem --key /etc/ssl/etcd/client-key.pem --endpoints=https://192.16.32.87:2379,https://192.16.32.89:2379 put firstname 'zhang' ETCDCTL_API=3 etcdctl --cacert /etc/ssl/etcd/ca.pem --cert /etc/ssl/etcd/client.pem --key /etc/ssl/etcd/client-key.pem --endpoints=https://192.16.32.87:2379,https://192.16.32.89:2379 get firstname
在本地执行以下命令进行客户端请求验证.
curl --cacert /path/ca.pem --cert /path/client.pem --key /path/client-key.pem \ -L https://192.16.32.87:2379/v2/keys/foo -XPUT -d value=bar -v