Working draft of Atomic fleet config

This is the first draft at making a repeatable workflow, I’m looking to get a few more eyes to make sure I’ve captured everything.  My next step is getting this into an ansible playbook.  Replace the inline IPs with the eth0 IPs of the VMs you build.  Hit the link to see the steps and very spare comments:

F21 Master

Build (or use Cloud Base) of F21 Server, yum updated to latest.
[fedora@f21master ~]$ sudo yum -y install kubernetes etcd ansible git
[fedora@f21master kubernetes]$ sudo vi config
KUBE_ETCD_SERVERS=”–etcd_servers=http://10.4.0.88:4001″

[fedora@f21master kubernetes]$ sudo vi apiserver
# The address on the local server to listen to.
KUBE_API_ADDRESS=”–address=0.0.0.0″
# How the replication controller and scheduler find the kube-apiserver
KUBE_MASTER=”–master=10.4.0.88:8080″

[fedora@f21master kubernetes]$ sudo vi controller-manager
# Comma seperated list of minions
KUBELET_ADDRESSES=”–machines=10.4.0.78,10.4.0.79,10.4.0.80″

[fedora@f21master ~]$ sudo systemctl start etcd kube-apiserver kube-controller-manager kube-scheduler
[fedora@f21master ~]$ sudo systemctl status etcd kube-apiserver kube-controller-manager kube-scheduler

[fedora@f21master ~]$ vi flanneld-conf.json
{
“Network”: “172.16.0.0/12”,
“SubnetLen”: 24,
“Backend”: {
“Type”: “vxlan”
}
}
[fedora@f21master ~]$ curl -L http://localhost:4001/v2/keys/coreos.com/network/config -XPUT –data-urlencode value@flanneld-conf.json

[fedora@f21master ~]$ curl -L http://localhost:4001/v2/keys/coreos.com/network/config | python -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 221 0 221 0 0 33408 0 –:–:– –:–:– –:–:– 36833
{
“action”: “get”,
“node”: {
“createdIndex”: 11,
“key”: “/coreos.com/network/config”,
“modifiedIndex”: 11,
“value”: “{n “Network”: “172.16.0.0/12”,n “SubnetLen”: 24,n “Backend”: {n “Type”: “vxlan”n }n}nn”
}
}

Atomic Minions

Build latest F21 Atomic base, use virt platform of choice. Tested with KVM and EC2. Repeat for each Atomic minion.
[fedora@f21master ~]$ ssh 10.4.0.120 sudo atomic upgrade
[fedora@f21master ~]$ ssh 10.4.0.120 sudo systemctl reboot

[fedora@f21master ~]$ rpm -q etcd docker-io kubernetes
etcd-0.4.6-6.fc21.x86_64
docker-io-1.4.0-1.fc21.x86_64
kubernetes-0.7.0-18.0.git52e165a.fc21.x86_64
[fedora@f21master ~]$ ssh 10.4.0.120 sudo rpm -q flannel etcd docker-io kubernetes
flannel-0.2.0-1.fc21.x86_64
etcd-0.4.6-6.fc21.x86_64
docker-io-1.4.0-1.fc21.x86_64
kubernetes-0.7.0-18.0.git52e165a.fc21.x86_64

[fedora@f21master ~]$ ssh 10.4.0.120 curl -L http://10.4.0.88:4001/v2/keys/coreos.com/network/config
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 221{“action”:”get”,”node”:{“key”:”/coreos.com/network/config”,”value”:”{n “Network”: “172.16.0.0/12”,n “SubnetLen”: 24,n “Backend”: {n “Type”: “vxlan”n }n}nn”,”modifiedIndex”:11,”createdIndex”:11}} 0 221 0 0 50769 0 –:–:– –:–:– –:–:– 73666

Atomic minion configuration, repeat on every minion.
[fedora@atomic01 ~]$ sudo vi /etc/sysconfig/flanneld
FLANNEL_ETCD=”http://10.4.0.88:4001″

[fedora@atomic01 ~]$ sudo cp /usr/lib/systemd/system/docker.service /etc/systemd/system/
[fedora@atomic01 ~]$ sudo vi /etc/systemd/system/docker.service
After=flanneld.service docker.socket
Requires=flanneld.service docker.socket

EnvironmentFile=/run/flannel/subnet.env
ExecStartPre=-/usr/sbin/ip link del docker0
ExecStart=/usr/bin/docker -d -H fd:// –bip=${FLANNEL_SUBNET} –mtu=${FLANNEL_MTU} $OPTIONS $DOCKER_STORAGE_OPTIONS

[fedora@atomic01 ~]$ sudo vi /etc/kubernetes/kubelet
# The address for the info server to serve on (set to 0.0.0.0 or “” for all interfaces)
KUBELET_ADDRESS=”–address=10.4.0.120″

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME=”–hostname_override=10.4.0.120″

[fedora@atomic01 ~]$ sudo vi /etc/kubernetes/config
# Comma seperated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS=”–etcd_servers=http://10.4.0.88:4001″
[fedora@atomic01 ~]$ sudo systemctl daemon-reload

[fedora@atomic01 ~]$ sudo systemctl start flanneld docker kubelet kube-proxy

[fedora@atomic01 ~]$ ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
link/ether 0a:45:46:8d:6a:de brd ff:ff:ff:ff:ff:ff
inet 10.4.0.120/24 brd 10.4.0.255 scope global dynamic eth0
valid_lft 3570sec preferred_lft 3570sec
inet6 fe80::845:46ff:fe8d:6ade/64 scope link
valid_lft forever preferred_lft forever
3: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UNKNOWN group default
link/ether 1a:50:6d:23:5d:a2 brd ff:ff:ff:ff:ff:ff
inet 172.16.36.0/12 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::1850:6dff:fe23:5da2/64 scope link
valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 56:84:7a:fe:97:99 brd ff:ff:ff:ff:ff:ff
inet 172.16.36.1/24 scope global docker0
valid_lft forever preferred_lft forever

[ Repeat on all minions ]

Kubernetes testing on Atomic fleet, use the latest upstream guestbook example

[fedora@f21master ~]$ kubectl get minions
[fedora@f21master ~]$ git clone https://github.com/GoogleCloudPlatform/kubernetes.git
[fedora@f21master ~]$ cd kubernetes/examples/guestbook
[fedora@f21master guestbook]$ kubectl create -f redis-master.json
[fedora@f21master guestbook]$ kubectl get pod redis-master
[fedora@f21master guestbook]$ kubectl create -f redis-master-service.json
[fedora@f21master guestbook]$ kubectl get service redis-master
[fedora@f21master guestbook]$ kubectl create -f redis-slave-controller.json
[fedora@f21master guestbook]$ kubectl get replicationController redisSlaveController
[fedora@f21master guestbook]$ kubectl get pods
NAME IMAGE(S) HOST LABELS STATUS
redis-master dockerfile/redis 10.4.0.120/ name=redis-master Running
067a3550-9b39-11e4-bd02-0af36a4797bc brendanburns/redis-slave 10.4.0.121/ name=redisslave,uses=redis-master Pending
067a746a-9b39-11e4-bd02-0af36a4797bc brendanburns/redis-slave 10.4.0.122/ name=redisslave,uses=redis-master Pending

Add publicIPs line for an “external” service, match with eth0 IP of a minion
[fedora@f21master guestbook]$ cat frontend-service.json
{
“id”: “frontend”,
“kind”: “Service”,
“apiVersion”: “v1beta1”,
“port”: 80,
“publicIPs”: [“10.4.0.118”],
“containerPort”: 80,
“selector”: {
“name”: “frontend”
},
“labels”: {
“name”: “frontend”
}
}

Leave a Reply

Your email address will not be published. Required fields are marked *