OVN Kubernetes插件

ovn-kubernetes提供了一个ovs OVN网络插件,支持underlay和overlay两种模式。

  • underlay:容器运行在虚拟机中,而ovs则运行在虚拟机所在的物理机上,OVN将容器网络和虚拟机网络连接在一起
  • overlay:OVN通过logical overlay network连接所有节点的容器,此时ovs可以直接运行在物理机或虚拟机上

Overlay模式

OVN Kubernetes插件 - 图1

(图片来自https://imgur.com/i7sci9O)

配置master

  1. ovs-vsctl set Open_vSwitch . external_ids:k8s-api-server="127.0.0.1:8080"
  2. ovn-k8s-overlay master-init \
  3. --cluster-ip-subnet="192.168.0.0/16" \
  4. --master-switch-subnet="192.168.1.0/24" \
  5. --node-name="kube-master"

配置Node

  1. ovs-vsctl set Open_vSwitch . \
  2. external_ids:k8s-api-server="$K8S_API_SERVER_IP:8080"
  3. ovs-vsctl set Open_vSwitch . \
  4. external_ids:k8s-api-server="https://$K8S_API_SERVER_IP" \
  5. external_ids:k8s-ca-certificate="$CA_CRT" \
  6. external_ids:k8s-api-token="$API_TOKEN"
  7. ovn-k8s-overlay minion-init \
  8. --cluster-ip-subnet="192.168.0.0/16" \
  9. --minion-switch-subnet="192.168.2.0/24" \
  10. --node-name="kube-minion1"

配置网关Node (可以用已有的Node或者单独的节点)

选项一:外网使用单独的网卡eth1

  1. ovs-vsctl set Open_vSwitch . \
  2. external_ids:k8s-api-server="$K8S_API_SERVER_IP:8080"
  3. ovn-k8s-overlay gateway-init \
  4. --cluster-ip-subnet="192.168.0.0/16" \
  5. --physical-interface eth1 \
  6. --physical-ip 10.33.74.138/24 \
  7. --node-name="kube-minion2" \
  8. --default-gw 10.33.74.253

选项二:外网网络和管理网络共享同一个网卡,此时需要将该网卡添加到网桥中,并迁移IP和路由

  1. # attach eth0 to bridge breth0 and move IP/routes
  2. ovn-k8s-util nics-to-bridge eth0
  3. # initialize gateway
  4. ovs-vsctl set Open_vSwitch . \
  5. external_ids:k8s-api-server="$K8S_API_SERVER_IP:8080"
  6. ovn-k8s-overlay gateway-init \
  7. --cluster-ip-subnet="$CLUSTER_IP_SUBNET" \
  8. --bridge-interface breth0 \
  9. --physical-ip "$PHYSICAL_IP" \
  10. --node-name="$NODE_NAME" \
  11. --default-gw "$EXTERNAL_GATEWAY"
  12. # Since you share a NIC for both mgmt and North-South connectivity, you will
  13. # have to start a separate daemon to de-multiplex the traffic.
  14. ovn-k8s-gateway-helper --physical-bridge=breth0 --physical-interface=eth0 \
  15. --pidfile --detach

启动ovn-k8s-watcher

ovn-k8s-watcher监听Kubernetes事件,并创建逻辑端口和负载均衡。

  1. ovn-k8s-watcher \
  2. --overlay \
  3. --pidfile \
  4. --log-file \
  5. -vfile:info \
  6. -vconsole:emer \
  7. --detach

CNI插件原理

ADD操作

  • ovn annotation获取ip/mac/gateway
  • 在容器netns中配置接口和路由
  • 添加ovs端口
  1. ovs-vsctl add-port br-int veth_outside \
  2. --set interface veth_outside \
  3. external_ids:attached_mac=mac_address \
  4. external_ids:iface-id=namespace_pod \
  5. external_ids:ip_address=ip_address

DEL操作

  1. ovs-vsctl del-port br-int port

Underlay模式

暂未实现。

参考文档