i4box’s Blog

What, Why, How ?

Dpdk对接openstack

安装

编译安装DPDK + OVS

参考 [DPDK安装详解与问题记录](http://www.i4box.com/blog/2017/08/21/dpdk-plus-ovsan-zhuang-xiang-jie/)

启动OVS

1.  ./configure --with-dpdk=$DPDK_BUILD --prefix=/usr --localstatedir=/var --sysconfdir=/etc
2.  make & make install
3.  export PATH=$PATH:/usr/share/openvswitch/scripts
    export DB_SOCK=/var/run/openvswitch/db.sock
4.  ovs-ctl --no-ovs-vswitchd start
5.  ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
    ovs-ctl --no-ovsdb-server --db-sock="$DB_SOCK" start

到此,支持dpdk的ovs启动成功。

创建相关网桥

1、创建br-int网桥

ovs-vsctl --may-exist add-br br-int -- set Bridge br-int datapath_type=netdev -- br-set-external-id br-int bridge-id br-int -- set bridge br-int fail-mode=standalone

2、创建br-phy网桥

ovs-vsctl --may-exist add-br br-phy -- set Bridge br-phy datapath_type=netdev -- br-set-external-id br-phy bridge-id br-phy \
            -- set bridge br-phy fail-mode=standalone other_config:hwaddr=<mac address of eth1 interface>

说明:这个网桥是ovs运行在用户态下,多添加的,使ovs用户态能够使用linux内核态协议栈进行路由和arp解析。(原文如下) This additional bridge is required when running Open vSwitch in userspace rather than kernel-based Open vSwitch. The purpose of this bridge is to allow use of the kernel network stack for routing and ARP resolution. The datapath needs to look-up the routing table and ARP table to prepare the tunnel header and transmit data to the output port.

3、将eth1添加到br-phy上

这里eth1已结通过dpdk绑定igb_uio驱动

# ovs-vsctl --timeout 10 add-port br-phy dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:06:00.0
# ip addr add 172.168.1.1/24 dev br-phy
# ip link set br-phy up
# iptables -F

配置openvswitch_agent

ovs配置项下配置如下配置

[ovs]
datapath_type=netdev
vhostuser_socket_dir=/usr/local/var/run/openvswitch

设置nova flavor大页

# sudo openstack flavor create --id 3 --ram 64 --vcpus 1 hugepage
# sudo openstack flavor set --property  hw:mem_page_size=large hugepage

创建虚机,观察

  1. dashboard上创建虚机,选择hugepage flavor;
  2. 创建成功后,ovs-vsctl show观察 vhostuser nic in ovs

问题记录

[1] libvirtError: internal error: process exited while connecting to monitor: 2017-11-20T12:18:04.693502Z qemu-kvm: -chardev socket,id=charnet0,path=/var/run/openvswitch/vhu360cc5c5-c9,server: Failed to bind socket to /var/run/openvswitch/vhu360cc5c5-c9: Permission denied

原因:由于我的Openstack环境是docker部署的,ovs被我移到宿主机上部署,docker中的nova没有权限操作宿主机的/var/run/openvswitch目录创建unix socket文件, 因此简单解决方法:chmod +777 /var/run/openvswitch

[2] 两台compute节点上的虚机不通

排查方法: 检查配置在br-phy上的两个ip在宿主机上是否能够ping通。

参考

http://docs.openvswitch.org/en/latest/intro/install/dpdk/
http://docs.openvswitch.org/en/latest/howto/dpdk/
http://docs.openvswitch.org/en/latest/intro/install/general/#general-building
http://syswift.com/183.html
http://docs.openvswitch.org/en/latest/howto/userspace-tunneling/