hmmm本来计划研究完双network nodes情形的,但是在学习过程中发现双网络节点模式还是有很多奇怪的问题和一些想法没有理顺,索性跳到LBaaS来吧(⊙o⊙)…这是不是我学习openstack的驱动力?。。。。
openstack的一大优势就是容许很多第三方软硬件通过plugin的方式与openstack集成起来。LBaaS,VPNaaS,FWaaS都是这样。由于目前从事于adn行业,所以还是关注LBaaS吧。首先学习openstack最容易集成的Loadbalancer haproxy,haproxy是一个软件级的LB,虽然它给专业的F5比还是小player,但是作为研究学习LBaaS是一个不错的开始,这里并不打算深入研究haproxy本身,而且关注其与openstack的结合,理解其工作原理,为后面引入F5 LBaaS打基础。。。。
在此次实验的安装过程中,默认并没有安装LBaaS服务,所以首先需要安装lb-agent 以及haproxy软件。实验环境如下:
将lbaas-agent和haproxy安装到 network节点:
apt-get install neutron-lbaas-agent haproxy
安装完毕后,需要做一些配置:
1. 配置contoller节点,启用lbaasservice
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
root@compute:/home/mycisco# egrep -v "^#|^$" /etc/neutron/neutron.conf [DEFAULT] state_path = /var/lib/neutron lock_path = $state_path/lock core_plugin = ml2 service_plugins = router,lbaas auth_strategy = keystone allow_overlapping_ips = True rpc_backend = neutron.openstack.common.rpc.impl_kombu rabbit_host = 192.168.232.138 notification_driver = neutron.openstack.common.notifier.rpc_notifier [quotas] [agent] root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf [keystone_authtoken] auth_host = 192.168.232.138 auth_port = 35357 auth_protocol = http admin_tenant_name = service admin_user = neutron admin_password = service_pass signing_dir = $state_path/keystone-signing [database] connection = sqlite:////var/lib/neutron/neutron.sqlite [service_providers] service_provider=LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default service_provider=VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default |
修改service_plugins,添加 lbaas(实际上实验一开始就添加了)
修改service_provider 增加 loadbalancer一行
重启服务
2. horizon界面启用lb,在主控节点:
vi /etc/openstack-dashboard/local_settings.py
找到enable_lb 项目,设置为True,并重启apache
3. 配置network 节点
-首先像配置主控节点的neutron.conf一样,进行相同配置
-再次,配置lbaas-agent配置文件,设置如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
root@network:/etc/neutron# vi lbaas_agent.ini [DEFAULT] # Show debugging output in log (sets DEBUG log level output). # debug = False # The LBaaS agent will resync its state with Neutron to recover from any # transient notification or rpc errors. The interval is number of # seconds between attempts. # periodic_interval = 10 # LBaas requires an interface driver be set. Choose the one that best # matches your plugin. # interface_driver = # Example of interface_driver option for OVS based plugins (OVS, Ryu, NEC, NVP, # BigSwitch/Floodlight) interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver # Use veth for an OVS interface or not. # Support kernels with limited namespace support # (e.g. RHEL 6.5) so long as ovs_use_veth is set to True. # ovs_use_veth = False # Example of interface_driver option for LinuxBridge # interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver # The agent requires drivers to manage the loadbalancer. HAProxy is the opensource version. # Multiple device drivers reflecting different service providers could be specified: # device_driver = path.to.provider1.driver.Driver # device_driver = path.to.provider2.driver.Driver # Default is: device_driver = neutron.services.loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver [haproxy] # Location to store config and state files # loadbalancer_state_path = $state_path/lbaas # The user group # user_group = nogroup ~ |
由于实验环境是希望haproxy工作在ovs内,因此interface driver 配置是for ovs。
启动Lbaas-agent服务,确认服务:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
root@network:/etc/neutron# neutron agent-list +--------------------------------------+--------------------+----------+-------+----------------+ | id | agent_type | host | alive | admin_state_up | +--------------------------------------+--------------------+----------+-------+----------------+ | 0e330cd6-5108-45e9-940f-6414a17eb636 | Open vSwitch agent | network | :-) | True | | 0e59f1c8-381c-4951-8086-535a72587510 | Open vSwitch agent | compute2 | xxx | True | | 1e4e16e0-7507-4cce-ac43-c6539028130a | Open vSwitch agent | network2 | xxx | True | | 4a9c7571-ab9f-456d-af50-5ce9486a9946 | Open vSwitch agent | compute | :-) | True | | 4c5e4a0e-df2f-4a76-80fc-a996b641fa60 | L3 agent | network2 | xxx | True | | 68ab802a-7e46-4030-8d82-aae689d9f875 | Metadata agent | network2 | xxx | True | | 800f56c4-e879-4bc2-bc5c-04605796892c | DHCP agent | network | :-) | True | | 82bc5d54-ee4d-49ee-827c-176636f71b02 | Metadata agent | network | :-) | True | | 97207b3a-0695-4927-819a-57e6fdd33189 | L3 agent | network | :-) | True | | bc2ed1a8-03b3-40ab-bf69-14346b3b6b16 | Loadbalancer agent | network | :-) | True | | c5bcad49-9732-431c-9a1d-408ad2c3f668 | DHCP agent | network2 | xxx | True | +--------------------------------------+--------------------+----------+-------+----------------+ |
输出中的network2 host因为没有启动,忽略。
至此,配置完毕。进入horizon界面,配置lb看看:
首先,有这样一个租户的网络,假设希望给两个server做负载均衡,设置LB位于同一网段,执行snat,客户端也位于同一网段:
进入 loadbalancer界面,点击add pool 添加一个pool
provider选择 haproxy, 网段选择服务器所在网段。可以理解这里的pool就是一个vs的配置,只是还没有配置vip。
接着进入 member 界面,为此pool添加members:
回到pool界面,给pool设置一个vip和将monitor应用到该pool上
配置完毕,由于vip也是落在同一网段,因此理论上br-int桥中应该为该vip对应一个接口:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
Bridge br-int fail_mode: secure Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "tap0d2b2134-a0" tag: 1 Interface "tap0d2b2134-a0" <<<<<<<vip port type: internal Port "tap587e26fe-41" tag: 1 Interface "tap587e26fe-41" type: internal Port "qr-33d58f14-1f" tag: 1 Interface "qr-33d58f14-1f" type: internal Port br-int Interface br-int type: internal |
ssh vip,可以联通
此时的整个数据流path是:
compute节点上的 client vm---通过tunnel---访问network node上的vip---network node---通过tunnel---访问compute节点上的server1和server2
看一下此时haproxy的进程以及其配置文件:
1 2 |
neutron 6843 0.1 1.7 106180 35108 ? Ss 05:57 0:10 /usr/bin/python /usr/bin/neutron-lbaas-agent --config-file=/etc/neutron/lbaas_agent.ini --config-file=/etc/neutron/neutron.conf --log-file=/var/log/neutron/lbaas-agent.log nobody 20140 0.0 0.0 20404 1204 ? Ss 08:13 0:01 haproxy -f /var/lib/neutron/lbaas/51f5f7eb-cbe1-4a61-b90a-693603ba63ce/conf -p /var/lib/neutron/lbaas/51f5f7eb-cbe1-4a61-b90a-693603ba63ce/pid -sf 19945 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
root@network:/etc/neutron# cat /var/lib/neutron/lbaas/51f5f7eb-cbe1-4a61-b90a-693603ba63ce/conf global daemon user nobody group nogroup log /dev/log local0 log /dev/log local1 notice stats socket /var/lib/neutron/lbaas/51f5f7eb-cbe1-4a61-b90a-693603ba63ce/sock mode 0666 level user defaults log global retries 3 option redispatch timeout connect 5000 timeout client 50000 timeout server 50000 frontend ceb2b159-9e3a-4fde-9fe0-fe778ec6fa63 option tcplog bind 10.10.10.200:22 mode tcp default_backend 51f5f7eb-cbe1-4a61-b90a-693603ba63ce backend 51f5f7eb-cbe1-4a61-b90a-693603ba63ce mode tcp balance roundrobin timeout check 10s server 7b7d6c65-cc84-4135-b8fb-5f53b6ee6489 10.10.10.201:22 weight 1 check inter 5s fall 5 server ef6468b9-8eca-4899-a07c-8f695ed632c6 10.10.10.6:22 weight 1 check inter 5s fall 5 |