[转]Consul服务自动化发现及F5动态配置更新

Last update from July 20th, 2015 - I gave a tech talk at Airbnb office related to this topic. you may find the presentation here at my github

In these days where microservices are becoming big, a compulsory modification of traditional architectural approaches are becoming evident. A more automatic distribution of application-afine services, such as load balancing, is required, as demands increase for agile software feature deployments. Traditional, manual configurations of load balancing services are becoming less of a viable option, as it becomes a major road block for organizations to deploy new application services or get new features to today’s rapidly changing software market.

In my quest to develop a solution for automate Virtual IPs deployment for F5 BIG-IP platforms, I’ve been abusing a lot of the iControl REST API available since 11.5.1 code, and I am really happy and pleased. It is powerful and allows safe and almost full interaction with the chassis config – tested for LTM and AFM modules, with a Ruby Sinatra API user interface.

In other hand, the approach with Hashicorp tools allows easily auto-scaling configuration management more close to DevOps world, extending all the good benefits from this hardware based load balancer, in the same way they would do with ha-proxy or keepalived software solutions. BIG-IP solutions has been proven stable and robust, with a good support, nice features with ASIC based routing, which performs awesome with FastL4 type profiles (no TCP proxying).

One of the things I love in BIG-IP platforms is not only they are based on Linux, but also they do not do stupid things to lock the system shell for the admin. Other nice feature is the concept of Partition, which allows you to create namespaces with different objects, roles, and last but not least, configuration files!

But the real BIG WEAPON we are using today is consul-template, which basically query a Consul instance and update a local fs config file based on Go templates, as well can execute arbitrary commands when template updates occurs.

Consul is used for service discovery, health check, and k/v registry for datacenter nodes, allowing query via REST API or DNS.  There is a good tutorial at Hashicorp Consul website about how to install a Vagrant Consul cluster if you’re not familiar with it and want to poke around.

Enough chit chat, lets get hacking!

First, create a different partition at BIG-IP – I am calling my new partition consul at my BIG-IP lab. In my case, the load balancer config file will be stored at /config/partitions/consul/bigip.conf.

Considering a simple scenario where you have at your datacenter a Consul server and a pool of application servers running consul agents publishing a web app at tcp/port 80, as well the app health check URL.

At this point Consul can provide reliable information about our app pool health. If a node fail or the health check times out, Consul will remove the service from the healthy pool. Here is an example of a raw query for my consul server:
curl http://localhost:8500/v1/catalog/service/web

This output provides all the information we need for generating BIG-IP nodes and pool objects, so we are ready to setup consul-template for updating and reloading our BIG-IP consul partition config at every Consul detected event, like node down, service failure, as well include new servers (or containers) brought in rotation to consul service pool.

Now we download consul agent and consul-template binaries for Linux AMD64 platform, and place both at our F5 BIG-IP /sbin folder. Make sure you setup your F5 BIG-IP /etc/consul.conf.json to join to the consul cluster, like following example:

As well create /etc/consul.d and /etc/consul-templates directories, placing the following template file at /etc/consul-template/bigip.ctmpl

Now create a /sbin/reload-consul.sh file with this content:

Finally, add these lines to your /config/startup:

And the magic happens when consul template starts:

Yay! Our F5 BIG-IP is being auto configured based on consul service catalog. As soon as you bring up more app servers and spin up their consul agent publishing their services, it’ll be automatically added to our load balancer pool. Beautiful isn’t it ?

The good news is that BIG-IP parses the config before applying it, so if you break your template after a change, BIG-IP wont overwrite the running-config with the faulty new one, until you provide a valid config. Safe is priceless!BIG-IP config file generated by consul-template: