Microservices are a hot topic, and there are many challenges you'll face when adopting this way of building software. In this blogpost we'll tackle the challenge of configuration. In particular how to manage the configuration of many different services located in a cluster where services are running in different datacenters, on a multitude of host machines or containers, and even potentially with different roles. Using Consultant - a newly open-sourced Magnet.me project - we'll show you how to load your service's configuration with zero effort and subscribe to updates.

Consul

We’ve been experimenting with Consul lately to manage service discovery, routing, and configuration for our (micro)services. Consul is a service developed by HashiCorp which you can run in your cluster. Consul uses Serf to gossip between multiple running instances of Consul to share the location and state of services running in the cluster. In addition to this, Consul also provides a key/value store which allows you to store configuration or meta-data for the operation of your services. Consul is also datacenter aware, meaning that the you can not only search for the locations of a particular service in the entire cluster, but also limit that search to a particular datacenter to avoid long roundtrips. We’ll save the service discovery awesome-ness for another blogpost. Let’s focus on configuration!

Configuration

Configuration is an integral part of many services. If your service needs to talk to a database, it’s likely you’ll want to store the username, password, database name, and perhaps even the location of the database server if you’re not using service discovery. These settings are typically static, and don’t change while the service is running. There are also settings which are more dynamic and can change while a service is running, like settings regarding the number of workers for a certain task. All these settings are typically stored inside one or more configuration files which the service reads on startup. If the service supports dynamic configurations, typically the service watches these configuration files for changes, and reloads the configuration from disk when a change is detected.

The problem is managing all this configuration for every instance of the service in the entire cluster. For instance, what do you need to do if you want to alter the config of all service instances in a particular datacenter, running on a particular host, or with a particular role? With microservices this can be a bit of challenge as it’s likely the service instances might be running just about anywhere in your cluster. The solution to this problem is to make your configuration available from a service such as Consul, and make your services look their own configuration up depending on their location and role in the cluster.

Integrating Consultant with a service

Assuming you’ve already got Consul setup for your cluster, let’s look at how we can integrate Consultant into our JVM-based services, and let them retrieve their own configuration from Consul. Consultant was designed to be extremely simple to use, and to integrate with existing services. Below you’ll see how we can create a Consultant object for our news service, and retrieve a Properties object from it:

Consultant consultant = Consultant.builder()
   .identifyAs("news")
   .build();

Properties config = consultant.getProperties();

It’s highly recommended to provide environment variables specifying the datacenter, host, and role of this particular service instance. In case of our news service we’d want to specify the following environment variables: SERVICE_DC=eu-central to let it know it’s running in our EU datacenter, and SERVICE_HOST=host-12.magnet.me to let it know that it’s running on a specific host.

Consultant will instantly retrieve the configuration applicable for this news service running in that particular datacenter and on that particular host.

Subscribing to changes

But that’s not all Consultant does. Should we change any of settings in Consul’s key/value store, Consultant is immediately notified of this, and will update the Properties object. Should you wish to validate any configuration changes, then you can do that too:

Consultant consultant = Consultant.builder()
   .identifyAs("news")
   .validateConfigWith((config) -> {
      Preconditions.checkNotNull(config.getProperty("database.username"));
      Preconditions.checkNotNull(config.getProperty("database.password"));
      Preconditions.checkNotNull(config.getProperty("database.url"));
   })
   .build();

Should the validation fail, the Properties object will not be updated. This means that your service remains unaware of these changes in its configuration. Should you wish to be notified of when the service’s configuration is successfully changed you can also subscribe to these changes:

Consultant consultant = Consultant.builder()
   .identifyAs("news")
   .onValidConfig((config) -> {
      System.out.println("Yay, we've got a new config!");
   })
   .build();

Or if you’d like to be notified of a particular setting being changed:

Consultant consultant = Consultant.builder()
   .identifyAs("news")
   .onSettingUpdate("database.password", (key, oldValue, newValue) -> {
      System.out.println("Hey the db password changed to: " + newValue + "!");
   })
   .build();

These listeners will be called every time Consultant is able to retrieve a configuration from Consul’s key/value store which passes the validation (if you’ve specified any validation checks).

Storing the config in Consul’s key/value store

Now that we can have our services retrieve their configuration from Consul’s key/value store, it’s time to see how we can store our settings in there. The easiest way to do this, is to manually add the configuration through Consul’s web UI. Simply by adding a new entry with a key matching the pattern: config/<service_name>/<setting_name> you can define a setting with that particular name, for that particular service.

If you wish to define a setting which overrides the default value for a particular datacenter, host, or instance you can do so by defining a selector block. In that case the key should follow one of these patterns:

config/<service_name>/[dc=<datacenter_name>].<setting_name>
config/<service_name>/[host=<host_name>].<setting_name>
config/<service_name>/[instance=<instance_name/role>].<setting_name>

Note that you can even mix and match these to define even more specific groups by for instance specifying a selector such as: [dc=eu-central,host=app-12.magnet.me].

Consul's web UI

Summary

Using Consul and Consultant, Magnet.me is able to have each service retrieve its own configuration from Consul. By subscribing to changes in that configuration, we are able to alter the behaviour of our services at runtime, tweak the performance of our services at runtime, and even run the same configuration in development environments as we do in production environments. We hope you’ve enjoyed this blogpost, and that you’ll give Consul and Consultant a try! Consultant is available from Maven’s Central Repository!