Micro
Go Config - A pluggable config framework
Application configuration has remained largely static for most of our lifetime, using flags, environment variables and files.
Any change has usually required restarting the application or significant complexity in code to signal and reload the config.
Today though, we need more. Restarting an application for a minor configuration seems like overkill. If only one property needs to change, should there not be a better way? Dynamic configuration provides us this option.
Dynamic Config
Dynamic configuration is the concept by which a system can be modified or extended while it’s running. Or in simpler terms the values you require for configuration are automatically updated as they change.
There’s an emergence of tooling for dynamic config mostly driven by key-value stores as the source of truth. Yet when we then need to extract out these values it requires reading individual keys, extracting bytes and handling decoding manually.
In an ideal world, the configuration system should handle all of these concerns leaving you to focus on what’s most important. Your code.
Go Config is a pluggable config framework which looks to solve these problems.
Go Config
Go Config was born out of a need to simplify configuration for users of micro. Once companies had started building real products that were going to production they echoed the need for the ability to provide configuration for business logic related code or higher level config that shouldn’t really necessitate the need for a restart.
In some cases they had opted for the kubernetes config map or a key-value store such as consul or etcd but handling this configuration in code was not pleasant. They were looking for abstractions similar to Go Micro for their config.
And so Go Config was created to help with this and much more.
Go Config is:
- Dynamic - Config is updated transparently in the background
- Pluggable - Backend sources can be swapped out
- Mergeable - Multiple sources are merged into a single source of truth
- Observable - Actively watch the config for changes if you need
- Safe - Default fallback values can be specified in case they don’t exist
It also has the benefit of supporting multiple backend sources and config encoding formats out of the box. See the project readme for details.
Here’s the top level interface which encapsulates all the features mentioned.
Ok so let’s break it down and discuss the various concerns in the framework starting with the backend sources.
Source
A source is a backend from which config is loaded. This could be command line flags, environment variables, a key-value store or any other number of places.
Go Config provides a simple abstraction over all these sources as a simple interface from which we read data or what we call a ChangeSet.
The ChangeSet includes the raw data, it’s format, timestamp of creation or last update and the source from which it was loaded.
There’s also an optional md5 checksum which can be recalculated using the Sum() method.
The simplicity of this interface allows us to easily create a source for any backend, read it’s values at any given time or watch for changes where possible.
Encoding
Config is rarely available in just a single format and people usually have varying preferences on whether it should be stored in json, yaml, toml or something else. We make sure to deal with this in the framework so almost any encoding format can be dealt with.
The encoder is a very simply interface for handling encoding and decoding different formats. Why wouldn’t we reuse existing libraries for this? We do beneath the covers but to ensure we could deal with encoding in an abstract way it made sense to define an interface for it.
The current supported formats are json, yaml, toml, xml and hcl.
Reader
Once we’ve loaded backend sources and developed a way to decode the variety of config formats we need some way of actually internally representing and reading it. For this we’ve created a reader.
The reader manages decoding and merging multiple changesets into a single source of truth. It then provides a value interface which allows you to retrieve native Go types or scan the config into a type of your choosing.
Our default internal representation for the merged source is json.
Example
Let’s look at how Go Config actually works in code. Starting with a simple example, let’s read config from a file.
Read Config
Step 1. Define a config.json file
Step 2. Load the file into config
Step 3. Read the values from config
And that’s it! It’s really that simple.
Watch Config
If the config file changes, the next time you read the value it will be different. But what if you want to track that change? You can watch for changes. Let’s test it out.
In this example rather than getting a value, we watch it. The next time the value changes we’ll receive it and can update our Host struct.
Merge Config
Another useful feature is the ability to load config from multiple sources which are ordered, merged and overridden. A good example of this would be loading config from a file but overriding via environment variables or flags.
Fallback Values
In the event some values may not exist or config does not load due to an error, you can set fallback values at time of getting them.
Summary
The way in which config is managed and consumed needs to evolve. Go Config looks to do this by drastically simplifying use of dynamic configuration with a pluggable framework.
Go Config currently supports a number of configuration formats and backend sources but we’re always looking for more contributions. If you’re interested in contribution please feel free to do so by with a pull request.
Let Go Config managed the complexity of configuration for you so you can focus on what’s really important. Your code.
To learn more about micro check out the website, follow us on twitter or join the slack community.