layered_config package

Tools for working with the ConfigParser instances.

load_cake() - Load layers of configuration files with optional environment variable overrides.

munchify_config() - copy a ConfigParser instance to a munch to support attribute access to configuration data. (Note: qe-config[munch] must be installed)

Layers of configuration files? Yes! Read on!

Application or test configuration is often dependent upon the specific environment being used. Across many environments, there can be a good deal of common configuration as well. Cut’n’paste-ing configuration files and trying to maintain the various duplicated configurations is a challenge, and is not DRY. load_cake gives you a way to layer configurations so you can have common configurations in one place, and use them, with variations, in many places. For convenience a layering of configuration files is called a config cake, or just a cake.

Additionally, load_cake’s configuration file can declare that environment variables can be used to override configuration settings. This makes it easy to have a cake used by your CI/CD server and still permit configuration values to be defined/overridden with manually entered job parameters.

load_cake returns a plain old Python ConfigParser instance that we already know and love. Or at least know.

The built-in Python ConfigParser class already supports layering multiple configuration files; load_cake allows your code to specify your layering in an external ‘master’ config file. The ‘master’ config file also lets you explicitly configure if, and how, environment variable overrides work. Note that environment variable overrides are processed only once, when the configuration cake’s layers are processed. If you want to change the configuration dynamically after that, use ConfigParser’s methods, as you would with any ConfigParser instance. load_cake merges configuration files just as the ConfigParser.read() method does, but, unlike read(), all the files must exist.

The ‘master’ config file has sections that represent your various cakes. Each cake section must have a layers key defining the relative paths of configuration files to be loaded, from left to right. All the keys and their values from the cake’s section of the ‘master’ config file will be set in the defaults dictionary of the ConfigParser instance before the config files from the layers key are loaded.

If a section for defining how environment variable overrides work is included in the ‘master’ config file, or any of the the config files from the layers, then config values can be overridden with environment variables.

Here is a simple example.

base.config:

[Section-A]
key0 = value0
key1 = value1
.  .  .

staging.config:

[Section-A]
key0 = staging_value0
.  .  .

demo.config:

[Section-A]
key0 = demo_value0
. . .

master.config:

[ENVIRONMENT VARIABLE OVERRIDE INFO]
# Override configuration data by setting environment variables with this prefix;
prefix=MyPrefix

# Environment variable overrides will be of the form:
#     <prefix><separator><section name><separator><option name>
# In this example:
#     MyPrefix__Section-A__key3=env_var_value_here
separator=__

[env1]
# Each section name is of the form: <thing>
# You probably want <thing> to be helpful; this example shows that the
# section name is arbitrary and doesn't affect or determine any of the
# config file names.

# layers is a comma separated list of configuration files to load,
# in order from left to right.
layers = base.config, staging.config

[env2]
layers = base.config, demo.config
key1 = value1
all extra keys = and values will be in the defaults of the new config parser

How to use it:

from qe_config import load_cake

# Load a configuration from the master config file for environment 1:
my_config = load_cake('master.config', 'env1')
layered_config.load_cake(master_config_path, cake_name, into_config=None)[source]

Load a layered configuration cake by name from a master config file.

All the keys and their values from the cake_name section of the master_config file will be added to the defaults dictionary of the into_config config parser before any of the layers config files are loaded.

Parameters

  • master_config_path (str) – the file path for the master configuration file.

  • cake_name (str) – the particular configuration to load as described in the master configuration file. This is the name of a section of the master configuration file.

  • into_config (ConfigParser, optional) – A pre-created config parser to load the cake in to. If None, a plain ConfigParser is created. Use this if want a different variant of a ConfigParser to be used; for example, if you don’t want the default interpolation that a plain ConfigParser uses.

Returns

A ConfigParser instance.

Raises

  • NoSectionError – If cake_name is not found in the master config file.

  • NoOptionError – If any required options are missing, either from the environment override section, or any individual CAKE section.

layered_config.munchify_config(config_parser)[source]

Create a munch version of the config_parser’s data.

munch allows both attribute and item access to it’s data. See the munch documentation for details.

Note: This is a one-time conversion, if the config_parser is subsequently updated, the returned munch will be unaffected.

layered_config.string_to_list(source, sep=',', maxsplit=- 1, chars=None)[source]

.split() a string into a list and .strip() each piece.

For handling lists of things, from config files, etc.

Parameters

  • source (str) – the source string to process

  • sep (str, optional) – The .split sep (separator) to use.

  • maxsplit (int, optional) – The .split maxsplit parameter to use.

  • chars (str, optional) – The .strip chars parameter to use.