This package gathers values (typically used for application settings and configuration) from various sources outside of the application, and layers them together in a single output struct (supplied as an argument) for use by the application. This package supports 12-factor configuration use cases to facilitate cloud-native API and application development.
The package will first attempt to load settings from the following sources in the order arranged below:
- a base file (in
yamlorjsonformat) - from the default values map (if provided in
ReadOptions) - from any command line provided override files (if
ArgsFileOverrideswitches are defined inReadOptions) - from any environment override files (if
EnvOverrideandEnvSearchPathsare provided inReadOptions) - from command line arguments (auto-mapped from
argstruct tags) - from environment variables (auto-mapped from
envstruct tags) - from any additional manual mappings you add via
SetArgsMap/SetVarsMap
go get -u go.jtlabs.io/settingsThe preferred way to wire overrides is to tag your config struct with arg and env. Gather will read those tags and automatically map CLI flags and environment variables to fields:
package main
import (
"log"
"go.jtlabs.io/settings"
)
type config struct {
Data struct {
Name string `json:"name" arg:"--data-name" env:"DATA_NAME"`
Host string `json:"host" arg:"--data-host" env:"DATA_HOST"`
Port int `json:"port" arg:"--data-port" env:"DATA_PORT"`
}
Logging struct {
Level string `json:"level" env:"LOG_LEVEL"`
}
Server struct {
Address string `json:"address" env:"SERVER_ADDRESS"`
}
}
func main() {
var c config
options := settings.Options().
SetBasePath("./defaults.yaml").
SetEnvOverride("GO_ENV"). // optional: look for environment-specific files
SetEnvSearchPaths("./", "./config"). // optional: where to look for those files
SetDefaultsMap(map[string]interface{}{
"Server.Address": ":3080",
})
// read in configuration from all sources
if err := settings.Gather(options, &c); err != nil {
log.Fatal(err)
}
}Now you can override with either CLI flags or env vars:
LOG_LEVEL=debug go run main.go --data-host=db.internal --data-port=5432Gather will merge, in order: defaults → base file → env override files → CLI args → env vars → any additional maps you add (next section).
For a more verbose example along with execution instructions, see examples/example.go.
If you want aliases or mappings beyond tags, you can still use SetArgsMap / SetVarsMap to add or rewrite entries. These are layered on top of the tag-derived maps:
options := settings.Options().
SetBasePath("./defaults.yaml").
SetArgsMap(map[string]string{
"-h": "Data.Host", // alias for --data-host
}).
SetVarsMap(map[string]string{
"HOST_OVERRIDE": "Data.Host",
})ReadOptions are used to instruct the package where to find override values from a base file, a command line override file, an environment override file, command line arguments, or from environment variables.
This helper wires a sensible baseline for environment-based overrides, similar to settings-lib. It:
- adds
GO_ENVtoEnvOverridesoGatherwill look for environment-specific files (e.g../testing.yaml,./settings/testing.json) - sets default search paths to
./,./config, and./settings
You can start with EnvDefault() and layer on additional options or tags:
options := settings.Options().EnvDefault()
settings.Gather(options, &config)If you need a different variable name or paths, call SetEnvOverride / SetEnvSearchPaths after EnvDefault() to override or extend the defaults.
Adds a single CLI flag mapping. Useful if you prefer to configure mappings in code rather than struct tags, or if you need to supplement the tag-derived map.
options := settings.Options().
SetArg("--a-flag", "Field.Name")
settings.Gather(options, &config)When providing a value to this method, one can override the underlying settings via one or more specific files that are provided via command line arguments.
options := settings.
Options().
SetArgsFileOverride("./path/to/first-file.yml", "./path/to/another/second-file.json")
settings.Gather(options, &config)The first-file.yml will be read and applied, and then the second-file.json will be read and applied over the top of the first. These files can be partial files with a subset of the fields from the out struct defined as &config in the example above if desired.
The arguments map is used by Gather to determine how command line switches can be applied to specific out struct fields. Tag-derived mappings are added first; SetArgsMap can add or rewrite entries.
options := settings.
Options().
SetArgsMap(map[string]string{
"--switch-to-look-for": "CaseSensitive.Field.Where.Hiearchy.Is.Noted.By.Dot",
"--logging-level": "Logging.Level",
"-l": "Logging.Level",
})
settings.Gather(options, &config)Any switches that are provided in the map that do not appear in the list of os.Args for the application are effectively ignored. If the desired outcome is to have an alias for a command line argument (i.e. --logging-level and -l both capable of overriding Logging.Level), each value can be independently added to the map. When processing arguments, --some-switch=value (notice the = character) is processed the same as --some-switch value so that the value will properly read and applied in either scenario.
The base path for settings is the initial (yaml or json) file that is loaded to populate the out argument to the gather method. As with the command line override file and with the environment override file, this base settings file is not required to be a complete serialization of the out struct... it can be partially defined if desired. If a file is specified, and the file can't be found or read, the Gather method will return a file doesn't exist (i.e. os.ErrNotExist) or a SettingsFileReadError in the event there is some other read problem.
options := settings.
Options().
SetBasePath("./settings.yml")
settings.Gather(options, &config)The defaults map is used by settings to apply default values to fields in the out struct. These defaults are applied immediately after the base settings (if provided) are applied.
options := settings.
Options().
SetArgsMap(map[string]interface{}{
"CaseSensitive.Field.Where.Hiearchy.Is.Noted.By.Dot": true,
"Data.Port": 27017,
"Server.Address": ":8080",
"Logging.Level": "trace",
"Name": "cool name",
})
settings.Gather(options, &config)The string value of the map is the field path where hierarchy / depth is noted by the . character.
Environment override and search paths can be provided to the package to enable virtually named environment level overrides at a partial or complete configuration level.
options := settings.
Options().
SetEnvSearchPaths("./", "./settings"). // look for files in "./" and "./settings
SetEnvOverride("GO_ENV", "GO_ENVIRONMENT")
settings.Gather(options, &config)In the above example, if a value is set in the GO_ENV or GO_ENVIRONMENT variables for the application, the value will be used in a search for matching yaml or json files that exist in the paths provided as search paths (in the above example, ./ and ./settings). To illustrate:
GO_ENV=testing go run cmd/app.goThe GO_ENV value is testing. Combined with the code snippet above, the app would search for the following files:
./testing.yml./testing.yaml./testing.json./settings/testing.yml./settings/testing.yaml./settings/testing.json
Upon finding a file that matches (the first match), that file is read and the fields defined therein are applied to the out struct.
If SetEnvSearchPattern is used to defined a file name pattern, in addition to the above steps, files are searched using the file name pattern provided...
options := settings.
Options().
SetEnvSearchPaths("./", "./settings"). // look for files in "./" and "./settings
SetEnvSearchPattern("config.%s").
SetEnvOverride("GO_ENV", "GO_ENVIRONMENT")
settings.Gather(options, &config)Using the above example, when the following is used to start the application:
GO_ENV=testing go run cmd/app.goThe settings package will look for the following files:
./config.testing.yml./config.testing.yaml./config.testing.json./settings/config.testing.yml./settings/config.testing.yaml./settings/config.testing.json
In this scenario, if both ./testing.yml and ./config.testing.yml are found, only the ./testing.yml will be loaded.
Adds a single environment variable mapping. Useful if you prefer to configure mappings in code rather than struct tags, or if you need to supplement the tag-derived map.
options := settings.Options().
SetVar("FIELD_NAME", "Field.Name")
settings.Gather(options, &config)Similar to the Args map, the Vars map can be used to override individual fields with values defined as environment variables. Tag-derived mappings are added first; SetVarsMap can add or rewrite entries.
For our use case, we desired a simple and extensible configuration mechanism that spiritually adheres to 12-factor principles and facilitates the layered specification of nn settings / configuration. An initial base configuration file to be the start, with an ability to specify override files as a layer of additional settings (either full or partial) on top of the base file (the locations for which specified via command line arguments or environment variables). Finally, having an ability to override individual keys within the configuration with specific environment variables or command line arguments.
This settings library was built in the spirit of https://github.com/brozeph/settings-lib, which in many ways could be considered this package's Node.js flavored older sibling. This package is not a direct port of settings-lib, though, and makes use of Go specific idioms.
How is this different than https://github.com/spf13/viper?
Viper is an incredible and feature rich configuration utility that also aligns, philosophically, with 12-factor principles. Viper supports several features that this package does not:
- loading configuration from external sources (i.e. Consul, etcd, and k/v stores, etc.)
- reading configuration from more sources (i.e. HCL, INI, TOML, dotenv files, etc.)
- saving configuration back out to a destination
Where Viper differs is in the order in which configuration is loaded. Additionally, to load additional full or partial files specified through command line arguments or environment variables, custom code is required.
Ultimately, Viper is a great choice for configuration as well. This package provides a subset of the functionality of Viper, and approaches the loading of configuration layers in a different order and with a completely different underlying approach.
TL;DR: yes, but this isn't really applicable in this package.
The approach within Viper involves loading configuration from various sources and each source is a potential origin of the configuration value. It would be a non-trivial matter for Viper to support case-sensitve key lookup.
In this package, there is a single source of truth for the final configuration, which is the out interface{} struct provided to the Gather method. All other sources for configuration are mapped to the desired output struct.