Introduction
Go‘s standard library support for JSONis an expedient method of implementing configuration files.An application can define a configuration struct typecontaining all the tunable elements, and load JSON from a file into aninstance of this struct, for example:
type Config struct {
ServerUrl string
APIKey string
MaxSessions int
}
func readConfig(filename string) (*Config, error) {
// initialize conf with default values.
conf := &Config{Url: "http://localhost:8080/", MaxSessions: 10}
b, err := ioutil.ReadFile("./conf.json")
if err != nil {
return nil, err
}
if err = json.Unmarshal(b, conf); err != nil {
return nil, err
}
return conf, nil
}
This is already an improvement over loading JSON into an untypeddictionary structure, as would happen in Javascript or Python, inthat
json.Unmarshal
provides some minimal validation. If configured with:
{ "MaxSessions": "quite a few" }
json.Unmarshal
will return a error indicating that
"quite a few"
is nota valid integer, and the program can handle this error at startup ratherthan later in the runtime.
Customizing
The Go JSON library supports Unmarshaling into custom types throughan interface type
json.Unmarshaler
:
type Unmarshaler interface {
UnmarshalJSON([]byte) error
}
If our config struct contains a field of a type satisfying
json.Unmarshaler
,that field’s
UnmarshalJSON
method will be called to fill in the field.This has a number of uses which can make config handling more pleasant.
Loading Values From Elsewhere
One issue with the above example is the API key in the config. Configsget copied and pasted and checked in to version control, and over timethis runs the risk of leaking API keys. A simple solution is to havethe API key stored in a separate file, and read it in, and store thisfilename in the config instead:
type Config struct {
...
APIKeyFile string
}
but this requires a separate step to load the API key from the fileand store it elsewhere. If you have multiple config values needingthe same handling, this gets repetitive. A custom
Unmarshaler
streamlines this process dramatically.
type FileString string
func (f *FileString) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
f, err := ioutil.ReadFile(s)
if err != nil {
return err
}
val := strings.TrimSpace(string(f))
*f = FileString(val)
return nil
}
type Config struct {
// ...
APIKey FileString
// ...
}
Now, the
Config
APIKey
field is a string read from the file namedin the config.
Validation and Parsing
The above example adds some extra error conditions toloading the config, namely returning appropriate errorsif the API key file does not exist or can’t be read. Wecan take this effect a step further and validate thatthe
ServerUrl
parameter contains a valid URL usingthe
net/url
library.
type Url string
func (u *Url) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
*u = Url(s)
_, err = url.Parse(s)
return err
}
type Config struct {
ServerUrl Url
// ...
}
With this, if the user supplies an invalid URL string for the URLparameter, it will be flagged as an error. The
ServerURL
fieldwill always be populated with a valid URL string
Note that we are calling the parser above, but throwing away its results.Those results are useful, we should keep them around!
type Url struct { *net.URL }
func (u *Url) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
u.URL, err = url.Parse(s)
return err
}
Now, the
ServerURL
field is a struct embedding a
*net.URL
,with all the fields and methods of that type.The string value is available with
.ServerUrl.String()
.
Bringing it all together
The above two techniques can be combined to load more complicatedconfigurations. For example, the crypto/tls library providesa
Config
structure that many TLS-supporting libraries use. The
Config
structure contains parsed forms of certificates andCA certificates. With a custom
Unmarshaler
, we can have theuser supply file names for certificates and keys, and load theminto a
tls.Config
:
type certFiles struct {
KeyFile, CertFile string
}
type tlsConfigFiles struct {
CertFile string
KeyFile string
CACertFile FileString
}
type TLSConfig struct { *tls.Config }
func (t *TLSConfig) UnmarshalJSON(b []byte) error {
var cf tlsConfigFiles
err := json.Unmarshal(b, &cf)
if err != nil {
return err
}
pool := x509.NewCertPool()
if !pool.AppendCertFromPEM([]byte(cf.CACertFile)) {
return errors.New("invalid CA Cert")
}
cert, err := tls.LoadX509KeyPair(cf.CertFile, cf.KeyFile)
if err != nil {
return err
}
t.Config = &tls.Config{
RootCAs: pool,
Certificates: []Certificate{cert},
}
return nil
}
struct Config {
//...
TLS TLSConfig
}
With this, any library call needing a tls config can use
conf.TLS.Config
.
Conclusion
The
Unmarshaler
interface in the
encoding/json
Golibrary is a powerful abstraction. It allows you tointercept the parsing of a loosely-structured JSONobject and validate or transform the underlying valuesinto the form the application needs them.
Although this post was JSON-centric, analogous
Unmarshaler
interfaces appear in the standard XMLlibrary and the leading third party YAML librariesso the above techniques would work for XML and YAMLconfigs.
Chris Mikkelson is a Senior Distributed Systems Engineer at Farsight Security, Inc.
Heading
