Previous Up Next

Chapter 4  Comparison with Other Technologies

4.1  Introduction

Config4* is not the only option you have for making applications configurable. Many other choices exist, including command-line arguments, environment variables, writing your own configuration-file parser, using Java properties files, or using XML files. In this chapter, I compare these alternative approaches to Config4*.

4.2  Command-line Options & Environment Variables

When you start writing an application, you might think the application needs only a small amount of configuration information and conclude that this need can be met by the use of command-line options. Over time, the amount of configuration information used by an application tends to grow. If the number of command-line options grows to more than, say, 10, then users are likely to think the application is difficult to configure. In contrast, a configuration file containing potentially hundreds of entries can be easy to use and maintain, especially if fallback configuration is used to provide useful default configuration values.

The use of environment variables to store configuration information shares a limitation with the use of command-line options: if an application uses more than a handful of environment variables, then most users will think the application is difficult to configure.

Environment variables suffer from another drawback, which is that for a time it was difficult for a Java-based application to access environment variables. The first version of Java provided System.getenv() for accessing environment variables. However, later on Sun realized that some operating systems, such as MacOS, did not support environment variables. Sun was promoting Java as a highly portable language, and had even coined a slogan to reflect this: Write Once, Run Anywhere. Sun decided it did not make sense to encourage people to rely on a not-universally-supported concept such as environment variables. Because of this, Sun deprecated the use of System.getenv() and encouraged developers to use Java system properties as a replacement. A Java system property can be set by supplying a command-line option of the form -D<name>=<value> to the Java interpreter. Several years later, Apple decided to re-implement the Macintosh operating system on top of a UNIX base; doing so introduced environment variables to the Macintosh, but there may still be other operating systems that do not support environment variables. Java 1.5 has undeprecated system.getenv() and this method now returns null if the environment variable does not exist or if the operating system does not support environment variables.

As a side note, developers using a pre-1.5 version of Java can access environment variables, albeit in an indirect manner. This is done by calling java.lang.Runtime.getRuntime().exec() to execute an external command (for example, "cmd /c set" on Windows or "env" on UNIX) that prints name=value pairs for all environment variables. You parse the output of the executed command to gain read-only access to the environment variables. The Ant utility (http://ant.apache.org) uses this technique so that environment variables can be used in its build files. Config4J uses the same technique to enable access to environment variables from configuration files.

4.3  Writing Your Own Configuration Parser

Many developers think to themselves “It’s not that difficult to write a configuration-file parser; I could hack together something in a few hours and with just a few hundred lines of code”. However, a configuration-file parser built in such a short period of time is likely to offer very limited functionality. For example, perhaps the parser accepts name=value statements but the value must be a string literal that is terminated by the end of line. This means that none of the following are supported: (1) long values that extend over several lines, (2) values that are lists, (3) values defined in terms of environment variables or previously defined configuration variables, (4) scopes for grouping related name=value statements, (5) if-then-else statements that enable a file to encapsulate configuration information that varies between users or machines, (6) the ability for one configuration file to include other files, or (7) the ability to obtain configuration from, say, a website or a database.

Another problem with writing a configuration-file parser is that it is not sufficient to just write the code. You also need to test and document it; ideally, you should write both a user guide and programming guide. Writing tests and documentation might not be particularly difficult, but these tasks require time. An initial plan to “hack together something in a few hours and with just a few hundred lines of code” can easily turn into a multi-week job. And the likely result is that you still have a configuration-file parser with the numerous limitations listed in the previous paragraph.

A final problem with this approach is that development teams for countless projects have written their own configuration-file parsers. Each development team’s parser tends to have slight differences in the syntax it accepts. The result is that users have to learn a different configuration syntax for each application they use.

A better approach is for development teams, globally, to standardize on using the same configuration-file syntax in all their applications. Of course, if one configuration-file syntax is to be used by many applications, then the syntax needs to be very flexible, parsers for that syntax need to be available in different languages, there should be good user- and programmer-oriented documentation available, and the parsers should be available under a license that does not hinder their use in open-source or proprietary projects. These are precisely the goals of Config4*.

4.4  Java Properties Files

The standard Java class library provides a configuration-file parser, although Java uses the term properties file rather than configuration file. The syntax rules of a Java properties file are as follows.

Lines starting with # or ! denote comments.

The backslash character ("\") is used as an escape mechanism. For example, "\t" and "\n" denote tab and newline characters. A backslash at the end of a line denotes line continuation, thus enabling a long string to be split across several lines. The input file is assumed to be in the ISO-Latin-1 encoding (ISO-Latin-1 is an 8-bit character set that contains the characters of US-ASCII plus accented characters for some European languages). Internally, Java stores strings in a Unicode format, so each ISO-Latin-1 character is converted to its Unicode equivalent while the properties file is being parsed. The escape sequence \uxxxx, where each x denotes a hexadecimal digit, can be used to represent an arbitrary Unicode character by its hexadecimal code point.

Entries in a Java properties file consist of name-value pairs. The name and the value can be separated by "=" or ":", with optional whitespace surrounding the separator. Alternatively, the name and value can be separated by just whitespace, that is, without "=" or ":". If a name but no value is specified, then an empty string is used as the value.

There are several significant problems with Java properties, as I discuss in the following subsections.

4.4.1  Unwanted Whitespace at the End of a Line

Consider the following line from a properties file.

logFile = /tmp/foo.log

That line appears to set logFile to the value "/tmp/foo.log". However, if there are any spaces at the end of the line, then those spaces become part of the value, which is almost certainly not what is desired. Trailing whitespace is a common cause of misconfigured Java applications, and it can be difficult to diagnose such misconfiguration because the whitespace is invisible in most text editors. In Config4*, strings are enclosed within double quotes which prevents this cause of misconfiguration.

4.4.2  Lack of Syntax Checking

Figure 4.1 shows an example of a Java properties file. The first two lines use "=" and ":" to separate the name and value. The third and fourth lines use just a space to separate the name and value. For example, in line 3, the name is Java and its value is "properties accepts this text". The last line of the properties file contains just a name (&*%!)1 so an empty string is used as its value.


Figure 4.1: Example of input acceptable to Java properties
Roses=red
Violets : better
Java properties accept this text
Without raising an error
&*%!

The syntactic flexibility permitted in a Java properties file is problematic because it means that very little syntax checking is performed. In fact, almost any file — whether it be a text file or a binary file — is acceptable as input to the Java properties parser. The only syntactic error that might be raised is if the file contains the characters "\u" and these are not followed by four hexadecimal digits.

One principle of good programming is fail fast [], which means that if a program is going to fail, then it should fail at the earliest opportunity because this makes it easier for users to diagnose the problem. The Java properties file parser ignores the fail-fast principle, instead accepting almost any garbage as input and letting other parts of an application report an error when they find that an expected property has not been set.

4.4.3  Semantically Poor

The names in the name=value pairs of a properties file are in a flat namespace. The names can have embedded periods (".") but that seems to be a side-effect of allowing arbitrary characters (such as &*%!) in names rather than an intent to support hierarchical names. Certainly, there is nothing in the Java properties API that supports the concept of hierarchical names. When you have only a flat namespace in a configuration file, then you typically end up having a separate configuration file for each application. In contrast, Config4* provides explicit support for hierarchical scopes through its syntax and API; this makes it feasible to store configuration information for several (usually related) applications in a single file.

Another limitation of properties files is that there is no support for values that are lists of strings. It is common for developers to work around this limitation by writing code that splits a property value into a list based on occurrences of, say, a comma (","). However, having to do this is tedious. The approach taken in Config4* of providing explicit support for lists is better.

Other useful capabilities of Config4* have no counterpart in Java properties files. For example, there is no ability to define a variable in terms of other variables, and there is no support for reusing configuration information, such as the @include and @copyFrom statements in Config4*. Also properties files have no support for accessing centralized configuration information by executing an external command, such as "exec#..." in Config4*, and no support for adaptive configuration (Section 2.11).

4.4.4  Type-unsafe Lookup API

The API of the Properties class enables developers to access property values only as strings. It is common for a property to be a non-string type, such as an integer, floating-point number, boolean or list. Whenever type-safe access to a property value is required, developers must write code that retrieves the property value as a string, converts it to another data type, and throws a suitable exception if the data-type conversion fails. Code for doing this is not difficult to write, but it is tedious and the need to write such code is common enough that it would have been better for the Properties class to provide this functionality. Config4* provides such type-safe lookup operations.

4.5  Platform-specific Configuration Files

It is common for a software company to write a configuration-file parser suitable for the needs of its own products. Having done this, the company may then decide to expose the API of the configuration-file parser so that their customers can use it too. Two famous examples come from Microsoft. Microsoft wrote a parser for its ".ini" files that were used in Windows 3.1, and exposed the API of this parser so companies that wrote applications for Windows could make use of it. Then when Microsoft released Windows 95, it switched from ".ini" files to the Windows Registry. Again, it exposed the API for this and many companies made use of it in their products. Another famous example is X11, a windowing system widely used on UNIX machines. X11 exposes an API for parsing its configuration files, called X resource files. Countless other examples can be found in companies that sell framework libraries.

If you are writing a Windows-based application, then it is easier to use the Windows-supplied API for accessing the registry rather than write your own configuration-file parser. Likewise, if you are writing an X11-based application, then it is easier to retrieve your application-specific configuration information from an X11 resource file rather than write your own configuration-file parser. And if you are writing an application that uses a framework library that happens to provide a configuration-file parser, then… Well, you get the idea.

If your application runs on only one platform (operating system, windowing system, framework library and so on), then making use of that platform’s configuration-file parser seems like a great idea. However, it is common for an application to be developed initially for one platform and later be ported to other platforms. As soon as that happens, your use of a platform-specific configuration-file parser becomes a liability. Countless software teams have run into this problem over the years. If you use an Internet search engine, then you will find evidence of different groups who have implemented their own parser for, say, ".ini" files because they have ported a Windows-specific application to another operating system. Likewise, some groups have implemented a Java properties file parser in other languages because their initially Java-only project grew to include components written in other languages.

I am not going to critique numerous platform-specific, configuration-file parsers. Aside from me never having seen one that has as much flexibility as Config4*, they all suffer from the same fundamental limitation, which is that they are platform specific.

4.6  XML-based Configuration Files

XML parsers are available for a wide variety of programming languages and operating systems. This means that XML provides a platform-neutral configuration file format. The platform-neutral characteristic of XML is probably a significant reason why more and more developers are using XML for storing configuration information. However, there are some significant drawbacks to using XML to store configuration information, as I now discuss.

4.6.1  Verbosity

XML can be easy to read when it is used for infrequent markup in a text-oriented document. However, when XML is used for structured data, then the amount of syntactic baggage imposed by the start tags and end tags becomes visually distracting. You can see this by comparing the Config4* file in Figure 4.2 with similar information expressed in XML format in Figure 4.3.


Figure 4.2: Example Config4* document
fooSrv {
    timeout = "2 minutes";
    log {
        dir = "C:\foo\logs";
        level = "0";
    }
}


Figure 4.3: Example XML document
<fooSrv>
    <timeout>2 minutes</timeout>
    <log>
        <dir>C:\foo\logs</dir>
        <level>0</level>
    </log>
</fooSrv>

Some people may think the XML would be more compact if it was written to make use of attributes, but a quick look at Figure 4.4 shows that this is not necessarily the case.


Figure 4.4: Example XML document using attributes
<scope name="fooSrv">
    <property name="timeout" value="2 minutes"/>
    <scope name="log">
        <property name="dir" value="C:\foo\logs"/>
        <property name="level" value="0"/>
    </scope>
</scope>

Many developers who work with structured XML files on a daily basis claim “you eventually get used to the verbosity”. And, of course, even if some developers do not get used to it, they may still endure it simply because they are paid to do what their employers tell them to do. However, if end users dislike XML’s verbosity, then they may decide that an application with an XML-based configuration file is too difficult to use, and so go in search of simpler-to-use application from a competing vendor.

4.6.2  Limited Functionality

Once you look beyond the syntactic differences of XML and Config4*, it is possible to compare the functionality they offer.

Several pieces of functionality are similar. First, the ability to nest elements in an XML file provides functionality similar to the nested scopes of Config4*. Second, in Config4*, a value can be either a string or a list, while in XML, a value can be a string, and it is possible to use nested elements to denote a list. Third, an XML document may contain several identically-named elements. Config4* provides a similar capability through the "uid-" prefix on identifiers (Section 2.12). Finally, an application can iterate over the elements in an XML document in the order in which they appeared in the source document. In contrast, Config4* stores parsed information in hash tables; these provide fast lookup times but they do not preserve the original order in which the entries appeared within the source file. However, if processing entries in order is important, then this can be achieved through use of the "uid-" prefix on the names of entries.

Several capabilities in Config4* have no counterpart in XML. These are: (1) statements, such as @include or @copyFrom, that enable configuration to be reused; (2) the ability to define one configuration variable in terms of other configuration variables; or (3) adaptive configuration (Section 2.11). None of these capabilities is provided by an out-of-the-box XML parser. Instead, countless XML-based projects have had to implement such functionality by parsing an XML document and then doing some post-processing of the generated DOM tree.

4.6.3  Checking the Correctness of Input Files

Writing code to check the validity of an XML file is very tedious. To avoid this, some developers use XML Schema to define what can legally appear in an XML-based configuration file and then parse configuration files with a schema-validating XML parser. However, this approach has several drawbacks. First, XML Schema has a steep learning curve.2 Second, schema files are extremely verbose. Third, it can be quite difficult to understand some of the error messages produced by schema validators.

In contrast, the Config4* schema validator has a trivial learning curve (it is described completely in 12 pages in Chapter 9), its schema definitions are very compact, and its error messages are easy to understand.

4.6.4  Memory Usage

Use of Config4* adds a few hundred KB to an application’s executable size, and the amount of RAM consumed when a configuration file is parsed is quite small: about 2.5 times the size of the configuration file. In contrast, the library for a schema-validating XML parser can add several megabytes to an application. In addition, the DOM tree built by most parsers can also consume a lot of memory. Many functionally-rich applications may not care about a few megabytes of overhead for parsing XML-based configuration files. However, such overhead is unacceptable for many smaller applications.

4.7  A Critique of Config4*

So far in this chapter I have compared Config4* with some other configuration technologies and, in so doing, have pointed out what I feel are the drawbacks of these other technologies. I now turn my focus on Config4* to discuss its drawbacks.

One significant drawback of Config4* is that, currently, there are implementations for only two languages: C++ and Java. I hope implementations for other languages will follow in time.

A second significant drawback of Config4* is that its internationalization and localisation support is a work in progress (see the Config4* Maintenance Guide for a discussion of these issues). I hope that, over time, these deficiencies will be addressed with the support of the open-source community.

These drawbacks are due to the young age of Config4*, so a bit of maturing should resolve both drawbacks. In contrast, the other configuration technologies discussed in this chapter have been around for over a decade. The drawbacks of those other technologies are due to inherent design limitations rather than immaturity.

4.8  Summary

In this chapter I have compared Config4* with alternative technologies for accessing configuration information. The purpose of the comparisons is not to ridicule or demonise any of the alternative approaches to providing configuration information. Rather, my purpose is to show that, when it comes to configuration parsers, there is a widespread under-appreciation for the importance of issues such as: (1) adaptable configuration, (2) centralizable configuration, (3) hierarchical scopes, (4) type-safe access, (5) ease of use for developers, (6) ease of use for end users, and (7) portability across operating systems and languages. Most existing configuration technologies score poorly against most of these criteria. Config4* is useful in its own right, but by scoring high in these seven criteria, it raises the bar for other (present and future) configuration technologies.

I am writing this paragraph in 2011 and, as far as I know, Config4* is by far the best configuration parser in the world. I will be disappointed if I can still make that claim in, say, 5 years time. By highlighting the ways in which Config4* is superior, I am laying down a challenge to the developers of other configuration technologies. Please, take the best ideas from Config4* and innovate to produce something better. Configuration parsers have been depressingly mediocre for decades. It is time for us to make them better. Significantly better.


1
&*%! is not really a name. It’s more of a curse at the lack of error checking in a Java properties file.
2
If you wish to learn XML Schema, then I recommend Definitive XML Schema by Priscilla Walmsley []. The book is excellent, but the fact it is about 500 pages long indicates that XML Schema is complex and has a steep learning curve.

Previous Up Next