Standardizing Linux Configuration Files
Erik Moeller
One problem I have experienced during my use of the "lower level" Linux
applications like vi and Apache is the wide-spread use of text-based
configuration files that often seem to be following different standards,
or no standards at all. My $HOME dir is also cluttered with .*rc files
from various applications, in various different formats. Some authors
have written separate GUI-frontends for some of these files, but overall
the situation is quite chaotic.
It seems like a tremendous undertaking to get application developers to
agree on a common standard, given that distributors seem incapable of
even following the simplest LSB requirements (no offense intended).
Nevertheless, in my opinion, a reasonably well-defined configuration
file standard would be a tremendous advantage. It would make
- learning Linux configuration easier
- merging different applications simpler
- building a standard GUI for configuration of all applications
possible.
So I brainstormed a little about this problem and thought about possible
solutions. This may be nonsense, or a similar project may already exist,
so I appreciate your feedback on this.
My basic idea, which I call "Qonfig", is to define a format which can be
both intermediary or the actual native configuration file format for an
application. Let's call this format "QFG" for the time being. That would
mean that at first you would mostly write import/export filters for
existing configuration files. Using these filters, a user could import,
say, a .vimrc file, edit the QFG version with a special QFG editor,
graphical or text-based, and then export it again to .vimrc. As QFG
would gain acceptance, it could both be extended and used natively by
more and more applications.
One GUI for all
---------------
While QFG should be easy to edit manually (otherwise many people would
refuse to use it), which may or may not rule out something XML-based, it
should also be possible to generate a standardized graphical
configuration dialog from any QFG file.
The respective graphical editor would automatically generate tabs in the
GUI from the different sections of QFG files, only allow certain values
for certain fields as specified in the QFG file, automatically show help
for certain options, allow the user to annotate all options, and would
allow to create new "records" as defined within the QFG file for certain
entries (e.g. create a new email account for a .fetchmailrc file).
The advantage of having one GUI for all configuration files would be
huge from the user's POV. By executing a single program, he would be
able to change the configuration of all supported applications, within a
standard interface that would follow the conventions of his
desktop/toolkit of choice.
This would be a quite complex tool, which should be clearly separated
into its different modules, the user interface(s), the QFG parser(s),
and the import/export filters plus the respective tool(s) to interpret
them. So it should be relatively easy to create a QFG-compliant GTK
front-end, a Qt front-end, a web-based front-ent etc. It should also be
possible to write a C, C++, Java etc. parser for the QFG files, and to
convert source files into QFG files (and vice versa) by different means,
with a simple conversion format as part of the QFG standard.
Let me try to discuss some more requirements for the different levels.
Templates and Filters
---------------------
As I explained above, in the early project stage the success of QFG
would mostly depend on being able to successfully and easily read and
write files for other applications. In order to accomplish this, you
would need a simple but powerful conversion standard. This could be
based on something existing, e.g. sed/awk, or be something entirely new
(if new, however, it should support standards like regular expressions).
A filter would consist of two parts:
I. A QFG TEMPLATE. This would define ALL possible configuration options
for a specific version of the program in question, filled with the
respective default values. It would also split the configuration into
logical sections, which would have STANDARDIZED NAMES that could easily
be localized. Let's illustrate this with some pseudo-format:
[Section $DISPLAY]
$FONT = "Arial" <@FONTS>
-> "Which font do you want to use in the mail display window?"
$SIZE = 12 <@FONTSIZES>
-> "Which font size should be used?"
$COLOR = "Black" <@COLORS>
-> "Which color should be used (white background)?"
Renderer = "HTML" <"Text","HTML","RTF">
-> "Which renderer do you want to use? Please be aware that the HTML
renderer is still beta, and the RTF renderer requires an additional
conversion
step, which slows things down a bit. Text is recommended."
Now what does this pseudo-format mean?
1. After each option, you would define help information which could be
interpreted in various ways by programs that read QFG files.
2. Any scalar like $FONT would be part of the QFG-standard. This would
have the advantage that the parser could provide localized versions of
some fields automatically, e.g. "Font" = "Schriftart" in German.
3. Options which are not part of the QFG-standard, like Renderer in the
example above, would have to be written as literals. If you wanted to
translate this QFG-template, you could basically leave everything as is
except for the help texts and the non-localized options, e.g. you would
write something like
...
Renderer = "Plain Text" <"Plain Text","HTML","RTF">
German: Darstellung <"Reiner Text","HTML","RTF">
-> "Welchen Renderer ..."
...
(optimally in a separate copy of the file). Now, if you read this QFG
template on a German system, it would *display* "Reiner Text", but would
write the matching English "Plain Text" into the resulting QFG file.
Similarly, if you exported this QFG file using the export filter into
some obscure .*rc standard, it would write the native-language version,
not the translated one.
So, as a user, you could create an English language configuration file
without speaking English!
4. The <...> after each option means "the allowed values" for this
option. There again you would have both pre-defined sets of allowed
values for the respective options, like the @FONTS and @COLORS arrays,
or user-provided arrays like the Renderer-array. These "allowed values"
would be useful both to the user who edits manually and to the UI. The
UI could use them to restrict the information you can select/enter in
certain fields.
Now once you have defined a template for a program, which is a lot of
work for complex applications like vim (but you could omit certain
options), you need to define the actual converter:
II. THE IMPORT/EXPORT FILTER. As I said above, this would use some
regex-based language to scan the configuration file and to convert the
options into the equivalents as defined in the QFG template, in
pseudo-format once again:
set font {Arial}
-> [Section $DISPLAY] $FONT = "\1"
and so on. It may be possible to define this converter in such a fashion
that it reliably works in both directions, otherwise it would be
necessary to define it separately. This is of course far more complex
than I make it sound. One question is how to treat unknown options. I
suggest simply returning a parsing error and asking the user to either
upgrade the filter or remove the unsupported option from the source
file.
Another question is how to deal with comments. Generally I would tend to
opine that comments in the source-file should be ignored for reasons of
simplicity (via a special code in the filter), and comments should be
made in the QFG file exclusively.
A more complex subject are user-defined records in the configuration
file like the fetchmail-example above. What do you do when you have
multiple entries for a given option? In the QFG file, using
pseudo-format again, we could write them simply like this:
[Recordsection Mail-Servers]
Record $SERVER <@ADDRESS>
Record $USERNAME <varchar,20>
Record $PASSWORD <varchar,20>
Record "Delete mail from server" = 0 <1,0>
...
and in another format for the actual content:
RecordEntry $SERVER="bla.com"
...
(Some content could even be included in the QFG template.)
Respectively, you would have to be able to define import filters which
a) create a new record based on a certain string / location in the input
file, b) represent the different fields. I think this is tricky for
different config files, but not impossible (anything is possible with
multiline regexps).
The ease of developing filters would be absolutely essential. It should
be a trivial process to write new filters/templates when you understand
the configuration file of a program, so that many users participate in
the process of writing new filters.
Parsing QFG files
-----------------
Once you have generated a QFG file from a template and, optionally, an
input file, you can handle it in various ways, e.g. you can generate a
GUI from it. A general parser library which would provide
structures/objects to use for other modules would make sense here.
Generating GUIs
---------------
This is of course one of the most exciting parts: Using a standard
config file format to generate a pre-defined GUI. As outlined above,
different sections in the config file could be represented as tabs in
the GUI. The value-restrictions would be handled in various ways,
through dropdowns or requesters (e.g. file/color/font requesters). Some
dropdowns would have to be editable, but still provide certain default
options.
The standard GUI should provide certain advanced features, like
annotation, or highlighting changes by coloring the tabs in which they
are made. Search/Replace would also be nice ("Search only in this
section - all sections ..") Records would be handled in a grid-like
manner, which would allow the user to add, modify and delete records,
using the same GUI elements as elsewhere.
Handling files
--------------
After the user accepts the changes, using the filters, the loaded QFG
file would be replaced with a new one, with annotations written into the
respective sections. Then the imported source config file would have to
be replaced as well using the export filter. Here annotations would also
be written next to the respective options, if the config file allows
comments, but this would only be for information purposes. The program
would automatically back-up the source configuration file during export
(generating a limited amount of parallel backups for each config file).
The next time you edit the config, if a timestamp-identical QFG version
exists, this would be used instead of running the import filter again.
Thereby the annotations would also be imported.
Generally speaking, QFG files should be stored in a .qonfig directory,
possibly with subdirectories for images / translations / backups etc.
This directory tree would need to be standardized.
Native files
------------
The situation would of course be much easier for programs which use QFG
natively. These could use the standard QFG graphical configuration tool,
or even text-based tools, within the program itself. They could even
offer the user several kinds of configuration from within the program,
e.g. "Configure CoolKDEApplication with your webbrowser". The number of
programs which use the format natively should grow as the standard
becomes more wide-spread thanks to more and more filters.
Using the front-end
-------------------
How would the user operate the graphical front-end for non-natively
supported applications? Basically, he would start "qonfig" with the
program as a parameter, or without any parameters. When started without
parameters, the front-end would offer the user a selection of supported
programs. This selection would have to be easily filterable/searchable.
From it, the user would also be able to "Get new filters" -- the program
would then automatically fetch the newest filters from the project's
website.
Regarding different program versions, the QFG templates would contain
version information. If several versions exist, these would all be
offered in the selector, newest versions first. Using outdated
filters/templates on newer config files should work as long as these new
files don't access any of the new features, which would result in
parsing errors, so it could easily be caught. (There may be some tricky
situations, but these should be the exception.)
Gimmicks
--------
There are lots of ways to extend this concept. Two nice gimmicks: USER
MODES, which would allow a QFG template to not only be split in sections
but also in different degrees of complexity, e.g. Novice, Advanced,
Expert etc. (These levels should have standardized names again.) Thereby
a user could filter the UI to only include basic options. INDEXING.
Large QFG files may need an additional index file to speed up the
parsing process.
Conclusion
----------
While the project would be quite complex, it is not a huge undertaking.
It would allow a single configuration standard for all applications, a
standard that could be built upon and extended gradually and
collaboratively. It would allow the user to use a multitude of different
front-ends if he so chooses, even for remote configuration using a
web-browser. The first implementation would mainly provide the parser,
some filters, and a good Qt or GTK-based GUI-generator. Using a
collaborative submission/selection process, new filters could be added
constantly. As more and more filters would be added, the program and
format would become more sophisticated, and developers would start
supporting it natively.
If successful, Linux and other Unixes which would support this standard
could eventually surpass competitors in usability/configurability, where
they are currently somewhat inferior with a multitude of different file
formats and front-ends.
Implementation
--------------
I throw these ideas out for general feedback. I'm primarily interested
in using the result, but I would make my role in the implementation
dependent on the feedback I get. I would certainly be willing to help
coordinate any efforts in this area, so if you plan to do something like
this, or know something like this, you can mail me and maybe I can bring
the right people together.
Sincerely:
Erik Moeller
(You may copy/modify this proposal as you wish, as long as you properly
credit the author, including e-mail address for feedback. I encourage
you to especially send it to people who you think may be interested in
implementing it, e.g. desktop developers, distributors, or alternative
operating system projects.)
cbbr...@acm.org
> I throw these ideas out for general feedback. I'm primarily
> interested in using the result, but I would make my role in the
> implementation dependent on the feedback I get. I would certainly be
> willing to help coordinate any efforts in this area, so if you plan
> to do something like this, or know something like this, you can mail
> me and maybe I can bring the right people together.
The _only_ way that your proposal is likely to get implemented is if
_you_ choose to implement it.
The only way it is likely to get used by anyone else is if you
demonstrate that your approach works (again, by _you_ implementing
it). At that point, there would be "several users."
The only way that it gets used widely is if you progress from having
an implementation to having a _mature_ implementation that is widely
usable. This is not likely to take place before 2004 (just to pick a
date a fair bit off into the future).
If you find that project plan daunting, as others have, that may
represent reason to give up on it, as others generally have. You are
_not_ going to convince others to join in until you have a workable
implementation.
--
(concatenate 'string "cbbrowne" "@cbbrowne.com")
http://www.cbbrowne.com/info/linuxsysconfig.html
Rules of the Evil Overlord #123. "If I decide to hold a contest of
skill open to the general public, contestants will be required to
remove their hooded cloaks and shave their beards before entering."
<http://www.eviloverlord.com/>
piani
another. Sure it may make editing configuations easier, but its adding
a layer of complexity. Another problem is that filters will need
continual maintainence.
It seems like very little gain at this point.
Going a step farther. If instead of playing with text files, all the
various configuation options were stored as key/attribute pairs in a
database, the gain is much greater.
1. You can point arbitrary keys at arbitrary attributes, giving the
ability to maintain a common set of preferences across multiple
applications.
2. Better granularity of changes and security. Includes less
application overhead for changing a single value in many
configuarations.
3. The opportunity to maintain many seperate configuation databases.
Configure multiple systems from a single application.
This is much like adding a Windows like registry to linux. But it is
much better because it is not tied to the operating system, just the
applications.
Adding registry support to an application is easier too.
GetRegistryKey and SetRegistryKey only need to popen executables of
the same name with the key/value as arguments.