[izpack-dev] New feature: Configuration file patching and merging
René Krell
and merging of configuration files, thus primarily
- properties (and two dimensional parameter assignment), as "key = value",
"key: value",
- INI files (3-dimensional - section, key, value) including registry entries
in Windows, which can be handled similarly (key, value, data),
- XML files.
It would bring IzPack more far to be convenient mfor ore complex "real-world"
install tasks.
For this purpose, at the first step, the old location or values of the
previous configurations must be gathered on the system, possibly installed in
a different location than the new one or from the registry in Windows. This
should be the first main part of the concept. For that purpose I would enhance
the dynamic variable concept from plain values to something like that (without
breaking older installer descriptions):
<dynamicvariables>
<variable name=_name_ [ checkonce="true"|"false ]
[ value=_plain_value_ ] |
[ environment=_env_variable_name_ ] |
[ file=_file_name_ type="properties" key=_key_ ] |
[ file=_file_name_ type="ini" section=<section> key=_key_ ] |
[ file=_file_name_ type="xml" key=_key_ ] |
[ regkey=_regkey_ regvalue=_regval_ ]
/>
</dynamicvariables>
Possible nested element for each variable is a regular expression filter using
Java regular expressions (inspired by and partly taken from the propertyregex
task from the ant-contrib framework):
<regex regexp=_expr_
[ select=_select_from_expr_ |
replace=_replace_in_expr_ [ global="true"|"false" ] ]
[ defaultValue=_plain_value_ ]
[ casesensitive="true"|"false" ]/>
Plain values, file and key names should be substitutable, so you can use other
variables (dynamic or compiler variables - to discuss) in it. This means also,
that previously defined variables in the install xml can be used in subsequent
definitions after resolving them on the target system. So you can at first
read and parse a path from a registry entry and after that read a certain
value from a configuration file to be found in the resulting location of the
above registry access.
This is already implemented here as a first pilot. I used a thin 3rdparty
framework, ini4j, with a small footprint, which does exactly the job we need
here and for the next parts in this concept.
BTW, ini4j can also replace the conventional registry access by JNI as it is
implemented now, because it uses simple "reg" system calls with a compatible
command line to even much older Windows versions.
After that it should be possible to use them to patch them. For that purpose I
thought it could be fine to have a new element in the packs section of the
installer descriptor file, for instance:
<configurable
[keepOldKeys="true|false"]
[keepOldValues="true|false"]
fromfile=_targetfile_ type=_type_ | fromregkey=_regkey_
targetfile=_targetfile_ type=_type_ | targetregkey=_regkey_
condition=_condition_id_>
<set [section=_sec_] key=_key_ value=_value_ [create="true"|"false]/> |
<set regvalue=_regval_ data=_data_ [create="true"|"false]/>
<delete [section=_sec_ [key=_key_]] | [key=_key_]/> |
<delete regkey=_regkey_ | regvalue=_regval_]/>
</configurable>
where _type_ := "properties" | "ini" | "xml" or
<configurableset
[keepOldKeys="true|false"]
[keepOldValues="true|false"]
[create="true"|"false"]
fromdir=_targetdir_ | fromregkey=_regrootkey_
targetdir=_targetdir_ | targetregkey=_regrootkey_
condition=_condition_id_>
[ <include=_includes_/>, ... ]
[ <exclude=_excludes_/>, ... ]
</configurableset>
to select a set of configurable files or registry root keys with the same name
relatively to source and target roots and provide standard actions with their
contents
NOTES:
Elements:
* <configurable> should concern only a single file or registry key
* <configurableset> should concern a set of configuration files or registry
keys under a root key. There are no single keys or value manipulable within a
configurableset, but the behaviour results from the mergemode given.
Attributes:
* keepoldkeys - If this option is false (default), parameters which do not
exist in the new parameter file or registry key will not be overtaken from old
files or registry keys, otherwise a full merge of parameters will be done.
Thus, this option decides about which parameters have to be in the target, not
their values. This attribute can be overriden by nested <delete> elements.
* keepoldvalues - If this option is true, parameter values or registry data
from old configurables override the values coming with the installation. Thus,
this option decides about the parameter values of the parameters, which are
meant to be in the target configurable at the end. This attribute can be
overriden by nested <set> elements.
* create - If this option is true a target configurable is created in the
target, if it exists in a previous installation in a different path given by
the fromdir attribute but doesn't come with the new installation. Otherwise
the installer will fail, if the configurable can't be found and keys are to be
set in it. A newly created configurable is added to the uninstaller file list
even if it doesn't come with the installer itself.
There might be some leaks in this concept, please tell them to me.
Maybe we can rename some elements or attributes to get the user better
understanding them.
Thanks,
René
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
Tino Schwarze
On Thu, Nov 26, 2009 at 07:02:51PM +0100, René Krell wrote:
> We need an important feature for handling software updates by IzPack: Patching
> and merging of configuration files, thus primarily
> - properties (and two dimensional parameter assignment), as "key = value",
> "key: value",
> - INI files (3-dimensional - section, key, value) including registry entries
> in Windows, which can be handled similarly (key, value, data),
> - XML files.
> It would bring IzPack more far to be convenient mfor ore complex "real-world"
> install tasks.
I've had this requirement as well. And the only clean solution is to
split configuration into two parts:
1. default configuration which will be overwritten on install time and
should not be altered by the user
2. local/user configuration which will only be installed once and is
under user-control later.
That way, it's also transparent to the user what he/she is allowed to
alter and that these settings won't get lost.
Of course, this requires support within the application to first read
the default configuration, then the user configuration which overrides
the default settings.
Automated patching of configuration will always be prone to errors and
may lead to unexpected behaviour - what do you do with values the user
altered?
HTH,
Tino.
--
"What we nourish flourishes." - "Was wir nähren erblüht."
www.lichtkreis-chemnitz.de
www.tisc.de
René Krell
thanks for your notes.
On Thursday 26 November 2009 19:27:12 Tino Schwarze wrote:
> ...
> I've had this requirement as well. And the only clean solution is to
> split configuration into two parts:
>
> 1. default configuration which will be overwritten on install time and
> should not be altered by the user
> 2. local/user configuration which will only be installed once and is
> under user-control later.
>
> That way, it's also transparent to the user what he/she is allowed to
> alter and that these settings won't get lost.
You are right, this is more comfortable, but let me leave some notes on that:
1. Often there is no chance to change the design of a certain application how
it handles it configuration. You get just the task "just to install and update
it".
2. To speak in terms of IzPack, the advantage of dividing "firmware
parameters" and "user parameters" of an application into different
configuration files for configuration automation is limited to being able to
overwrite the firmware parameters file with the new one. The user parameters
must still be patched somehow, at the moment I can see only the possibility of
using parsables and putting placeholders into the plain files, thus, keep the
contents of plain configuration files in sync with the install.xml.
BTW1: I mean that parsables are still good for plain files without a well-
known and OS-independent structure or format, as shell scripts. This concept
speaks about handling certain types (formats) of configuration resource in a
special manner - properties, ini, registry, maybe in future even more, as
system environment variables, which can be handled like key-value-pairs, too.
BTW2: What an installer never will be able to do - upgrading parameters
outside of the system scope, in the home directories of all users, for several
reasons, security, users might use different versions of the application and
possibly many more. Parameters in the users home directory are always
dynamically created by the application and must be handled and migrated by the
application itself. An installer can only pre-set system defaults.
3. To mix firmware and user parameters into one single configuration files
doesn't conflict with the described concept. You can delete obsolete
parameters which do no longer occur in the new version and their comments in
the target file by keepOldKeys="false", or set or delete some explicitely,
either by plain values or by user inputs.
>
> Of course, this requires support within the application to first read
> the default configuration, then the user configuration which overrides
> the default settings.
>
> Automated patching of configuration will always be prone to errors and
> may lead to unexpected behaviour ...
Not necessarily if it is well designed. This is not patching in a conventional
way of plain files using the 'diff' and 'patch' command, where you assume an
entry to be in a certain line at a certain position. For configurables there
should be use the processor from the ini4j framework, which keeps comments and
tolerates one ore more empty lines, whitespaces, user comments or different
parameter ordering coming from a manual change or different versions of the
previously installed files. The only requirement is to keep the interface -
property or ini file or a registry entry.
> what do you do with values the user altered?
Precisely therefore I intended to use dynamic variables, which are to be set
at install time and can alter on each panel change by definition. To set a
dynamic variable only once at the beginning of an install process I would use
the attribute checkonce="true", for instance to read a configuration entry
from a certain pre-installed file. You can give a dynamic variable an initial
value and overload it on each panel change by the appropriate user setting.
> ...
Perhaps you can give me an use-case or an example, and we will see if the
concept fits on it or whether there are leaks. Nobody is perfect :-)
Anyway I would maybe reduce the number of attributes, because in the most
common way, there are to be handled two kinds of configurables:
- key-value pairs (properties, options, environment variables)
- section-key-value triples (ini files or registry entries, where
section=registry key, key=registry value, value=registry data of a value)
René