Configuration database
Mats Liljegren
It has been said many times: One of Unix's weaknesses is the difficulty
of configuration. For home users, this means that Unix is only available
for those who are prepared to allocate a significant amount of time
learning it. For companies, this means high administration costs.
One thing that bugs me, is that it's very hard to get any overview of all
configuration. Also, not all configuration is shared between applications.
Only the basic configurations are, these that most applications can assume
exists on all systems that this application will be installed on.
This means two things: Configuration files is laying on many places (every-
one don't agree on where configuration files should reside), and the same
data has to be entered several times. The last one will be a source of many
troubles when something changes...
So I thought, why not centralize it? Making a central configuration data-
base? The best thing would be if all applications would access this database
directly. However, backward compatibility must be achieved. The easiest
way to solve this problem would be to convert what's in the database into
necessary configuration files.
Next question is, what should be stored in this database? And how? Using
textfiles is a de facto standard in the Unix community. Let's stick to that.
If binary configuration files would be needed for performance reasons, the
text files could be automatically compiled whenever something changes.
(Dates could be compared). So let's use text files.
Also, I would like to split the database into two parts: Current
configuration database, and history database. The current configuration
database would hold the actual configurations, and the history database
would store information about what applications needs. This would mean that
we could tell the history database that we've removed application X. The
history configuration database could then update the current configuration
database, in whatever way is appropriate.
With this system we get several things:
* A standardized way of storing configuration. Applications that use the
configuration database directly woun't have to bother about how to
implement this for themselves.
* Centralized configuration. This means you know where the configuration
files resides...
* You also have great possibilities to implement user friendly and generic
APIs to the database. If each applications told the database its name,
which configurations it needs, and some information about what these
configurations are and what they do, user friendly Linux configuration
would become a reality. Even when doing more advanced configurations, like
setting up a full implementation of a DNS server.
* Better configuration overview. This can mean less administration costs for
companies. It can also mean better security, as it would become easier to
spot configuration mistakes.
* If it's possible to implement dependencies for configurations, the chances
of doing mistakes would drop dramatically. However, I don't currently know
if this can be done in a generic way.
Should this become a project, or is it high time to forget the whole idea?
Or is there better ways to solve the problems this idea addresses?
/Mats
G Sumner Hayes
Mats Liljegren <mats.li...@abc.se> writes:
>
> I think you missed my intention: The database would live side-by-side
> with the configuratin files that exists today. Import and export
> functions would keep respective configurations up-to-date.
That's the worst of both worlds. The _only_ right ways to do this are
to either scrap the traditional config files and use only databases
--- which requires rewriting many utilities and adding plenty of
sysadmin tools --- or having your programs read and write the textual
config files --- which shouldn't be that hard because the utilities
that run from those config files are able to read them. What you
propose is absolutely awful, though; imagine what happens when I edit
one of the config files with emacs or vi. Everything's out of sync.
:(
Quite frankly, making sure that the files remain human readable and
human editable with standard text utilities should be a top priority;
a lot of automated code has been written in perl, sed, awk, and other
tools which would need to be re-written if you broke this assumption.
Cordially,
Sumner
Please don't e-mail postings to me, my mailbox is already full enough.
James Hughes
Kay Winkler wrote:
>
> Mats Liljegren wrote:
> >
> > It has been said many times: One of Unix's weaknesses is the
> > difficulty of configuration. For home users, this means that
> > Unix is only available for those who are prepared to allocate
> > a significant amount of time learning it. For companies, this
> > means high administration costs.
>
> the UNIX from IBM: AIX. They are doing this and it's horrible.
>
> We would need a generic configuration tool, with a description
> file for the possible entries (e.g generic entries) and
> an import and an export script, which will convert to or from
> the original files.
>
> Don't touch the configuration files that are established and are
> used by many people doing configuration by hand/feet.
I think we would not be able to do much with most text files anyway
other than stick them in a tar backup, as they are mostly from
applications.
The proc filesystem could be converted into a binary format though. If
it were done, the format should be the ASN1 standard used for SNMP
network management. An SNMP agent could allow management of "farms" of
Linux boxes across the network, or a local utility could a) browse the
MIB like an SNMP manager or b) spit out the MIB into text files like
todays proc filesystem c) produce html output for lynx for text or one
of the X based W3 browsers for X.
A small library for driver/kernel developers could provide the routines
to set/read the configuration and provide pointers to callbacks for
configuration changes. The apropriate OID could be set and the driver's
callback routine could make the changes on the fly.
This would really benefit those who want to use Linux as
gateway/router/middleware machines and workgroup servers in Intranets.
-James
Mats Liljegren
Kay Winkler wrote:
> Don't do this configuraton in a database maner. Have a look at
> the UNIX from IBM: AIX. They are doing this and it's horrible.
>
> We would need a generic configuration tool, with a description
> file for the possible entries (e.g generic entries) and
> an import and an export script, which will convert to or from
> the original files.
>
> Don't touch the configuration files that are established and are
> used by many people doing configuration by hand/feet.
I think you missed my intention: The database would live side-by-side
with the configuratin files that exists today. Import and export
functions would keep respective configurations up-to-date. Eventually,
most applications would hopefully use the database, as it can store
other parameters than just pure configurations. These parameters could
make it possible to de-install components without breaking anything.
/Mats
Kay Winkler
Mats Liljegren wrote:
>
> It has been said many times: One of Unix's weaknesses is the
> difficulty of configuration. For home users, this means that
> Unix is only available for those who are prepared to allocate
> a significant amount of time learning it. For companies, this
> means high administration costs.
Don't do this configuraton in a database maner. Have a look at
the UNIX from IBM: AIX. They are doing this and it's horrible.
We would need a generic configuration tool, with a description
file for the possible entries (e.g generic entries) and
an import and an export script, which will convert to or from
the original files.
Don't touch the configuration files that are established and are
used by many people doing configuration by hand/feet.
Gruss
Kay
============================================================
Kay Winkler Planckstrasse 1 D-64291 Darmstadt
(49)6159/71-2551 Fax:(49)6159/71-2785 K.Wi...@gsi.de
============================================================
o r c e l l . c h i . i l . u s
In article <wpLcyMKF...@abc.se>, Mats Liljegren <m8...@abc.se> wrote:
>It has been said many times: One of Unix's weaknesses is the difficulty
>of configuration. For home users, this means that Unix is only available
>for those who are prepared to allocate a significant amount of time
>learning it. For companies, this means high administration costs.
.
.
>So I thought, why not centralize it? Making a central configuration data-
>base? The best thing would be if all applications would access this database
>directly. However, backward compatibility must be achieved. The easiest
>way to solve this problem would be to convert what's in the database into
>necessary configuration files.
Or, alternatively, build a configuration database into a filesystem that
you can mount (/config) and which contains file-level representations of
the configuration data in the traditional way. Then you can symlink those
files into the conventional places, use regular backups and restores to
save and modify configurations, read info from the configuration with the
existing tools, and still have your centralised database to get at with
the new tools.
So you'd have your configdb in /etc/config.db, you'd have a configuration
filesystem userfs-mounted, and you've have links from fstab to /config/fstab,
mount to /config/mount, passwd to /config/passwd, and so on. It does beg
the question of why bother, though? If you're going to write a library to
read and write config to your centralised database, you can also do as I've
done and write a library that gives you a consistant interface to read and
write directly to the offending configuration files. This gives you a method
to look at and set the configuration without knowing about the config files,
but those files remain in case you need to get at them to repair them, you
don't need to modify the world to use them (same as a config-fs), and you
don't have some daemon (== another point of failure) doing the writing and
mirroring in database formats.
I agree with you that the existing setup is horribly nonintuitive and
downright confusing, but it is traditional and there are a lot of people
who are used to looking for things in those places. If you look at
ftp://gehenna.pell.chi.il.us/~ftp/pub/configure, I've copied sources to
some of the low-level libraries I'm using to implement the scheme I've
described above. (No documentation as of yet, because they were used
internally before I used them for WebShield.)
____
david parsons \bi/ o...@pell.chi.il.us
\/
Ian Burrell
In article <0mRGMD200...@andrew.cmu.edu>,
G Sumner Hayes <sum...@CMU.EDU> wrote:
>
>That's the worst of both worlds. The _only_ right ways to do this are
>to either scrap the traditional config files and use only databases
>--- which requires rewriting many utilities and adding plenty of
>sysadmin tools --- or having your programs read and write the textual
>config files --- which shouldn't be that hard because the utilities
>that run from those config files are able to read them. What you
>propose is absolutely awful, though; imagine what happens when I edit
>one of the config files with emacs or vi. Everything's out of sync.
>:(
>
The only way to do it is to write the library that does the standard
configuration file parsing and database management. Then offer it to
new software authors and porters of old software to gradually migrate
it over. It will never be universally accepted, but I am sure many
programmers would like to have a standard library for doing simple
configuration files so they don't have to write their own.
It still probably wouldn't be used to most standard utilities, just
for backward compatibility reasons. You want to keep the config files
in the same format as other Unix platforms, until you can convince the
software writes and users to accept the standard.
>Quite frankly, making sure that the files remain human readable and
>human editable with standard text utilities should be a top priority;
>a lot of automated code has been written in perl, sed, awk, and other
>tools which would need to be re-written if you broke this assumption.
>
Text files are a definite plus. Especially since most configuration
files are just key, value pairs with ASCII characters. Even short
binary strings can be encoded in text. BTW, the database consistency
problem is solved by having an in-memory daemon/server that re-reads the
configuration when it changes and serves requests from client
programs. But that is pretty remote from the Unix way of doing
things.
- Ian
--
-- Ian Burrell == ibur...@leland.stanford.edu **
<URL:http://www-leland.stanford.edu/~iburrell/>
You are only young once, but you can stay immature indefinitely.
Hans-Joachim Baader
In article <wpLcyMKF...@abc.se>, Mats Liljegren <m8...@abc.se> wrote:
>It has been said many times: One of Unix's weaknesses is the difficulty
>of configuration. For home users, this means that Unix is only available
>for those who are prepared to allocate a significant amount of time
>learning it. For companies, this means high administration costs.
True. Linus wants to place Linux on the desktop OS market.
But currently Linux-based systems are too hard to manage. Some
distributions try to be easy to use, but the distributions I have
seen so far are not much help with networking, shadow passwords,
logging and logfile rotation, etc.
[..]
>This means two things: Configuration files is laying on many places (every-
>one don't agree on where configuration files should reside), and the same
>data has to be entered several times. The last one will be a source of many
>troubles when something changes...
>
>So I thought, why not centralize it? Making a central configuration data-
>base? The best thing would be if all applications would access this database
>directly. However, backward compatibility must be achieved. The easiest
>way to solve this problem would be to convert what's in the database into
>necessary configuration files.
Your thought is quite reasonable; that must be the reason why it has
already been discussed by many people :-)
Backward compatibility would be hard to achieve because for each
program you would need another (small) program that converts the
config database entries into the program's config file format.
The opposite conversion would be even worse: manual changes of the
program's configuration files would bring them out of sync with
the config database, without a chance to automatically resychronize
them.
>Next question is, what should be stored in this database? And how? Using
>textfiles is a de facto standard in the Unix community. Let's stick to that.
That goes without saying.
>If binary configuration files would be needed for performance reasons, the
>text files could be automatically compiled whenever something changes.
>(Dates could be compared). So let's use text files.
I don't know a single unix program that uses binary config files.
>Also, I would like to split the database into two parts: Current
>configuration database, and history database. The current configuration
>database would hold the actual configurations, and the history database
>would store information about what applications needs. This would mean that
>we could tell the history database that we've removed application X. The
>history configuration database could then update the current configuration
>database, in whatever way is appropriate.
You mean reference counting and such? This would give us the same
problems as Windows, ie. being wrong most of the time.
>With this system we get several things:
>* A standardized way of storing configuration. Applications that use the
> configuration database directly woun't have to bother about how to
> implement this for themselves.
This would be a good thing even if it had no other benefits.
However it would not be easy to convert existing applications
to make use of it.
>* Centralized configuration. This means you know where the configuration
> files resides...
Even then, it would be hard enough to find what you search for.
But it would surely be easier to implement a search tool :-)
>* You also have great possibilities to implement user friendly and generic
> APIs to the database. If each applications told the database its name,
> which configurations it needs, and some information about what these
> configurations are and what they do, user friendly Linux configuration
> would become a reality. Even when doing more advanced configurations, like
> setting up a full implementation of a DNS server.
"User friendly" (whatever that means) configuration could be achieved
without centralization, too.
And why Linux? While many Linux distributions could make use of
it and support of the distribution maintainers would be helpful,
there's no point in restricting the idea to Linux. Most Linux
software runs on many other systems, too. So should the config
database.
>* Better configuration overview. This can mean less administration costs for
> companies. It can also mean better security, as it would become easier to
> spot configuration mistakes.
With a properly designed database and an appropriate tool to
use it, this may (and should) be the case.
>* If it's possible to implement dependencies for configurations, the chances
> of doing mistakes would drop dramatically. However, I don't currently know
> if this can be done in a generic way.
What kind of dependencies do you mean? If a program A depends on B,
does the configuration of A depend on the configuration of B? Or does
it only depend on the existence of B?
>Should this become a project, or is it high time to forget the whole idea?
There was the linux-conf mailing list which seems to be dead now
although I never received an obituary.
However, I have started development of a config database and a
library to manipulate it. It is comparable to the Windows registry,
only much more powerful (The Windows registry seems to be rather
primitive, from what I have seen).
It is system independent, public domain, and very incomplete. The
base code already runs, however. But it still needs some days of
work; and I have very little time.
And what is worse, I'm not really sure if such an approach will be
useful...
Hans-Joachim
--
Uncle Ed's Rule of Thumb: Never use your thumb for a rule.
You'll either hit it with a hammer or get a splinter in it.
Dimitri Maziuk
Mats Liljegren wrote:
>
... The database would live side-by-side
> with the configuratin files that exists today. ...
Developing import/export scripts that can work with every imaginable
layout of a given config file does not sound like an exciting job
to me. Keeping the two in sync may be more interesting: somebody
has to sit in background and keep checking for modifications in one
or the other. Oh, and you'd want rollback/recovery, too...
Something along the lines of Norton's INI tuner may be a better
option, I think.
Dimitri
--
An interesting sig that I saw recently:
"By US Code Title 47, Sec.227(a)(2)(B), a computer/modem/printer meet
the
definition of a telephone fax machine. By Sec.227(b)(1)(C), it is
unlawful
to send any unsolicited advertisement to such equipment. By
Sec.227(b)(3)(C),
a violation of the aforementioned Section is punishable by action to
recover
actual monetary loss, or $500, whichever is greater, for each
violation."
Mats Liljegren
david parsons wrote:
> Or, alternatively, build a configuration database into a filesystem that
> you can mount (/config) and which contains file-level representations of
> the configuration data in the traditional way. Then you can symlink those
> files into the conventional places, use regular backups and restores to
> save and modify configurations, read info from the configuration with the
> existing tools, and still have your centralised database to get at with
> the new tools.
This would be used instead of importing/exporting configurations files?
Hmm, this would actually make things more automatic, and give less
dependencies problems. (Sooner or later you'll forget to update
something, even if you use scripts for configuration!)
> So you'd have your configdb in /etc/config.db, you'd have a configuration
> filesystem userfs-mounted, and you've have links from fstab to /config/fstab,
> mount to /config/mount, passwd to /config/passwd, and so on. It does beg
> the question of why bother, though? If you're going to write a library to
> read and write config to your centralised database, you can also do as I've
> done and write a library that gives you a consistant interface to read and
> write directly to the offending configuration files. This gives you a method
> to look at and set the configuration without knowing about the config files,
> but those files remain in case you need to get at them to repair them, you
> don't need to modify the world to use them (same as a config-fs), and you
> don't have some daemon (== another point of failure) doing the writing and
> mirroring in database formats.
But you could make a consistent interface into a configuration library,
which uses the techniques you're describing. Having one big
configuration file in etc, or at least having all configuration files in
etc, would be great. To help administer these files, a frontend would be
nice. If you've got this interface library, this would be a piece of
cake.
A mistake I feel some authors are doing is assuming that some
configurations only applies for their program. Imagine the assumption
"using shadow passwords?", this is something shared among all programs.
Yet, I have to recompile/configure each and every program to make this
work. (Ok, some of them already have shadow support when shipped...).
Imagine a cacheing proxy server. A lot of configurations! And the author
thinks to himself "these configurations are unique for a cacheing proxy
server, no need for any standards here!". Wrong! What if I change to
another proxy server? Which happens to do cacheing...
> I agree with you that the existing setup is horribly nonintuitive and
> downright confusing, but it is traditional and there are a lot of people
> who are used to looking for things in those places. If you look at
> ftp://gehenna.pell.chi.il.us/~ftp/pub/configure, I've copied sources to
> some of the low-level libraries I'm using to implement the scheme I've
> described above. (No documentation as of yet, because they were used
> internally before I used them for WebShield.)
Your address didn't work. It said it couldn't find that page.
/Mats
Mats Liljegren
James Hughes wrote:
> The proc filesystem could be converted into a binary format though. If
> it were done, the format should be the ASN1 standard used for SNMP
> network management. An SNMP agent could allow management of "farms" of
> Linux boxes across the network, or a local utility could a) browse the
> MIB like an SNMP manager or b) spit out the MIB into text files like
> todays proc filesystem c) produce html output for lynx for text or one
> of the X based W3 browsers for X.
I don't quite follow you here. What has the proc filesystem to with
this? You can't store things there, so it could only be used for static
configurations, or configurations lying in memory.
As it is now, some configurations must be written via scripts.
(ifconfig, route, etc). Some configurations resides in more or less
standardized text files in /etc. Some configurations is declared
"private" and reside who-nows-where (only the imagination of the author
limits this).
Using proc for this, would mean that you had to make a gigantic script
for giving all configurations to the proc filesystem. Also, this would
mean that all configuration would reside in memory. But ok, memory can
be swapped, so using a little larger swap partition and armed with some
patience at boot-up, this is no problem.
But is it the right way?
I would like kernel-specific data to reside in proc. Things that kernel
has to have to operate.
If we used a userfs for the configuration database, it would be simple
to have links between the configuration filesystem and proc.
> A small library for driver/kernel developers could provide the routines
> to set/read the configuration and provide pointers to callbacks for
> configuration changes. The apropriate OID could be set and the driver's
> callback routine could make the changes on the fly.
You could do this with a configuration userfs.
> This would really benefit those who want to use Linux as
> gateway/router/middleware machines and workgroup servers in Intranets.
How?
/Mats
Mats Liljegren
Dimitri Maziuk wrote:
> Developing import/export scripts that can work with every imaginable
> layout of a given config file does not sound like an exciting job
> to me. Keeping the two in sync may be more interesting: somebody
> has to sit in background and keep checking for modifications in one
> or the other. Oh, and you'd want rollback/recovery, too...
I agree. Using userfs for this might be a solution. But it would be
better if everybody used the database directly, in one way or the other
(depending on which way is the best).
> Something along the lines of Norton's INI tuner may be a better
> option, I think.
I'm not familiar with Norton.
/Mats
James Hughes
Mats Liljegren wrote:
>
> James Hughes wrote:
> > The proc filesystem could be converted into a binary format though. If
> > it were done, the format should be the ASN1 standard used for SNMP
> > network management. An SNMP agent could allow management of "farms" of
> > Linux boxes across the network, or a local utility could a) browse the
> > MIB like an SNMP manager or b) spit out the MIB into text files like
> > todays proc filesystem c) produce html output for lynx for text or one
> > of the X based W3 browsers for X.
>
> I don't quite follow you here. What has the proc filesystem to with
> this? You can't store things there, so it could only be used for static
> configurations, or configurations lying in memory.
If you have Java on you machine, you have to tell the kernel where to
find the appletviewer by: 'echo /usr/local/java/bin/appletviewer >
/proc/sys/kernel/java-appletviewer'. When you use netstat, route, ps or
other tools, they read information from the /proc filesystem. The
CMU-SNMP agent for Linux is reading all this stuff when you poll it. You
can also set some things with it.
>
> As it is now, some configurations must be written via scripts.
> (ifconfig, route, etc). Some configurations resides in more or less
> standardized text files in /etc. Some configurations is declared
> "private" and reside who-nows-where (only the imagination of the author
> limits this).
These end up in the proc filesystem anyway try...
cat /proc/sys/kernel/hostname
(YourHostname Here)
hostname
(YourHostname Here)
echo testing > /proc/sys/kernel/hostname
cat /proc/sys/kernel/hostname
(testing)
hostname
(testing)
hostname (YourHostname Here)
hostname
(YourHostname Here)
cat /proc/sys/kernel/hostname
(YourHostname Here)
>
> Using proc for this, would mean that you had to make a gigantic script
> for giving all configurations to the proc filesystem. Also, this would
> mean that all configuration would reside in memory. But ok, memory can
> be swapped, so using a little larger swap partition and armed with some
> patience at boot-up, this is no problem.
>
> But is it the right way?
>
I think It depends on what you try to do with it. I don't think you
should use it to control the features of vi or application programs. I
think that's what I said at the onset of my post. To make all
applications programmers store configuration in the proc filesystem
would make Linux a more proprietary system than DOS or Windows.
> I would like kernel-specific data to reside in proc. Things that kernel
> has to have to operate.
>
Exactly!
> If we used a userfs for the configuration database, it would be simple
> to have links between the configuration filesystem and proc.
>
This might be useful! I was responsable for several networked computer
labs in a school. People would come in and play with the systems, which
was OK. The two most common problems were 1) people changing things and
not knowing how to fix it back and 2) the person who came behind this
guy could not recognize the system ro run programs.
To fix this I stored the configuration information in a password
protected, hidden ZIP file. Each time the system re-boots, the
information gets restored. I could see somthing like userfs (or
loopback) for saving application specific configuration data.
> > A small library for driver/kernel developers could provide the routines
> > to set/read the configuration and provide pointers to callbacks for
> > configuration changes. The apropriate OID could be set and the driver's
> > callback routine could make the changes on the fly.
>
> You could do this with a configuration userfs.
>
Best to keep kernel/process things in procfs.
> > This would really benefit those who want to use Linux as
> > gateway/router/middleware machines and workgroup servers in Intranets.
>
> How?
>
An exerpt from /usr/src/linux/Documentation/networking/Configurable...
# cat /proc/sys/net/ipv4/arp_timeout
6000
# echo 7000 > /proc/sys/net/ipv4/arp_timeout
# cat /proc/sys/net/ipv4/arp_timeout
7000
Look at the file '/usr/src/linux/net/TUNABLE'. Most of these parameters
(and probably others if given the capability) could probably be adjusted
on the fly if the driver had an interface to the outside world. Say we
want to make some adjustments to improve performance for a single
machine. We do a set command for the OID of teh parameter we want to
change. The Agent authenticates us, changes the value in the procfs and
finally signals the driver. The driver then responds by reading the new
value in the procfs and making the requested adjustment. We monitor the
machine for a while and it seems to be working better. Now we use our
management station to grab a group of the gateway machines and issue the
changes to all. They respond as the sample machine did and everyone
lives happily ever after.
This would work for ANY driver. WAN interfaces, HD subsystems, Ethernet
adapters, Routing, Firewalling............
I guess the configuration could get hard coded somewhere (maby in the
userfs) for boot time?
For local management of machines or for machines not on the network, a
utility which could manipulate the data the same way. The driver authors
could write modules for the 'Agent' instead of writing utility programs
like 'hostname, tune2fs, ipfwadm...'
> /Mats
I think a lot of this has been done already, it just needs a unified
interface and some changes here and there. Anyway, an interface like
this would blow anything like NT or other Unixes away in the management
arena. You can manage NT from NT, but not from Unix. This system would
allow management of Linux from any platform.
-James
Mats Liljegren
G Sumner Hayes wrote:
> Quite frankly, making sure that the files remain human readable and
> human editable with standard text utilities should be a top priority;
> a lot of automated code has been written in perl, sed, awk, and other
> tools which would need to be re-written if you broke this assumption.
userfs might solve the synch problem, and yet give perl etc access to
configurations. And this would provide for backward compatibility.
Can perl use library functions? If so, perl could be made to work with
the database.
/Mats
o r c e l l . c h i . i l . u s
In article <32766D...@abc.se>,
Mats Liljegren <mats.li...@abc.se> wrote:
>david parsons wrote:
>> ftp://gehenna.pell.chi.il.us/~ftp/pub/configure, I've copied sources to
>> some of the low-level libraries I'm using to implement the scheme I've
>> described above. (No documentation as of yet, because they were used
>> internally before I used them for WebShield.)
>
>Your address didn't work. It said it couldn't find that page.
Try, try, try again. The vagaries of dealing with web stuff is
a pain and it takes a while to get things working. (I resolve
ftp: urls by firing up ftp and going in by hand. This is not
quite the same as launching a browser.)
Dimitri Maziuk
Mats Liljegren wrote:
>
> Dimitri Maziuk wrote:
...
>
> > Something along the lines of Norton's INI tuner may be a better
> > option, I think.
>
> I'm not familiar with Norton.
>
It is basically a text editor with bells and whistles. Never mind,
its main feature is help on (often undocumented) windoze settings
-- not relevant in our case.
Dimitri
Mats Liljegren
Hans-Joachim Baader wrote:
> True. Linus wants to place Linux on the desktop OS market.
> But currently Linux-based systems are too hard to manage. Some
> distributions try to be easy to use, but the distributions I have
> seen so far are not much help with networking, shadow passwords,
> logging and logfile rotation, etc.
In this case, configuration database would be very necessay, IMHO.
> Backward compatibility would be hard to achieve because for each
> program you would need another (small) program that converts the
> config database entries into the program's config file format.
But if this approach is primarily directed towards normal desktop users,
this would mean that they probably would stay with distributions. And
with a distribution, you know what software you've got, and can create
tools to make them compatible with the configuration database.
When we implement the database, we wouldn't break anything existing.
Let's say we implement a userfs for the backward compatibility (which
would make synchronization automatic). All the standard packages with
their configurations (passwd, shadow, group, exports etc) would be
implement in this userfs, and linked from /etc.
Other programs using their own configurations, could still do this. And
also depend on standard configuration without any problems.
> The opposite conversion would be even worse: manual changes of the
> program's configuration files would bring them out of sync with
> the config database, without a chance to automatically resychronize
> them.
Some kind of automation must be done, at least using scripts. But I
think userfs would be a leaner solution.
> That goes without saying.
I come from DOS and OS/2 world, so for me, configurations in text files
isn't takened for granted. Yet...
> I don't know a single unix program that uses binary config files.
Which doesn't mean that could be done. See this like Java: You change a
textfile, but you run a binary file. The compilation could be done
automatically, though.
Another solution could be using binary index files into the text files,
to speed things up.
> You mean reference counting and such? This would give us the same
> problems as Windows, ie. being wrong most of the time.
Nah, not quite. I was more thinking of each program saying what they
need (I need a path to this location, for example), and then the
configuration database compiles all programs requirements into a
complete configuration.
> This would be a good thing even if it had no other benefits.
> However it would not be easy to convert existing applications
> to make use of it.
If we set up a standard interface I think this might become a standard,
sooner or later.
> Even then, it would be hard enough to find what you search for.
> But it would surely be easier to implement a search tool :-)
That's exactly what I'm after. Having huge amount of configuration will
always mean that it's hard to know where things are. Keeping everyting
central, would mean that you could automate the search.
> "User friendly" (whatever that means) configuration could be achieved
> without centralization, too.
Sure, but it would be much harder and much less stable. (People would
easily get their own idea how to do things...)
> And why Linux? While many Linux distributions could make use of
> it and support of the distribution maintainers would be helpful,
> there's no point in restricting the idea to Linux. Most Linux
> software runs on many other systems, too. So should the config
> database.
I'm thinking of Linux because this is a Linux conference and I'm using
Linux. However, it wouldn't hurt if it got portable, too...
> With a properly designed database and an appropriate tool to
> use it, this may (and should) be the case.
The design of the database is important, and I think much discussion
need to be done about it.
> What kind of dependencies do you mean? If a program A depends on B,
> does the configuration of A depend on the configuration of B? Or does
> it only depend on the existence of B?
More like common configurations. Let's say configuration a is common to
program X and Y. If we remove Y, can we remove configuration a? No, not
until we remove program X too...
> There was the linux-conf mailing list which seems to be dead now
> although I never received an obituary.
Do you think it's time to start a mailing list for this project?
> However, I have started development of a config database and a
> library to manipulate it. It is comparable to the Windows registry,
> only much more powerful (The Windows registry seems to be rather
> primitive, from what I have seen).
I saw some other text mentioning about a configuration database project,
was that you too, or is somebody else also doing this? In that case, it
seems like it's high time to get together instead of reinventing the
wheel all by ourselves. What a luck that I haven't started my project
yet!
> It is system independent, public domain, and very incomplete. The
> base code already runs, however. But it still needs some days of
> work; and I have very little time.
I know, I haven't got a lot of time either. But you're code could be a
good place to start with. Especially documents about how you did the
interface are interesting!
> And what is worse, I'm not really sure if such an approach will be
> useful...
If you make it public, maybe you'll find out... ;-)
/Mats
Markus Koskimies
In article <5537uj$k...@grumbeer.inka.de>,
ha...@grumbeer.inka.de (Hans-Joachim Baader) writes:
>>If binary configuration files would be needed for performance reasons, the
>>text files could be automatically compiled whenever something changes.
>>(Dates could be compared). So let's use text files.
>
>I don't know a single unix program that uses binary config files.
And that's extremely good. Never again turn back to binary files.
Text files are, of course, slower to handle by program than binary
files, but there are some really good points on these:
* You don't need that user-friendly configuration kit to edit it.
What about if you accidently destroy your configuration kit's
configuration?
* You don't need that kit, if your hard disk space is limited,
like in some embedded computers.
* ASCII-protocol is easy to debug
* It's flexible (there's much less troubles adding new features)
* Speed is not critical in configurations. 99.99 % of time spent
by program is surely something else than configuration
reading.
IMO, if configuration kits are developed, they should handle these
text files.
It was time when I thought that binary files are better than
ASCII-files. But that would not happen any more (except in certain
rare cases).
Albert D. Cahalan
ma...@maahinen.oulu.fi (Markus Koskimies) writes:
> In article <5537uj$k...@grumbeer.inka.de>,
ha...@grumbeer.inka.de (Hans-Joachim Baader) writes:
>>> reasons, the text files could be automatically compiled whenever
>>> something changes. (Dates could be compared). So let's use text
>>> files.
>>
>>I don't know a single unix program that uses binary config files.
Minicom uses one. Both sendmail and LILO compile text files into
binary ones.
> And that's extremely good. Never again turn back to binary files.
> Text files are, of course, slower to handle by program than binary
> files, but there are some really good points on these:
> * You don't need that user-friendly configuration kit to edit it.
> What about if you accidently destroy your configuration kit's
> configuration?
Don't do that. What if you destroy your text editor? Same deal.
My text editor won't run without a text config file in /usr/lib.
So what? I don't ever destroy that by accident, and even if I did
I can replace it from a backup, boot disk, or archive.
> * You don't need that kit, if your hard disk space is limited,
> like in some embedded computers.
Text often requires more space. Numbers in ASCII take 8/3 of the
space they would require in binary.
The config tool can be much smaller than the editor required to
modify text config files. Emacs is not small!
> * Speed is not critical in configurations. 99.99 % of time spent
> by program is surely something else than configuration reading.
Speed is critical if the program starts often and/or has a large
configuration file. Many people use Linux because it is fast.
It would be great to make Linux run even faster than it does now.
> It was time when I thought that binary files are better than
> ASCII-files. But that would not happen any more (except in
> certain rare cases).
Look in /proc. You will find a few binary files which were added
for performance reasons.
I think that a standard database should be used. Then we can query
it in the standard database way. Use gdbm or a plain SQL database.
If not that, then at least readable binary files. By that I mean
the files look like text, but they use space-padded fixed-width
fields and zero-padded hex numbers. You may carefully edit such
a file with an editor in overstrike mode, and the file format
does not impede performance.
--
--
Albert Cahalan
acahalan at cs.uml.edu (no junk mail please - I will hunt you down)
Sascha Ziemann
Mats Liljegren wrote:
>
> It has been said many times: One of Unix's weaknesses is the difficulty
> of configuration. For home users, this means that Unix is only available
> for those who are prepared to allocate a significant amount of time
> learning it. For companies, this means high administration costs.
This is a strong need! Consider GUILE for this job. It is intended for
such tasks.
-- bis später...
- Sascha ---<~>=( http://www.ping.de/sites/aibon/ )=<~>---
() Free speech online
/\ http://www.eff.org/BlueRibbon/bluehtml.html
Hans-Joachim Baader
In article <327798...@abc.se>,
Mats Liljegren <mats.li...@abc.se> wrote:
>When we implement the database, we wouldn't break anything existing.
That's right, but if the information of the traditional config
files is simply duplicated the system will be even harder to manage.
>Let's say we implement a userfs for the backward compatibility (which
>would make synchronization automatic). All the standard packages with
>their configurations (passwd, shadow, group, exports etc) would be
>implement in this userfs, and linked from /etc.
How portable would this be? If it can't be ported to all Unix systems
it wouldn't be of much use because many programs run on a wide range
of Unix systems. The maintainers of these programs are not very likely
to support a solution that only runs on the free Unixes.
>Which doesn't mean that could be done. See this like Java: You change a
>textfile, but you run a binary file. The compilation could be done
>automatically, though.
>
>Another solution could be using binary index files into the text files,
>to speed things up.
Could be done, of course. But it would cost additional disk space
without much benefits. Reading a config file with mmap() and a
handwritten parser, as it is done in my implementation, should
be fast enough.
>The design of the database is important, and I think much discussion
>need to be done about it.
There was already much discussion about it. I have saved most
of it. The summary of it, along with some of my own thoughts,
went into my implemtation.
>Do you think it's time to start a mailing list for this project?
Feel free to create a new list. I can't, for various reasons...
>> However, I have started development of a config database and a
>> library to manipulate it. It is comparable to the Windows registry,
>> only much more powerful (The Windows registry seems to be rather
>> primitive, from what I have seen).
>
>I saw some other text mentioning about a configuration database project,
>was that you too, or is somebody else also doing this? In that case, it
>seems like it's high time to get together instead of reinventing the
>wheel all by ourselves. What a luck that I haven't started my project
>yet!
I've read a statement from someone else. In fact, I guess there are
several projects which implement a kind of configuration database.
The question is, are they powerful enough? I don't only think of
configuring a local machine. Things become much more interesting
when it comes to network-wide configuration. Network administration
is very expensive. So there's a lot of money to save with the
proper tools.
There are already some network management tools around. SNMP would
be something to consider. I haven't played much with it, though.
>> It is system independent, public domain, and very incomplete. The
>> base code already runs, however. But it still needs some days of
>> work; and I have very little time.
>
>I know, I haven't got a lot of time either. But you're code could be a
>good place to start with. Especially documents about how you did the
>interface are interesting!
I have some poorly formatted ASCII documents. I'll pack it all
together and try to make it available by FTP if I have the time.
But this may take a while.
Regards
Mats Liljegren
Sascha Ziemann wrote:
> This is a strong need! Consider GUILE for this job. It is intended for
> such tasks.
Where can I get more information about Guile?
/Mats
Riku Saikkonen
Mats Liljegren (mats.li...@abc.se) wrote:
>Hans-Joachim Baader wrote:
>> I don't know a single unix program that uses binary config files.
>Which doesn't mean that could be done. See this like Java: You change a
>textfile, but you run a binary file. The compilation could be done
>automatically, though.
Part of sendmail was like that (the checkcompat() code, where you had to
modify the source and recompile), until they started changing it in version
8.8...
>Another solution could be using binary index files into the text files,
>to speed things up.
Is text file configuration parsing too slow? I haven't noticed, not even on
a 386SX laptop...
>That's exactly what I'm after. Having huge amount of configuration will
>always mean that it's hard to know where things are. Keeping everyting
>central, would mean that you could automate the search.
Most things _are_ central already -- they're either in /etc or in
/usr/local/etc...
Automate the search? Will you implement find, grep, awk and perl into your
binary configuration database?
(And, if you think about it, currently configuration files are in a database
too. The filesystem is a general-purpose database. If you want to make
accessing /etc/foobar.config faster, enhance the filesystem code, don't
reinvent and replace it.)
>> "User friendly" (whatever that means) configuration could be achieved
>> without centralization, too.
>Sure, but it would be much harder and much less stable. (People would
>easily get their own idea how to do things...)
Well, tell people who want to use `user-friendly' configuration that they
shouldn't modify the text files. The binary configuration scheme would force
_everyone_ to use the `(l)user-friendly' configuration... Now, only people
who want to use `user-friendly' configuration software have to worry about
not touching their configuration files manually.
Okay, so I'm critical. But I just don't see an advantage in binary
configuration files, and I see a lot of disadvantages. What do you gain in
binary configuration files?
- Speed? Text file parsing doesn't look so slow. And the fastest possible
system is to put configuration in a header file, so that the parsing gets
done when the software is compiled, and the program doesn't need to parse
anything at run-time. But there is a reason why it's not considered good
programming practice to embed configuration in source...
- Space? Binary files generally take less space than text, but the programs
to maintain those binary files more than make up for it (and, anyway, you
have to have some way to convert the binary configuration to text, since
the user has to be able to see what the configuration is). And if your
primary concern is space, simply using a compressed file system solves
most of this problem.
- Ease of maintaining? That has little to do with text vs. binary, it has
to do with what `user-friendly' configuration helpers are available.
However you store your configuration file, I don't think it's possible to
have configuration software that is both user-friendly (the traditional
GUI way?) and anywhere near as flexible as a text file.
For example, how do you have comments in a binary configuration file? At
least I use a lot of comments in my text configuration files, both
commenting out old or default settings and as a remainder to myself why I
chose some option and what other choices I think I can use. Commenting
out one configuration directive ("Papersize: a4" or whatever) while using
another is very useful, and not really possible in a binary file (how do
you `comment out' a dialog box in a configuration program?).
--
-=- Rjs -=- r...@spider.compart.fi, r...@lloke.dna.fi
Software patents do not further technology.
Anselm Lingnau
In article <327A5727...@interpath.com>,
James Hughes <jam...@interpath.com> wrote:
> I have seen this topic discussed all over the place. I'm sure if we
> gather enough interest, someone would sponsor us with a place to work on
> the project. This is a very important area for Linux's future growth!
You can use the list that *does* exist, which is at
linux...@tm.informatik.uni-frankfurt.de
Send (un)subscription requests to
linux-con...@tm.informatik.uni-frankfurt.de
There is an archive site containing past postings etc. at
http://www.tm.informatik.uni-frankfurt.de/linux-conf/
Anselm
--
Anselm Lingnau ......................... lin...@tm.informatik.uni-frankfurt.de
And whatever you do, never, ever, play Trivial Pursuit against a reference
librarian. --- Jim Frost
Albert D. Cahalan
In article <E06wM...@lloke.dna.fi> r...@lloke.dna.fi (Riku Saikkonen) writes:
> Mats Liljegren (mats.li...@abc.se) wrote:
>> Hans-Joachim Baader wrote:
>>
>>
>> textfile, but you run a binary file. The compilation could be done
>> automatically, though.
>
> Part of sendmail was like that (the checkcompat() code, where you
> had to modify the source and recompile), until they started changing
> it in version 8.8...
That works.
>> Another solution could be using binary index files into the
>> text files, to speed things up.
> Is text file configuration parsing too slow? I haven't noticed,
> not even on a 386SX laptop...
It can be. Don't think "it only takes 0.1 seconds to parse this",
but instead think "this must be parsed 5000 times an hour".
That is why sendmail uses some binary files.
>> That's exactly what I'm after. Having huge amount of configuration
>> will always mean that it's hard to know where things are. Keeping
>> everyting central, would mean that you could automate the search.
>
> Most things _are_ central already -- they're either in /etc or in
> /usr/local/etc...
>
> Automate the search? Will you implement find, grep, awk and perl
> into your binary configuration database?
No, perl gets a module with functions that access the database.
Maybe a database cat program would exist. SQL might work.
> Okay, so I'm critical. But I just don't see an advantage in binary
> configuration files, and I see a lot of disadvantages. What do you
> gain in binary configuration files?
>
> - Speed? Text file parsing doesn't look so slow.
Not too slow if you only do it once... Think 100 times/second.
> - Space? Binary files generally take less space than text, but the programs
> to maintain those binary files more than make up for it (and, anyway, you
> have to have some way to convert the binary configuration to text, since
> the user has to be able to see what the configuration is). And if your
> primary concern is space, simply using a compressed file system solves
> most of this problem.
Emacs is required to maintain text files. Emacs is NOT small!
> - Ease of maintaining? That has little to do with text vs. binary, it has
> to do with what `user-friendly' configuration helpers are available.
> However you store your configuration file, I don't think it's possible to
> have configuration software that is both user-friendly (the traditional
> GUI way?) and anywhere near as flexible as a text file.
> For example, how do you have comments in a binary configuration file? At
> least I use a lot of comments in my text configuration files, both
> commenting out old or default settings and as a remainder to myself why I
> chose some option and what other choices I think I can use. Commenting
> out one configuration directive ("Papersize: a4" or whatever) while using
> another is very useful, and not really possible in a binary file (how do
> you `comment out' a dialog box in a configuration program?).
You click on the `comment out' button.
James Hughes
Mats Liljegren wrote:
>
> Hans-Joachim Baader wrote:
> > That's right, but if the information of the traditional config
>
> > Feel free to create a new list. I can't, for various reasons...
>
> I can't either...
I have seen this topic discussed all over the place. I'm sure if we
gather enough interest, someone would sponsor us with a place to work on
the project. This is a very important area for Linux's future growth!
> > I've read a statement from someone else. In fact, I guess there are
> > several projects which implement a kind of configuration database.
> > The question is, are they powerful enough? I don't only think of
> > configuring a local machine. Things become much more interesting
> > when it comes to network-wide configuration. Network administration
> > is very expensive. So there's a lot of money to save with the
> > proper tools.
>
> I agree, a database that could be shared over a network would be much
> more interesting. Feeding bootp, DHCP, NIS etc with this information
> would be interesting. Also, having a netwide protocol of your own, where
> all information could be obtained, would also be nice.
>
> This means that the database have to keep track of:
>
> What settings there is.
> Some kind of description for each setting.
> Which programs/applications that need this setting.
> The value of the setting.
> Which computer this is for.
>
> We also have the question how to deal with the users. Should each user
> have their own database file in their home directory, replacing all
> these ".<application>" configuration files? Or should this be in the
> system wide configuration database?
>
Smells like the MS Win32 registration database to me! This is actually
a powerful tool in their OS, it's just cryptic and not documented very
well (?Where Have I Heard That One Before?). Theirs is HUGE! It has
everything in there (Even passwords to the E-Mail system!) YUK!
It does allow for roving users which can be controlled tightly by the
system policies. This is a database which can reside on a network
computer and be accessed from the workstations. In this database, you
can set limits for each user and machine. You force the workstation to
be authenticated off the Domain controller (Running Linux Of Course;).
When they log in, the registry data for the machine/user is downloaded
into the workstation. The user can change preferences as allowed by the
policies in force. These are saved on the users home directory (On The
Linux Server;).
This thing (the registry), on the workstation, is a concatination of
two files, SYSTEM.DAT and USER.DAT. The USER.DAT gets written to the
users home directory for roving/multi-user functionality.
The problem is that you can only manage this from another Win32 machine
and some things can only be done if you have NT managing your user
passwords. We can make this so that it can be managed from any platform.
> > There are already some network management tools around. SNMP would
> > be something to consider. I haven't played much with it, though.
>
> I don't know what this protocol can do, actually. Haven't look at its
> capabilities.
>
SNMP would work well for controlling things like tuneable parameters in
the procfs and monitoring performance. Some work is being done towards
re-formatting the procfs. If this is done right, the developer will be
able to easily place both reporting and control features in the code
which could be accessed via an SNMP agent.
I think a lot of the groundwork has been done, or is in progress.
Finding out 'who is doing what' in relation to this project and getting
the parts together will be the hardest part:)
-James
PS: The next time some Wind-blows user says that Unix is cryptic, tell
them that they've never had to do any real work with their OS!
Hans-Joachim Baader
In article <327A25...@abc.se>,
Mats Liljegren <mats.li...@abc.se> wrote:
>Hans-Joachim Baader wrote:
>> That's right, but if the information of the traditional config
>
>would just mirror the contents of the database.
I agree. But the traditional config files may be altered by hand.
These changes would not be reflected into the config database, so
a config tool would see things not as they are. Maybe this problem
is not as big as it seems, but it would be quite an effort to
reflect such manual changes in the database.
>> Could be done, of course. But it would cost additional disk space
>> without much benefits. Reading a config file with mmap() and a
>> handwritten parser, as it is done in my implementation, should
>> be fast enough.
>
>If you would use text files with static width strings, there would be
>nearly no speed penalty. However, there would be some problems with
>this. Which length to use would become the biggest problem.
I'd like to avoid them all ;-). My implementation has no length limits
at all.
>> There was already much discussion about it. I have saved most
>> of it. The summary of it, along with some of my own thoughts,
>> went into my implemtation.
>
>Where has this been discussed?
On the linux-conf mailing list which is probably dead now.
>I agree, a database that could be shared over a network would be much
>more interesting. Feeding bootp, DHCP, NIS etc with this information
>would be interesting. Also, having a netwide protocol of your own, where
>all information could be obtained, would also be nice.
>
>This means that the database have to keep track of:
>
>What settings there is.
>Some kind of description for each setting.
>Which programs/applications that need this setting.
>The value of the setting.
>Which computer this is for.
SNMP does this all. But there are other protocols which do similar
things, it even seems that CMIS/CMIP has come to life ;-)
>We also have the question how to deal with the users. Should each user
>have their own database file in their home directory, replacing all
>these ".<application>" configuration files? Or should this be in the
>system wide configuration database?
IMHO, every user should have his own database. The values in this
database override those in the system database. Some care must be
taken so that a user cannot override values that only privileged
users may alter.
James Hughes
Anselm Lingnau wrote:
>
>
> You can use the list that *does* exist, which is at
>
> linux...@tm.informatik.uni-frankfurt.de
>
> Send (un)subscription requests to
>
> linux-con...@tm.informatik.uni-frankfurt.de
>
> There is an archive site containing past postings etc. at
>
> http://www.tm.informatik.uni-frankfurt.de/linux-conf/
>
Good Deal!
-James
Roger Espel Llima
In article <vc7k9s5...@jupiter.cs.uml.edu>,
Albert D. Cahalan <no_junk_m...@jupiter.cs.uml.edu> wrote:
>In article <E06wM...@lloke.dna.fi> r...@lloke.dna.fi (Riku Saikkonen) writes:
>> - Speed? Text file parsing doesn't look so slow.
>Not too slow if you only do it once... Think 100 times/second.
Text parsing isn't considerably slower than binary parsing if done
properly; the right tools ([f]lex, ...) are readily available.
Besides, there may be exceptions, but I'd say that just about any
program that needs to parse 100s of lines of config-file text a second
has broken design. Have it re-read it only when it gets a signal or
something.
>Emacs is required to maintain text files. Emacs is NOT small!
Required? I sure hope not. (Not to start an editor flame war; I find
it great that people use whatever tools they like and are used to).
$ ll -L =vim
-rwxr-xr-x 1 root users 316556 Oct 31 1996 /usr/bin/vim*
$ ll -L =elvis
-rwxr-xr-x 1 root root 272334 Jun 10 1996 /usr/bin/elvis*
$ ll =jed
-rwxr-xr-x 1 root bin 187512 Jun 13 1995 /usr/bin/jed*
$ ll =joe
-rwxr-xr-x 1 root root 172716 Aug 10 17:43 /usr/bin/joe*
$ ll -L =ed
-rwxr-xr-x 1 root root 43793 Oct 8 18:48 /usr/bin/ed*
Is that small enough for you? I've seen perfectly functional and usable
editors in the 32k range for DOS, there is no reason the same couldn't be
done for Unix/Linux, and in fact I think some Linux distribs come with a
full-screen editor that is very small.
More generally, I see no reason to move from text to binary for config
files. The only advantage binary has is that it's a little smaller, but
disk space is cheap these days and we're talking a few K's anyway.
Anything (automated config tools, whatever) that can be done w/ binary
files can also be done, and will probably be easier (if nothing, easier
to debug) with text files. ASCII text is *the* standard, and it leaves
the most freedom to the user.
-Roger
--
e-mail: roger.es...@ens.fr
WWW page & PGP key: http://www.eleves.ens.fr:8080/home/espel/index.html
Neil Moore
no_junk_m...@jupiter.cs.uml.edu (Albert D. Cahalan) writes:
> r...@lloke.dna.fi (Riku Saikkonen) writes:
> > to maintain those binary files more than make up for it (and, anyway, you
> > have to have some way to convert the binary configuration to text, since
> > the user has to be able to see what the configuration is). And if your
> > primary concern is space, simply using a compressed file system solves
> > most of this problem.
>
Emacs is NOT required! I suspect vi-lovers around the world (not
me, I like Emacs) will be smacking you upside the head for saying
that :)
--
Version: 3.12
GCS/M/L d- s-:+ a--- C+++(++)>$ UL+(++)>++++$ P+++ L+++>++++$ E++ W-(--)
N++ o-- K-- w--- O- M-- V-- PS+++? PE Y+ PGP-- t+* 5- X R tv--? b+>+++ DI
D--- G e-(*)>++++ h r++>+++ y?
John Gouk
I haven't included all the various original posts here - apologies.
It seems to me that the point of the configuration stuff is not really
the underlying technology; if "we" want to make Linux simpler to
configure and use, it's more a case of presenting the correct level of
information and control at the appropriate time to the user. The actual
configuration could be stored on paper tape for all the user cares.
Having said that, in keeping with the original kind of Unix ethic, it
makes more sense to use ASCII text files. These can be edited with no
special tools at all if you're really stuck. They can be read without
any tools, and as someone pointed out, all those old configs can be left
in there as comments with no special magic. Impressive looking as the
Win95 registry is, it doesn't actually store any more information than
the collection of text files used by Linux; it's just the centralised
availability and presentation that is useful.
SNMP? Great idea. CMIP? Good too, if you can get your head round it. But
it's just presentation with a different interface - the semantics maybe
more or less enabling.
This is interesting. Keep talking!
Hans-Joachim Baader
In article <E0Ar0...@lloke.dna.fi>, Riku Saikkonen <r...@lloke.dna.fi> wrote:
>Albert D. Cahalan (no_junk_m...@jupiter.cs.uml.edu) wrote:
>>In article <E06wM...@lloke.dna.fi> r...@lloke.dna.fi (Riku Saikkonen) writes:
>>> Part of sendmail was like that (the checkcompat() code, where you
>>> had to modify the source and recompile), until they started changing
>
>Works, but compilation is slow, you have to do it for every architecture
>(for heterogenous networks), and you have to keep around a copy of the
>configured source...
If the compiled config file only contains fixed-with text fields, it
would be sharable between architectures. And compilation can be done
automatically. But I don't speak for binary files nevertheless.
>I think it's a sign of broken design if you have to read your configuration
>5000 times an hour. But if you really want that, you could have the program
>parse the text configuration, do a core dump, and then run the program from
>that core dump 5000 times an hour... (Kind of like emacs does with its
>standard elisp code.)
Well, nearly every tool can make good use of config files. If it is
called often, you reach 5000 quite quickly. Consider the 'ls' program.
It currently *does* read /etc/passwd and /etc/group on every run.
If this information were in the config database instead, this would
result in quite a lot of reading, but certainly no more than currently.
To improve throughput further, I consider having a config database
daemon that processes requests from client programs. This way only
one program would actually access the config database. It could
use a cache for config files, so that the most often accessed files
are already parsed and available in the daemon's memory. On the other
hand, there would be the additional overhead of communication between
daemon and clients, and memory copy operations. But for network-wide
configuration, a daemon would be needed anyway (again SNMP is an
example).
>>> - Space? Binary files generally take less space than text, but the programs
>>> to maintain those binary files more than make up for it (and, anyway, you
Actually I think binary config files would be bigger than ASCII ones
because you need fixed-with fields and you must make these fields
large enough to hold all information.
What's more important than size, binary config files have *limited*
field widths, and by Murphy's law there will be applications for
which they will be too small. In contrast, the implementation I work on
doesn't have any limitations except available memory. But then it
mallocs and frees a lot. But I think it's still very fast.
Binary config files? Not with me. It would be as silly as buying
Windows...
Regards
Mats Liljegren
Riku Saikkonen wrote:
> Is text file configuration parsing too slow? I haven't noticed, not even on
> a 386SX laptop...
The configuration database can get pretty big, and if there'll be many
questions from different programs, the delay can be noticeable, But
we'll find out, and do something about it, when the problem occurs.
> Most things _are_ central already -- they're either in /etc or in
> /usr/local/etc...
Not on my machine. I've got named configuration files in /var/named.
I've got configuration files for squid (proxy server) in
/usr/local/squid/etc. And so on...
> Automate the search? Will you implement find, grep, awk and perl into your
> binary configuration database?
What's wrong with the text database? The binary database would just be a
mirror of the text database, so it would be up to date. What's more of a
concern is the binary database, and keeping _that_ up to date.
> (And, if you think about it, currently configuration files are in a database
> too. The filesystem is a general-purpose database. If you want to make
> accessing /etc/foobar.config faster, enhance the filesystem code, don't
> reinvent and replace it.)
This means very hard standardization of where these files should be, and
what they should contain. As it is today, only a few files meet these
criteria. Today, we've even got problems with the password file! Not
even that file has a uniform standard, as it might be shadowed or
non-shadowed. And now there's PAM, but I don't know how PAM is working.
Using several files could mean duplicate data. It's very easy to do
that. And that would make things harder to change in the future. It's
easier using only one file. No hazzle when you have to move the
configuration file(s).
> Well, tell people who want to use `user-friendly' configuration that they
> shouldn't modify the text files. The binary configuration scheme would force
> _everyone_ to use the `(l)user-friendly' configuration... Now, only people
> who want to use `user-friendly' configuration software have to worry about
> not touching their configuration files manually.
Use the text editor against the text configuration file, if you don't
like user friendliness.
> Okay, so I'm critical. But I just don't see an advantage in binary
> configuration files, and I see a lot of disadvantages. What do you gain in
> binary configuration files?
> - Speed? Text file parsing doesn't look so slow. And the fastest possible
> system is to put configuration in a header file, so that the parsing gets
> done when the software is compiled, and the program doesn't need to parse
> anything at run-time. But there is a reason why it's not considered good
> programming practice to embed configuration in source...
With binary index file, we would gain speed. That could be enough.
> - Space? Binary files generally take less space than text, but the programs
> to maintain those binary files more than make up for it (and, anyway, you
> have to have some way to convert the binary configuration to text, since
> the user has to be able to see what the configuration is). And if your
> primary concern is space, simply using a compressed file system solves
> most of this problem.
Space is not an issue, if we're going to use binary files. As I made
very clear from the very beginning, we're going to have text files
no-matter-what. If we're using binary files also, space is apparantely
no problem.
> - Ease of maintaining? That has little to do with text vs. binary, it has
> to do with what `user-friendly' configuration helpers are available.
> However you store your configuration file, I don't think it's possible to
> have configuration software that is both user-friendly (the traditional
> GUI way?) and anywhere near as flexible as a text file.
You can have a front-end against the text file, so this has nothing to
do with binary files.
> For example, how do you have comments in a binary configuration file? At
> least I use a lot of comments in my text configuration files, both
> commenting out old or default settings and as a remainder to myself why I
> chose some option and what other choices I think I can use. Commenting
> out one configuration directive ("Papersize: a4" or whatever) while using
> another is very useful, and not really possible in a binary file (how do
> you `comment out' a dialog box in a configuration program?).
The comments would stay in the text file, you would have to search there
to find them.
It's pretty clear your against any form of binary files. And as we're
going to use text files (no one has AFAIK mentioned replacing these with
binary files, only using them for speed-up), we can wait about deciding
wether to use binary files or not. Using index files only could be one
alternative. But as I said, this decision doesn't have to be made today.
If we've got a library interface against this configuration database,
we'll have the freedom to change our minds later.
/Mats
stephen farrell
no_junk_m...@jupiter.cs.uml.edu (Albert D. Cahalan) writes:
>
> > - Space? Binary files generally take less space than text, but the programs
> > to maintain those binary files more than make up for it (and, anyway, you
> > have to have some way to convert the binary configuration to text, since
> > the user has to be able to see what the configuration is). And if your
> > primary concern is space, simply using a compressed file system solves
> > most of this problem.
>
> Emacs is required to maintain text files. Emacs is NOT small!
>
what are you talking about? have you ever heard of vi?
Sam Glasby
In article <558o1q$m...@grumbeer.inka.de>,
Hans-Joachim Baader <ha...@grumbeer.inka.de> wrote:
>In article <327798...@abc.se>,
>Mats Liljegren <mats.li...@abc.se> wrote:
[SNIP]
On a quite-related note to this
configuration database proposal, there exists
a thread on comp.os.linux.advocacy
(heck, who am I kidding... *.advocacy :-)
regarding a Hypothetical NT Registry filesystem.
This would be a filesystem view (text
tools, using standard POSIX API) onto the NT Registry,
in the style of the /proc filesystem.
We have discussed various issues
relevant to this related Linux proposal.
I recommend that you peruse said thread.
On another quite-related note,
it occurs to me to wonder if any of you are
familiar with the NetInfo database approach
used in NeXTSTEP for similar purposes.
Not sure what documentation exists on NetInfo
on the 'Web/etc, but for good or ill, it
is an existing system of similar nature
you may want to look at (to steal good
ideas, or run screaming from botched
implementations, as the case may be! :-)
Hope this is helpful...
>Hans-Joachim
--
Sam Glasby <sgl...@primenet.com>
Pagan, RPG designer, and user of Linux and Emacs
Sam Glasby
In article <327A25...@abc.se>,
Mats Liljegren <mats.li...@abc.se> wrote:
>Hans-Joachim Baader wrote:
>> That's right, but if the information of the traditional config
>> files is simply duplicated the system will be even harder to manage.
>
>The idea was that only the database would be altered, everything else
>would just mirror the contents of the database.
A perhaps helful idea someone suggested
on the .advocacy thread about NT Registry filesystem:
an earlier OS coexisting on same HW as a Unix flavor
shared binary configuration database information
by means of a read-only Unix special file
/etc/passwd (etc)
which appeared as a normal /etc/passwd
This approach is similar to a /proc filesystem
or the like.
Vital, of course, is to provide tools to (re)generate
the database from textfiles, and to (re)generate textfiles
from a database, in any such scheme.
NeXT's NetInfo is an example to look at, also.
Hope this is helpful...
Kyle R. Hofmann
In article <327E78...@abc.se>, Mats Liljegren wrote:
: Riku Saikkonen wrote:
: > Most things _are_ central already -- they're either in /etc or in
: > /usr/local/etc...
: Not on my machine. I've got named configuration files in /var/named.
: I've got configuration files for squid (proxy server) in
: /usr/local/squid/etc. And so on...
So what if we move all of the config files to somewhere else?
Currently, the FSSTND doesn't seem to specify such a place. /var looks
like the best place to put such a directory, and /var/config would be
fairly intuitive. Creation of a new directory for config files would
also be easier than creating a central configuration database --- it
would only require that references to old directories are changed,
instead of having to rewrite large portions of programs.
It does present a potential problem if /var is mounted on a separate
drive and the drive dies. Anything which needs its config file may not
work right. This shouldn't be much of a problem, AFAICT, because there's
nothing that's truly vital that can't have its config file
reconstructed. It would be safer to keep essential system config files
on root, perhaps in /etc/config. And even with two places for config
files, it would still be easier than the current "anything goes" approach
and would, IMHO, be better than one monstrous config file.
-Kyle R. Hofmann
Mats Liljegren
Hans-Joachim Baader wrote:
> I agree. But the traditional config files may be altered by hand.
> These changes would not be reflected into the config database, so
> a config tool would see things not as they are. Maybe this problem
> is not as big as it seems, but it would be quite an effort to
> reflect such manual changes in the database.
Userfs could solve this.
> I'd like to avoid them all ;-). My implementation has no length limits
> at all.
Having limits on how long the lines should be is a bad idea, in my
opinion too...
> On the linux-conf mailing list which is probably dead now.
No use joining that one, then?
> SNMP does this all. But there are other protocols which do similar
> things, it even seems that CMIS/CMIP has come to life ;-)
CMIS/CMIP??
> IMHO, every user should have his own database. The values in this
> database override those in the system database. Some care must be
> taken so that a user cannot override values that only privileged
> users may alter.
This means we've got a multi-layered distributed database with privilege
control over the elements. Better think twice before doing this, or
it'll end up more complicated than anything else ;-)
Have the layering generic would be nice, this would make it possible to
have a networld configuration, which is more and more detailed the
nearer the applications we get.
/Mats
Mats Liljegren
Kyle R. Hofmann wrote:
> So what if we move all of the config files to somewhere else?
> Currently, the FSSTND doesn't seem to specify such a place. /var looks
> like the best place to put such a directory, and /var/config would be
> fairly intuitive. Creation of a new directory for config files would
> also be easier than creating a central configuration database --- it
> would only require that references to old directories are changed,
> instead of having to rewrite large portions of programs.
That would make things eaiser. But why use /var? /var is specified to be
used for data, not configuration. The /var directory should consist of
information that is constantly changing. Configuration changes only
sporadically.
And if you really want to put your configuration in /var, use a link.
/etc is the standard, let's use it!
> It does present a potential problem if /var is mounted on a separate
> drive and the drive dies. Anything which needs its config file may not
> work right. This shouldn't be much of a problem, AFAICT, because there's
> nothing that's truly vital that can't have its config file
> reconstructed. It would be safer to keep essential system config files
> on root, perhaps in /etc/config. And even with two places for config
> files, it would still be easier than the current "anything goes" approach
> and would, IMHO, be better than one monstrous config file.
This solves problems with system and user specific configurations. But
if I want network wide configurations?
/Mats
Hans-Joachim Baader
In article <3289CF...@ebc.ericsson.se>,
Mats Liljegren <Mats.Li...@ebc.ericsson.se> wrote:
>
>> On the linux-conf mailing list which is probably dead now.
>
>No use joining that one, then?
I stand corrected. The list still exists and I have re-subscribed
to it. I haven't received a single article yet, however. Feel free
to continue the discussion in this list.
>> SNMP does this all. But there are other protocols which do similar
>> things, it even seems that CMIS/CMIP has come to life ;-)
>
>CMIS/CMIP??
The ISO/OSI standard for network management. I don't know anything
about it, I only know that it exists ;-)
>> IMHO, every user should have his own database. The values in this
>> database override those in the system database. Some care must be
>> taken so that a user cannot override values that only privileged
>> users may alter.
>
>This means we've got a multi-layered distributed database with privilege
>control over the elements. Better think twice before doing this, or
>it'll end up more complicated than anything else ;-)
Already thought about it 3 times :-) It's not that difficult.
>Have the layering generic would be nice, this would make it possible to
>have a networld configuration, which is more and more detailed the
>nearer the applications we get.
Regards
Kyle R. Hofmann
In article <3289D2...@ebc.ericsson.se>, Mats Liljegren wrote:
: Kyle R. Hofmann wrote:
: > So what if we move all of the config files to somewhere else?
: > Currently, the FSSTND doesn't seem to specify such a place. /var looks
: > like the best place to put such a directory, and /var/config would be
: > fairly intuitive. Creation of a new directory for config files would
: > also be easier than creating a central configuration database --- it
: > would only require that references to old directories are changed,
: > instead of having to rewrite large portions of programs.
: That would make things eaiser. But why use /var? /var is specified to be
: used for data, not configuration. The /var directory should consist of
: information that is constantly changing. Configuration changes only
: sporadically.
:
: And if you really want to put your configuration in /var, use a link.
: /etc is the standard, let's use it!
But /etc is for system configuration, i.e., lilo.conf, modules.conf,
inittab, fstab, etc.. Configuration is a type of data, so /var makes sense.
: This solves problems with system and user specific configurations. But
: if I want network wide configurations?
That's a good point. So maybe we should have the config directory on
/usr? This is probably a better idea, as configurations should be static
for the most part, and /usr can be mounted rw for when configurations
need to be changed. And looking at the FSSTND, I see that /usr/etc is
already specified for non-vital system-wide configuration files. It
seems that we wern't the only people to think of this.
-Kyle R. Hofmann