For people that still don't know the Elektra Project (http://elektra.sourceforge.net), it provides an alternative back-end for text configuration files. So instead, for example, the human-readable-only xorg.conf file, we'll have a hierarchy of key/value pairs easily and precisely accessible by X.org and also any other programs through an API or command, and human beings too. Same concept as GConf, but designed for system-wide use.
We've been working on X.org source code to do exactly what we just described above. And it is ready for use, tested even with very complex X.org configurations. A patched X server will convert your xorg.conf file automatically, even the most complex ones, including non-documented parameters.
It is available at http://sourceforge.net/project/showfiles.php?group_id=117521 http://sourceforge.net/project/showfiles.php?group_id=117521&package_id=...
You'll find there the patch against X.org source, the patch against Fedora Core 3 RPM spec file (to see how to build it), and the README file, which is pasted in this e-mail bellow.
We'll write patches for other software, and everybody is invited to join this integration effort.
Thank you, Avi Alkalay Elektra Project http://elektra.sf.net
----------------------README------------------------- ELEKTRIFIED X.ORG: PRECISE AND PROGRAMATICALY EASY X CONFIGURATION
Your X server has to work with your installed video card, monitor, find your font server and dirs, modules, extensions, drivers, plugins. You have to tell him how to integrate it all through its xorg.conf file. If you need to change something, you start a text editor and use your human brain and eyes to find the correct line, understand it, have the skills to change it, and change it in order to work.
This is good for advanced technical guys, but very bad for people that don't have this skills, and in fact, don't really want to. He just wants to change the screen resolution to make that projector work with his laptop, and go ahead with his sales presentation. This is just an example.
The point is: it is very difficult to make simple programs or scripts that make simple changes to X configuration. Another example is a monitor vendor that wants to support X, and for this he'd like to provide easy installation of his product, without having to ask his user to read documentation about horizontal Sync, and vertical refresh rates. For him again is difficult to write some simple software that preciselly changes X configuration to work correctly with his product.
The xorg.conf file (as most Unix configuration files) was designed for human beings.
The Elektra Project (http://elektra.sourceforge.net) introduces a new way to handle configurations through a clean API (or command line tool) that accesses atomic configuration data, into a standarized hierarchycal tree of key-value pairs. It is similar to GConf, but designed for system-wide use, which also implies it does not have dependencies.
And this is what this patch is about: to make the X server look for its configurations into a machine-ready tree of key-value pairs, instead of the human-ready xorg.conf.
So where you had to look for "Device radeon" into a "Section Device", with the key/value tree, X and other programs can look for it preciselly at
system/sw/xorg/current/Devices/Videocard0/Driver = radeon
Where you once had to "vi" your "Section Monitor", now X and other programs can do it accessing the keys:
system/sw/xorg/current/Monitors/Monitor0/HorizSync = 31.5 - 48.5 system/sw/xorg/current/Monitors/Monitor0/VertRefresh = 40.0 - 70.0 system/sw/xorg/current/Monitors/Monitor0/ModelName = IBM T40 LCD Panel system/sw/xorg/current/Monitors/Monitor0/VendorName = IBM system/sw/xorg/current/Monitors/Monitor0/Options/dpms
Where once the salesman above had to "vi" the Screen Section to change the resolution, color depth, etc, a program can help him accessing:
system/sw/xorg/current/Screens/Screen0/Displays/00/Depth=24 system/sw/xorg/current/Screens/Screen0/Displays/00/Modes=1024x768
And so on....
We believe an elektrified X server can leverage more plug-and-play configurations, providing configuration power to HW vendors in a very simple way, and making users experience less painfull.
BEHAVIOR OF AN ELEKTRIFIED X SERVER
A patched X server will look for its configuration keys under the namespace:
system/sw/xorg/current first, then if not found it tries system/sw/xorg
If not found, it will default to some xorg.conf file, parse it, and store in its internal structures, then convert and commit it to a set of keys under system/sw/xorg/current, and reload these keys.
So you get automatic one-time conversion from xorg.conf to the hierarchycal configuration key/value pairs
Very complex examples of xorg.conf files were tested for conversion. Even undocumented configuration parameters (because the original source was used as the reference).
The Elektrifyied X server also works for the Red Hat Graphical Boot, where you still don't have mounted partitions, network, etc.
ELEKTRIFING X.ORG
You'll need the elektra-devel package installed in order to build X with Elektra support.
1. Download and unpack X.org source code from 2. Download the xorg-x11-6.8.1-elektra.patch file from http://sourceforge.net/project/showfiles.php?group_id=117521&package_id=... 3. Apply it: ~/src/xc$ cd .. ~/src$ patch -p0 < xorg-x11-6.8.1-elektra.patch ~/src$ cd xc ~/src/xc$ # ...configure your build in host.def 4. Enable the patch: ~/src/xc$ echo "#define UseElektra" >> config/cf/host.def 5. Build X.Org
You'll find the new X server as file xc/programs/Xserver/Xorg .
The patch will add the following files:
xc/programs/Xserver/hw/xfree86/parser/ elektra.h (exported methods) elektra.c (key fetching and X structs integration business logic) xorg-example.conf (a very complex conf file to test conversion) xelektrify.c (cmd to convert xorg.conf->keys and vice-versa) README.Elektra (this file)
And it will instrument xc/programs/Xserver/hw/xfree86/common/xf86Config.c::xf86HandleConfigFile() to trigger the one-time conversion, and key fetching logic.
And instrument the Imakefiles for correct builds:
xc/programs/Xserver/hw/xfree86/parser/Imakefile xc/programs/Xserver/hw/xfree86/common/Imakefile xc/programs/Xserver/Imakefile
If "#define UseElektra" is not present in host.def, the patch is completelly disabled, and you'll get a binary-identicall built as before applying the patch. All patched code are surrounded by #ifdefs.
ELEKTRA MEETS X.ORG SOURCE CODE or how we wrote the patch....
X.org has an xorg.conf parser that takes this steps to handle configuration:
1. Lexically parse the xorg.conf file 2. Put each Section info in an equivalent struct 3. Encapsulate all structs together and pass them to a validator 4. Use structs to define X behavior
This process is triggered by the xf86HandleConfigFile() method from
xc/programs/Xserver/hw/xfree86/common/xf86Config.c
Each xorg.conf Section has an equivalent structure defined in
xc/programs/Xserver/hw/xfree86/parser/xf86Parser.h
and the lexycall analyzer code to parse each Section is under
xc/programs/Xserver/hw/xfree86/parser/
A fully parsed file has its equivalent structures encapsulated in a parent XF86Config struct. We have:
struct XF86ConfModuleRec for the "Section Modules" struct XF86ConfMonitorRec for the "Section Monitor" struct XF86ConfDeviceRec for the "Section Device" etc...
These structs are a pure computer representation of the text in each Section, so the methods under "parser/" convert text to structs, and the structs to text. This is how original X.org source handles xorg.conf.
The Elektrification add methods that act in steps 1 and 2 above. And also include methods to convert each struct to a KeySet. Both old (xorg.conf) and new (Elektra) ways to get configuration information can live together and they are actually used to automatically convert xorg.conf to keys. So at the first time you'll run your elektrified X server, it will:
1. Not find configuration keys (because it is the first time) 2. Parse xorg.conf into structs 3. Convert structs to Keys 4. Commit the keys to key database 5. Reload configurations from the key database
See the behavior in the previous section.
After assembling the XF86Config C structure, X will decode all its info into more practicall parameters for its further operation.
As a side note, with a key/value pair configuration hierarchy paradigm, the XF86Config assembling code (the parser) could be avoided, making X look for its configurations in a programatically easier, yet human-readable, configuration organization. We worked at the parser level to keep compatibility and to not go too deep in X.org source code.
http://elektra.sourceforge.net The Elektra Project Avi Alkalay November 2004
On Tue, Nov 30, 2004 at 10:25:09AM -0300, Avi Alkalay wrote:
The xorg.conf file (as most Unix configuration files) was designed for human beings.
It was designed for both humans and software and includes a bundled library for working with the file itself.
The Elektrifyied X server also works for the Red Hat Graphical Boot, where you still don't have mounted partitions, network, etc.
But presumably breaks system-config-display and all the related tools ?
On Tue, 30 Nov 2004 09:49:13 -0500, Alan Cox alan@redhat.com wrote:
On Tue, Nov 30, 2004 at 10:25:09AM -0300, Avi Alkalay wrote:
The xorg.conf file (as most Unix configuration files) was designed for human beings.
It was designed for both humans and software and includes a bundled library for working with the file itself.
Yes, and I used this library, and I can say it is completely dependent on X source. It is a very complex lexical analyzer, that works with many complex C structures.
A 7-button mouse vendor won't spend time to learn it to make a sort of driver installation program. In fact, I don't know anybody outside the X.org project that uses it.
The Elektrifyied X server also works for the Red Hat Graphical Boot, where you still don't have mounted partitions, network, etc.
But presumably breaks system-config-display and all the related tools ?
Of course. This is a first step. Does system-config-display uses the X.org library? I think this tool's developer simply learned the xorg.conf format and wrote code to generate it.
And system-config-display, as far as I know, is a monolitic tool that don't permit HW vendors, or X plugin writers, to independently extend it. Whit a key/value pair paradigm, these folks can write a 10-line Shell script to correctly integrate itself in X, without having to deal with a lexical parser, complex structs, and C.
Regards, Avi
The Elektrifyied X server also works for the Red Hat Graphical Boot, where you still don't have mounted partitions, network, etc.
But presumably breaks system-config-display and all the related tools ?
Of course. This is a first step. Does system-config-display uses the X.org library? I think this tool's developer simply learned the xorg.conf format and wrote code to generate it.
Actually s-c-d uses the Python binding to this X.org library (pyxf86config).
And system-config-display, as far as I know, is a monolitic tool that don't permit HW vendors, or X plugin writers, to independently extend it. Whit a key/value pair paradigm, these folks can write a 10-line Shell script to correctly integrate itself in X, without having to deal with a lexical parser, complex structs, and C.
There's a small tool, written in Python against pyxf86config, that the rpm.livna.org ATI and Nvidia graphics driver packages uses for autoconfiguring xorg.conf. It works fairly well, and pyxf86config is easy to use.
However, I do agree that s-c-d should be extendable by 3:rd party components.
/Peter Backlund
On Tue, Nov 30, 2004 at 12:18:21PM -0300, Avi Alkalay wrote:
don't permit HW vendors, or X plugin writers, to independently extend it. Whit a key/value pair paradigm, these folks can write a 10-line Shell script to correctly integrate itself in X, without having to deal with a lexical parser, complex structs, and C.
In all the systems I've seen so far trying to just burn disk and performance with simple key/value pairs this isn't true because
1. You need to do locking 2. You need to do atomic changes of multiple keys 3. You might run out of disk space 4. You need rollback
The current xorg.conf certainly doesn't solve these either but anything trying to be a grad unified config scheme just won't work with one file per key. Gconf proved that pretty conclusively.
On Tue, 30 Nov 2004 11:23:38 -0500, Alan Cox alan@redhat.com wrote:
In all the systems I've seen so far trying to just burn disk and performance with simple key/value pairs this isn't true because
1. You need to do locking
It does.
2. You need to do atomic changes of multiple keys
Of course. And in current xorg.conf text file you have to rewrite it all.
3. You might run out of disk space 4. You need rollback
So the app handling the keys must be mature enough to roll it back.
The current xorg.conf certainly doesn't solve these either but anything trying to be a grad unified config scheme just won't work with one file per key. Gconf proved that pretty conclusively.
Again this detail? How many time I'll have to say that a key/value hierarchy paradigm value is a unified namespace and API? The way it deals with storage is an implementation detail that can be reimplemented without breaking the applications.
About storage, 1 file per key or 1 file per folder uses almost the same amount of disk space. Do a 'du -sk' in gconf storage folders to see that.
On Tue, Nov 30, 2004 at 01:34:47PM -0300, Avi Alkalay wrote:
3. You might run out of disk space 4. You need rollbackSo the app handling the keys must be mature enough to roll it back.
In ten lines of shell script. Rollback belongs in the configuration libraries so you can do commit(list of key changes). That handles out of disk for free also.
About storage, 1 file per key or 1 file per folder uses almost the same amount of disk space. Do a 'du -sk' in gconf storage folders to see that.
In a db4 hash overhead seems to be percentage points. In flat file its about 3.5K per key for single key per file. It is implementation detail compared to the fundamental goal.
Overall I'm glad to see this work being done. I'm not convinced the format is right but the essential goals of unifying configuration spaces, improving tools and also just going out and doing it are all good things.
On Tue, 2004-11-30 at 12:17 -0500, Alan Cox wrote:
On Tue, Nov 30, 2004 at 01:34:47PM -0300, Avi Alkalay wrote:
3. You might run out of disk space 4. You need rollbackSo the app handling the keys must be mature enough to roll it back.
In ten lines of shell script. Rollback belongs in the configuration libraries so you can do commit(list of key changes). That handles out of disk for free also.
Indeed. Any configuration system like Elektra basically needs some real database functionality. This isn't a problem - it's 100% possible. Elektra just needs to export the API, and leave the implementation details of it to the backend. A Berkeley DB backend for example could easily implement transactions and such. (Or do you need one of the commercial BDB variants for those?)
Looking at the API, it appears that Elektra is already pretty much "there". Just grab a KeySet, set the keys, and save the whole set. You just need to guarantee that saving a set is atomic. i.e., make sure everyone backend does this, and clearly document this.
So far as implementation, this really is easy even with a (single) file- based backend. Robustness might take some work, but the basics are easy. You simply keep a list of all they keys that have been set, and when the Commit command comes along, lock the file(s) in question and modify them. On Rollback, just "forget" all the key changes that haven't been committed yet. Basically the hard part for the simple file backend is getting the commit to stay atomic on errors when you use multiple files. If you have multiple files being changed, I don't believe it's actually possible to be 100% atomic - you can reduce the chances of something breaking, but there will always be various race conditions in regards to when errors occur that could leave you in a half-committed state.
Getting intricate behaviour with shell scripts is also quite possible. You would probably need some kind of service daemon. So you'd write something like:
#!/bin/sh
# this would print out code to set an environment variable, so the code # gets executed by eval, setting the variable eval(kdb begin)
# set some keys - the command would check for the environment variable # defined by the shell service daemon, and communicate with the daemon kdb set "key1" "value" kdb set "key2" "value"
# do some other stuff, including checking values of keys, which would # also communicate with the daemon if the env-var is set if [ `kdb get 'key2'` = 'value' ] ; then do-some-stuff set-some-other-keys fi
# commit/rollback/whatever - again, checks env var, daemon, etc kdb commit
#eof
The daemon would take some work to get working robustly (i.e., noticing when its parent exists without shutting it down, and thus shutting itself down cleanly after doing a rollback op, etc.) but its certainly possible.
About storage, 1 file per key or 1 file per folder uses almost the same amount of disk space. Do a 'du -sk' in gconf storage folders to see that.
In a db4 hash overhead seems to be percentage points. In flat file its about 3.5K per key for single key per file. It is implementation detail compared to the fundamental goal.
The author has this unfortunate tendency to use ReiserFS and other non- mainstream file-systems and claim everyone else should too (Linux Registry, the previous name for Elektra, was shot down repeatedly due to his disregard for real-life systems) so he probably doesn't see any of the over-head for small files.
A db4-baed system certainly has technical advantages in terms of efficiency, but stop and think - you're basically now running the WIndows Registry. The Registry's *largest* problem is that it's a binary format that easily gets corrupted - a lot like Berkeley DB (hell, I wonder if that's what the Windows Registry's code is based off of ~_^ )
The best format is probably something like XML, or something very XML- ish, because it can easily store a hierarchy in a single file, is "safe" in terms of value escaping (many custom formats are not), and there are tons of libraries to read/write it efficiently. It isn't *easily* edited by hand, but if it's "pretty-printed" it can be easy enough.
Overall I'm glad to see this work being done. I'm not convinced the format is right but the essential goals of unifying configuration spaces, improving tools and also just going out and doing it are all good things.
Definitely. Elektra still has major short comings (no transactional guarantees, no notification, no real access control, no temporary keys... mostly things Elektra can never do since it refuses to use a daemon), but it's at least a tiny step in teh right direction that might open some peoples' eyes.
Sean Middleditch wrote:
[...]
So far as implementation, this really is easy even with a (single) file- based backend. Robustness might take some work, but the basics are easy. You simply keep a list of all they keys that have been set, and when the Commit command comes along, lock the file(s) in question and modify them. On Rollback, just "forget" all the key changes that haven't been committed yet. Basically the hard part for the simple file backend is getting the commit to stay atomic on errors when you use multiple files. If you have multiple files being changed, I don't believe it's actually possible to be 100% atomic - you can reduce the chances of something breaking, but there will always be various race conditions in regards to when errors occur that could leave you in a half-committed state.
rename is atomic. You can always use this fact if you need to.
On Tue, 2004-11-30 at 14:24 -0500, Neal D. Becker wrote:
Sean Middleditch wrote:
[...]
So far as implementation, this really is easy even with a (single) file- based backend. Robustness might take some work, but the basics are easy. You simply keep a list of all they keys that have been set, and when the Commit command comes along, lock the file(s) in question and modify them. On Rollback, just "forget" all the key changes that haven't been committed yet. Basically the hard part for the simple file backend is getting the commit to stay atomic on errors when you use multiple files. If you have multiple files being changed, I don't believe it's actually possible to be 100% atomic - you can reduce the chances of something breaking, but there will always be various race conditions in regards to when errors occur that could leave you in a half-committed state.
rename is atomic. You can always use this fact if you need to.
Yes. That doesn't help with the problem of making atomic commits to *multiple* files. You would have to perform multiple renames, of which some may succeed and others may fail for any of a number of reasons, and then you can't necessarily undo the ones you already did. Unless there is a way to do an atomic rename of multiple files, my point definitely still stands - multiple files cannot be modified atomically.
Sean Middleditch wrote:
On Tue, 2004-11-30 at 14:24 -0500, Neal D. Becker wrote:
Sean Middleditch wrote:
[...]
So far as implementation, this really is easy even with a (single) file- based backend. Robustness might take some work, but the basics are easy. You simply keep a list of all they keys that have been set, and when the Commit command comes along, lock the file(s) in question and modify them. On Rollback, just "forget" all the key changes that haven't been committed yet. Basically the hard part for the simple file backend is getting the commit to stay atomic on errors when you use multiple files. If you have multiple files being changed, I don't believe it's actually possible to be 100% atomic - you can reduce the chances of something breaking, but there will always be various race conditions in regards to when errors occur that could leave you in a half-committed state.
rename is atomic. You can always use this fact if you need to.
Yes. That doesn't help with the problem of making atomic commits to *multiple* files. You would have to perform multiple renames, of which some may succeed and others may fail for any of a number of reasons, and then you can't necessarily undo the ones you already did. Unless there is a way to do an atomic rename of multiple files, my point definitely still stands - multiple files cannot be modified atomically.
Are you sure? What if you: 1) duplicate the directory (using hardlinks to files) 2) atomic rename directory
On Tue, 2004-11-30 at 14:55 -0500, Neal D. Becker wrote:
Sean Middleditch wrote: Are you sure? What if you:
- duplicate the directory (using hardlinks to files)
- atomic rename directory
I don't believe that actually works. You can't rename one directory onto another already existing directory. You'd have to move the original out of the way, creating a window for disaster to strike.
Now, you could lock the entire hierarchy, start the move, and if it fails in the middle (power outage, whatever) have the next process that attempts to access the DB to "fix it."
Something like: lock db copy(hardlinks) db to db-work modify db-work rename db-work db-ready # begin danger rename db db-old rename db-ready db delete db-old unlock db
Now, if at any point between the begin/end danger lines, if the system power shuts off or the process doing the modification crashes, a later process can "fix" the system. Basically, if it sees a db-ready directory, finish up replacing db with it.
Assuming that all works as intended and doesn't have some other race I'm not seeing, then yes, I was wrong - you *can* atomically modify multiple files. Sort of. Assuming that everything that accesses them does so using the entire process above. Modifying even a single file would require locking the whole DB. Reading a file would likewise require it. That could potentially result in a lot of over-head. The locking could be a huge problem for some systems running over NFS.
All in all, I'm fairly sure it's not nearly robust enough - not compared to just a single rename of a single file. If you're gonna go through all that trouble, deny users the ability to just edit any of the files directly, and so on, why not just use an existing, tested, stable database for the backend? BDB, SQLite, whatever - they do the same thing the multiple text files do, plus they're a lot more efficient about it.
Once upon a time, Neal D. Becker ndbecker2@verizon.net said:
Are you sure? What if you:
- duplicate the directory (using hardlinks to files)
- atomic rename directory
You can't do step #2. You can only rename over an existing directory if the directory is already empty.
The only way you can do something like what you say is if you create the directories under a different name and use symlinks to point to them (you can replace a symlink atomically by creating a new symlink under a new name and then renaming the symlink over the old one).
If you go that way, you have to have a clean-up program. Otherwise, unclean shutdowns (either of the management program(s) or of the system) will leave stuff all over the place. You could use a log to make the clean-up faster, and rotate the log out when it is processed (i.e. along the lines of db_checkpoint and db_archive from BDB).
On Tue, 2004-11-30 at 14:16 -0600, Chris Adams wrote:
Once upon a time, Neal D. Becker ndbecker2@verizon.net said:
Are you sure? What if you:
- duplicate the directory (using hardlinks to files)
- atomic rename directory
You can't do step #2. You can only rename over an existing directory if the directory is already empty.
The only way you can do something like what you say is if you create the directories under a different name and use symlinks to point to them (you can replace a symlink atomically by creating a new symlink under a new name and then renaming the symlink over the old one).
Would this actually be atomic? What if two apps tried doing this at once? One makes db-1 as the temp work directory, another makes db-2 as the temp dir, and then whoever makes the symlink last gets the symlink pointing to their data, resulting in the previous process' changes being silently lost.
You'd have to do locking, which I noted the problems with (NFS in particular) in my other mail.
Or, if Elektra were to use a daemon that centralizes access, you wouldn't need the locking (only one process doing this at a time), but Elektra won't do that for several sort-of-almost good reasons of early bootup. (I suppose if the DB is read-only you can do without the daemon, and early bootup probably would be read-only, so you can have Elektra start the daemon later on, but I'm not sure how well that would work either - yet more race conditions between apps reading something when the daemon starts and potentially starts changing stuff.)
If you go that way, you have to have a clean-up program. Otherwise, unclean shutdowns (either of the management program(s) or of the system) will leave stuff all over the place. You could use a log to make the clean-up faster, and rotate the log out when it is processed (i.e. along the lines of db_checkpoint and db_archive from BDB).
-- Chris Adams cmadams@hiwaay.net Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Once upon a time, Sean Middleditch elanthis@awesomeplay.com said:
You'd have to do locking, which I noted the problems with (NFS in particular) in my other mail.
I was ignoring that. I was presuming write locking was already being done somewhere; I just described what would have to be done to have atomic updates such that the data stays internally consistent across a crash.
Reading would also require locks, unless you can only fetch a single value at a time.
Basically, trying to use the filesystem to implement a transactional database is not a good idea. If you want a transactional database, use a transactional database library (i.e. BDB) and supply a few command-line dump and restore type tools for emergency editing (as long as the database doesn't get internally corrupted).
On Tue, 30 Nov 2004 13:57:01 -0500, Sean Middleditch elanthis@awesomeplay.com wrote:
Getting intricate behaviour with shell scripts is also quite possible. You would probably need some kind of service daemon. So you'd write something like:
#!/bin/sh
# this would print out code to set an environment variable, so the code # gets executed by eval, setting the variable eval(kdb begin)
# set some keys - the command would check for the environment variable # defined by the shell service daemon, and communicate with the daemon kdb set "key1" "value" kdb set "key2" "value"
# do some other stuff, including checking values of keys, which would # also communicate with the daemon if the env-var is set if [ `kdb get 'key2'` = 'value' ] ; then do-some-stuff set-some-other-keys fi
# commit/rollback/whatever - again, checks env var, daemon, etc kdb commit
#eof
Instead of doing a lot of 'kdb set', you can generate an XML file with the keys you want changed, and do 'kdb import keys.xml'. It will be more atomic, and will use the kdbSetKeys() call which wraps transaction support.
The daemon would take some work to get working robustly (i.e., noticing when its parent exists without shutting it down, and thus shutting itself down cleanly after doing a rollback op, etc.) but its certainly possible.
About storage, 1 file per key or 1 file per folder uses almost the same amount of disk space. Do a 'du -sk' in gconf storage folders to see that.
In a db4 hash overhead seems to be percentage points. In flat file its about 3.5K per key for single key per file. It is implementation detail compared to the fundamental goal.
The author has this unfortunate tendency to use ReiserFS and other non- mainstream file-systems and claim everyone else should too (Linux Registry, the previous name for Elektra, was shot down repeatedly due to his disregard for real-life systems) so he probably doesn't see any of the over-head for small files.
I don't have a tendency to use ReiserFS, but it showed to be more performatic and less space consuming then any other FS. And I do see the overhead for small files. The point is: only discussing ideas is very different that writing code. In this last case you'll find issues that destroy entire architectures you built in your head. A single DB file and a deamon was obviously the first reasonable thing to think and to implement, so read bellow.
A db4-baed system certainly has technical advantages in terms of efficiency, but stop and think - you're basically now running the WIndows Registry. The Registry's *largest* problem is that it's a binary format that easily gets corrupted - a lot like Berkeley DB (hell, I wonder if that's what the Windows Registry's code is based off of ~_^ )
Yes, this is one problem with BDB. BDB was the first thing used, and droped in version 0.0.9. See the changelog: http://www.germane-software.com/repositories/elektra/trunk/ChangeLog
The major problems with BDB or a daemon (that was also the paradigm for the old old old versions) are: - It is very difficult to implement hierarchy in a consistent way inside a BDB - Elektra is system-wide by definition. So with a global daemon, security has to be handled in the user-space daemon. And security is a little thing that you may require this: is the best (kernel space) and you have it, or you simply don't have it. - Besides, you can't authenticate against a system daemon. Authentication means to have a user to *type* a password or *read* it from a file. Think about how authentication works in HTTP sessions, LDAP, DB servers, etc. I'm open to hear ideas on this point. - A daemon is a single point of failure.
These were the reasons why daemon and a single file (binary or text) is accepted only in non-critical, desktopish, single user environment (read GConf, that really makes its job!). Not systemwide.
The best format is probably something like XML, or something very XML- ish, because it can easily store a hierarchy in a single file, is "safe" in terms of value escaping (many custom formats are not), and there are tons of libraries to read/write it efficiently. It isn't *easily* edited by hand, but if it's "pretty-printed" it can be easy enough.
Read/write in memory, not on the disk. On the disk you have to rewrite the entire file, so bye bye transactions.
Overall I'm glad to see this work being done. I'm not convinced the format is right but the essential goals of unifying configuration spaces, improving tools and also just going out and doing it are all good things.
Definitely. Elektra still has major short comings (no transactional guarantees, no notification, no real access control, no temporary keys... mostly things Elektra can never do since it refuses to use a daemon), but it's at least a tiny step in teh right direction that might open some peoples' eyes.
Notifications is there already. Check the code. Transactions needs some stabilization, yes. What do you mean with "no real access control"? This is completelly delegated to the best guy that can do it: the Kernel. And what do you mean about "temporary keys" ?
Sean Middleditch elanthis@awesomeplay.com AwesomePlay Productions, Inc.
Thank you, Avi
On Wed, 2004-12-01 at 14:33 -0300, Avi Alkalay wrote:
Definitely. Elektra still has major short comings (no transactional guarantees, no notification, no real access control, no temporary keys... mostly things Elektra can never do since it refuses to use a daemon), but it's at least a tiny step in teh right direction that might open some peoples' eyes.
Notifications is there already. Check the code.
Avi, correct me if I'm misinterpretting, but the API docs seem to say that Elektra has "monitoring" rather than true notification. Notification as it exists in GConf is event driven (which is easy for a programmer to use) and reasonably performant. The monitoring of keys that Elektra's API implements places a much higher burden on the programmer and seems like it would consume more system resources (scanning for changes instead of being notified when a change occurs.)
If the goal of Elektra truly is every program should use it to config, then this is going to need some rethinking because few programmers who need notification (at minimum, desktop apps) will choose the extra work involved in Elektra over GConf. If Elektra is only aiming at the current system services which don't change state dynamically (The app "resyncs" with its config via restart or reload of all config info rather than changing on the fly as each key is modified.) then it might not need true notification... although I think there are system services could benefit from notification; it's just that the capablitity to do it isn't attractive with the current /etc files scheme.
-Toshio
On Wed, 01 Dec 2004 21:43:24 -0500, Toshio toshio@tiki-lounge.com wrote:
On Wed, 2004-12-01 at 14:33 -0300, Avi Alkalay wrote:
Definitely. Elektra still has major short comings (no transactional guarantees, no notification, no real access control, no temporary keys... mostly things Elektra can never do since it refuses to use a daemon), but it's at least a tiny step in teh right direction that might open some peoples' eyes.
Notifications is there already. Check the code.
Avi, correct me if I'm misinterpretting, but the API docs seem to say that Elektra has "monitoring" rather than true notification. Notification as it exists in GConf is event driven (which is easy for a programmer to use) and reasonably performant. The monitoring of keys that Elektra's API implements places a much higher burden on the programmer and seems like it would consume more system resources (scanning for changes instead of being notified when a change occurs.)
If the goal of Elektra truly is every program should use it to config, then this is going to need some rethinking because few programmers who need notification (at minimum, desktop apps) will choose the extra work involved in Elektra over GConf. If Elektra is only aiming at the current system services which don't change state dynamically (The app "resyncs" with its config via restart or reload of all config info rather than changing on the fly as each key is modified.) then it might not need true notification... although I think there are system services could benefit from notification; it's just that the capablitity to do it isn't attractive with the current /etc files scheme.
-Toshio
Toshio toshio@tiki-lounge.com
The way GConf works is compatible with the layer GConf sits on: It is high level, (I think) it is integrated with an event driven framework (Gnome), etc etc.
On the other hand, Elektra is very low level, in the same level as libc functions. If it was notification instead of monitoring, how it will notify a program that doesn't have an event loop? One way I can think is to install a callback and Elektra will create a new thread or process to wait for changes (remember we aren't in an event loop context) and call your callback function, but it will work inside a different thread and it can cause problems to your logic. So the solution I was able to think is to give you the basics (monitoring), and based on your program logic, you'll or wait for a change, or wrap it in a thread context that your program can handle. The way monitoring is implemented in Elektra is the best I could think for its descentralized nature.
I'm open for other ideas in this space.
Thank you, Avi
I'm having a look at inotify, which is the basis for gamin.
Thank you, Avi
On Sun, 5 Dec 2004 22:57:42 +0800, Jeff Pitman symbiont@berlios.de wrote:
On Sunday 05 December 2004 06:41, Avi Alkalay wrote:
If it was notification instead of monitoring, how it will notify a program that doesn't have an event loop?
What about gamin? And then reload the appropriate config upon alteration.
-- -jeff
Avi Alkalay wrote:
I'm having a look at inotify, which is the basis for gamin.
Don't tie your code to inotify, tie it to something like gamin. Gamin can abstract away the different OS interfaces, so that machines without inotify will still have some form of notification (dnotify, polling, whatever). There's no good reason to reimplement that code yourself.
On Sun, 05 Dec 2004 12:57:30 -0500, Sean Middleditch elanthis@awesomeplay.com wrote:
Avi Alkalay wrote:
I'm having a look at inotify, which is the basis for gamin.
Don't tie your code to inotify, tie it to something like gamin. Gamin can abstract away the different OS interfaces, so that machines without inotify will still have some form of notification (dnotify, polling, whatever). There's no good reason to reimplement that code yourself.
Yeah I know.... I'll try to stick in the high level, but I'm worried about dependencies. It makes no sense to have /lib/libkdb.so depending on libs under /usr/lib.
The 5 minutes I spent reading about inotify made me believe it is a kernel module, but I couldn't find it, nor the /dev entry, nor the dmesg messages in the FC2 machine I have close to me right now. Is this stuff already included in regular distros kernels ?
I'm still studing all of this....
Regards, Avi
Avi Alkalay wrote:
On Sun, 05 Dec 2004 12:57:30 -0500, Sean Middleditch elanthis@awesomeplay.com wrote:
Avi Alkalay wrote:
I'm having a look at inotify, which is the basis for gamin.
Don't tie your code to inotify, tie it to something like gamin. Gamin can abstract away the different OS interfaces, so that machines without inotify will still have some form of notification (dnotify, polling, whatever). There's no good reason to reimplement that code yourself.
Yeah I know.... I'll try to stick in the high level, but I'm worried about dependencies. It makes no sense to have /lib/libkdb.so depending on libs under /usr/lib.
That does make sense, good point.
The 5 minutes I spent reading about inotify made me believe it is a kernel module, but I couldn't find it, nor the /dev entry, nor the dmesg messages in the FC2 machine I have close to me right now. Is this stuff already included in regular distros kernels ?
It's a patch, not a module. it will, eventually, be part of the mainline kernel, but at the moment, you have to patch it in yourself. Or you can grab pre-patched kernels - for example, if you're running Netware Linux Desktop, RML has some patched kernel RPMS here: http://primates.ximian.com/~rml/kernel-rml/nld-9-i586/?C=N;O=A
I'm still studing all of this....
Regards, Avi
On Sun, Dec 05, 2004 at 04:17:36PM -0300, Avi Alkalay wrote:
On Sun, 05 Dec 2004 12:57:30 -0500, Sean Middleditch elanthis@awesomeplay.com wrote:
Avi Alkalay wrote:
I'm having a look at inotify, which is the basis for gamin.
Don't tie your code to inotify, tie it to something like gamin. Gamin can abstract away the different OS interfaces, so that machines without inotify will still have some form of notification (dnotify, polling, whatever). There's no good reason to reimplement that code yourself.
Yeah I know.... I'll try to stick in the high level, but I'm worried about dependencies. It makes no sense to have /lib/libkdb.so depending on libs under /usr/lib.
The 5 minutes I spent reading about inotify made me believe it is a kernel module, but I couldn't find it, nor the /dev entry, nor the dmesg messages in the FC2 machine I have close to me right now. Is this stuff already included in regular distros kernels ?
Use gamin. If inotify is present it will use it, if not it will fallback to whatever is available. And if you're on Solaris, you can still use FAM to provide the equivalent API as gamin. And gamin only dependancy on the client side is libc, you really have no excuse to not use it :-)
Daniel
On Tue, 7 Dec 2004 19:49:52 -0500, Daniel Veillard veillard@redhat.com wrote:
Use gamin. If inotify is present it will use it, if not it will fallback to whatever is available. And if you're on Solaris, you can still use FAM to provide the equivalent API as gamin. And gamin only dependancy on the client side is libc, you really have no excuse to not use it :-)
OK about the client. But to really have it, I imagine you need the server too. So then what early bootup programs (when you still don't have /usr/lib mounted or any services running) will have ?
Thank you, Avi
On Tue, 7 Dec 2004 23:14:24 -0300, Avi Alkalay avibrazil@gmail.com wrote:
On Tue, 7 Dec 2004 19:49:52 -0500, Daniel Veillard veillard@redhat.com wrote:
Use gamin. If inotify is present it will use it, if not it will fallback to whatever is available. And if you're on Solaris, you can still use FAM to provide the equivalent API as gamin. And gamin only dependancy on the client side is libc, you really have no excuse to not use it :-)
OK about the client. But to really have it, I imagine you need the server too. So then what early bootup programs (when you still don't have /usr/lib mounted or any services running) will have ?
Well, maybe not depend on gamin, FAM etc, through dlopen, and handle linking errors, and not providing notification in early boot stage.....
On Tue, Dec 07, 2004 at 11:17:28PM -0300, Avi Alkalay wrote:
On Tue, 7 Dec 2004 23:14:24 -0300, Avi Alkalay avibrazil@gmail.com wrote:
OK about the client. But to really have it, I imagine you need the server too. So then what early bootup programs (when you still don't have /usr/lib mounted or any services running) will have ?
Well, maybe not depend on gamin, FAM etc, through dlopen, and handle linking errors, and not providing notification in early boot stage.....
Maybe gamin should be moved to /lib?
On Tue, 2004-11-30 at 11:23 -0500, Alan Cox wrote:
On Tue, Nov 30, 2004 at 12:18:21PM -0300, Avi Alkalay wrote:
don't permit HW vendors, or X plugin writers, to independently extend it. Whit a key/value pair paradigm, these folks can write a 10-line Shell script to correctly integrate itself in X, without having to deal with a lexical parser, complex structs, and C.
In all the systems I've seen so far trying to just burn disk and performance with simple key/value pairs this isn't true because
- You need to do locking
- You need to do atomic changes of multiple keys
- You might run out of disk space
- You need rollback
The current xorg.conf certainly doesn't solve these either but anything trying to be a grad unified config scheme just won't work with one file per key. Gconf proved that pretty conclusively.
Yes. http://www.gnome.org/projects/gconf/plans.html
The issue isn't really one file per key (though I think that's wrong - even gconf is one file per directory, not per key). It's that you need to address transactions, locking, error handling, and notifications.
Havoc
Hey,
I'm all for improving the situation with around auto configuration of hardware, but with all due respect, I think you guys are trying to solve the symptom, not the real problem. In my view you really want the X server to be able to export an API for software higher up the stack (GNOME, KDE, etc.) to configure the X server. You also want to reconfigure it while it's running. It seems to me, that putting in an mediator, for basically writing out configuration files, is not the best API for doing this. I could be wrong though. Ideally the X server wouldn't even touch hardware before someone used that API to say "Add monitor, Add input device, blah blah".
Anyway, with the right API in the X server (which would probably be exported through D-BUS), I should be able to write a daemon, let's call it gnome-input-manager, that runs in the desktop session as user davidz. This would also allow said daemon to disable the touchpad when I connect an external mouse or, for more fun, to disable it around intervals where I'm punching the keys. The reason you want this in the desktop session is that you want to query the locally logged in users preferences from e.g. gconf or whatever.
Just what I personally think.
Have fun, David
On Tue, 2004-11-30 at 12:43 -0500, David Zeuthen wrote:
I'm all for improving the situation with around auto configuration of hardware, but with all due respect, I think you guys are trying to solve the symptom, not the real problem. In my view you really want the X server to be able to export an API for software higher up the stack (GNOME, KDE, etc.) to configure the X server. You also want to reconfigure it while it's running. It seems to me, that putting in an mediator, for basically writing out configuration files, is not the best API for doing this. I could be wrong though. Ideally the X server wouldn't even touch hardware before someone used that API to say "Add monitor, Add input device, blah blah".
Elektra does not prevent any of what you describe.
The current implementation does appear to assume (I've not tried it) that the current X config should map to neatly to key/value pairs in a similarly shaped namespace. The namespace of those pairs seems to be a sticking point. Seems that once you separate input config from monitor config at the file level, keeping them together as "X" config doesn't make so much sense anymore.
Elektra also doesn't solve world hunger, but it definitely makes some exsiting issues stink more.
-- Darrin
On Tue, 2004-11-30 at 13:14 -0500, Darrin Thompson wrote:
On Tue, 2004-11-30 at 12:43 -0500, David Zeuthen wrote:
I'm all for improving the situation with around auto configuration of hardware, but with all due respect, I think you guys are trying to solve the symptom, not the real problem. In my view you really want the X server to be able to export an API for software higher up the stack (GNOME, KDE, etc.) to configure the X server. You also want to reconfigure it while it's running. It seems to me, that putting in an mediator, for basically writing out configuration files, is not the best API for doing this. I could be wrong though. Ideally the X server wouldn't even touch hardware before someone used that API to say "Add monitor, Add input device, blah blah".
Elektra does not prevent any of what you describe.
Not sure, I see that the Elektra API is key/value based and the only two types supported appears to be UTF-8 strings and Blobs. Perhaps it would be useful with other fundamental data types on grounds of type safety.
The current implementation does appear to assume (I've not tried it) that the current X config should map to neatly to key/value pairs in a similarly shaped namespace. The namespace of those pairs seems to be a sticking point. Seems that once you separate input config from monitor config at the file level, keeping them together as "X" config doesn't make so much sense anymore.
The API I'd like to see the X server export would definitely include methods/facilities that you cannot express in terms of 'set one more more keys atomically'.
For instance, shouldn't the X server expose a method Degauss() if the hardware is capable of degaussing a monitor?
(locking down who and what is permitted to invoke Degauss() is another matter entirely - I suggest that people take a lot at D-BUS and some of the policy you can apply there; e.g. only allow console user to do it. In fact, it might be too dangerous to expose such a method as repeated use might damage the hardware, but you get the point)
Cheers, David
On Tue, Nov 30, 2004 at 12:43:16PM -0500, David Zeuthen wrote:
exported through D-BUS), I should be able to write a daemon, let's call it gnome-input-manager, that runs in the desktop session as user davidz. This would also allow said daemon to disable the touchpad when I connect an external mouse or, for more fun, to disable it around intervals where I'm punching the keys. The reason you want this in the desktop session
You pretty much have to. I've got a hotplug video card here for example. That really ruins the current configuration system. OTOH you have to start somewhere and if the goal is to unify configuration then you want to let someone else rewrite X. However as you and Havoc both point out it has to handle notifiers
On Tue, 30 Nov 2004 13:56:54 -0500, Alan Cox alan@redhat.com wrote:
On Tue, Nov 30, 2004 at 12:43:16PM -0500, David Zeuthen wrote:
exported through D-BUS), I should be able to write a daemon, let's call it gnome-input-manager, that runs in the desktop session as user davidz. This would also allow said daemon to disable the touchpad when I connect an external mouse or, for more fun, to disable it around intervals where I'm punching the keys. The reason you want this in the desktop session
You pretty much have to. I've got a hotplug video card here for example. That really ruins the current configuration system. OTOH you have to start somewhere and if the goal is to unify configuration then you want to let someone else rewrite X. However as you and Havoc both point out it has to handle notifiers
Good. Elektra has call to get notification. Check the kdbMonitorKey() and kdbMonitorKeys() methods in the Doxygen: http://elektra.sourceforge.net/elektra-api/html/
Is the cross posting to four lists *really* necessary for this discussion?
On Wed, 2004-12-01 at 13:45 -0300, Avi Alkalay wrote:
On Tue, 30 Nov 2004 13:56:54 -0500, Alan Cox alan@redhat.com wrote:
On Tue, Nov 30, 2004 at 12:43:16PM -0500, David Zeuthen wrote:
exported through D-BUS), I should be able to write a daemon, let's call it gnome-input-manager, that runs in the desktop session as user davidz. This would also allow said daemon to disable the touchpad when I connect an external mouse or, for more fun, to disable it around intervals where I'm punching the keys. The reason you want this in the desktop session
You pretty much have to. I've got a hotplug video card here for example. That really ruins the current configuration system. OTOH you have to start somewhere and if the goal is to unify configuration then you want to let someone else rewrite X. However as you and Havoc both point out it has to handle notifiers
Good. Elektra has call to get notification. Check the kdbMonitorKey() and kdbMonitorKeys() methods in the Doxygen: http://elektra.sourceforge.net/elektra-api/html/
From the docs: "This method will block your program until one of the
folowing happens:"
I'm imaging it works by just re-reading the DB on each iteration. kdbMonitorKeys has to be a performance monster... Especially if you want to get notified about a key change in a reasonable period of time. Let's not even get into the disk seek implications we have here.
This API needs to be efficient and reasonably easy to integrate with existing applications main loops.
Seriously - suck it up and write a kdbd daemon for this kind of stuff. There's no good reason not to have one. For times when the daemon isn't around (and there's no reason why this should ever be the case, even during early early bootup, but I have the feeling you'll make that argument anyhow) you can have the client library access the kdb in read- only mode and disable features like key notification.
Take a look at how the GConf client interface is structured. You get a socket to the configuration server. If you have a custom main loop, you can poll/select/epoll/whatever the socket to wait for updates (while in the meantime blocking on other important file descriptors or running various timeouts). When the descriptor has data available, you tell the library to read as much as it can - if the library gets a complete notification message, it invokes your callback(s). This is the absolute best you can get. It makes the API *easy* to integrate with Qt, glib, or any other custom mainloop.
xdg mailing list xdg@freedesktop.org http://freedesktop.org/mailman/listinfo/xdg
Sean
From the docs: "This method will block your program until one of the
folowing happens:"
I'm imaging it works by just re-reading the DB on each iteration. kdbMonitorKeys has to be a performance monster...
From looking at the sources, I think it does. But that's not the point. And it's a bit harsh to judge the API on it's first implementation.
I think it's useful work to try and port apps to a common config API, so long as the API is capable. Elektra seems to be written with the assumption that the back end will be reimplemented, so maybe one day it will. Until then, poor performance of some less-used parts of the API shouldn't be a problem.
-Cam
On Wed, 2004-12-01 at 18:00 +0000, Cam wrote:
Sean
From the docs: "This method will block your program until one of the
folowing happens:"
I'm imaging it works by just re-reading the DB on each iteration. kdbMonitorKeys has to be a performance monster...
From looking at the sources, I think it does. But that's not the point. And it's a bit harsh to judge the API on it's first implementation.
I think it's useful work to try and port apps to a common config API, so long as the API is capable. Elektra seems to be written with the assumption that the back end will be reimplemented, so maybe one day it will. Until then, poor performance of some less-used parts of the API shouldn't be a problem.
Having seen the first few incarnations of Elektra (then "Linux Registry") and the initial community discussions with the author, I'll have to disagree. Applications' needs came second, after the initial design. Elektra is several iterations into the design. The goal needs to stop being "get all applications to share a config database" and become "how can we make configuring applications easier and more consistent." A shared key-based configuration database is simply a means to that end, and an implementation choice, not the end goal itself. If it is made the end goal then you end up with, well, Elektra.
The API needs to be designed first and then the backend needs to be made to fit. Currently, the API is built entirely around the idea of the flat text key files. The KeyMonitor API is a perfect example of that fact.
-Cam
camilo@mesias.co.uk <--
tir, 30.11.2004 kl. 18.43 skrev David Zeuthen:
Hey,
I'm all for improving the situation with around auto configuration of hardware, but with all due respect, I think you guys are trying to solve the symptom, not the real problem. In my view you really want the X server to be able to export an API for software higher up the stack (GNOME, KDE, etc.) to configure the X server. You also want to reconfigure it while it's running. It seems to me, that putting in an mediator, for basically writing out configuration files, is not the best API for doing this. I could be wrong though. Ideally the X server wouldn't even touch hardware before someone used that API to say "Add monitor, Add input device, blah blah".
Anyway, with the right API in the X server (which would probably be exported through D-BUS), I should be able to write a daemon, let's call it gnome-input-manager, that runs in the desktop session as user davidz. This would also allow said daemon to disable the touchpad when I connect an external mouse or, for more fun, to disable it around intervals where I'm punching the keys. The reason you want this in the desktop session is that you want to query the locally logged in users preferences from e.g. gconf or whatever.
Just what I personally think.
Hmm... Would it be possible to restart *X11* without killing the current session? So that the user would only se a "blink" and then whatever was working?
tir, 30.11.2004 kl. 14.25 skrev Avi Alkalay:
For people that still don't know the Elektra Project (http://elektra.sourceforge.net), it provides an alternative back-end for text configuration files. So instead, for example, the human-readable-only xorg.conf file, we'll have a hierarchy of key/value pairs easily and precisely accessible by X.org and also any other programs through an API or command, and human beings too. Same concept as GConf, but designed for system-wide use.
We've been working on X.org source code to do exactly what we just described above. And it is ready for use, tested even with very complex X.org configurations. A patched X server will convert your xorg.conf file automatically, even the most complex ones, including non-documented parameters.
It is available at http://sourceforge.net/project/showfiles.php?group_id=117521 http://sourceforge.net/project/showfiles.php?group_id=117521&package_id=...
You'll find there the patch against X.org source, the patch against Fedora Core 3 RPM spec file (to see how to build it), and the README file, which is pasted in this e-mail bellow.
We'll write patches for other software, and everybody is invited to join this integration effort.
Thank you, Avi Alkalay Elektra Project http://elektra.sf.net
----------------------README------------------------- ELEKTRIFIED X.ORG: PRECISE AND PROGRAMATICALY EASY X CONFIGURATION
Your X server has to work with your installed video card, monitor, find your font server and dirs, modules, extensions, drivers, plugins. You have to tell him how to integrate it all through its xorg.conf file. If you need to change something, you start a text editor and use your human brain and eyes to find the correct line, understand it, have the skills to change it, and change it in order to work.
This is good for advanced technical guys, but very bad for people that don't have this skills, and in fact, don't really want to. He just wants to change the screen resolution to make that projector work with his laptop, and go ahead with his sales presentation. This is just an example.
The point is: it is very difficult to make simple programs or scripts that make simple changes to X configuration. Another example is a monitor vendor that wants to support X, and for this he'd like to provide easy installation of his product, without having to ask his user to read documentation about horizontal Sync, and vertical refresh rates. For him again is difficult to write some simple software that preciselly changes X configuration to work correctly with his product.
The xorg.conf file (as most Unix configuration files) was designed for human beings.
The Elektra Project (http://elektra.sourceforge.net) introduces a new way to handle configurations through a clean API (or command line tool) that accesses atomic configuration data, into a standarized hierarchycal tree of key-value pairs. It is similar to GConf, but designed for system-wide use, which also implies it does not have dependencies.
And this is what this patch is about: to make the X server look for its configurations into a machine-ready tree of key-value pairs, instead of the human-ready xorg.conf.
So where you had to look for "Device radeon" into a "Section Device", with the key/value tree, X and other programs can look for it preciselly at
system/sw/xorg/current/Devices/Videocard0/Driver = radeonWhere you once had to "vi" your "Section Monitor", now X and other programs can do it accessing the keys:
system/sw/xorg/current/Monitors/Monitor0/HorizSync = 31.5 - 48.5 system/sw/xorg/current/Monitors/Monitor0/VertRefresh = 40.0 - 70.0 system/sw/xorg/current/Monitors/Monitor0/ModelName = IBM T40 LCD Panel system/sw/xorg/current/Monitors/Monitor0/VendorName = IBM system/sw/xorg/current/Monitors/Monitor0/Options/dpmsWhere once the salesman above had to "vi" the Screen Section to change the resolution, color depth, etc, a program can help him accessing:
system/sw/xorg/current/Screens/Screen0/Displays/00/Depth=24 system/sw/xorg/current/Screens/Screen0/Displays/00/Modes=1024x768And so on....
We believe an elektrified X server can leverage more plug-and-play configurations, providing configuration power to HW vendors in a very simple way, and making users experience less painfull.
BEHAVIOR OF AN ELEKTRIFIED X SERVER
A patched X server will look for its configuration keys under the namespace:
system/sw/xorg/current first, then if not found it tries system/sw/xorgIf not found, it will default to some xorg.conf file, parse it, and store in its internal structures, then convert and commit it to a set of keys under system/sw/xorg/current, and reload these keys.
So you get automatic one-time conversion from xorg.conf to the hierarchycal configuration key/value pairs
Very complex examples of xorg.conf files were tested for conversion. Even undocumented configuration parameters (because the original source was used as the reference).
The Elektrifyied X server also works for the Red Hat Graphical Boot, where you still don't have mounted partitions, network, etc.
ELEKTRIFING X.ORG
You'll need the elektra-devel package installed in order to build X with Elektra support.
1. Download and unpack X.org source code from 2. Download the xorg-x11-6.8.1-elektra.patch file from http://sourceforge.net/project/showfiles.php?group_id=117521&package_id=136552 3. Apply it: ~/src/xc$ cd .. ~/src$ patch -p0 < xorg-x11-6.8.1-elektra.patch ~/src$ cd xc ~/src/xc$ # ...configure your build in host.def 4. Enable the patch: ~/src/xc$ echo "#define UseElektra" >> config/cf/host.def 5. Build X.OrgYou'll find the new X server as file xc/programs/Xserver/Xorg .
The patch will add the following files:
xc/programs/Xserver/hw/xfree86/parser/ elektra.h (exported methods) elektra.c (key fetching and X structs integration business logic) xorg-example.conf (a very complex conf file to test conversion) xelektrify.c (cmd to convert xorg.conf->keys and vice-versa) README.Elektra (this file)And it will instrument xc/programs/Xserver/hw/xfree86/common/xf86Config.c::xf86HandleConfigFile() to trigger the one-time conversion, and key fetching logic.
And instrument the Imakefiles for correct builds:
xc/programs/Xserver/hw/xfree86/parser/Imakefile xc/programs/Xserver/hw/xfree86/common/Imakefile xc/programs/Xserver/ImakefileIf "#define UseElektra" is not present in host.def, the patch is completelly disabled, and you'll get a binary-identicall built as before applying the patch. All patched code are surrounded by #ifdefs.
ELEKTRA MEETS X.ORG SOURCE CODE or how we wrote the patch....
X.org has an xorg.conf parser that takes this steps to handle configuration:
1. Lexically parse the xorg.conf file 2. Put each Section info in an equivalent struct 3. Encapsulate all structs together and pass them to a validator 4. Use structs to define X behaviorThis process is triggered by the xf86HandleConfigFile() method from
xc/programs/Xserver/hw/xfree86/common/xf86Config.cEach xorg.conf Section has an equivalent structure defined in
xc/programs/Xserver/hw/xfree86/parser/xf86Parser.hand the lexycall analyzer code to parse each Section is under
xc/programs/Xserver/hw/xfree86/parser/A fully parsed file has its equivalent structures encapsulated in a parent XF86Config struct. We have:
struct XF86ConfModuleRec for the "Section Modules" struct XF86ConfMonitorRec for the "Section Monitor" struct XF86ConfDeviceRec for the "Section Device" etc...These structs are a pure computer representation of the text in each Section, so the methods under "parser/" convert text to structs, and the structs to text. This is how original X.org source handles xorg.conf.
The Elektrification add methods that act in steps 1 and 2 above. And also include methods to convert each struct to a KeySet. Both old (xorg.conf) and new (Elektra) ways to get configuration information can live together and they are actually used to automatically convert xorg.conf to keys. So at the first time you'll run your elektrified X server, it will:
1. Not find configuration keys (because it is the first time) 2. Parse xorg.conf into structs 3. Convert structs to Keys 4. Commit the keys to key database 5. Reload configurations from the key databaseSee the behavior in the previous section.
After assembling the XF86Config C structure, X will decode all its info into more practicall parameters for its further operation.
As a side note, with a key/value pair configuration hierarchy paradigm, the XF86Config assembling code (the parser) could be avoided, making X look for its configurations in a programatically easier, yet human-readable, configuration organization. We worked at the parser level to keep compatibility and to not go too deep in X.org source code.
http://elektra.sourceforge.net The Elektra Project Avi Alkalay November 2004
While i am all for the *idea*, what happens if your X11 server breaks down and you *have* to use VIM to "save" it? While i am not opposed to creating an easily learned api for config-file configuration, i am *strongly* opposed to not be able to hack into that xorg.conf file from a crude command line (hell, i have vim'ed config files with /bin/bash as my init process and a half borked kernel that would boot 1/5 times...).
I don't want my config's to be put all in the hands of some program, to *have* to use a config program (with all the odds and ends it migth have...) in order to fix a broken config. Now imagine that program needed another program, which needed a service wich needed ... the thing you where just trying to fix.
Kyrre
On Tue, 2004-11-30 at 19:05 +0100, Kyrre Ness Sjobak wrote:
While i am all for the *idea*, what happens if your X11 server breaks down and you *have* to use VIM to "save" it? While i am not opposed to creating an easily learned api for config-file configuration, i am *strongly* opposed to not be able to hack into that xorg.conf file from a crude command line (hell, i have vim'ed config files with /bin/bash as my init process and a half borked kernel that would boot 1/5 times...).
I don't want my config's to be put all in the hands of some program, to *have* to use a config program (with all the odds and ends it migth have...) in order to fix a broken config. Now imagine that program needed another program, which needed a service wich needed ... the thing you where just trying to fix.
You don't *have* to use a config program, the keys are all vim hackable. It might be a royal pain, however, to add a bunch of keys with an editor. Tweaks would be easy enough though.
http://elektra.sourceforge.net/#rgfacts (third bullet, also seventh)
-- Darrin
tir, 30.11.2004 kl. 19.22 skrev Darrin Thompson:
On Tue, 2004-11-30 at 19:05 +0100, Kyrre Ness Sjobak wrote:
While i am all for the *idea*, what happens if your X11 server breaks down and you *have* to use VIM to "save" it? While i am not opposed to creating an easily learned api for config-file configuration, i am *strongly* opposed to not be able to hack into that xorg.conf file from a crude command line (hell, i have vim'ed config files with /bin/bash as my init process and a half borked kernel that would boot 1/5 times...).
I don't want my config's to be put all in the hands of some program, to *have* to use a config program (with all the odds and ends it migth have...) in order to fix a broken config. Now imagine that program needed another program, which needed a service wich needed ... the thing you where just trying to fix.
You don't *have* to use a config program, the keys are all vim hackable. It might be a royal pain, however, to add a bunch of keys with an editor. Tweaks would be easy enough though.
http://elektra.sourceforge.net/#rgfacts (third bullet, also seventh)
-- Darrin
Sometimes you will have to do that.
What about a plugin-based system, which used "regular" configs as back end, and exported them as an (consistent over different configs) API?
On Tue, 30 Nov 2004 13:22:53 -0500, Darrin Thompson darrint@progeny.com wrote:
On Tue, 2004-11-30 at 19:05 +0100, Kyrre Ness Sjobak wrote:
While i am all for the *idea*, what happens if your X11 server breaks down and you *have* to use VIM to "save" it? While i am not opposed to creating an easily learned api for config-file configuration, i am *strongly* opposed to not be able to hack into that xorg.conf file from a crude command line (hell, i have vim'ed config files with /bin/bash as my init process and a half borked kernel that would boot 1/5 times...).
I don't want my config's to be put all in the hands of some program, to *have* to use a config program (with all the odds and ends it migth have...) in order to fix a broken config. Now imagine that program needed another program, which needed a service wich needed ... the thing you where just trying to fix.
You don't *have* to use a config program, the keys are all vim hackable. It might be a royal pain, however, to add a bunch of keys with an editor. Tweaks would be easy enough though.
http://elektra.sourceforge.net/#rgfacts (third bullet, also seventh)
Also this screenshot: http://elektra.sourceforge.net/#edit Which shows using any text editor (including vi) to edit an export of a subtree.
The idea is to create such a level of integration between applications, that you probably won't need to edit configuration by hand. Software will do this for you. How many times per year you edit modern windows registry by hand? How many times per year you edit GConf keys by hand ?
Regards, Avi
--On Wednesday, December 01, 2004 1:41 PM -0300 Avi Alkalay avibrazil@gmail.com wrote:
How many times per year you edit modern windows registry by hand?
Quite a lot. The ability to grep the registry would be really nice, but no built-in tool offers that. One has to buy commercial add-ons for that. Backing up the whole registry is also a PITA.
But I don't think Elektra suffers these problems. As long as it's all text files and state isn't held in memory, unavailable to backup programs, I'm fine with it.
-
Alan Cox -
Avi Alkalay -
Camilo Mesias -
Chris Adams -
Chuck Anderson -
Daniel Veillard -
Darrin Thompson -
David Zeuthen -
Havoc Pennington -
Jeff Pitman -
Kenneth Porter -
Kyrre Ness Sjobak -
Neal Becker -
Peter Backlund -
Sean Middleditch -
Toshio Kuratomi