(Discussion continued from: https://github.com/pydoit/doit/issues/32#issuecomment-87821389)
@schettino72 The introduction of a 2nd configuration file for me is the most problematic point. I will give 3 examples:1. Have a `foo/dodo.py` import tasks from another `foo/dodo.py` that depend on their companion `foo/doit.cfg` (ie for a "pickling" db-backend).It is too late to read companion `foo/doit.cfg`, the user should have already merge it with its own.
2. Use `doit -d <dir>` option. Which `doit.cfg` should it read? cwd or `-d` dir?
3. What about interactive consoles (ie IPython %doit)? Should they read cwd? home-dir (ie `~/.doit.cfg`), ipython profile-dit, or all of them together in some deterministic format?In master this issue is still unresolved.
I have seen this pattern of dual-config-files in maven-1x, they had exactly this setup, one for the project and another one for the plugin-properties, and it was too cumbersome to the users.Eventually they scrapped the whole thing in maven-2 for a *single* pom.xml file.
And exactly this is a key point about the current architecture:It complicates the life to the user of the plugin, for the benefit of the plugin developer (where it should be the other way round).
Concerning now your 2 objections to setuptools, the 2nd one (for an extra depepnency to `setuptools`) was invalidated by @kwpolska.
Regarding the 1st, that you have to have the plugin installed somehow, and hence "distributed".No, it is not neccessary to have a package in *pypi* for to be installed with pip.You can just use: ``python setup.py develop`` on the private copy of your code.Of course you have to have a `setup.py` file.
Finally, regarding your question of what i'm trying to achieve with the sample "interface", a general answer would be: giving a chance to plugins to configure themselves in the context of other plugins.
Of course we can discuss this later if you deem worthwhile to proceed to a different approach.
Kind regards,kostis
On Tue, Mar 31, 2015 at 4:41 PM, Kostis ankostis <anko...@gmail.com> wrote:(Discussion continued from: https://github.com/pydoit/doit/issues/32#issuecomment-87821389)
Finally, regarding your question of what i'm trying to achieve with the sample "interface", a general answer would be: giving a chance to plugins to configure themselves in the context of other plugins.Yes the current architecture does not allow "plugins to configure themselves in the context of other plugins".doit components are well defined and independent... Can you give an example that applies to doit?
On Tue, Mar 31, 2015 at 11:48 PM, Kostis ankostis <anko...@gmail.com> wrote:
On Tuesday, March 31, 2015 at 2:16:18 PM UTC+2, schettino72 wrote:On Tue, Mar 31, 2015 at 4:41 PM, Kostis ankostis <anko...@gmail.com> wrote:(Discussion continued from: https://github.com/pydoit/doit/issues/32#issuecomment-87821389)Finally, regarding your question of what i'm trying to achieve with the sample "interface", a general answer would be: giving a chance to plugins to configure themselves in the context of other plugins.Yes the current architecture does not allow "plugins to configure themselves in the context of other plugins".doit components are well defined and independent... Can you give an example that applies to doit?I have not an code-example as all this is new stuff.But I did contemplated about splitting the `cmd_graphx` plugin into ...more plugins.Currently, cmd_graphx requires *optionally* matplotlib and graphviz, meaning that it can also provide results without them. The "normal" way to deal with optional requirements is to split them in dependent projects (backends).So you have cmd_graphx <-- (graphx_matplotlib, graphx_graphviz).For this, I have at least 2 alternatives:A. Parse the same `doit.cfg` file, or parse a different one.B. Use the setuptools extension mechanism and let the backends modify available graph-cmd's options (ie add 'matplotlib' into the dict of available outputs).Lets leave case2 aside for the moment.A.1) For the 1st case, if I choose to use a 2nd cfg-file, I contribute into the proliferation of config-files, making harder the life of my users, i don't want to do this; and you don't want to do it either, trust me, people hated the guts out of how maven-1 forced them to keep multiple files spread in several folders.A.2) If I choose to reuse the `doit.cfg` file, there are also 2 alternatives, α) access the parsed file from doit-base (needs modifications to your plugin code), or β) re-parse the file, which means reproducing your reading code.In all cases (A & B) we need to allow "plugins to configure themselves in the context of other plugins".But in the A case we are in the "maven-1" world, where for each project the user has to maintain 2 files, the `dodo.py` and the `doit.cfg`;that's the "companion cfg-file" i was talking about. It is not always obvious how how to handle it, ie i still don't know how to resolve the IPython case because it requires the user to manually edit the `doit.cfg` in ipython profile, and I consider this awkward)Truth is that I did not understand most of what you are saying
but it seems there is confusion: Who write a `doit.cfg` is a plugin user, not a plugin author...
The plugins were designed to handle the extension points doit already provides.
You are talking about a plugin to extend something that doit doesnt support at all.
This complete out of what i am trying to do.
And I do not agree the `graph` command should be split into many plugins...
You can just detect which libraries are available and expose just those. For example doit checks if multiprocess module is available or not. I dont understand how plugins could improve that.
I suggest you just get the `graph` command working fine first (it still doesnt work for me and I didnt have time to check why)
than think if its functionality should be further split.
cheers
It took only 10 lines of code to integrate support setuptools plugins! [1]I added support for auto-loading plugins that are registered using setuptools.I plan to implement task-loaders plugin before next release...At https://github.com/pydoit/doit-plugin-sample there is a minimal plugin to serve as an example.Hi,I completed a good enough implementation of doit plugin system.Of course you need the master from github to test it.So far on `doit` master there is support for 3 categories of plugins: Command, Reporter and DB backend.Answering Kostis concerns:## setuptools support
Local plugins without using setuptools are also supported.One question: Can I install a package and instruct pip/setuptools to not save the entry points?
Yes, it goes against the Zen of Python "There should be one-- and preferably only one --obvious way to do it."## doit having multiple sources of configurationthe plugins.but if for *your* plugin category, this is important, it is up to you to decide when and how you load and initializeFor the plugin categories doit recognizes the order in which they are loaded/initialized does not matter...For example the REPORTER plugins are only used by the RUN command.doit will collect the entry points but it will not load/import plugin from categories it does not recognize.It is possible for a plugin to rely/use other plugins that doit has no idea about.
## plugins relying on other pluginsIt doesnt really bother me, and IMO in this case, it is a feature :)If it bothers you just stick to just one of them.
Kind regards,