New configuration system
Marty Alchin
list just to step on toes, but I assure you that's not my intention. I
just like hacking around the insides of projects and seeing what makes
them work. And I try to make them better where I can. I only just
today started looking at Satchmo, and I notice you guys just did a
substantial merge of a new configuration system, and I had some
questions about it.
Is there a particular reason Satchmo didn't go with a
declarative-syntax approach like how Django does models and newforms?
It looks like the registration portion of the config system would be
very well suited for this, and it could cut a bit of code out of the
individual modules.
Of course, this is probably just personal preference, and since you
already merged it, I'm not trying to change anything, I'm just
curious. I have a couple apps of my own that use declarative syntax,
and if your approach has clear benefits, I may consider alternative
approaches in the future.
Just as an example of what I'm talking about, could configuration have
been handled like this?
class PaypalConfig(config.Group):
# More here...
live = config.BooleanValue(_("Accept real payments"),
help_text=_("False if you want to be in test mode"),
default=False)
module = config.ModuleValue(_("Implementation module"),
hidden=True,
default='satchmo.payment.modules.paypal')
# More here...
Again, I'm not trying to step on toes, or question your decisions, I'm
just trying to learn.
-Gul
Bruce Kroeze
Anyway, the answer is that the config system supports a couple features which would be difficult to achieve with a straight declarative syntax.
- Ability to add more options to a group. For example, the payment modules all contribute choices to the payment module select box. Registering them all at once would preclude supporting user-written payment modules.
- Ability to specify requirements before a specific setting is enabled/visible.
If you see ways to achieve those requirements, then I would actually prefer a more standard syntax.
-Bruce Kroeze
Marty Alchin
> It would be my toes you are stepping on, and I've got workboots. :)
Well, nice to meet you, then! Glad to hear you're wearing workboots,
I'd feel bad if you were in sandals.
> - Ability to add more options to a group. For example, the payment modules
> all contribute choices to the payment module select box. Registering them
> all at once would preclude supporting user-written payment modules.
Ah, that certainly makes a lot of sense. I hadn't yet dealt with
something that had to extend another declaration, so that would indeed
be tricky.
> - Ability to specify requirements before a specific setting is
> enabled/visible.
Oh, excellent! You guys really are thinking of the average person,
aren't you? Few things bother me more than documentation saying, "Oh,
but if you disabled option X, just ignore options Y and Z." Having
that built-in is a nice touch.
> If you see ways to achieve those requirements, then I would actually prefer
> a more standard syntax.
I'll toy around and see what I can come up with, but it's certainly
not high on my list of things. I'll be poking around in Satchmo a bit
anyway for my RPG-economy (thanks for making that so easy, by the
way), so I may well find a way to manage it.
-Gul
Marty Alchin
> > a more standard syntax.
>
> I'll toy around and see what I can come up with, but it's certainly
> not high on my list of things. I'll be poking around in Satchmo a bit
> anyway for my RPG-economy (thanks for making that so easy, by the
> way), so I may well find a way to manage it.
The more I think about a declarative syntax for Satchmo's
configuration system, the more it looks like an extension of my own
dbsettings[1] application. It looks like the basic goal is the same:
allowing settings to be changed while the server is up and running,
storing in the database for persistence across server instances, and
caching the settings so they don't have to hit the database when they
don't change that often.
To be honest, I'm wondering if it might be worth my while to extend
dbsettings to include the requirements you mentioned as being
essential to Satchmo. It'd certainly be a faster way to go about it,
as opposed to a drastic rewrite in Satchmo, it'd be a feature upgrade
for dbsettings. The biggest downside is that it would impose yet
another dependency on a Satchmo installation, which is probably a big
concern.
I haven't written any code either way, but what would you think about
merging your needs into dbsettings, then using that to power Satchmo's
configuration? I can provide more details on dbsettings if you're
unfamiliar with it.
-Gul
Bruce Kroeze
I added:
- requirements - some settings only appear if requirements are met. These requirements are set up at register time, and can be applied to either individual settings or the group as a whole.
- formal registration of settings - which you were explicitly avoiding.
- lots of ways to handle multiple settings - serializing to the DB as JSON.
- ability to retrieve choice/value pairs
- default values - really persistent default settings. This gives the ability to use settings directly on models, as long as they have defaults.
- post-creation addition of choices to settings.
- much more extensive caching, using a caching framework rather than the small stub you had. This allows some nice features such as administrative cache purges.
- a "module" field.
- hidden fields
I removed:
- any way to declare as part of a model. I didn't need it, and it was easier to skip that, since it was a lot of the trickiness of the original.
- your caching system.
Take a look at it if you like, and merge in any of the parts that make sense. I really like the defaults and the way I handle them. The requirements feature is equally nice. The code is in the satchmo.configuration module.
Marty Alchin
> Well, I actually already did it. It is basically a gigantic fork of your
> app, really only using the values module, and extending that pretty heavily.
I wondered if that was the case, once I noticed that they were named *Value. :)
> I added:
> - requirements - some settings only appear if requirements are met. These
> requirements are set up at register time, and can be applied to either
> individual settings or the group as a whole.
And I very much like this idea. I think I have a decent way to go
about this in a dbsettings manner, so I'll probably work something
like that in.
> - formal registration of settings - which you were explicitly avoiding.
Well, it depends somewhat on how you define "registration". dbsettings
allows apps to either register settings immediately, or to just
provide a group of settings and leave it up to the project to register
them.
Personally, I'm not even the biggest fan of the syntax dbsettings
ended up using in this regard, but there was significant interest
among the core devs in getting it into core, so I was willing to make
that change in hopes of making available to more people.
I think the biggest thing that Satchmo's configuration has over
dbsettings in this area is the ability to register settings in a
particular organization, which makes them make a lot more sense to the
end user. I'll definitely see if there's a way to do this in
dbsettings, but it might require too significant of an overhaul.
And, of course, if I'm misunderstanding what you meant by this, I
apologize. I'm eager to hear all the improvements you've made.
> - lots of ways to handle multiple settings - serializing to the DB as JSON.
I admit, I had considered doing something like this, but I couldn't
think of a good way to do so. I even had a MultipleStringValue at one
point, but it just used a comma to separate values, which obviously
isn't very robust.
I haven't looked in much detail at the implementation though, so I'm
not sure how you handle situations where the JSON string exceeds 255
characters. Once you mentioned JSON, I considered ways to do it, and
it makes me wish I had started with TextField instead of CharField. Oh
well.
> - ability to retrieve choice/value pairs
Yeah, this becomes essential once you allow choices to be modified at run-time.
> - default values - really persistent default settings. This gives the
> ability to use settings directly on models, as long as they have defaults.
I'll have to look into this a bit more to understand what exactly this means.
> - post-creation addition of choices to settings.
This no doubt comes out of Satchmo's need to extend existing settings
and groups, which is something I hadn't considered when I wrote
dbsettings initially. I like this very much, and I'll definitely work
it in somehow.
> - much more extensive caching, using a caching framework rather than the
> small stub you had. This allows some nice features such as administrative
> cache purges.
Well, that stub was running into problems in multi-process
environments, so I had to move to Django's cache anyway. But it's just
a simple tie-in, nothing like the framework you put together. I'll
have to do some more research into that to understand it better.
> - a "module" field.
One of my goals for dbsettings was the ability to add new value types,
so I'm glad to hear you came up with one!
> - hidden fields
I hadn't had a use for hidden fields, but I'm all for it if it helps you out. :)
> I removed:
> - any way to declare as part of a model. I didn't need it, and it was
> easier to skip that, since it was a lot of the trickiness of the original.
Yeah, adding stuff to models would be of almost no use to Satchmo, so
this makes sense.
> - your caching system.
Ditto.
> Take a look at it if you like, and merge in any of the parts that make
> sense. I really like the defaults and the way I handle them. The
> requirements feature is equally nice. The code is in the
> satchmo.configuration module.
I'm definitely looking through it, and I appreciate hearing your
explanation of your added features. I'm beginning to think that even
though I could incorporate some of your features into dbsettings, it
wouldn't make much sense to replace yours even after that.
You have a good system in place, so the only thing I'd offer is help
turning it into a declarative syntax. But, of course, that's all just
a matter of taste, so it's really up to you guys whether you want to
do that.
-Gul