Profile Layer
Dominic König
I just want to share a vision for the future development of the Eden
architecture which Fran and I had discussed recently.
Currently, Sahana Eden consist mostly of three different "layers":
1) the "kernel" layer, also called "S3 framework"
This layer provides all the intrinsic functionality for our use-cases
including tools for rapid modeling and deployment. Despite often seen as a
pure back-end machinery, it actually facilitates 99.9% of all interfaces of
Eden - including the web GUI.
2) the "application" layer, also called "modules"
I'm not a friend of the term "modules" - actually, the "application" layer
consists of data models and controllers. This layer implements the specific
business logic and processes.
3) the "configuration" layer
Currently, this layer consists of a number of different configuration and data
files - most prominently the "deployment settings" and the data sets for "DB
pre-population". Other pieces like language files, UI layouts and styles, or
user role definitions can also be counted into this layer - all of which is
meant to adapt the application to a specific deployment.
---
We have come to a point where the "application" layer is largely implemented,
i.e. new "modules" are rarely going to be added. Instead there is an
increasing number of variations (branches) of Eden with specializations of
existing modules.
Now, our vision is to develop the current "configuration" layer in a way that
it can largely remove the need for specialization branches - replacing them by
a single branch containing multiple "deployment profiles".
Of course this requires to give the configuration layer much more shape -
especially in view of model specializations and workflow configuration. But
there's also a need for better integration of the different specialization and
configuration aspects - all in a way that it is relatively easy to set up and
maintain by non-technical people, and the ultimate goal, of course, is to be
able to "develop" (customize) most of a deployment profile through a web setup
GUI.
I've coined "profile layer" to reflect the vision, however, this shall of
course not lead into confusion with "user profiles" - we're talking of
application setup here.
Discussion welcome!
Regards,
Dominic
Fran Boon
> We have come to a point where the "application" layer is largely implemented,
> i.e. new "modules" are rarely going to be added.
I don't think I agree with this part actually ;)
> Instead there is an
> increasing number of variations (branches) of Eden with specializations of
> existing modules.
& I really want to minimise this - instead we're putting effort into
using deployment_settings to allow a single codebase to support
multiple workflows & labelling schemes
Currently branches should only contain variations in 000_config &
theme (including frontpage, etc content)
Otherwise I agree with the thread....so what seems to be the next
steps to achieve this?
(1) Move the 000_config settings from a single template to the
prepopulate folders (renaming them away from just 'demo')
(2) Separate out the theme in a similar manner
F
Fran Boon
> so what seems to be the next steps to achieve this?
> (1) Move the 000_config settings from a single template to the
> prepopulate folders (renaming them away from just 'demo')
Note that I don't want to simply move 000_config.py from templates to
a copy inside each 'demo' (sic) folder.
Rather I'd like the version inside the 'demo' folders to simply be the
'override' settings which are required for that profile to run.
I think this would always contain the list of enabled modules (& their
names & menu positions)
This would never contain the public_url or email settings.
Since most settings in 000_config default correctly, the default file
should be able to be mostly just commented options.
The override settings could then be copied to a separate
000_config_profile.py file, or something,
Obviously we'd want a nice mechanism to ensure that this only happened
when-needed & not every request - similar to 1st_run or update_check.
F
Dominic König
3 mar 2012 kl. 21:36 skrev Fran Boon <franc...@gmail.com>:
> 2012/3/4 Dominic König <dom...@nursix.org>:
>> We have come to a point where the "application" layer is largely implemented,
>> i.e. new "modules" are rarely going to be added.
>
> I don't think I agree with this part actually ;)
Rather an observed tendency, not a reasoning.
>
>> Instead there is an
>> increasing number of variations (branches) of Eden with specializations of
>> existing modules.
>
> & I really want to minimise this - instead we're putting effort into
> using deployment_settings to allow a single codebase to support
> multiple workflows & labelling schemes
> Currently branches should only contain variations in 000_config &
> theme (including frontpage, etc content)
>
> Otherwise I agree with the thread....so what seems to be the next
> steps to achieve this?
> (1) Move the 000_config settings from a single template to the
> prepopulate folders (renaming them away from just 'demo')
> (2) Separate out the theme in a similar manner
The first thing that needs to be made clear is how we select the profile in general. If this shall happen via a Python variable, then it maybe the right thing to move the actual instance of the config into the prepop ("profile") folders.
It could though also be solved by a custom import - which would be more consistent and less fragile, and would not need to happen every request.
However, before we make a strategy here, I would actually want to discuss the goals first.
Also - one thing that would be reasonable to do first is to complete the move of the models into modules.
Dominic
Pat Tressel
>> (1) Move the 000_config settings from a single template to the
>> prepopulate folders (renaming them away from just 'demo')
>
> Note that I don't want to simply move 000_config.py from templates to
> a copy inside each 'demo' (sic) folder.
> Rather I'd like the version inside the 'demo' folders to simply be the
> 'override' settings which are required for that profile to run.
> Obviously we'd want a nice mechanism to ensure that this only happened
> when-needed & not every request - similar to 1st_run or update_check.
Tossing out two things:
1) Maybe look at how other frameworks handle this. (For instance,
Wordpress stores customization info in the database, IIUC. That's
slow if css or other static view files have to be generated on the
fly. Dunno how Wordpress handles that.)
2) Is there a GSoC project in here somewhere? Fran, I forget -- did
you say there's prior GSoC work for web setup to integrate?. If we
finish web setup, we could have that do whatever's needed to write
files or write to the db. As you say, we could use something like
update_check to only generate files when needed. (That was intended
to be merged w/ web setup in any case.)
-- Pat
Fran Boon
>>> so what seems to be the next steps to achieve this?
>>> (1) Move the 000_config settings from a single template to the
>>> prepopulate folders (renaming them away from just 'demo')
>> Note that I don't want to simply move 000_config.py from templates to
>> a copy inside each 'demo' (sic) folder.
>> Rather I'd like the version inside the 'demo' folders to simply be the
>> 'override' settings which are required for that profile to run.
>> when-needed & not every request - similar to 1st_run or update_check.
> Wordpress stores customization info in the database, IIUC. That's
> slow if css or other static view files have to be generated on the
> fly. Dunno how Wordpress handles that.)
Right - we'd prefer to avoid the DB vs simple Python dict if possible.
Happy for people to do research on how other systems do this in general :)
> 2) Is there a GSoC project in here somewhere?
We have a community meeting on Wednesday - I was going to make GSoC
ideas top of the agenda.
Theme is 1 of the ideas that I have for that...although we need a
mentor to step forward for it.
> Fran, I forget -- did
> you say there's prior GSoC work for web setup to integrate?.
Yes. Shikhar: Were you going to get a merge proposal ready for that?
> If we
> finish web setup, we could have that do whatever's needed to write
> files or write to the db.
Yes, however I don't want that to be required for this process.
I want everything to be possible through the CLI - e.g. this will
allow Testing using CI of all the different profiles.
In general I prefer a back-end system doing 1st & then adding a GUI 2nd.
F
Dominic König
That is, I think, instead of starting the discussion at where and how to store the options I'd like to get some more input on what should be made configurable at all, and from there analyze the best methods. I'm saying that because for example menu configuration involves more than just variable values - meaning: code. Same for layouts (renderers), export/import stylesheets, or workflow and model specialization. I don't think we can handle everything in Python dicts.
Dominic
Fran Boon
> That is, I think, instead of starting the discussion at where and how to store the options I'd like to get some more input on what should be made configurable at all, and from there analyze the best methods. I'm saying that because for >example menu configuration involves more than just variable values - meaning: code. Same for layouts (renderers), export/import stylesheets, or workflow and model specialization. I don't think we can handle everything in Python >dicts.
I don't see much need to have Stylesheets in the Profile - I'd rather
make them as flexible as possible.
Workflow/Model customizations I'd see as just flipped by switches in
the short term
- all fields are present & just made readable/writable=False when not
used by a specific deployment.
- whilst a full-blown workflow engine would be nice, it isn't really
required at this time - instead simply being able to select via
deployment_setting between 2 alternates seems just fine.
Another example of something that we'd like to be able to have in the
profile is the handling of Inbound Messages (e.g SMS)
Currently this is done here:
https://github.com/flavour/eden/blob/master/modules/s3/s3msg.py#L192
I'm thinking that moving this out & adding the CERT profile could be a
good GSoC.
F
Michael Howden
My only question is why we are calling this the "Profile Layer", when "Configuration Layer" seems like the more obvious name and you've already pointed out the potential for confusion with "user profiles" (which is what I thought that this email was about)
In terms of what should be made configurable:
- Workflows (currently create_next/ update_next / etc, but I suspect at some point we'll need to look more seriously at workflows)
- Disabling/Enabling Fields
- Field Labels
- CRUD Strings
I'm sure there's more...
Cheers
-- Michael Howden skype: michael.howden twitter:@michaelhowden Sahana Software Foundation Member Sahana Eden Project Management Committee Sahana Community Development Committee
Dominic König
> This sounds like a great idea - especially separating out the theme design.
Yeah, can imagine that particularily this attracts your attention... :P
>
> My only question is why we are calling this the "Profile Layer", when
> "Configuration Layer" seems like the more obvious name
No no, it's intentionally "profile" and not "config" - it *includes*
configuration (and customization elements) *and* data.
Dominic
Michael Howden
> No no, it's intentionally "profile" and not "config" - it *includes*
> configuration (and customization elements) *and* data.
under-sells the functionality you're talking about. I still think that
configuration is the more descriptive term - configuration can include
data. Or what about "Deployment Layer"?
Fran Boon
>> No no, it's intentionally "profile" and not "config" - it *includes*
>> configuration (and customization elements) *and* data.
> Hmm... I still think that "profile" is a bit too ambiguous and under-sells
> the functionality you're talking about. I still think that configuration is
> the more descriptive term - configuration can include data. Or what about
> "Deployment Layer"?
Deployment is the specific parts unique to a single deployment - a
profile allows as much as possible to get inherited from a profile
common to other deployments of that type, yet distinct from
deployments of other types.
F
Dominic König
it feels so totally superfluous to discuss the term here and spend a lot of
emails on it - I don't know what you're after, but we chose this term merely
to introduce and discuss an architecture vision, not to name anything after it
or sell something by it.
And if we had called it the "outfit layer" or "bodywork layer", then that'd
still be good enough - so can we please return to the actual discussion and
talk about reason, strategy and methods to get this baby born?
How do we structure this layer (i.e. into which components), and how are we
going to connect this layer to the application (do we do this dynamically per
request, or per 1st run, or both)? Is any additional framework required for
this (likely)? How do we resolve dependencies? Are we going to introduce plug-
in architecture for profiles? In a staged implementation, which are the most
important things that need to be solved? What is the envisioned timeframe?
Which general strategy (e.g. which kind of things in the application layer
need to be parametrized so that they can inherit from the profile) should we
pursue? And WHO is going to implement it and how and when - is there a GSoC
opportunity?
Well, and so forth.
Dominic
Dominic König
it would be better to rephrase the discussion items a little bit to make the
controversies more visible:
1) The profile layer is meant to contain the configuration, the theme and data
sets for database pre-population. Is that all? Wouldn't it also contain model
specializations and workflows? How do we parametrize them?
2) Profiles shall be cascadable - or not? As far as I understood Fran, it is
intended to have an architecture where you can not only choose a profile, but
can choose a "base profile" and additional profiles for overlays, which
constructs a fallback-cascade.
3) How would profiles be developed? By whom? And which means are needed to
streamline this? Text files? GUI? External applications (it is very well
possible to develop workflows with an external application and import them)?
4) Some parts of the profile will contain (or produce) Python code - e.g.
layouts, and maybe workflows and model specializations. What kind of tool is
needed to make this manageable by less experienced developers or even
sysadmins?
Besides that, I understood from Fran that currently the menu structure, the
theme (layout+style), the configuration (deployment-settings) and the prepop-
data are the most prominent parts of what shall form a profile.
What I did not understand yet is who is to maintain the profile and how. The
question that comes up here is: aren't these multiple different sources?
Aren't prepop-data and theme and workflow likely to be maintained by three
different people? Certainly, a manager would design the workflows, a designer
would develop the theme(s), and yet someone else would provide/prepare the
prepop data (maybe different people) - how does that all come together? And
who does that?
Dominic
Fran Boon
> 1) The profile layer is meant to contain the configuration, the theme and data
> sets for database pre-population. Is that all? Wouldn't it also contain model
> specializations and workflows? How do we parametrize them?
Workflows definitely - e.g. the simple create_next
The 'model specializations' I see are mainly 2-fold:
* Hiding of optional fields which aren't wanted
* Labelling of Fields
* Online Help for fields
Whilst a branded theme could be seen as separate - for a
single-instance deployment this could be a deployment-specific thing,
for a multi-instance deployment (offline clients) then it could be a
profile.
Even with a single instance, many smaller users might not invest in a
custom design, but simply want to pick from 2-3 available designs (or
ideally be able to select a base colour & get matches) & then upload
their Logo & change the site name.
> 2) Profiles shall be cascadable - or not? As far as I understood Fran, it is
> intended to have an architecture where you can not only choose a profile, but
> can choose a "base profile" and additional profiles for overlays, which
> constructs a fallback-cascade.
Just like we have for Demos currently where we inherit settings from
various folders.
We also do this for deployment_settings to some extent: the branch
defines the profile settings in the template, which are further
refined by a deployment in the models/ instance.
> 3) How would profiles be developed? By whom? And which means are needed to
> streamline this? Text files? GUI? External applications (it is very well
> possible to develop workflows with an external application and import them)?
Overall profile I see as being assembled from components into
something like the current tasks.cfg
Each component can be configured in the most suitable way for the component.
I like the current demos approach where a profile can be contained
within a single folder &/or inherit from other folders.
> 4) Some parts of the profile will contain (or produce) Python code - e.g.
> layouts, and maybe workflows and model specializations. What kind of tool is
> needed to make this manageable by less experienced developers or even
> sysadmins?
I think that front-end tools to make this more accessible to less
technical people is some way in the distance.
The first step is separation of the elements whilst retaining the
ability to pull them together into a single 'profile'
> Besides that, I understood from Fran that currently the menu structure, the
> theme (layout+style), the configuration (deployment-settings) and the prepop-
> data are the most prominent parts of what shall form a profile.
Along with the things I mentioned above: field visibility, labels
(fields & CRUD Strings), workflows (which currently would be just
deployment_settings configuring create_next).
A new one here which relates to a GSoC project is inbound message-parsing:
http://eden.sahanafoundation.org/wiki/BluePrint/Messaging/Parsing
> What I did not understand yet is who is to maintain the profile and how. The
> question that comes up here is: aren't these multiple different sources?
> Aren't prepop-data and theme and workflow likely to be maintained by three
> different people? Certainly, a manager would design the workflows, a designer
> would develop the theme(s), and yet someone else would provide/prepare the
> prepop data (maybe different people) - how does that all come together? And
> who does that?
As I say, I see something like the current tasks.cfg pulling stuff
together from different sources.
I like a single folder for a profile being possible (even if it pulls
stuff in from other folders)
If components require their own sub-folders, that would be fine too.
F