active template

[graeme]

Hi GSoC students,

I know of two projects which are trying to extract information from the active template:

• Translation
• Automatic Testing
• ... do we have any others?

Now rather than inventing the wheel twice it would be good to share plans.

So:

1. I want Nikhil and Somay to list what information your project will need to extract from the template.
2. How essential is this to your project (i.e where does it fit in your timeline)
3. Then maybe outline the approach that you were thinking of using to get that data

Once we have a clear idea of each others requirements I think that we can come up with a common approach to tackling the problem.

Graeme.

[Nikhil Goyal]

Hi Graeme,

Regarding Extracting list of active modules.

Importance :

It is important to extract only the relevant strings for translation rather than extracting all.

My approach :
 

* First we parse 000_config.py file using parse tree approach and retrieve the currently active template
( settings.base.template =  "Current_Template" )

* Next we parse eden/private/templates/Current_Template/config.py to retrieve the list of active modules in the Current_Template, again using parse tree approach.
( settings.modules = OrderedDict[] )

Nikhil

--
You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAE8saEN2G7hR0MhBAnZUb7Uy31LeGYF7bRKYYtHYMtuuqZp2JQ%40mail.gmail.com?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[somay jain]

Hi Graeme,

The information I need from the tempate are -

1. Template Name ofcourse
2. List of modules enabled
3. The type of form fields used in the pages.

Importance :

It is important to run the Selenium tests independent of the template.

Approach -

1. The template name can be obtained from settings.base.template
2. List of modules enabled can be obtained from private/templates/current_template/config.py
3. The form field types can be obtained by first running the function stored in variable settings.ui.customize_<table_name>, which is indicated in the config file of the respective template. The widgets used in the template can then be acquired from current.s3db.<table_name>.<field_id>.widget
Using the widgets, we can know which type of form field is being used in the form.

Regards

Somay

[graeme]

Hi Guys,

Thanks for you replies. You both have some overlap in what you are interested in getting. Nikhil look at Somay's approach, this will be much easier. I'd suggest that you both look at S3Model, and specifically mandatory_models, and has_module, which is defined in s3cfg.

Somay, I'm not sure about the need for using settings.ui.customize_<table_name>, can't you just get the widget details directly from current.s3db.<table_name>.<field_id>.widget ?

Graeme.

--

You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/cf3d4a98-530e-484b-b8f0-37a453f90a7f%40googlegroups.com?hl=en-US.

[Fran Boon]

On 2 June 2013 03:07, graeme <foster...@gmail.com> wrote:
> Somay, I'm not sure about the need for using
> settings.ui.customize_<table_name>, can't you just get the widget details
> directly from current.s3db.<table_name>.<field_id>.widget ?

The value of the widget depends on the context
The value is originally set (or not) in the model, however it can be
adjusted in the controller or in a template (in
settings.ui.customize).
The widget can potentially vary based on context in either of these 3
locations too, but this is rarer....other than commonly only setting
the widget in a custom prep when r.interactive == True, so I agree
that reading settings.ui.customize is essential, it can't simply be
run as-is, but must have a fake S3Request to work on...

F

[graeme]

Thanks Fran,

How easy would it be to get this to work when you have two servers one running the test and the other serving-up the web pages? Is it possible once the page has been loaded to identify the widget from the HTML, rather than try and identify the widget from the code & context?

Graeme.

F

--
You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzktLN%3Du%2BSPLaqiEn2zg5xynd94fSr7aSkuFV15WCcPWLvpw%40mail.gmail.com?hl=en-US.

[Fran Boon]

On 2 June 2013 10:41, graeme <foster...@gmail.com> wrote:
> How easy would it be to get this to work when you have two servers one
> running the test and the other serving-up the web pages?

I'm not sure this makes too much difference if they both have access
to the same code (& indeed can run in the same context)

> Is it possible once
> the page has been loaded to identify the widget from the HTML, rather than
> try and identify the widget from the code & context?

I would imagine this to be possible, yes...we might need to seed the
HTML a little with appropriate classes, but that's definitely not a
barrier...

F

[Fran Boon]

On 2 June 2013 10:54, Fran Boon <franc...@gmail.com> wrote:
> On 2 June 2013 10:41, graeme <foster...@gmail.com> wrote:
>> How easy would it be to get this to work when you have two servers one
>> running the test and the other serving-up the web pages?
> I'm not sure this makes too much difference if they both have access
> to the same code (& indeed can run in the same context)

Note that I envisage a remote testing including the ability to SSH
into the remote system...this is required as part of deployment (for
setting up test systems & then throwing them away...this can then
allow testing multiple test configurations on either limited hardware
or hardware which costs to maintain 24x7 such as AWS).

F

[Nikhil Goyal]

Hi Graeme,

I don't see any difference in the approach mentioned by Somay and my approach (apart from his 3rd point which I won't be requiring). Am I missing something here?

Also, Somay - How do you plan to extract the active template and the modules inside it (from private/templates/current_template/config.py file) ?  Using the parse tree or do you have some other method in mind?

Thanks,

Nikhil

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAE8saEMfnt8dFkm1hrRb5eYdH2rgc_Fd-ik04qD2R%3DGpZiYmxg%40mail.gmail.com?hl=en-US.

[Fran Boon]

Does it need parsing especially?

Aren't we just running inside it's context anyway to read the
appropriate settings?

So for Translation, we normally want to just include strings in the
running template, so we can just use settings.get_template() to know
which template folder to include & then exclude all others?

Once we're reading prepop files, then some of these may exist in other
files, so we may need to parse the tasks.cfg inside the running
template.

F

> https://groups.google.com/d/msgid/sahana-eden/CAAu86T73%2BQKWxgBs7wZgxfcjMSSOmx6vSR1BJO-AKAOowHtRqg%40mail.gmail.com?hl=en-US.

[Nikhil Goyal]

Hi Fran,

Yes, we can use that to get the name of the current template but what about the modules which it uses? Won't that require parsing of private/templates/current_template/config.py? ( or is there a similar way to get all active modules too?)

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzkt%2BET1TyGbz4bWhAwsRQm9p%2BGq6RyvXQHkH1DRH9P3maqg%40mail.gmail.com?hl=en-US.

[Fran Boon]

On 2 June 2013 12:10, Nikhil Goyal <nown...@gmail.com> wrote:
> Yes, we can use that to get the name of the current template but what about
> the modules which it uses? Won't that require parsing of
> private/templates/current_template/config.py? ( or is there a similar way to
> get all active modules too?)

if we're running inside that template (the usual case) then since
we've already run this template, we can just use settings.modules

F

> https://groups.google.com/d/msgid/sahana-eden/CAAu86T67O9J-9b_VevbwaXMeE%2B9oScDc1YjD2XjU6f0%3DVA%2B4rw%40mail.gmail.com?hl=en-US.

[Nikhil Goyal]

Oh alright, that's helpful!
I guess we will generally be running the translation feature while using the required template, so this works :)

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzktJmahxN%2BtxCBn_bBWBchWLKA%3DNiBvrgO0VDDox_yum-jg%40mail.gmail.com?hl=en-US.

[somay jain]

Fran, could you explain the need for the fake S3Request? I did not understand this part.

[Fran Boon]

You need to read the settings which apply within a prep r.interactive == True

You can either do this by faking r (the S3Request) or doing some other
kind of parsing.
The former seems more reliable to me.

F

> --
> You received this message because you are subscribed to the Google Groups
> "Sahana-Eden" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sahana-eden...@googlegroups.com.
> To post to this group, send email to sahan...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sahana-eden/54c5b477-c14e-4212-a327-cd6d863190ef%40googlegroups.com?hl=en-US.

[Nikhil Goyal]

Hi Fran,

Regarding getting active modules in template, you suggested we use settings.modules. This will give info about the currently active template. What if multiple templates are selected ? In that we will have to parse the template config files anyway.
Is it an important usecase or shall we proceed without parsing and just using the settings variables ?

Nikhil

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzktJmahxN%2BtxCBn_bBWBchWLKA%3DNiBvrgO0VDDox_yum-jg%40mail.gmail.com?hl=en-US.

[Fran Boon]

On 4 June 2013 10:44, Nikhil Goyal <nown...@gmail.com> wrote:
> Regarding getting active modules in template, you suggested we use
> settings.modules. This will give info about the currently active template.
> What if multiple templates are selected ? In that we will have to parse the
> template config files anyway.
> Is it an important usecase or shall we proceed without parsing and just
> using the settings variables ?

I don't think this is an important usecase.

I see 2 usecases:
* My template (& hence only the modules active in my template)
* All templates (& all modules)

F

[Nikhil Goyal]

Hi Fran,

" I merged.
The approach seems good for getting us a useful step closer very easily :)
The "this template" part works perfectly, which is great, thanks :)
However the list of preselected modules wasn't accurate on my system though for some reason....I was running the DRMP template at the time. "

It works well on my system. The only explanation I can think of is that maybe the the list of modules from the settings variable is not consistent with the list of modules that we get from controller.

"After fixing that, the next steps would be to hide core modules:
Auth/Default should be inside Core
Appadmin/Errors/Sync should be inside 'Admin' "

I think it would be important to give the admin the option of deselecting these modules also, in case he knows that modules are already translated or for some other reason.

" We should probably have name_nice displayed rather than the actual names if this is to be used by non-core devs. "

Yeah I l do that, we can display name_nice with the acutal name in bracket.

" We should also build a dependency system, so that e.g. 'supply' gets selected when modules which use it are selected (i.e. Inv/Req). "

Is there any particular way to detect these dependencies or should we hard code these in our code ?

" Another example of dependencies:
modules/s3survey should only be included if the Survey module is selected "

I think this is being already done in the current code.

https://github.com/nownikhil/eden/blob/master/modules/s3/s3translate.py#L278

Nikhil

F

--
You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzktKa5PexxwMzAJek6nwx2JHVVAjMosNRrh92VnSap_hsqA%40mail.gmail.com?hl=en-US.

[Fran Boon]

On 16 June 2013 21:38, Nikhil Goyal <nown...@gmail.com> wrote:
> The "this template" part works perfectly, which is great, thanks :)
> However the list of preselected modules wasn't accurate on my system though
> for some reason....I was running the DRMP template at the time. "
> It works well on my system. The only explanation I can think of is that
> maybe the the list of modules from the settings variable is not consistent
> with the list of modules that we get from controller.

Actually this may have been a problem with my system - I have the
DVI/HMS modules locally enabled on top of what templates say since
they are used by the Unit Tests (would love to see those
decoupled...unit tests should use core modules like Org/PR)

> "After fixing that, the next steps would be to hide core modules:
> Auth/Default should be inside Core
> Appadmin/Errors/Sync should be inside 'Admin' "
> I think it would be important to give the admin the option of deselecting
> these modules also, in case he knows that modules are already translated or
> for some other reason.

I quite strongly disagree actually - we should make the system simple
to use which means providing less options.
Sync could perhaps be separated as may well not be used by many
deployments, however 'errors' & 'appadmin' are very much part of
Admin.
If there are already-translated parts then we pick these up in the
XLS/PO generated anyway...

> " We should probably have name_nice displayed rather than the actual names
> if this is to be used by non-core devs. "
> Yeah I l do that, we can display name_nice with the acutal name in bracket.

Fine :)

> " We should also build a dependency system, so that e.g. 'supply' gets
> selected when modules which use it are selected (i.e. Inv/Req). "
> Is there any particular way to detect these dependencies or should we hard
> code these in our code ?

I think ideally we should build a dependency system which could be
used by Translate initially & then potentially other components.
i.e. build the system into S3Model rather than internally within S3Translate
This could be something very simple like:
self.depends = []
or
self.depends = ["stats"]

> " Another example of dependencies:
> modules/s3survey should only be included if the Survey module is selected "
> I think this is being already done in the current code.
> https://github.com/nownikhil/eden/blob/master/modules/s3/s3translate.py#L278

I didn't see it take effect though: DRMP template had strings from
s3survey despite survey module not being enabled.
- probably because s3survey isn't inside modules/s3 but just modules/

F

[Nikhil Goyal]

Hi Fran,

Can you explain a little more about dependency system ? Is there anything that I need to look into ?

Nikhil

F

--
You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzkt%2B7ZBSwB9szou-BAo4jy8V%3DQaDvhYaJTn4po7-BwkPfwA%40mail.gmail.com.

[Fran Boon]

On 17 June 2013 18:30, Nikhil Goyal <nown...@gmail.com> wrote:
> Can you explain a little more about dependency system ? Is there anything
> that I need to look into ?

'Explain a little more' is very vague/unactionable, so hard for me to work with.

There currently is no dependency system, but there are dependencies.
I proposed a very simple way to mark these so that these can be made
use of in both Translate & potentially future things (like WebSetup).

Do you not understand the proposal?

I'm unclear on what exactly you want help with...

F

> https://groups.google.com/d/msgid/sahana-eden/CAAu86T52WSDCokZt9UqLNuqVaUb9LbX3qxsKNgrHLS38nDweqg%40mail.gmail.com.

[graeme]

Nikhil,

I am missing some of this conversation. To help me get up to speed can you first show me the code changes that you have done. Next what is the problem that you are trying to solve.

Graeme

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAPKzktL2Ez1aGbni62fU%3Ddw3MHZvF1Xck2ug6jTij%3D3QDiaVeA%40mail.gmail.com.

[Nikhil Goyal]

Hi Graeme,

https://github.com/nownikhil/eden/commit/fc6b428dc911a1e4c00038f2b79b3c8eba68d62d

These are the changes I made to the code :

* Active Modules are checked by default

* In case only one template is selected, strings from /private/templates/current_template are also selected.

Right now I am working on building a dependency system.

Nikhil

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAE8saEMcJDiQTCHRJphsZGy4wrbGU%2BnABPCMFF%3DPmGd3eBSKVw%40mail.gmail.com.

[graeme]

Nikhil,

Thanks for sending me the link to github, I'll look at that later today when I have time. You told me what you are doing, building a dependency system, but what is the problem this is going to solve? I need to understand what the problem is.

Graeme.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAAu86T4M2EoKncsxRYumWM7Juwvv8v02Hchnovj%3Dd4Hzug423Q%40mail.gmail.com.

[Nikhil Goyal]

Graeme,

We are doing this so that if a particular module is selected then all the modules using it should also be selected.

Fran : " We should also build a dependency system, so that e.g. 'supply' gets selected when modules which use it are selected (i.e. Inv/Req). "

Nikhil : " Is there any particular way to detect these dependencies or should we hard code these in our code ? "

Fran : I think ideally we should build a dependency system which could be used by Translate initially & then potentially other components.

i.e. build the system into S3Model rather than internally within S3Translate
This could be something very simple like:
self.depends = []
or
self.depends = ["stats"]

Nikhil

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAE8saEO%3D%2BfpKTjEDGmbCutKw-nDL0jmMffinAqLM9fm%2BXdp6uA%40mail.gmail.com.

[Nikhil Goyal]

Hi Fran & Graeme,

I tried the following methods for the dependency system and would like to know your thoughts.
(I read through an old mail by Dominic about accessing s3db variables :

http://groups.google.com/group/sahana-eden/tree/browse_frm/thread/b5793a6da58f131b/1e97f1f83cb63a0c?rnum=1&_done=/group/sahana-eden/browse_frm/thread/b5793a6da58f131b/1e97f1f83cb63a0c?tvc%3D1%26fwc%3D1%26&pli=1 )

Method 1 : We can make changes in s3model.py and add self.depends = { ‘inv’ : ‘supply’ , ‘req’ : ‘supply }. This Dictionary will be global and can be accessed by current.s3db.depends. This is one way to set the dependencies and access them when required.

Method 2 : Now, if we want to set the dependencies individually for each module in its s3db/<module>.py, then I am not able to access them using current.s3db[<table_name>]. depends. For eg, if we see file https://github.com/nownikhil/eden/blob/master/modules/s3db/inv.py we see inv.py has 4 instances of S3Model. Key being used is the TableName. eg :- current.s3db[‘inv_warehouse’]. So firstly, the table names do not follow a pattern for each module and secondly, I am not able to access the “.depends” list for the table.

 

Nikhil

[graeme]

Hi Nikhil,

I've just been looking at the code, which is something I've not done for a while. What we want to do is if we want to translate one module then we want to ensure that a dependent module is also translated. This can be a little confusing since when we get to the code level a module  such as inv will have multiple models which will each define multiple tables. So putting in into one (or all) s3model classes is going to be confusing and doesn't address your need. However, I don't like having a different file to add this data (s3model) since that becomes easy to forget so I would prefer to add this as close to the source as possible.

At this stage my suggestion would be to make it a global at the module level. So taking inv as an example you can add a global list: inv_depends = ["supply"] this will be similar to the way that the global inv_ship_status works. This will need to be exported and then in your translate controller you can find the value as models.inv.inv_depends

To me this look simple to implement and I trust meets your needs.

Graeme

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CAAu86T7nodLF-93g%3Do%2BUWNLcF-ZJ7YEdC%3D4Ya%3DEo%3DPCjbLXY8A%40mail.gmail.com.

[Fran Boon]

On 23 June 2013 08:34, graeme <foster...@gmail.com> wrote:
> What we want to do is if we want to translate one module then we want
> to ensure that a dependent module is also translated. This can be a little
> confusing since when we get to the code level a module such as inv will
> have multiple models which will each define multiple tables. So putting in
> into one (or all) s3model classes is going to be confusing and doesn't
> address your need. However, I don't like having a different file to add this
> data (s3model) since that becomes easy to forget so I would prefer to add
> this as close to the source as possible.

+1 - exactly my steer too.

> At this stage my suggestion would be to make it a global at the module
> level. So taking inv as an example you can add a global list: inv_depends =
> ["supply"] this will be similar to the way that the global inv_ship_status
> works. This will need to be exported and then in your translate controller
> you can find the value as models.inv.inv_depends

That works for me :)

F

[Pat Tressel]

I'm confused about two things here:

At this stage my suggestion would be to make it a global at the module level. So taking inv as an example you can add a global list: inv_depends = ["supply"] this will be similar to the way that the global inv_ship_status works. This will need to be exported and then in your translate controller you can find the value as models.inv.inv_depends

Is "models" s3db?

I'm guessing you don't mean "add a global list: inv_depends" in the Python sense of "global", i.e. something declared in the global namespace because you then say "models.inv.inv_depends" which means inv_depends is defined in models.inv (or in s3db.inv?) (and even models / s3db is probably gotten from current, and not global either).  Besides, we started getting rid of globals some while back.  So...where are you thinking of declaring these?  At file scope in modules/s3db/xxx.py?  Or as class constants in the S3Model subclasses in each modules/s3db/xxx.py?

-- Pat

[graeme]

The example I gave was to add it to modules/s3db/inv.py, similar in working to inv_ship_status which is also in that file.

Graeme.

--

You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CABT-%2B2pLwiptdrg77xb29AD9baEVzQ7BEeBXCEk6L8V5uJXGiA%40mail.gmail.com.

[Pat Tressel]

The example I gave was to add it to modules/s3db/inv.py, similar in working to inv_ship_status which is also in that file.

Ok, so that's at file scope, and ends up directly in s3db, so inv_depends would be used as:

s3db.inv_depends

Is that what you want, not s3db.inv.inv_depends or s3db.inv.depends?  If it were me, and there were already an s3db.inv, then I'd prefer a common name for the attribute like "depends", not a different name per module.  But I see a lot of flattened inv_xxx already, and only one instance of any s3db.<module>.<something> so maybe there is already a convention for a flattened namespace.

-- Pat

[graeme]

I tested this in the debugger before posting, models.inv.inv_depends works.

I was following the approach to naming, which is currently in use.

Graeme.

--

You received this message because you are subscribed to the Google Groups "Sahana-Eden" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sahana-eden...@googlegroups.com.
To post to this group, send email to sahan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sahana-eden/CABT-%2B2qJxqxb%2B-5%2BcUVeUBEpQcXqe7MSj_oNZFXTGUVdQDUfuA%40mail.gmail.com.

[Pat Tressel]

Graeme -

I tested this in the debugger before posting, models.inv.inv_depends works.

I was following the approach to naming, which is currently in use.

Again, what is "models"?  I can't find any examples -- can you point some out?  And some of "models.inv.<something>"?

-- Pat