Just capturing some thoughts...
1 view
Skip to first unread message
Derek O'Connell
Sep 17, 2014, 8:24:55 AM9/17/14
to aush...@googlegroups.com
I have other priorities so can't contribute much atm but wanted to
capture some thoughts for when I have do have time. Sorry for the length
of this post. I was tempted to split it up but most of this is
interrelated, which I hope becomes clear by the end of it...
Custom Forms
One of Au's objectives is flexibility for users and one specific feature discussed is allowing some customization of form contents & layout. Dj doesn't (easily/officially) support dynamic forms so the current thinking is to provide a mechanism to create a form spec that is then used to build Dj models before Au is started. This seems do-able but I have some reservations about it being practical even for restrictive use, ie, in addition to pre-spec'd Au forms. A form-designer utility would be need and a way to keep models sync'ed with a form-spec or to extract a form-spec from existing models. This sync'ing requirement seems a little fragile to me in a way similar to migrations (well maybe to pre-1.7 assuming 1.7 is a big improvement). I suspect creating even a basic form-designer is no small feat so I did a quick search for existing solutions. I have yet to fully evaluate these options but here's some quick info and links...
Orbeon Forms: http://www.orbeon.com/
Very mature with open source edition. Uses XForms standards so...
Introduction To XForms: http://www.w3ctutorial.com/xforms-tutorial/xforms_intro
XForms Essentials (on-line book): http://xformsinstitute.com/essentials/browse/
Maqetta: http://maqetta.org/
WYSIWYG visual authoring of HTML5 user interfaces, originally from Dojo creators, mature but no longer supported. Might be easier to start here and remove functionality/ re-skin rather than build one from the ground up. Easy to install, give it a try.
Django app's (not checked): https://www.djangopackages.com/grids/g/form-builder/
Even if this type of custom-forms can be provided for restrictive use I think there's a danger of feature creep in the direction of creating a WYSIWYG Django App Designer (I foresee the obvious question: if it's possible for one part of Au then why not all of it?). I'm not arguing for/against such a thing, just pointing out that it's a major effort and would probably need a different approach.
Consideration of custom-forms also leads to the question of best choice of back-end database to support more flexibility. This question is mostly side-stepped by *not* attempting dynamic run-time form creation but it does clarify the importance of data...
Back-end Database
We have already moved to PostgreSQL for better multi-user support and also because it makes sense to develop & test using the same DB back-end as used for deployment. For an application like Au that deals with personal (and possibly critical) medical data it also needs to be emphasized that "data is king" and any/all design decisions should first be evaluated with this in mind. That alone rules out other DB options that might offer more flexibility but at the expense of data safety/ease-of-management.
Even with licence clauses that absolve developers from any responsibility, I still believe the integrity of data is of prime importance even if only to give potential users confidence that Au is a viable option. The key to this is of course is comprehensive testing before rubber stamping any release as being suitable for for general use...
Releases, Continuous Integration
Easwar is aiming for a first release but that is still some way off depending on your criteria. I think a continuous-integration (CI) way of doing things might be a better fit than trying to get a feature freeze. That requires tests and IMHO it also requires someone dedicated specifically to creating those test and setting up the CI side of things. That task would be made much easier with a development freeze to perform some clean-up & specific improvements (basically anything that reduces the amount of code. For an example see what I did in the PCS pane).
Related to data integrity & testing: we have started using IPython notebooks for pulling in external data such as Drugbank, ICD10-PCS, etc. For this specific use I think the IPy notebook format makes good sense and that they should ship with Au. The reason is that end-users may be tempted to update this type of data themselves and it is better if they are guided through the process and be able to spot problems before Au data becomes tainted, which might lead to Au itself becoming unstable. Code in notebooks would first need exporting for automated tests but this should only be for the duration of such tests to avoid the possibility of having multiple copies of the code. This assumes notebook code has sensible defaults and can run without further manual input. TBD: in-notebook testing?
Issues related to data integrity and testing is why I would prefer any releases to be tagged as "PREVIEW" until such time we can demonstrate via tests that Au is robust in this regard. Which leads to the format of releases...
Distro-type "official" Releases
Without imposing restrictions, I think "official" releases should come in the form of a distro that pre-packages not only Au and current external data but also any other requirements such as a form-designer, viewers for medical data (eg, MMR images) and maybe even "certified" versions of various browsers (may as well throw in IPython ;-) ). In other words, a complete tool-kit/eco-system to support Au out-of-the-box and which offers an implicit guarantee of end-user experience. This also makes sense for sites that have no internet access or slow speeds/ high costs.
System Design
Not specifically related to the above but best factored-in earlier than later...
We made some attempts at creating a system diagram for Au which IMO is needed for future versions of Au if flexibility is a key feature, ie, it's the sort of thing that has to be incorporated from the ground up. What flexibility: separation of data/UI to support independently managed DB's, a variety of front-end's, more localized customization, live integration with external data sources, integration with other systems (eg, billing), etc. While working on an unrelated project I came across "Kolab", discovered it's history (more interesting than it might first seem) and a video presentation that provides a good example of the importance and benefits of a good overall system design:
https://www.youtube.com/watch?v=pVU0y-SKXnE
http://kolab.org/
https://kolabsys.com/
Respect for reading this far :-)
PS: this is not some sort of "goodbye!" post ;-) My contribution has been minor so far despite long stretches getting familiar with XML data sources. Easwar is doing an amazing one-man-band job of progressing Au and I still hope to make a more significant contribution in the future when I have more time.
-D
Custom Forms
One of Au's objectives is flexibility for users and one specific feature discussed is allowing some customization of form contents & layout. Dj doesn't (easily/officially) support dynamic forms so the current thinking is to provide a mechanism to create a form spec that is then used to build Dj models before Au is started. This seems do-able but I have some reservations about it being practical even for restrictive use, ie, in addition to pre-spec'd Au forms. A form-designer utility would be need and a way to keep models sync'ed with a form-spec or to extract a form-spec from existing models. This sync'ing requirement seems a little fragile to me in a way similar to migrations (well maybe to pre-1.7 assuming 1.7 is a big improvement). I suspect creating even a basic form-designer is no small feat so I did a quick search for existing solutions. I have yet to fully evaluate these options but here's some quick info and links...
Orbeon Forms: http://www.orbeon.com/
Very mature with open source edition. Uses XForms standards so...
Introduction To XForms: http://www.w3ctutorial.com/xforms-tutorial/xforms_intro
XForms Essentials (on-line book): http://xformsinstitute.com/essentials/browse/
Maqetta: http://maqetta.org/
WYSIWYG visual authoring of HTML5 user interfaces, originally from Dojo creators, mature but no longer supported. Might be easier to start here and remove functionality/ re-skin rather than build one from the ground up. Easy to install, give it a try.
Django app's (not checked): https://www.djangopackages.com/grids/g/form-builder/
Even if this type of custom-forms can be provided for restrictive use I think there's a danger of feature creep in the direction of creating a WYSIWYG Django App Designer (I foresee the obvious question: if it's possible for one part of Au then why not all of it?). I'm not arguing for/against such a thing, just pointing out that it's a major effort and would probably need a different approach.
Consideration of custom-forms also leads to the question of best choice of back-end database to support more flexibility. This question is mostly side-stepped by *not* attempting dynamic run-time form creation but it does clarify the importance of data...
Back-end Database
We have already moved to PostgreSQL for better multi-user support and also because it makes sense to develop & test using the same DB back-end as used for deployment. For an application like Au that deals with personal (and possibly critical) medical data it also needs to be emphasized that "data is king" and any/all design decisions should first be evaluated with this in mind. That alone rules out other DB options that might offer more flexibility but at the expense of data safety/ease-of-management.
Even with licence clauses that absolve developers from any responsibility, I still believe the integrity of data is of prime importance even if only to give potential users confidence that Au is a viable option. The key to this is of course is comprehensive testing before rubber stamping any release as being suitable for for general use...
Releases, Continuous Integration
Easwar is aiming for a first release but that is still some way off depending on your criteria. I think a continuous-integration (CI) way of doing things might be a better fit than trying to get a feature freeze. That requires tests and IMHO it also requires someone dedicated specifically to creating those test and setting up the CI side of things. That task would be made much easier with a development freeze to perform some clean-up & specific improvements (basically anything that reduces the amount of code. For an example see what I did in the PCS pane).
Related to data integrity & testing: we have started using IPython notebooks for pulling in external data such as Drugbank, ICD10-PCS, etc. For this specific use I think the IPy notebook format makes good sense and that they should ship with Au. The reason is that end-users may be tempted to update this type of data themselves and it is better if they are guided through the process and be able to spot problems before Au data becomes tainted, which might lead to Au itself becoming unstable. Code in notebooks would first need exporting for automated tests but this should only be for the duration of such tests to avoid the possibility of having multiple copies of the code. This assumes notebook code has sensible defaults and can run without further manual input. TBD: in-notebook testing?
Issues related to data integrity and testing is why I would prefer any releases to be tagged as "PREVIEW" until such time we can demonstrate via tests that Au is robust in this regard. Which leads to the format of releases...
Distro-type "official" Releases
Without imposing restrictions, I think "official" releases should come in the form of a distro that pre-packages not only Au and current external data but also any other requirements such as a form-designer, viewers for medical data (eg, MMR images) and maybe even "certified" versions of various browsers (may as well throw in IPython ;-) ). In other words, a complete tool-kit/eco-system to support Au out-of-the-box and which offers an implicit guarantee of end-user experience. This also makes sense for sites that have no internet access or slow speeds/ high costs.
System Design
Not specifically related to the above but best factored-in earlier than later...
We made some attempts at creating a system diagram for Au which IMO is needed for future versions of Au if flexibility is a key feature, ie, it's the sort of thing that has to be incorporated from the ground up. What flexibility: separation of data/UI to support independently managed DB's, a variety of front-end's, more localized customization, live integration with external data sources, integration with other systems (eg, billing), etc. While working on an unrelated project I came across "Kolab", discovered it's history (more interesting than it might first seem) and a video presentation that provides a good example of the importance and benefits of a good overall system design:
https://www.youtube.com/watch?v=pVU0y-SKXnE
http://kolab.org/
https://kolabsys.com/
Respect for reading this far :-)
PS: this is not some sort of "goodbye!" post ;-) My contribution has been minor so far despite long stretches getting familiar with XML data sources. Easwar is doing an amazing one-man-band job of progressing Au and I still hope to make a more significant contribution in the future when I have more time.
-D
Reply all
Reply to author
Forward
0 new messages