Fwd: CouchDB/CouchApps Database Group

[Gregory Kelleher]

-- Greg

Begin forwarded message:

From: Gregory Kelleher <gr...@kelleher.cc>
Date: November 1, 2011 1:24:09 AM EDT
To: Manor Lev-tov <manor...@gmail.com>
Cc: Surajit Nundy <nun...@gmail.com>, "rd...@cdc.gov" <rd...@cdc.gov>, MattAdams <matt....@radicaldynamic.com>, Saptarshi Purkayastha <sun...@gmail.com>, Daniel Pepper <daniel...@gmail.com>
Subject: Re: CouchDB/CouchApps Database Group

Manor,

Good analysis.  Clearly it's difficult to estimate the relative effort when one approach is so foreign.  Two differences seem especially important: the learning curve for developers, and the rewrite of the.business logic.

 Saptarshi should be able to give some insight on the learning curve. We can't expect others to learn as quickly but he should brave a sense for the steepness of the learning curve and relative easy of use of the tools.

I find the concern over loosing the investment in business logic to be more nebulous. We certainly don,t want to spend substantial effort recreating the existing logic, unless it improves the end product.  But what is the business logic? Does a paper-based system have a similar layer? Could it be that the better fit of the JSON document reduces the need for business logic?  How much of OpenMRS business logic is reassembly instructions or interpretation rules for atomized records?  And if there is such logic required for couchdb, can it be converted?

-- Greg

On Oct 31, 2011, at 11:41 PM, Manor Lev-tov <manor...@gmail.com> wrote:

All,
I've been looking through the OpenMRS documentation/code and CouchDB/CouchApps documentation a bit.  I am still feeling my way around Coach, but I really don't think any sort of middle path is advisable, as it would negate a lot of the benefits of using this approach.  This means that we essentially have the three choices below going forward, with the major pros and cons.
1. Use OpenMRS. 
Pros:
-We know what we are dealing with.
-We currently have developers with the skills that we need to work on this path.
Cons:
-We will have to address or live with the known issues/limitations of OpenMRS.  As per Shuro these are (generally):
1. Long development learning curve with questionable technology investments (Groovy, etc)
2. UI layer obsolescence and inefficiencies (Server side web page generation)
3. Backend scalability issues.
4. Not designed to be a (responsive) point-of-care application

2. Use CouchDB/CouchApps. 
Pros:
-The document oriented approach is better suited for concept/observation data than the relational approach.
-Down the road, the skill set of developers who can continue/maintain the project will not have to be as high.  We could use more developers who are familiar only with JavaScript.  However, this is negated somewhat by the need to query data using map reduce views.
-More scalability.  This will not be of direct benefit to JSS, but will add value to project by making it more attractive to far larger organizations.
Cons:
-We will essentially be scrapping all of the work done in the OpenMRS project.  This means we will have to write an EMR system from the bottom up and solve a lot of the problems that have already been solved in OpenMRS: business logic, encryption, etc.  This is a lot of work (will take longer to deploy) and, as with rewriting any system, there is a lot of risk involved.
-We will have to do a fair amount of back end work upfront, so we may loose momentum and some developers might start to loose interest.
-This approach hasn't really been done before, so we don't know what other potential downsides there are.

3. Split up into two teams, one that works on the OpenMRS approach and one that works on the CouchApps approach.
Pros: 
We will be able to build the CouchApps approach with very little risk - if, for some reason we decide it doesn't suite our needs, we just use the OpenMRS approach.
Cons:
We will be doing duplicate work and a lot of work will not be used.  This is very disheartening, especially to volunteers.
The central idea behind this method is that we are attempting to build a CouchApps system, but are using OpenMRS as a fallback.  This means that we will either have to wait to deploy until the CouchApps approach is ready or we deploy the OpenMRS system and migrate the data to the CouchApps system when it's ready (this is not a trivial task).

The main takeaway is that using a CouchApps requires us to essentially change our path and build an EMR system from scratch, rather than simply building the front end components on top of an existing system.  There are some major potential advantages to it - it could be ground breaking, but there is a lot of work and risk involved.

-Manor

On Tue, Nov 1, 2011 at 9:08 AM, Manor Lev-tov <manor...@gmail.com> wrote:

Sure.  Saptarshi, sorry for sending you the email so many times - sent to the wrong group.
Hi Saptarshi,
The main concern that was raised during today's phone call was the potential of loosing the business knowledge that already exists in OpenMRS if we go with CouchDB.  Roger (I believe) brought up the point that OpenMRS has already been designed around the health care problem space - it knows about visits/encounters/observations and how they all relate, it has kept up with changing medical terminology, etc.  If we build CouchApps to replace OpenMRS, we will loose all of that work and have to redo a large part of it.  I am very weary of having to rewrite a large amount of business logic, as it is a lot of work and very error prone.

Could you give me a sense of the amount of business logic in OpenMRS between the REST API and the MySQL database and what exactly that business logic is?  Also, I just checked out the OpenMRS code, so if you can point me to the best places in the code to look, I will dig around a bit.
Your input on this is most valuable.
Thanks,
Manor

On Tue, Nov 1, 2011 at 8:50 AM, Surajit Nundy <nun...@gmail.com> wrote:

Hi Manor:

This is the list of people who were on the call yesterday and with whom it would be good to discuss database issues this week. Could you send yesterday's message to all of us here?

Thanks
Shuro

Sent from my mobile device

On 31-Oct-2011, at 8:55 AM, Surajit Nundy <nun...@gmail.com> wrote:

> Hi everyone:
>
> Saptarshi, too bad we couldn't have you on this call.  We have decided to create this group to decide a granular path to a potential CouchDB migration by this week.
>
> Shuro

[Gregory Kelleher]

-- Greg

Begin forwarded message:

From: Gregory Kelleher <gr...@kelleher.cc>
Date: November 1, 2011 5:37:31 AM EDT
To: Manor Lev-tov <manor...@gmail.com>
Cc: Surajit Nundy <nun...@gmail.com>, "rd...@cdc.gov" <rd...@cdc.gov>, MattAdams <matt....@radicaldynamic.com>, Saptarshi Purkayastha <sun...@gmail.com>, Daniel Pepper <daniel...@gmail.com>
Subject: Re: CouchDB/CouchApps Database Group

Manor,

Let,s take my profession of ignorance seriously and see if we can discover what the language of 'business rules', 'concepts', and 'relationships' mean when applied in this altered representation.  Surely we won't be surprised that 'relationships' has a different sense in a non-relational world and the term 'document' takes on different significance.

Suppose I say that I don't propose that we use CouchDB to model OpenMRS but instead we use it to model the paper-based system. Surely I need to have fixed referents for people, places and some things, but why do I need to distinguish visits and encounters if we have all the documents?

Maybe it would help to take a specific examples of clinically useful reports that go beyond just reproducing the original documents.  AMPATH is trying to implement patient summaries for return visits.   If we start with that and the JSS physiician list of analytical questions and compare the processes for answering those needs under the 2 approaches we might discover the necessary underpinnings.

This approach, pushed to reasonable limits for both systems, could illuminate for us the differences in concept definition, form development, report creation.  If we added the tracking for the screener and a tablet form for a mobile worker, we'd have a good base for due diligence.  I'd bet that the volunteers would understand and be willing to participate in this test even though they risked throwing out their code.

Wouldn't this be reportable and likely grant-worthy?  Or am I taking too much license under my role of ignorant customer?

-- Greg

On Nov 1, 2011, at 4:12 AM, Manor Lev-tov <manor...@gmail.com> wrote:

In terms of the business logic, it's very likely that the implementing an EMR system will be much simpler using CouchDB, but we will still need to re-examine the business rules, concepts, and relationships.  There are likely some nuances that were discovered when OpenMRS was developed that we will have to reconsider (and hopefully not miss).  Admittedly, I don't have a lot of experience in this industry, but this seems to me like it will be a fairly significant project on its own.  The question that we need to answer is: Are the benefits of switching to a CouchDB approach worth the added work and inherent risk?

[Friedman, Roger (CDC/CGH/DGHA) (CTR)]

This is sort of a heady topic to take up before my first cup of coffee, but here goes.

 

I think Greg is totally off base with his view of medical records.  He acts as if the doctor is the only client of the system and the only use of the medical record is to remind the doctor of what has come before.  But that is far from true.  The EMR is also a tracking system for service delivery, counting all the activities taking place in the facility; it is a quality control system, allowing the measurement of outcomes of patients with similar diagnoses and comparison with targets; it is an epidemiological system, tracking the outbreak and spread of disease; it is a patient protection system, flagging contraindications or allergies or medication conflicts in prescribing; it is a physician management system, identifying cases where the doctors did and did not adhere to treatment standards, and hopefully reminding them what the right thing to do is.  Physicians have a lot of things to remember -- in one study, one group of physicians used an EMR system with alerts and the other used the same system but without alerts; after three months, there were large, signficant differences in the administration of tests and prescribing of medications in the alerts; then the alerts were taken away and the differences disappeared.  So it is not lack of knowledge or lack of desire to implement best practices that prevents their implementation, it is the fact that there is so much medical knowledge that even the best physicians can't know it all, and the EMR should give them an assist.

 

In the 40 year history of medical informatics, a lot of work has been done to achieve a somewhat uniform representation of health care activities.  A lot of this work has focused on standardizing vocabulary, identifying process steps and disease stages, providing a structure for medical records both paper and electronic.  This has allowed the comparison of medical records from different facilities and promoted interoperability between systems.  Not that there hasn't been work done in natural language processing, just that standardization had to come first before the problem became at all tractable.  At present, the interoperability of systems is limited -- it is possible to exchange data elements.  A current goal of medical informatics is allow machine understanding of the data elements at a higher level -- to correlate observations, to diagnose, to propose treatments, to identify malpractice automatically.  But all of these efforts are limited to very small areas of practice, and attempts to extend them to practical utility have gone very slowly.  Why is this?  It is because of two characteristics of medical information.  First, the corpus of medical information is growing and changing.  The US National Library of Medicine has 11 people whose only job is to keep up with new medical terminology.  Second, the level of detail needed by medical practitioners varies immensely.  A general practitioner can identify a skin cancer; an oncologist can identify what type of tissue has become cancerous, the stages in its lifecycle that have been affected by the mutations, and its treatability by different modalities; today's cancer researchers and tomorrow's oncologists will know the particular mutations which have taken place and the vulnerabilities of each along with the consequences of targeting each on the rest of the body.

 

Greg seems to view the medical record as a collection of forms.  But actually, the forms are only an organization tool for the underlying observations.  This becomes particularly clear when you look at more specialized visits.  Consider, for example, an OPD visit by an adult whose main complaint is diarrhea; an antenatal care visit; an HIV care visit; and an antenatal care visit by an HIV-positive mother.  All four will have their vital signs taken; any new patient will have a history taken; all four will undergo a physical exam but to a different extent; each will be asked screening questions relevant to their condition(s) and answers which could indicate abnormal conditions are followed up with testing; the last three will undergo progress checks to assure that their conditions are following the normal course.  If the four visits were reduced to forms, there would be a lot of overlapping questions; the forms are only there to assure that the right questions are asked.  And the questions may have multiple parallel answers (WARM, RED) or structured answers (SEVERE allergy to PENECILLIN) or semi-structured answers (MODERATE allergy to OTHER: Peanuts) or answers that contain various dimensions of detail (WOUND [LACERATION], [EXTREMITY], [HAND, LEFT, PALMAR]) .  In this regard, it is important to note that medicine makes a lot of use of negative inference -- not just the consequences of a negative test, but the fact that a particular test or procedure was not done is a sign that it was contraindicated or that the patient's presentation did not support it.

 

Manor asks what business logic OpenMRS implements.  This is not quite what I meant by my remarks.  OpenMRS parses the subject matter in standard ways.   For example, the patient, patient address, patient relations, provider identification, encounter, observation and order objects correspond to message segments one finds in HL7.  Many of the objects (including problem lists, allergy lists and programs) come from previous work on structuring medical records.  This provides a fairly high level of confidence that there are not areas of medical practice which cannot be represented in the data model.  The only "business logic" that is implemented directly by OpenMRS is the cardinality relationships between the objects.  The more businessy logic (e.g. a person is considered cured of tuberculosis if s/he has a two negative tests at least a month apart without an intervening positive test) is found in the modules (e.g. tuberculosis module).  To take advantage of this logic under the current design, these modules would require rework to expose web services beyond the basic CRUD model of OpenMRS.  Logic rules and cohort definitions are another source of business rules which will probably also require the creation of additional web service functionality to access.

 

Greg seems to think that the OpenMRS data model is too complex.  But given the characteristics of medical data, the KAV model of OpenMRS is almost obligatory.  Maintenance of indexes indeed requires overhead, but given the nature of a typical search (an observation of a given concept, perhaps connected to an encounter of a particular type, perhaps related to a particular patient or set of patients), the searches can be done on the indexes alone.  Contrast that to Couch, with the document corresponding to an encounter.  The JSON of the document has to be parsed and the nodes representing observations have to be scanned for occurrences of the concept in question.  This just has to be substantially slower.

 

I have to take issue with the "pros and cons." 

·         As for the "long development learning curve," as one who has been making the transition from .NET to OpenMRS's J2EE, I agree that there are complexities.  However, it seems that most developers will be working at the UI level and accessing the OpenMRS DB via REST calls.  This will require recreation of validation rules and input widgets but not contact with OpenMRS.  

o   Where there is a need to create new modules (e.g. lab) or add services to existing modules (e.g. TB), there will be a need for developers who understand the way modules interface, the use of concept maps and global properties to provide parameters to modules, etc. 

o   Depending on whether the maintenance of metadata is done inside or outside OpenMRS, there may be a need to develop additional OpenMRS administration screens. 

o   I am concerned that the HTML5/REST interaction could be slow because every object will have to be completely retrieved before it can be sent out over the REST interface.  Consideration should be given to having a module exposing additional RESTful web services that can take advantage of Hibernate lazy loading by interacting at the API level.

·         As for "questionable technology investments," I think you need to focus on what's there that you need rather than that which you don't.  The entire scope of OpenMRS is available without knowing Groovy.  It's just that some people like Groovy, and so we see some features designed to use it -- but not to the exclusion of other means.  Similarly, OpenMRS supports Arden Syntax for writing logic rules.  This query language is not widely used so far as I know, but someone in the funding stream was interested in it.  I think in any open project you are going to find features that interest only a subset of users/developers but nonetheless get implemented. 

·         As for UI obsolescence, I think JSPs, ajax and jquery will be around for a long time; I fail to see how it's any barrier to development.  But in any event it's irrelevant since you don't intend to use that part of OpenMRS anyway.

·         As for backend scalability issues, I don't see any volume estimates for JSS.  Nor do I think that adoption at other facilities need cause scalability issues; at a certain point, one sets up a new instance and communicates via HL7 where necessary; all this capability is in work.  I think there is a lot of confusion of MySQL issues with OpenMRS issues.  OpenMRS will run on Postgres and SQL Server and I believe Oracle, all of which allow clustering (and Postgres at least is faster indexing).

·         As for not being designed as a point of care application, that just applies to the default webapp.  A number of places are using OpenMRS for direct entry, and a number of tools have been created to allow direct entry and bypass the default webapp interface.

·         As for the document approach being better for concept/observation data, I think that's wrong as described in the previous full paragraph.

·         As for CouchDB views, they are like rebuilding an index every time you want to do a query and show the weakness of the document model.

·         I don't see how anyone can say that CouchDB is more scalable under the constraint of a need for a singular master view representing the clinic.  The improvement in scalability comes from replication under the assumption that no node needs all the data at the same time.  I find it astonishing to hear people say they don't like OpenMRS because it uses an HL7 input queue for some types of form entry so that the DB is not instantly up to date, while saying that replication with daily synchronization would be just fine using Couch.

·         Those who advocate Couch have an obligation to show a working application of similar complexity to that of the JSS EMR.  In particular, it is important to see how fast Couch is in a metadata-driven application.  Also, we need internationalization and a security model at least as good as that of OpenMRS (and there have been a lot of calls for even more granularity within the security model).

 

I agree that these approaches are so different that trying to combine them would not allow either to work to full advantage; they need to be considered as separate tracks.  I think we can get a working, reliable EMR for JSS faster with OpenMRS than Couch.

 

Saludos, Roger

[david martin]

Dear list colleagues,
           I have hastily read the many informed comments in the list.
           There are far too many aspects and views to agree or refute them all in a short time.

I particularly was impressed by Roger Friedman's cri de coeur.  I can feel his pain of the .NET to OpenMRS's  J2EE transition and the prospect of yet another upheaval as many aspects of the user data driven paradigm approaches.

It seems to me that Roger in particular, and OpenMRS and HL7 in general are in a particularly good position to greatly benefit not only raxa-JSS but the whole issue of patient data driven medical records systems.

This concrete and simple task that would do the most in enabling the whole world to benefit from the 40 years of medical informatics could be done under the direction of persons such as Roger wielding the sharp blade of Occams Razor.

Occams razor applied to this Gordian knot says, "entities must not be multiplied beyond necessity" (entia non sunt multiplicanda praeter necessitatem). This could not be more apt in this case.

    Yes they say, but "what does that have to do with the price of tea?", or other similar irrelevancy.

The simple task, that has no real cost other than the motivation and will of qualified volunteers, is to tear out the essence of the 40 years of medical informatics and to codify it without the obscuring  structures.

  Even if the end result is only enables a clear holistic view to be taken by the various concerned decision makers, this task must be done, and now. It is not only a means to an end, It is an end to an end.

It signals an and to the bickering about the merits of various monstrous edifices and reduces them to a meaningful form understandable by all participants and not just those with a grip of the mysteries of todays commercial flavor of framework.

I have proposed that the database expert  pursues the following action list.

1. Obtain text dump of all required OpenMRS schemas and all diagrams that depict the relationships between entities.
1a. Obtain flat lists of all  required HL7 data exchange formats
2  Flatten as much as possible the multi-dimensional structurea.
3  Determine areas that are not immediately amenable to flattening.
4 Get rid of redundant fields and concepts.
5  Review the resulting data and decide how best to represent it in JSON format bearing in mind the relative ease of  accessing the different sub-structures within the overall document structure. some things are accessed knowing a key, some things have to be iterated through such as arrays with no key and some are iterated through because you do not know if the key exists.
6 Review all fields for use and ease of use of multi-language Regular expression (Regex) searches.
7 Present scenarios to access all data in all required user views of the resulting data.

All help, resource and guidance is requested and required to be given by all participants, both personal and corporate in achieving  this goal.

The rapid end result of this will enable real progress to be made in all aspects of a data driven system.

When the whole of this data has been made available, the depth to which it is pursued is an informed decision made looking at a readable structure understandable by anyone who reads the plain text that is before them. All other aspects of this venture are intimately  dependent on an understanding of this.

It is a great opportunity, and to quote the Bard,

Brutus:
There is a tide in the affairs of men.
Which, taken at the flood, leads on to fortune;
Omitted, all the voyage of their life
Is bound in shallows and in miseries.
On such a full sea are we now afloat,
And we must take the current when it serves,
Or lose our ventures.

Julius Caesar Act 4, scene 3, 218–224


Bon voyage!,

David M. W. Martin (davidoccam)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

--
You received this message because you are subscribed to the Google Groups "Raxa JSS EMR Database" group.
To post to this group, send email to raxa-jss-e...@googlegroups.com.
To unsubscribe from this group, send email to raxa-jss-emr-dat...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/raxa-jss-emr-database?hl=en.