Add ability to store complex obs in the database

[Lluis Martinez]

Hi

Regarding TRUNK-1928 I checked Victor's solution (https://github.com/ehasalud/openmrs-module-complexdatadb) and it seems easy to integrate in core. 

However, I wonder if we really need additional handlers just to decide the storage location. Wouldn't be enough with a global property to indicate this? 

e.g. COMPLEX_OBS_STORAGE_LOCATION = 1-File system, 2-Database, 3-JCR ... 

Cheers

[Burke Mamlin]

I believe Victor was providing some out-of-the-box generic handlers that could be extended as needed.  In the end, where the payloads are stored is up to the particular handler (e.g., one type of image handler might store in the file system, another in its own database table, and yet another might store & fetch images from a distant PACS system).

If you wanted to store scanned documents, electrocardiograms, and sound recordings by creating three nifty new custom datatypes (i.e., complex observations), I could see where all three could extend a generic binary file handler and focus just on any additional details relevant to their datatype (e.g., the electrocardiogram handler might render its HTML view as an embedded PDF where the sound recording handler might embed some HTML5 code to present an audio player).

Cheers,

-Burke

--
OpenMRS Developers: http://om.rs/dev
Post: d...@openmrs.org | Unsubscribe: dev+uns...@openmrs.org
Manage your OpenMRS subscriptions: http://om.rs/id
 
*** THIS GROUP IS MOVING to OpenMRS Talk effective 10 April 2015 at 19:00 UTC. Please visit https://talk.openmrs.org/t/openmrs-developers-group-changes-2015-04-10/1508 for details and to make necessary changes.

[Darius Jazayeri]

Hi Lluis,

To be explicit: there is no global answer to "where do you want to store your complex obs". It needs to depend on the handler.

It would be okay to have a GP that controls whether a specific handler stores in the DB vs filesystem. (I'm not sure how much it buys us, though. This doesn't seem like it would decrease complexity or lines-of-code significantly. Having an abstract base class with two concrete implementations could be just as good.)

-Darius

PS- Maybe Burke already covered all this. It's too early to be reading email... :-)

*** THIS GROUP WILL BE MOVED to OpenMRS Talk effective 10 April 2015 at 19:00 UTC. Please visit https://talk.openmrs.org/t/openmrs-developers-group-changes-2015-04-10/1508 for details and to make necessary changes.

[Lluis Martinez]

Hi

The main goal of the ticket is to allow Sync module to share complex obs (not possible in general if the storage is file system).

Currently all handlers extend an abstract one that saves the obs in the file system. If the requirement is to allow each handler to choose its storage location, I devise too options:

- One global property per handler

- A new metadata entity (complex obs handler)

I'll go with the simplest one for the moment. In the future we could include an open source CMS as the central repository (Alfresco?).

Cheers

To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@openmrs.org.

[Michael Seaton]

Hi Lluis,

Is that true that we can't sync complex obs if they are stored on the file system?  I wouldn't think that would matter.

Cheers,
Mike

[Lluis Martinez]

Hi Michael,

I'm not sure, I read it here https://issues.openmrs.org/browse/TRUNK-1928

Cheers

[Burke Mamlin]

Sync support may be another addition needed for the complex obs handler interface.

I know that we are missing bulk operations.  Without them, the only way to fetch 100,000 complex obs values is to loop through them one at a time.  Adding methods for bulk operations woudl allow complex obs handlers to perform these operations with much higher efficiency.

If it was widely needed, then we could consider adding sync-releated methods to the handler interface.  Since the handler interface is in core and sync in a module (we try to avoid coding against specific modules in core) and I'm not sure how many sites are using sync, it might make more sense to refactor specific complex obs handlers to be "sync aware" or "sync capable."  You won't be able to accomplish generically across all complex observations, since a handler can store its payload in the file system, in an external system, or even through a web service integration.

-Burke

[Michael Seaton]

I would think that if we had a generic interface or abstract Handler class, and this had a method for "byte[] getRawData()" that needed to be implemented, then this might be enough, at least for most use cases.

Basically, on saving an Obs, sync would check whether or not it is complex, and the type of handler it has.  If it was a handler of this nature, it would retrieve and encode the data in the sync payload, transfer it to the remote server, play this back, and let the handler take care of storing the data in the appropriate place.

I don't think this needs to be anything explicitly tied to Sync.  Just needs to be something that Sync can identify as using a Handler that gives access to retrieving and storing the raw data representation.

If implemented like this, whether we had a Handler that stored images in the DB or in the filesystem, either could meet the needs of such a requirement for Sync.

Mike