RESTWS release timing/content

[Friedman, Roger (CDC/CGH/DGHA) (CTR)]

     We have not had an up-to-date version of RESTWS in the module repo for quite a while, and there have been a lot of important changes.  Darius would like to get something into the repo, and I agree, but I don't think the current version is sufficiently complete and correct to just move on.  I have four reasons for this: (1) we don't have all the 1.8 data model in REST; (2) we have some known anomalies to fix; (3) we haven't had much experience with the subclass model; and (4) there will be a competition for resources due to the need for a version that supports the 1.9 data model.

 

    If I understand Darius correctly, he never thought version 1.0 would be a complete exposure of the data model, but only a sufficient proof of concept to assure that the interface would not change drastically.  But I believe everyone who has tried to use RESTWS has found a data object they needed which was not exposed.  If a needed object is not exposed, then the user can't really put RESTWS to the test.  I don't really care what the 1.8-compatible and the 1.9-compatible releases are called, I just want a clear path to a fully capable 1.8-compatible version.  I don't want issues (1)-(3) to be lost as we pursue 1.9.

 

    It is not clear how we are going to revise RESTWS as the data model changes.  If I understand Darius correctly, the "v1.0" portion of the URL will indicate that the module supports 1.8, and will be changed to another value (most likely "v2.0") when the module supports 1.9.  There will probably not be much backporting (maybe additional queries or custom reps or a resource-specific catalog call) because most of the changes will depend on the data model (in the case of 1.8-1.9, location attributes, providers, provider attributes, visits, multiple providers per encounter, concept mapping, complex concepts).  It seems to make sense to make a new 2.0 head and keep a 1.0 branch, but I don't think we've ever worked on both a branch and the head at the same time.  So I think it would be better to get 1.0 to a more complete, correct state before we start work on 2.0. 

 

    Maybe the sprint schedule needs to be revised to have an RESTWS 1.1 (or whatever) sprint sooner, and tickets that address 1.9 issues should be given a new future release fix 2.0 (or whatever).  Maybe we can have simultaneous 1.1 and 2.0 sprints.

 

 

 

[Darius Jazayeri]

Roger,

To clarify, I do not think we should have branches of the webservices.rest module for OpenMRS 1.8.x, 1.9.x, 1.10.x, etc. I assume that we will release webservices.rest v1.0, and this will work with OpenMRS 1.8.2+. However if you're using OpenMRS 1.9 with webservices.rest v1.0 you will not have access to anything that wasn't available in OpenMRS 1.8 (e.g. Visits, Location Attributes) I think we should do a "webservices.rest-19ext" module that adds support for all the new 1.9 features.

My initial thought is that adding new fields to existing resources (e.g. adding attributes to Location) should not increment the version number of the REST API. (But this is not well-thought out.)

Regarding whether to release 1.0 or not. I feel very strongly that we should release 1.0 ASAP, with the current functionality.

Almost a year ago we set out a limited set of functional requirements: we wanted to satisfy a few key user stories that were suggested by interested people. (See this wiki page.) We added to that the strategic requirement that whatever we release as "1.0" should be an API that we want to support going forwards.

As a result of that requirement we delayed doing a 1.0 release to make some fundamental design changes. (The bulk of these were spurred by you, Roger, because you insisted we deal with the issue of subclasses. I think you were absolutely right to do that, and I think it led to us doing some very-necessary refactoring of the code. So, a non-sarcastic "Thanks" for that.)

At this point I really think we need release our current code (just missing a couple of reviews) as 1.0, and commit to supporting it. Work can and should continue to happen on the module. It is straightforward to add some missing resources and release 1.1, 1.2, etc. (For the record those would be module versions, not API versions.)

-Darius

PS- Regarding competition of resources between finishing off the 1.8 API and doing the 1.9 API...I'd say "patches are welcome". Whatever tickets people create and vote on, and whatever tickets people claim and submit patches to, is what will get released first. :-)

---

Click here to unsubscribe from OpenMRS Developers' mailing list

[Spencer Kathol]

Darius, Roger, and Whoever,

I wanted to clarify the difference between module and API versions.  Based on the conversation, I think (and would like to confirm) the v1 part of the URL (and Java package) will stay the same as the module advances to 1.1, 1.2...  This is helpful to know since I'm working on a ticket to expose provider objects for v1.1. 

The 2.x module versions will add functionality, nested under /rest/v2...


Thanks,

-Spencer

[Burke Mamlin]

Correct.  We definitely do not want to be changing the REST URLs with each minor update of the module.

-Burke

[Saptarshi Purkayastha]

Darius,

So if I understand you correctly, you are saying that we will release v1.0 of the module the way it is currently with support for OpenMRS 1.8.1 and higher.

Then we will release v1.1 of the module (that will require OpenMRS 1.9 and higher) that will be compatible with OpenMRS 1.9, but you do not want branches for the module?? Or v1.1 will still support 1.8 and we will have another module that will provide 1.9 resources??

---
Regards,
Saptarshi PURKAYASTHA

My Tech Blog:  http://sunnytalkstech.blogspot.com
You Live by CHOICE, Not by CHANCE

[Friedman, Roger (CDC/CGH/DGHA) (CTR)]

On further review, it seems to me that while there are certain types of changes to the data model that can be made without breaking RESTWS, others cannot.  For example, adding a new nullable field to a table requires a change to the creatable and updatable field lists and to the could-be-missing field list within the resource (as well as corresponding changes in the .hbm and pojo), but existing calls to the resource will still work correctly, so it doesn't break anything.  But adding providers and using them as a mapped collection in encounters instead of a user reference does break things, you can no longer create an encounter with a user.

 

I am beginning to think that we should roll RESTWS into core so that it moves with its version-appropriate data model.  So 1.9.1 should include a RESTWS appropriate to its data model.  That would leave the "v1.0" portion of the URL to handle changes in the semantics of the calls, should that ever happen.  And 1.8 would be the only version to use the RESTWS module (or we could roll it into 1.8.4 or whatever is next).

[Darius Jazayeri]

Saptarshi,

I think that for the foreseeable future, all versions of the restws module (1.0, 1.1, 1.2, etc) will require OpenMRS 1.8.1.

So, webservices.rest 1.1 will still require OpenMRS 1.8.1. In addition we'll have a webservices.rest-19ext that requires 1.9 and provides the additional 1.9 resources.

Roger,

I actually think there's a benefit to having our REST API change more slowly than our java API. (It would be wonderful to be able to build a future version of sync based on having 1.8, 1.9, and 1.10 children all synching over web services.)

-Darius

[Saptarshi Purkayastha]

So is the plan that with every new data model change in the core, there will another web service extension module??

That means to get web services in 1.10, I'd have to install the webservices.rest, the webservices.rest-19ext and then webservices.rest-110ext (or the new webservices-110.ozip)

So, Spencer Kathol's work on the Provider should be in the webservices.rest-19ext module and not in 1.1

---
Regards,
Saptarshi PURKAYASTHA

My Tech Blog:  http://sunnytalkstech.blogspot.com
You Live by CHOICE, Not by CHANCE

[Darius Jazayeri]

Yes, my suggestion is that we'll have to manage up to a couple of -ext modules, for supported versions of OpenMRS. At some (way future) point we might decide to stop supporting 1.8 in active web service development, and we could fold -19ext into the main webservice module, and increase the required version to 1.9.

I think Ben prefers to branch the module instead (so we'd have a 1.8-compatible branch, a 1.9-compatible branch, etc).

Let's discuss this (in a time-boxed way!) on the design call today, as neither Ben or I are experts on this.

Another question: what happens when we introduce an incompatible variation of a resource (e.g. in 1.9 concept.mappings changes significantly)? Would we make that v2 of the concept resource only? Or does introducing v2 imply incrementing the version of the whole API? (And thus we might try to hide the new concept map structure under the hood.)

-Darius

[Ben Wolfe]

I don't see this in the notes for the design call today, so I assume it wasn't discussed.

http://notes.openmrs.org/Design-Forum-2012-05-16

I do NOT want branches for the module.  I also would prefer to not have the -ext modules.  Can someone confirm that the rest module def does not start up if the provider classes are just on the path?  If it fails, I assume it is because of classpath scanning by us or by spring.  The former could be caught and handled gracefully. 

Adding a v2 of a resource would come at any point, I think.  I don't the the v1/v2/v3 versions match the rest module version which doesn't match the openmrs version.  (We will need to document each rest method and keep it updated on the wiki to make alleviate developer version matchings)

Ben

[Spencer Kathol]

I coded a ProviderResource and ProviderController - untested - just to play around with versioning.

To resolve dependencies...
 - pom.xml: openMRSversion=1.9.0-SNAPSHOT
 - config.xml: require_Version=1.8.1

Building against 1.9 generates jUnit failures and errors, which I didn't investigate.  If I skip tests, the build succeeds.

I can load and start the module in 1.9 (No testing yet).
I can load in 1.8, but when I start it, I get an exception:

Unable to refresh the WebApplicationContext
Error creating bean with name 'org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping#0' defined in URL [jar:file:/C:/Users/Spencer/AppData/Local/Temp/1337224327577.openmrs-lib-cache/webservices.rest/webservices.rest.jar!/webModuleApplicationContext.xml]: Initialization of bean failed; nested exception is java.lang.ArrayStoreException
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:527)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:291)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:288)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:580)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
 ** org.openmrs.module.ModuleUtil.refreshApplicationContext(ModuleUtil.java:780)
 ** org.openmrs.module.web.WebModuleUtil.refreshWAC(WebModuleUtil.java:793)
 ** org.openmrs.module.web.WebModuleUtil.startModule(WebModuleUtil.java:317)
 ** org.openmrs.module.web.controller.ModuleListController.onSubmit(ModuleListController.java:222)
org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:272)
org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:268)
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
 ** org.openmrs.module.web.filter.ForcePasswordChangeFilter.doFilter(ForcePasswordChangeFilter.java:65)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
 ** org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:76)
 ** org.openmrs.module.web.filter.ModuleFilter.doFilter(ModuleFilter.java:58)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
 ** org.openmrs.web.filter.OpenmrsFilter.doFilterInternal(OpenmrsFilter.java:112)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
 ** org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:83)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
 ** org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:83)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
 ** org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:83)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:726)
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
org.mortbay.jetty.Server.handle(Server.java:324)
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843)
org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

I am still investigating, but maybe we could write an HandlerInterceptor to weed out mappings that would cause the handler to choke.  Thoughts?

-Spencer

[Saptarshi Purkayastha]

Ben,

Yes, it was not discussed since you were missing and we wanted you to be part of the decision... also time was short and it seemed like a decision on the list could be reached.

Yes, I can confirm that it doesn't work, if we've built the module using 1.9.0-RC3 and then say require version to be 1.8.1 and have the LocationAttributeResource that uses the LocationAttribute class. Spring doesn't like it and throws the following exception: http://pastebin.com/GBy9bRyW

I think resource versioning is fairly trivial. If the resource changes in its representation (or params it takes), we should make it v2 (or v1.1) and then only that resource has a v2 in its URL while others have v1. This is how many services exist where resources have different versions in the same release.

---
Regards,
Saptarshi PURKAYASTHA

My Tech Blog:  http://sunnytalkstech.blogspot.com
You Live by CHOICE, Not by CHANCE

[Ben Wolfe]

Thanks Saptarshi. So its thrown by spring, not openmrs. Thats unfortunate.  Is this because the controller has @Controller on it?  Or is it our @Handler annotation on the Resource?

I really want to try to keep all code in the rest module.  Perhaps building specifically for different versions is the solution?

1) If building against 1.8 don't include certain resources and controllers.  Annotate classes somehow to know what to exclude?
2) Hide certain bits of code using maven variables?
3) Add 1.9 specific resources/controllers at compile/runtime to avoid these classpath scanning errors?

Any maven experts out there know if this is possible?  Maven can do a lot, but I don't know if it can do this much.

Ben

[Friedman, Roger (CDC/CGH/DGHA) (CTR)]

I think both Darius and Saptarshi have highlighted the fact that there are two different aspects to the issue.  One is from the developer side, having to do with version control and packaging.  The other is from the client side, having to do with making the appropriate calls to each instance of OpenMRS it is accessing.  

 

From the packaging side, I think the most straightforward thing is to branch, so that it is always possible to package a version of the module with a version of core and have them work together.  I think this fits a use case where the client accesses only a single instance of OpenMRS, such as JSS or Andy's need to connect the child health program with OpenMRS.

 

For a use case such as Darius' metadata sharing or a central registry manager for the Rwanda enterprise architecture, we need the module to be totipotent, i.e., it needs to work with the version of OpenMRS being run on the particular instance it is trying to communicate with at any moment.  This goes with Saptarshi's idea of the version number changing whenever there is a change that breaks what was there before.    It also seems to be why Darius has the series of module extensions.  I still don't like the idea, because of the complicated packaging; but it does imply that the version of RESTWS that matches the 1.9 data model should consist of the 1.8 version as v1.0, plus v2.0 of the resources that have been superseded, such as encounter.  I think this use case could be helped by a resource version discovery mechanism, such as ws/catalog/<resource> returning version number and field names for the host.  I think we might also need to look at the routines in the abstract classes that find the appropriate resource to make sure that it finds the version-appropriate one.

 

But I still don't want to lose track of the need to have full coverage for the data model in a 1.8 version.

[Saptarshi Purkayastha]

On 17 May 2012 19:31, Friedman, Roger (CDC/CGH/DGHA) (CTR) <rd...@cdc.gov> wrote:

But I still don't want to lose track of the need to have full coverage for the data model in a 1.8 version.

We need to provide ConceptNumeric to be able to give ranges for the concepts available for client validation.

This is extremely important (should make it to v1.0) resource and I wonder how we never implemented this... 

It would be great if the Concept resource could show ranges automatically for the valueNumeric. 

[Ben Wolfe]

I agree that its important.  Can we just add the properties to the Concept resource to show up only when its a numeric concept?

Ben

-- OpenMRS Developers: http://go.openmrs.org/dev
Post: d...@openmrs.org
Unsubscribe: dev+uns...@openmrs.org

[Darius Jazayeri]

Back in my "REST WS and subclasses" email from a couple months ago, I said I thought Concept/ConceptNumeric should work like Order/DrugOrder in that it's a single-resource-per-class-hierarchy.

So, one approach would be to write a SubclassHandler for ConceptNumeric and register it with ConceptResource.

But Concept is a bit different from Order in that we expect and want to support modules defining new types of Orders, but we do not support modules defining new kinds of concepts. So, since the set of all available concept types is known ahead of time, we could take an alternative approach, and make the Concept resource transparently behave like a ConceptNumeric (i.e. show more properties) if it is one. This would work for GET. Someone should think through how that would work for POST...

-Darius