A little help orienting myself in the API documentation

67 views
Skip to first unread message

Eric Stephan

unread,
Sep 2, 2015, 4:52:13 AM9/2/15
to commcare-users

Hello folks --

I am in a situation where we are considering doing a bit of integration between our commcare project and a third-party web application (PHP/mySQL) that might, in certain circumstances, want to connect with the commcareHQ API, authenticate itself as a given user, and basically issue some commands to submit one form of a type that we have already of course defined and are making use of in our app.  Furthermore, that form, that we'd like this other app to complete and submit via the API, has its case management set up so that it requires an existing case, as well as in some conditions will create a new case of a different case type.

I am not the programmer that will do this coding, but I do want to be oriented as to the basic sections of the API documentation apply to the various steps, so that I can get this programmer (who doesn't know commcare) at least ... oriented to start off with. :)

I am looking at the table of about 20 APIs that I see listed on https://confluence.dimagi.com/display/commcarepublic/CommCare+HQ+APIs.

I'm a little bit uncertain about which API will be able to concretely carry out the following steps. I was hoping that someone could look at my crude list of expected steps below, and tell me which API has the commands to allow these to be invoked for that step.  Sorry if this seems needy or even wrong-headed -- it will just make it easier for me to be confident that I'm basically oriented, and also to start my team off in the right mindset.

This PHP application, after having collected a bunch of information in its own forms and variables, will then hopefully carry out these steps. For each one of these, can you tell me which API our programmer should look at to start learning how to make it happen?  Again, sorry if this seems painfully obvious to you:

1-- the third party app makes a call to authenticate itself as a given user, using its password ---> ________ api

[Now we ultimately want to complete a form whose name is "C1 Referral Followup" which however requires the selection of an existing case for which to complete the form. Therefore:]

2-- Their app asks the commcare system for the list of cases that the logged-in user has available to them that are allowed for that C1 Referral Followup form. (in other words, the list of available cases that the user is normally presented with when they go to the C1 form on the normal mobile phone app.)  ... so that their web app can then present that list to the user and allow them to select one of them. ----> __________ api

3-- Their app takes a number of fields/variables that they have already collected from the user, and transmits that data to the CommCareHQ server, specifying which of the C1 form's fields are being populated with what pieces of data, and resulting in the form being fully submitted and saved to the CommCareHQ server. ---> ______ api

4-- The CommCareHQ system, which has already been configured in its case management that under certain circumstances a new case should be created and passing certain values to the new case, does that as normal --- Does that process happen automatically in the CommCareHQ app since the Case Management for that form has been specified right there on the CommCareHQ server? (I hope!)   Or does the remote web app need to invoke something for that additional case to be created?

5-- And just to be clear, the other existing case management configurations on CommCareHQ will also handle the data submitted themselves, right? no additional things that the remote app would have to do for that to happen?

6-- Do  you have any other notes, or things that I should flag, about possible complexities in the story that I just told above?   

Thanks much for your inputs here, this will set me off on the right foot.

Eric

Cory Zue

unread,
Sep 2, 2015, 7:59:49 AM9/2/15
to commcar...@googlegroups.com
Hey Eric,

Apologies for brevity, but quick responses inline.

On Wed, Sep 2, 2015 at 4:52 AM, Eric Stephan <este...@fhi360.org> wrote:

Hello folks --

I am in a situation where we are considering doing a bit of integration between our commcare project and a third-party web application (PHP/mySQL) that might, in certain circumstances, want to connect with the commcareHQ API, authenticate itself as a given user, and basically issue some commands to submit one form of a type that we have already of course defined and are making use of in our app.  Furthermore, that form, that we'd like this other app to complete and submit via the API, has its case management set up so that it requires an existing case, as well as in some conditions will create a new case of a different case type.

I am not the programmer that will do this coding, but I do want to be oriented as to the basic sections of the API documentation apply to the various steps, so that I can get this programmer (who doesn't know commcare) at least ... oriented to start off with. :)

I am looking at the table of about 20 APIs that I see listed on https://confluence.dimagi.com/display/commcarepublic/CommCare+HQ+APIs.

I'm a little bit uncertain about which API will be able to concretely carry out the following steps. I was hoping that someone could look at my crude list of expected steps below, and tell me which API has the commands to allow these to be invoked for that step.  Sorry if this seems needy or even wrong-headed -- it will just make it easier for me to be confident that I'm basically oriented, and also to start my team off in the right mindset.

This PHP application, after having collected a bunch of information in its own forms and variables, will then hopefully carry out these steps. For each one of these, can you tell me which API our programmer should look at to start learning how to make it happen?  Again, sorry if this seems painfully obvious to you:

1-- the third party app makes a call to authenticate itself as a given user, using its password ---> ________ api

Authentication will happen on every request so this does not exist.
 

[Now we ultimately want to complete a form whose name is "C1 Referral Followup" which however requires the selection of an existing case for which to complete the form. Therefore:]

2-- Their app asks the commcare system for the list of cases that the logged-in user has available to them that are allowed for that C1 Referral Followup form. (in other words, the list of available cases that the user is normally presented with when they go to the C1 form on the normal mobile phone app.)  ... so that their web app can then present that list to the user and allow them to select one of them. ----> __________ api

List cases (filter by user id or owner id)  

3-- Their app takes a number of fields/variables that they have already collected from the user, and transmits that data to the CommCareHQ server, specifying which of the C1 form's fields are being populated with what pieces of data, and resulting in the form being fully submitted and saved to the CommCareHQ server. ---> ______ api


4-- The CommCareHQ system, which has already been configured in its case management that under certain circumstances a new case should be created and passing certain values to the new case, does that as normal --- Does that process happen automatically in the CommCareHQ app since the Case Management for that form has been specified right there on the CommCareHQ server? (I hope!)   Or does the remote web app need to invoke something for that additional case to be created?

You will need to submit valid casexml to create/modify cases.

5-- And just to be clear, the other existing case management configurations on CommCareHQ will also handle the data submitted themselves, right? no additional things that the remote app would have to do for that to happen?

Once the cases are created/updated they will look "normal" to CommCare, so if you set the type and properties the same the app will treat them the same.


6-- Do  you have any other notes, or things that I should flag, about possible complexities in the story that I just told above?   

The submission + casexml API will be the tricky part. 
 

Thanks much for your inputs here, this will set me off on the right foot.

Eric

--
You received this message because you are subscribed to the Google Groups "commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Eric Stephan

unread,
Oct 21, 2015, 1:16:17 AM10/21/15
to commcare-users


Hello Cory and all --  Thanks very much for your very useful info about the commcare API process earlier in this thread.  I now have a "second level" of questions to make sure I am oriented correctly about the API, so that when I work with the coding side of our team, I'm sure we are headed in the right direction.

(In fact, in another thread you have given help to my colleague working on this same project, who is learning about the API process so he can guide his coders. That thread is https://groups.google.com/forum/#!topic/commcare-users/IBZNG21oWmE .)

** As a reminder, we have a fully working commcare project that involves a few separate apps and two different levels of referrals. To over-simplify, our outreach workers fill out form A1 when they interview people on the street. When we determine that person needs a referral to HIV testing, it creates a referral by creating a new child case of type "referral."  Then, over in the HIV testing clinic, the clinicians pick up their commcare phone and go to form C1 which allows them to open up that "referral" case and fill out a form describing the results of the HIV test visit. And if the results of that test were HIV positive, then the app is designed to create a further referral to ART care, by creating a new child case, of type "art-client," which will later on be picked up over in the ART clinic in form D1. 

** That is all working fine. But now our goal is to make it so that over in that middle clinic -- the HIV testing clinic described above that would normally use our smartphone app to load up cases on form C1 and sometimes create child cases -- we can use the CommCare API to actually build that full "C1 form" process into a PHP-based web app that is currently operating in those clinics. Therefore, the web app would replace that smartphone in that one clinic's role in the chain that I describe above. 

(1) My first question is just confirming my understanding. Earlier, I had been hoping that the info that the web app would have to submit through the API would be somewhat minimal, and that much of the complex plumbing stuff would be taken care of on the configurations already existing on the CommCare servers. But now I am understanding that when the person using their PHP web app goes to do the C1 form, and they select which 'referral' case they want to open and fill out the form for, the web app will actually have to do a lot of loading up of values from that existing case... and then, after the web app contributes the few new pieces of info collected by the web app (like the results of the HIV test etc.), if a child case/referral is indeed required to be made, then their web app will also have to generate all the right code to fully create the new ongoing case ("art-client") and then submit that.
--> Other than confirming that I understand that correctly, my only question here is probably obvious to all -- can you remind me which API is used to load the values from the existing case into variables in the web app, so that they can later be submitted correctly in the XML?

(2) My second question is simply trying to orient myself in the XML that the C1 form created. Since the web app will have to output this XML faithfully, I wanted to make sure I understood the general overview, so that I and the coding team will be headed in the right direction with the same understanding. 
... So I filled out one of our C1 forms completely, giving responses in which the patient tested HIV positive and therefore a child case is indeed created. I am now attaching the XML that resulted from that C1 form submission -- I have highlighted with a little color. Here are some embarrassingly high-level questions to make sure I'm oriented OK. And if I appear to be misunderstanding things, please point that out too:

** The first section, in green -- this seems simply submitting exactly the fields that were completed when the C1 form was filled out. This seems straightforward.
** The second section, whose tags start n0, in blue, seems to be only those the fields that I had configured to save their values to the PARENT case rather than the case itself. That I also basically understand. 
** The third section, whose tags start n1, in yellow, seems to be all the info that is needed for the CHILD case that is being created because the person tested HIV positive. That also seems clear. 
** But the fourth section, whose tags start n2, in grey, is confusing to me. It seems that it is simply submitting a few fields that were already submitted up in the first section (the "simply the values from this form" one.) For example, it is submitting the "hct-services-provided" field again, exactly as it did up above. Can you tell what this grey "n2" section is for?
** The last section, n3, just seems like various metadata about versions and deviceIDs.

(3) A few odd questions:

-- When we go to analyze our app's data, we have very attentive to make sure we can do analysis that is able to recall the exact links between a given A1 form submission, its resulting exact C1 form submission, and its resulting D1 form submission. Looking at the data, I have found that the workable key value to connect all of those is the A1 form's info.caseid, the C1 form's parents.parent.case.@case_id, and the D1 form's parents.parent.parent.case.@case_id.   Is there any chance that when we implement this API to allow a web app to take the place of the mobile app's C1 form, that that chain of key values could be spoiled?  That method of linking the three forms will still work, right?

-- it's a little bit daunting to see all of those identifier values placed into the headers of those sections in the XML. for example, things like case_id, user_id, instanceID, owner_id, deviceID, etc. But I'm guessing that all those IDs are nicely obtained through the API calls that we will use to locate and load up the case. It can't hurt for me to ask, are any of those values not actually fully required? It is good for me to ask in case it saves us a bit of effort :)


Giant thanks for your help, and tolerance of my unbelievably verbose approach :)

Eric


xml-for-our-C1-form.docx

Simon Kelly

unread,
Oct 21, 2015, 3:55:27 AM10/21/15
to commcar...@googlegroups.com
Hi Eric

I'll take a quick stab at answering your questions. Here is another resource that may be helpful:
The general process that happens when you fill out a form is as follows (you may already know this):
  • Form session created with necessary data
  • During form entry the form can look up addition data via API's provided on the phone:
    • e.g. <bind nodeset="/data/mother_name" calculate="instance('casedb')/casedb/case[@case_id=instance('commcaresession')/session/data/case_id]/@case_name"/>
    • This is how case properties get loaded into the form
    • See https://github.com/dimagi/commcare/wiki/casedb
  • As questions get completed in the form the values in the main XML 'instance' (see xform-spec) get populated via the 'bind' expressions
  • Once the form is complete the instance XML is ready to submit to the server with all values populated.
Now let me try and answer your questions below.

On 21 October 2015 at 07:16, Eric Stephan <estep...@gmail.com> wrote:


Hello Cory and all --  Thanks very much for your very useful info about the commcare API process earlier in this thread.  I now have a "second level" of questions to make sure I am oriented correctly about the API, so that when I work with the coding side of our team, I'm sure we are headed in the right direction.

(In fact, in another thread you have given help to my colleague working on this same project, who is learning about the API process so he can guide his coders. That thread is https://groups.google.com/forum/#!topic/commcare-users/IBZNG21oWmE .)

** As a reminder, we have a fully working commcare project that involves a few separate apps and two different levels of referrals. To over-simplify, our outreach workers fill out form A1 when they interview people on the street. When we determine that person needs a referral to HIV testing, it creates a referral by creating a new child case of type "referral."  Then, over in the HIV testing clinic, the clinicians pick up their commcare phone and go to form C1 which allows them to open up that "referral" case and fill out a form describing the results of the HIV test visit. And if the results of that test were HIV positive, then the app is designed to create a further referral to ART care, by creating a new child case, of type "art-client," which will later on be picked up over in the ART clinic in form D1. 

** That is all working fine. But now our goal is to make it so that over in that middle clinic -- the HIV testing clinic described above that would normally use our smartphone app to load up cases on form C1 and sometimes create child cases -- we can use the CommCare API to actually build that full "C1 form" process into a PHP-based web app that is currently operating in those clinics. Therefore, the web app would replace that smartphone in that one clinic's role in the chain that I describe above. 

(1) My first question is just confirming my understanding. Earlier, I had been hoping that the info that the web app would have to submit through the API would be somewhat minimal, and that much of the complex plumbing stuff would be taken care of on the configurations already existing on the CommCare servers. But now I am understanding that when the person using their PHP web app goes to do the C1 form, and they select which 'referral' case they want to open and fill out the form for, the web app will actually have to do a lot of loading up of values from that existing case... and then, after the web app contributes the few new pieces of info collected by the web app (like the results of the HIV test etc.), if a child case/referral is indeed required to be made, then their web app will also have to generate all the right code to fully create the new ongoing case ("art-client") and then submit that.
--> Other than confirming that I understand that correctly, my only question here is probably obvious to all -- can you remind me which API is used to load the values from the existing case into variables in the web app, so that they can later be submitted correctly in the XML?


With reference to my explanation above, the form engine uses the 'external instance' API on the phone to load case properties and other values form the session: http://dimagi.github.io/xform-spec/#secondary-instances---external 
Without using a form engine you could manually insert values from the case into the XML by parsing the 'bind' expressions to see which values are required.


(2) My second question is simply trying to orient myself in the XML that the C1 form created. Since the web app will have to output this XML faithfully, I wanted to make sure I understood the general overview, so that I and the coding team will be headed in the right direction with the same understanding. 
... So I filled out one of our C1 forms completely, giving responses in which the patient tested HIV positive and therefore a child case is indeed created. I am now attaching the XML that resulted from that C1 form submission -- I have highlighted with a little color. Here are some embarrassingly high-level questions to make sure I'm oriented OK. And if I appear to be misunderstanding things, please point that out too:

** The first section, in green -- this seems simply submitting exactly the fields that were completed when the C1 form was filled out. This seems straightforward.
** The second section, whose tags start n0, in blue, seems to be only those the fields that I had configured to save their values to the PARENT case rather than the case itself. That I also basically understand. 
** The third section, whose tags start n1, in yellow, seems to be all the info that is needed for the CHILD case that is being created because the person tested HIV positive. That also seems clear. 
** But the fourth section, whose tags start n2, in grey, is confusing to me. It seems that it is simply submitting a few fields that were already submitted up in the first section (the "simply the values from this form" one.) For example, it is submitting the "hct-services-provided" field again, exactly as it did up above. Can you tell what this grey "n2" section is for?
** The last section, n3, just seems like various metadata about versions and deviceIDs.

The way we process the XML on the server is to extract all the 'case' elements and treat them as a transaction on the case to which the apply (via the @case_id attribute). The reason there are duplicate values is to avoid having to do any evaluation logic on the server to determine what the case values should be. All the logic happens in the form.

The n2 section corresponds to the 'main' case that was selected for this form.

(3) A few odd questions:

-- When we go to analyze our app's data, we have very attentive to make sure we can do analysis that is able to recall the exact links between a given A1 form submission, its resulting exact C1 form submission, and its resulting D1 form submission. Looking at the data, I have found that the workable key value to connect all of those is the A1 form's info.caseid, the C1 form's parents.parent.case.@case_id, and the D1 form's parents.parent.parent.case.@case_id.   Is there any chance that when we implement this API to allow a web app to take the place of the mobile app's C1 form, that that chain of key values could be spoiled?  That method of linking the three forms will still work, right?

Provided that the correct ID values are supplied in the form XML the server will process it as normal and the linking will work.
 

-- it's a little bit daunting to see all of those identifier values placed into the headers of those sections in the XML. for example, things like case_id, user_id, instanceID, owner_id, deviceID, etc. But I'm guessing that all those IDs are nicely obtained through the API calls that we will use to locate and load up the case. It can't hurt for me to ask, are any of those values not actually fully required? It is good for me to ask in case it saves us a bit of effort :)


I hope that answers your questions.
Best regards

--
Simon Kelly
Senior Engineer | Dimagi South Africa
Reply all
Reply to author
Forward
0 new messages