CommCare Integration with DHIS2 via OpenFn

[Chaiwa Berian]

Dear OpenFn Users,

In our project, we will need to be able to pull data from DHIS2 into CommCare and vice versa. The Government workers will be submitting data via their DHIS2 Mobile App to their DHIS2 server and we would like to be able to automatically push that data to CommCare for integrated data management and reporting. Similarly, we would like to be able to automatically push data collected via CommCare App to DHIS2.

 

I am new to OpenFn and tried to look at how that can be achieved via OpenFn but again it is not very clear to me yet.

 

Anyone with experience setting up such an integration?

 

I will greatly appreciate.

Thanks and kind regards,

Chaiwa.

[Taylor Downs]

Adding this to the forum. Thanks, Chaiwa.

@Santiago, could you take a look at the message below? Chaiwa is using the following job (with the inbox UUID removed):

fetchData({
  fields: {
    dataSet: 'SaIdNyi73XA',
    orgUnit: 'J7PQPWAeRUk',
    period: '201702'
  },
  postUrl: "https://www.openfn.org/inbox/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

});

-------- Original message --------

From: Taylor Downs <tay...@openfn.org>

Date: 16/02/2018 01:22 (GMT+02:00)

To: Chaiwa Berian <>

Subject: Re: CommCare-DHIS2 Integrations

Chaiwa, do you mind if I move this thread over to the google group? I'll add it to your previous question... I think it will benefit the community. We will respond over there.

On Tue, Feb 13, 2018 at 6:31 PM, Chaiwa Berian <> wrote:

Hi Taylor,

I hope you have been okay! I am still stuck on getting the basic setup to fetch data from DHIS2. As you requested in the previous email, please see below the the details :

1. Project Name: Mawa WASH

2. Job Name: Fetch Mawa WASH CLTS Data

3. I have also allowed "Support Access"

Note that I am able to get the response as a "200" but the message content in the inbox contains no expected data:

  a. Post Result:

Getting Data Value Sets:

Get Result: { dataSet: 'SaIdNyi73XA',

  period: '201702',

  orgUnit: 'J7PQPWAeRUk' }

Post Result: 200

Finished.

  b.  Message Body

{

  "period": "201702",

  "orgUnit": "J7PQPWAeRUk",

  "dataSet": "SaIdNyi73XA",

  "__query_params": {}

}

Thanks and kind regards,

Chaiwa 

[sgutier...@gmail.com]

Hi Chaiwa,

I've tested the fetchData function using this expression and this state:

fetchData(
  { fields: {
      "dataSet": 'pBOMPrpg1QX',
      "orgUnit": 'DiszpKrYNg8',
      "period": '201711'
  }}
)

{
  "data": {},
  "configuration": {
    "username": "admin",
    "password": "district",
    "apiUrl": "https://play.dhis2.org/demo"
  }
}

As you can see, I'm using the test account for DHIS2. I get this result when I run it:

Getting Data Value Sets:
Get Result: { dataSet: 'pBOMPrpg1QX',
  period: '201711',
  orgUnit: 'DiszpKrYNg8',
  dataValues: 
   [ { dataElement: 'Ix2HsbDMLea',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '2',
       storedBy: 'system',
       created: '2014-02-27T06:43:18.659+0000',
       lastUpdated: '2014-02-27T06:43:18.659+0000',
       followUp: false },
     { dataElement: 'MSZuQ1mTsia',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '5',
       storedBy: 'system',
       created: '2014-01-28T00:16:26.887+0000',
       lastUpdated: '2014-01-28T00:16:26.887+0000',
       followUp: false },
     { dataElement: 'r6nrJANOqMw',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '7',
       storedBy: 'system',
       created: '2014-01-28T00:16:25.438+0000',
       lastUpdated: '2014-01-28T00:16:25.438+0000',
       followUp: false },
     { dataElement: 'eY5ehpbEsB7',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '2',
       storedBy: 'system',
       created: '2014-02-27T06:43:18.184+0000',
       lastUpdated: '2014-02-27T06:43:18.184+0000',
       followUp: false },
     { dataElement: 'NpJtsQkMTm3',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '6',
       storedBy: 'system',
       created: '2014-01-28T00:16:25.158+0000',
       lastUpdated: '2014-01-28T00:16:25.158+0000',
       followUp: false },
     { dataElement: 'f7n9E0hX8qk',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '6',
       storedBy: 'system',
       created: '2014-01-28T00:16:25.983+0000',
       lastUpdated: '2014-01-28T00:16:25.983+0000',
       followUp: false },
     { dataElement: 'lXolhoWewYH',
       period: '201711',
       orgUnit: 'DiszpKrYNg8',
       categoryOptionCombo: 'bRowv6yZOF2',
       attributeOptionCombo: 'bRowv6yZOF2',
       value: '5',
       storedBy: 'system',
       created: '2014-01-28T00:16:26.415+0000',
       lastUpdated: '2014-01-28T00:16:26.415+0000',
       followUp: false } ] }

I get a full set of dataValues. However, when I tried a value of 201601 for period, I got your same results. So maybe you have to check that you fetch dataValues for other periods that has data attached to it.

Hope this info will be helpful for you. Let us know how this goes for you.

Bests,

Santiago

[Chaiwa Berian]

Hi @Santiago,

I replicated your steps, using the test account and I got the data as expected! But I am still wondering why I am not able to get the data when I authenticate using the "live user" on our server. I am able to confirm the data is there for the selected period and orgUnit, when I use DHIS2 data visualizer, but getting it via Browser API call or via OpenFn is not giving me the expected results.Are there special permissions(Authorities) the user needs to have, on DHIS2, for them to be able to pull the data via API calls? I know this might be better posted in the DHIS2 forum but just in-case you aware of such a requirement would be great to share!

Thanks and kind regards,

Chaiwa.

[Chaiwa Berian]

Dear Taylor and Santiago,

I have tested pulling data from DHIS2 Demo Instance, and all looks good. We are now getting ready for a live integration, would you kindly advise on what specific things we need to put in place for a successful live integration? Particularly, would you kindly highlight key things to put in place like the ones below(both on the DHIS2 side and CommCare side)?:

1. Access rights
• On the side of DHIS2, what level of access should the authenticating credentials have?
2. Organisation Units
3. Datasets
4. Period
• Is there a function that can be used to return current month and then pass the value?
• What happens when a job is run more than once in a given period? Data gets duplicated?
5. Jobs
• Can we pull more than one dataset in a single job run? How can that be configured?
• What are the Pros and Cons of using a free tier?
• Can we use the same job(same as used to pull data from DHIS2) to push data to CommCare?
6. Setup Costs
• If we were to consult you to help us setup-up the configurations, what is the expected total cost?

There could be many more questions that I may not have asked that can be critical to a successful integration and I would appreciate if you can highlight them for us as well.

Thanks and kind regards,

Chaiwa.

--
You received this message because you are subscribed to a topic in the Google Groups "OpenFn Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openfn/uNddpsBB3P4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openfn+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openfn/a9ba9e06-f0b7-43b7-bd9d-aa4c0760b93e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Taylor Downs]

Hey Chaiwa, these are all great questions but since we're starting to get into the specifics of your project it might be best to take this to email. I'll do my best to respond here, but I'll then follow up on email to set a time for us to speak.

1. If you've been managing this DHIS2 implementation for a while now, you've likely got a better understanding of what access rights are required to read or write the information you want to use for this project. Once you've configured a user with the proper rights, you can use that user's login information for your OpenFn credential.
2. Here's a link to DHIS2 Org Units documentation: https://docs.dhis2.org/2.23/en/user/html/ch04.html . Do you have a specific question?
3. Also not sure of the ask here: You'll notice that when writing to DHIS2 using the data values API, you'll can specify the data set: https://docs.dhis2.org/2.22/en/developer/html/ch01s13.html
4. Period
1. Absolutely—use the Javascript Date module: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/getMonth . You could write something like return new Date(Date.now()).getMonth()
2. Let's discuss—not sure which way you're sending data but you could configure OpenFn to always create new records, update existing records, or do some logic-based combination of the two.
5. Jobs
1. Yes. Job expressions consist of operations—so long as you use a single adaptor (HTTP, DHIS2, CommCare, etc.) you can chain together as many operations as you'd like and keep the relevant data accessible to you in that job's state. The only limit is that a single run must be completed in less than 100 seconds, unless you're using an enterprise plan.
2. The most obvious pro is that it's free! The drawback is that you can only have one job active at any given time, and execute no more than 100 runs each month.
3. You might be able to achieve this using the generic HTTP adapter, but then you'll loose the pre-configured functionality that handles authentication and wraps DHIS2 and CommCare's respective APIs.
6. Projects can range from less than an hour to several weeks of in-person work! I'm not sure what the scope of your integration project is, but we can certainly work to define it and estimate costs on a call.

Best,

Taylor

To unsubscribe from this group and all its topics, send an email to openfn+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openfn/a9ba9e06-f0b7-43b7-bd9d-aa4c0760b93e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "OpenFn Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openfn+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openfn/CAHSY2xvkq8zLDtLDKF-a_NpoU7ei%2BNQRBbEi8mMNfkRitFX3mg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Taylor Downs | Founder & Core Contributor
Open Function | Integrated Systems for Development

[Chaiwa Berian]

Thanks Taylor,

This is appreciated. I am sending you a separate email to schedule a call, shortly.

Thanks and kind regards,

Chaiwa.

To unsubscribe from this group and all its topics, send an email to openfn+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openfn/a9ba9e06-f0b7-43b7-bd9d-aa4c0760b93e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "OpenFn Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openfn+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openfn/CAHSY2xvkq8zLDtLDKF-a_NpoU7ei%2BNQRBbEi8mMNfkRitFX3mg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[yesomin...@gmail.com]