ToR - INDEPENDENT ASSESSMENT OF DATA QUALITY AND OPERATING COST OF MYCHILD SOLUTION IN THE GAMBIA
Rustam Nabiev
I. Introduction
Shifo, a not for profit social enterprise based in Sweden, develops and implements solutions and methods to strengthen health information systems and child health service delivery. Shifo focuses on ensuring children receive all essential health services, and work to ensure that every child is registered and followed-up to receive life saving health services. Shifo is currently working on the implementation of MyChild Solution, an innovative paper solution with a digital component based on Smart Paper TechnologyTM, that allows for beneficiaries' health data to be captured on paper by health workers and subsequently scanned and digitised.
In consultation with Ministry of Health and Social Welfare The Gambia (MoHSW), ActionAid International The Gambia (AAITG) and Shifo have conducted an internal evaluation of data quality and cost evaluation of the MyChild Solution in The Gambia.
This Terms of Reference outlines the nature of the evaluation and the scope and time line of the independent assessment.
II. Context
In 2017, MoHSW signed a Memorandum of Understanding together with ActionAid International The Gambia and Shifo Foundation to implement a new form of Health Management Information System (HMIS) in The Gambia. The intervention is implementing a hybrid paper-digital solution, named MyChild Solution, which collects, shares, stores and distributes health facility data by use of Smart Paper Technology. MyChild Solution is, as of February 2018, being used in 18 fixed and outreach health service delivery points in Western Region 1 and Western Region 2 to capture immunisation services provided to children. The data is digitised and stored in an electronic register from which HMIS reports are generated and shared on a beneficiary, health facility, regional and national level.
The key stakeholders in the implementation project are Ministry of Health and Social Welfare The Gambia (MoHSW), ActionAid International The Gambia (AAITG), The Vaccine Alliance (GAVI), IKARE, Swedish Postcode Foundation (PKS) and Shifo Foundation (Shifo).
III. Background
Shifo together with MoHSW and AAITG conducted an evaluation of data quality and the operating cost of MyChild Solution in The Gambia in order to inform stakeholders on the feasibility of the solution to be scaled up in the country.
The evaluation of data quality was based on information captured in MyChild Solution. The study was designed using the Data Quality Review (DQR) framework and toolkit developed by WHO, The Global Fund, Gavi, The Vaccine Alliance and United States Agency for International Development/MEASURE Evaluation. The toolkit is designed to support routine assessments of facility-reported data (WHO, 2017). The dimensions of data quality that were examined were: 1) completeness and timeliness of data, 2) internal consistency of reported data, 3) health worker compliance. The evaluation sought to answer the following questions:
1. What is the level of data completeness of immunisation sessions captured in electronic reports?
2. What is the level of timeliness for the submission of monthly HMIS reports?
3. What is the level of consistency between immunisation indicators and between Smart-Paper forms and electronic data?
4. What is the level of health worker compliance related to recording immunisation data?
Cost evaluation was performed to answer following question: What are the operating costs of MyChild Solution from the perspective of the Ministry of Health and Social Welfare? For cost evaluation the operating costs of MyChild Solution were identified and compared to current HMIS costs. Annual operating costs were calculated considering national scope in The Gambia with total of 57 fixed health facilities and 80 000 newborns per year. Costs were identified from MoHSW perspective. The data to estimate the costs of MyChild Solution was obtained from tracking the project activity records and resources used by the project team in implementing the system in Western Region, The Gambia. Operating costs of HMIS were identified in consultation with MoHSW.
IV. Scope of work of the independent assessment
The key stakeholders are seeking an independent assessment and verification of data quality with MyChild Solution and operating cost of MyChild Solution in The Gambia in order to inform further scale-up of MyChild Solution in Western Region 1 and Western Region 2. Building on the evaluation done, an independent assessor(s) will perform an independent review to assess the costs to sustain MyChild and requirements needed to assure data quality in The Gambia. The independent assessment will include an audit of data quality and cost evaluations of MyChild Solution collected previously. The Independent assessment of data quality will be done using the WHO Toolkit for self-assessment of data quality. The assessment is divided into two sections.
Section A
Data quality assessment
The data quality assessment will evaluate the data quality produced through the MyChild solution. The assessment will include an audit based on verification of data from different data sources and documentations. The independent assessor will perform the following procedures and report on factual findings:
· Asses the methods of MyChild to assure data quality
· Check that data quality evaluation methods were performed according to WHO Toolkit
· Verify whether there are supporting documents to data quality evaluation report
· Verify whether figures from the data quality evaluation report are correct based on supporting documents
· Assess whether there are any weaknesses in Shifo’s internal control regarding data quality evaluation
· Assess the time required to perform the data quality controls and required rectifications
The procedures that will be performed will constitute an independent review and audit made in accordance with WHO Toolkit on Data quality self-assessment of health facilities. Shifo will provide the independent assessor access to necessary data and documents in electronic format.
Section B
Cost evaluation assessment
Independent assessor will perform the following procedures and report on factual findings on:
· Asses costs required to implement and sustain MyChild
· Assess whether the methods used to identify the costs were appropriate
· Verify whether all important and relevant operating costs from MoHSW viewpoint were considered
· Check that the costs were measured accurately in appropriate physical units
· Assess whether the costs were valued credibly (source of values)
· Assess the costs of Shifo’s data quality control methods
V. Organization of the assessment
Different independent assessors can be selected for Sections A and B.
The deadline for proposal submission is 15 March 2018.
The deadline for the selection of the independent assessor is 30 March 2018.
VI. Deliverables
The independent assessor will be expected to produce the following deliverables, in line with the scope of work detailed above:
· Section A: Data Quality evaluation Report.
· Section B: Cost evaluation Report.
Work is expected to begin in March/April 2018.
Independent assessor will work closely with MoHSW, AAITG and Shifo Foundation. He/she can work remotely from their current location. For obtaining and verifying data sources the assessor will receive the documents in electronic format.
The key stakeholders are expected to be notified immediately of any concerns or delays that may arise during the course of the engagement.
VII. Proposal submission requirements
Interested parties are invited to submit their CV and a technical proposal (in Word or PDF format) to: MoHSW The Gambia EPI Secretariat gamb...@gmail.com, Secretariat of Shifo Foundation, shah...@shifo.org and Secretariat of AAITG, yorro...@actionaid.org
In the technical proposal include the evaluation methodology to be used, relevant experience, details of previously completed projects of similar scope and a detailed timeline of activities. Include a financial proposal with a line item budget (US dollars).
Proposals must be received no later than 15th of March 2018. You are solely responsible for ensuring that your proposal is delivered on time. Late proposals may be accepted at the stakeholder’s discretion. All costs incurred in preparing proposals will be borne by the applicant.
VIII. Proposal evaluation criteria
The assessors for part A and part B will be selected by MoHSW EPI Manager, MoHSW Director of Planning, MoHSW RMNCH, AAITG Project Coordinator, Shifo Foundation Project Coordinator and GAVI Project Coordinator. The selection will be based on the basis of the following criteria:
· Relevant qualification (Bachelor of Science in Public Health / Master of Science in Public Health/ equivalent)
· Relevant experience
· Previously completed projects of similar scope
· Knowledge of the health care domain (preventive child health)
· Knowledge of the setting
IX. Terms of payment
Payments will be initiated upon successful completion or documented progress made on tasks as mentioned in the above Terms of Reference, or as agreed with independent assessor.
X. References
World Health Organization. (2017) Data quality review: a toolkit for facility data quality assessment. Module 1. Framework and metrics. Geneva: World Health Organization.