Quality control on HDSS automated application systems
Ezekiel Chiteri
Authors: Ezekiel K. Chiteri, Frank Odhiambo, Gordon Orwa, Software
Developers, Kayla Laserson
The KEMRI/CDC Health and Demographic Surveillance System (HDSS),
situated in western Kenya, collect longitudinal data from over 240,000
individuals living in the Demographic Surveillance Area (DSA). In the
DSA, data is currently collected on handheld devices then transferred
to the main data reservoir in a MS SQL server database via a desktop
system. We have embedded various security measures and validations in
the handheld device application to ensure that the data collected is
integral and consistent by eliminating obvious human errors.
Quality control forms part of the measures taken to ensure that
quality data is collected. It started in 2001 with the inception of
HDSS; the quality control team was instituted in June 2001. Back then
until 2008, quality control involved doing repeat interview
verifications, spot checks, use of red herrings, and accompanied
interviews using the paper based system. Currently, the quality
control and software development teams work together to implement
methods of collecting, and managing quality data for analysis and
reporting using the automated systems. Quality control
functionalities have been implemented in the desktop and handheld
device applications. These functions include; Red herrings, capture-
recapture, spot checks, and follow-ups.
Red herring are fictitious individuals randomly inserted/placed in
households, in the handheld application. A fieldworker is expected to
indentify these fictitious individuals and report them as non-
existent; otherwise the fieldworker will have failed to detect the
fictitious individuals indicating that they didn’t physically visit
the assigned location to conduct a proper interview. This measure is
designed to identify data falsifiers.
Capture-recapture method involves collecting data by visiting the same
household twice on different dates or times by different field
workers. During the normal data capture, a visit is done by a
fieldworker. Another visit by a special team of quality control
fieldworkers is done to recapture data from the same household. The
data collected on the same households by different teams of
fieldworkers are then compared using a desktop system. The comparison
could produce similar or different results. Causes of different
results could be data falsification, respondent bias, or genuine human
error. The capture-recapture method enables the quality control team
to investigate any discrepancies arising after comparing captured and
recaptured data. A follow-up method of quality control simply means
carrying out revisits to the already visited households upon detection
of data inconsistencies during data cleaning by data specialists or
managers. It requires a quality control fieldworker to return to the
specific location and collect data again using a handheld device. The
resulting collected data is then used to correct the detected
inconsistency.
Spot checks involve selecting randomly already visited locations by a
fieldworker. The spot checks are carried out to reduce or stop
fabrication of data by the fieldworker, improve quality of data
collected and to ensure that the field worker actually visits the
assigned location.
Implementation of these methods has greatly improved the quality of
data collected, reduced data falsification by fieldworkers, and built
integrity in the KEMRI/CDC HDSS database. These successes have been
achieved as a result of a small committed team of quality control
personnel, timeliness of data collection for verification by the team
due to the automated systems, and improvements in field workers’
probing skills due to repeated training at the start of every round.
The data personnel in HDSS are constantly working together to improve
data collection, management, and analysis by devising detective,
preventive, and corrective methods of quality control to ensure
collection of high quality data.
However, there is still more that can be done to improve the automated
quality control system. So as to cover at least 5% of the total HDSS
population (the preferred quantity for more effective quality
control), the quality control team needs more personnel. The automated
quality control system should enable the data collected by a quality
control fieldworker with those collected by an ordinary field worker
to merge into the main database. The current methods used by the
quality control team can sometimes be breached. Invention of new
methods or some security measure can be employed to prevent breaching
of the currently used methods.
neal lesh
Thanks very much for this post! I've been keen for you to tell us about
your work for a while, since you have more experience in successfully
deploying mobile health systems than most organizations, and we have a lot
to learn from you. As usual, i have a few questions--but no need to
answer them all.
What changed in 2008? If I understand correctly, you are saying that in
2008 the software developers started working more closely on data quality,
and building in automated systems.
Do you have a sense of which of the methods you described are most
effective? If an organization was going to just implement two, which two
would you suggest they start with?
Many of your methods are meant to detect field workers who are entering
false data either intentionally or through human error. In general, once
you identify such cases, does a field worker tend to start entering higher
quality data? Or do you often need to hire new field workers?
For the capture-recapture methods, what level of discrepancy triggers follow
up? If the answers are 10% different between the initial and followup
sessions, is that good or is that a cause for further investigation?
I'm curious to hear more about the fieldworkers breaching your quality
control team's efforts. Does that mean that the field workers are
deliberating finding ways to enter false data but not get caught by your
quality control methods? Is it possible to breach capture-recapture?
Finally, do you have to have quality control methods to monitor the quality
control teams themselves?
thanks again!
neal
--
You received this message because you are subscribed to the Google Groups
"ict4chw" group.
To post to this group, send email to ict...@googlegroups.com.
To unsubscribe from this group, send email to
ict4chw+u...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/ict4chw?hl=en.