IWP: Data Capacity Building Case Study

[Nisha Thompson]

Hello All,

I have been working with the Keystone Foundation, in the Niligiris on how they collect and manage data.  

We have been putting together a case study on how water related NGOs in the field deal with data and how they manage it.  We decided to document why data is important to them, how it influences their work, and what problems they encounter.  We also are offering some solutions and tips to basic problems. 

The first three posts of the series is up. The first two are background on Keystone and their projects. The 3rd is a post on tips for using excel more effectively for data management and analysis.  

http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-i

http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-ii

http://www.indiawaterportal.org/news/tips-working-spreadsheets-data-entry-storage-and-retrieval

This series will continue over the next few months.  If anyone has any other tips or suggestions for excel or any other low cost, low tech solutions that could be worth exploring, please let me know!

Any comments welcome!

Nisha

--
Nisha Thompson
Mobile: 962-061-2245

[Samuel Rajkumar]

Hi Nisha,

Though 'spreadsheets are ubiquitous', I would think a better way of data gathering would be through an Android survey tool, given that smartphones are only slightly less ubiquitous than spreadsheets. Open Data Kit is probably a good free option.

Cheers,

--
For more details about this list
http://datameet.org/discussions/
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Gora Mohanty]

Hi,

At the risk of tooting our own horn, we have built a system for such smartphone/SMS based data collection, based on OpenDataKit, at http://www.fieldata.org . Anyone can register at the site, and the base tier is free of cost.

Please do note the disclaimer that Fieldata is a commercial venture, and I have a stake in it.

Regards,
Gora

[Nisha Thompson]

Thanks guys!

We are looking at teh experience local NGOs have with data, and I'm sorry to say technology is a huge issue.  Either these devices are too expensive or there are coverage problems.  So we are trying to look at many solutions across many technology fronts (including low tech). SMS data collection actually is more useful in a lot of places than an Android App.  

Whether we like it or not spreadsheets are how most data is collected.  We can tout new technology all we like but it's still a ways to go.  

Also a lot of data that is has been collected is sitting in excel sheets or word docs or paper formats that have been collected in the past.  This post has come from many experiences my team had with NGOs trying to clean up past data.  

The upcoming posts will look at new projects that require data collection and how we can help make it more streamline.  

I will add these tools to a list of resources though for further reference!

Nisha

Nisha Thompson
Mobile: 962-061-2245

[Samuel Rajkumar]

Respectfully disagree, Nisha. 

There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone.

Smartphones are now about 4k, much cheaper than a computer that can run Open Office / MS Office. Most survey apps take into account intermittent coverage and cache results locally till they can be uploaded, and so on.

Historical data can sit on a spreadsheet for now, but new data can and should go into a properly normalized database, and then exported to whatever format is convenient. And then there's also the wonderful side effect of not having to version spreadsheets, as anyone with 2013_new_project_methodology_x_survey_results_final-new-updated-2.ods will tell you.

Regards,

[Nisha Thompson]

We're not really disagreeing.   I agree with you, but you can't just say they should use Android App!  Technology adaption is still an issue with NGOs that work in communities, there are language issues (most data is collected in the local language on paper then translated), technology familiarity, and training and so on.  Data management is still done with spread sheets and paper because that is what is easiest to train people with.  You can say an Android is easy and cheap, but with a limited budget an NGO will say I'd rather spend the money on something else. 

The upcoming posts will deal with data standardization, and better ways to collect different types of data.  In which tips and solutions technology related will be discussed in this context.

Nisha

[Gora Mohanty]

On Oct 30, 2013 10:13 AM, "Samuel Rajkumar" <aadv...@gmail.com> wrote:
>
> Respectfully disagree, Nisha. 
>
> There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone.

[...]

Agreed. Our experience has been that most organisations are immediately appreciative of the benefits of data collection based on smartphones. As you note, Android smartphones are now cheap, and even with the one-time sunk cost of purchasing the devices, our platform is usually cheaper than paper-based methods, besides being less error-prone. SMS can be used if there is a big sensitivity to price.

Spreadsheets are also unstructured, and difficult to convert to API-based access, which is needed for any generic analysis methods not customised to a specific data format embodied in the spreadsheet. IMHO, this is the biggest problem with the current data sets on data.gov.in

Regards,
Gora

[Sailendra Mishra]

Hi Guys, 

Have you ever considered any OCR based survey?  OCR based survey may avoid technical and financial constraints (Popular with all major examinations conducted in India). Sample here (http://chemmunish.com/answer%20key%20+2%20Full%20Physical.JPG). 

It require one time investment on a good OCR machine (really not very expensive and  less error-prone).  If the machine is a constraint,  little programming with a fast scanner will do the job. 

However, the success with OCR based survey really depends on the designing of the survey.

Regards

Sailendra 

--

For more details about this list
http://datameet.org/discussions/
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Sailendra Prasanna Mishra

Linkedin: http://in.linkedin.com/in/sailendraprasannamishra

Twitter:  https://twitter.com/sailendram

Skype: Sailendra.prasannam

"Few people in this world who make mistakes in right direction."

[Samuel Rajkumar]

Hi Sailendra,

This is actually OMR - Optical Mark Recognition, which is much simpler and less error prone than OCR.

Regards,

[Gautam John]

In Akshara's experience, because we devolve data collection to the
teachers themselves and have no real control over them, giving them
smartphones would be risky for us because we'd probably lose many of
them. Also, with paper, the replacement cost and time is easy and it
is a known work flow for most government institutions - stamps, seals,
signatures and what not. Lastly, we currently have a error rate of
paper to digital transcription under 0.5% because our data entry work
flow uses a double entry system. Also, at the scale we operate at,
phone devices are expensive and it's much easier to scale paper and
data entry operators.

On Wed, Oct 30, 2013 at 10:37 AM, Nisha Thompson

[Sailendra Mishra]

Thanks Samuel for the correction. 

[Balachander T]

Hi

Data is still collected using pen and paper in most cases. The spreadsheet comes in when the data management part comes in. In most cases, electronic input devices can't straightaway replace this because of a number of factors

1. Data collectors may not be used to using computers/tablets etc.. Smart phone use with active internet usage is quite low.

2. Giving each data collector a device may not be possible given the costs involved.

3. NGOs typically don't have capacity to manage data, leave alone web based apps, tablets etc. Even if they all have android smartphones, someone needs to design a good database. This capacity is also quite rare.

4. Design of questionnaires is also often not done very well, so that you can't predict that the responses will always be from a predefined list etc.

The fundamental issue is the capacity to handle data from design to analysis and presentation. The android tech part is one piece of the puzzle. I see the fieldata type of initiatives helping solve part of the problem. Data literacy is a more urgently needed input.

I do agree that android devices and associated tech such as opendatakit have helped bring down the cost to a potential adopter. However, most NGOs are unlikely to have any budget allocation towards data collection devices, unless they do a lot of surveys.

As Nisha has mentioned, this is a first post in a series. We will be covering other aspects such as technology adoption as well. If any of you has any already documented cases on using opendatakit or any other such app we could benefit from your sharing the experience.

Regards

Bala