Odin Open Data
Nancie Fazzari
As the 2030 deadline for the Sustainable Development Goals (SDGs) approaches and the international community grapples with the impacts of a global pandemic, climate change, and multiple humanitarian disasters, the need for better data is greater than ever before. Since our founding, Open Data Watch (ODW) has advocated for resilient statistical systems that make data openly available to facilitate evidence-informed policymaking and good governance.
Many factors affect the scores of a country. For the 2022/23 round, the impact of the COVID-19 pandemic must be acknowledged. The ODIN 2020/21 assessments took place just as COVID-19 was spreading around the globe. Many NSOs struggled to adapt to working remotely, but they were still working with data collected prior to the pandemic. According to the United Nations Statistics Division (UNSD) and the World Bank, 96 percent of countries limited or stopped face-to-face data collection in May 2020. These disruptions persisted in 57 percent of countries the following year. The 2022/23 ODIN assessments clearly reflect the impact of these disruptions, particularly in countries without experience in remote data collection, signaling a strong need for increased investment to build resilient data systems. Recovery and further progress will require investments in staff and technology and a commitment to open data that serve the needs of all users.
ODIN 2022/23 overall scores ranged from 1.3 to 90.4. Figure 1 shows the scores for each country included in ODIN 2022/23. Hover over a country to see its coverage, openness, and overall scores or click the country name to see a detailed profile of its scores and comparisons with previous years.
More countries have made their data available online. ODIN 2022/23 includes 195 countries, a net increase of 8 from 2020/21: 7 are new to ODIN, 3 returned after an absence, and 2 countries were dropped from the assessment because their websites could not be reached. The median overall score of all countries has continued to improve, but 77 countries lost ground, and there is evidence that the rate of progress has slowed.
In 2022 official statistics were more accessible, but data for many indicators became less available. The median ODIN openness score, which measures the accessibility of official statistics, reached 52.6, 0.6 points higher than in 2020 for comparable countries. The median ODIN coverage score, which measures the availability and completeness of official statistics fell by 0.8 points to 47.5.
Changes in methodology in ODIN 2022/23 may have contributed to the decline in coverage scores. These include requiring more indicators to have sex disaggregated data and modifications to the energy category that require datasets to report on more than just electricity supply and consumption. However, the effect of these changes may have been somewhat offset by the first-time inclusion of data from SDG portals maintained by official government entities that were not linked from the NSO site. See the Methodology section for further information.
ODIN assesses the coverage and openness of 22 categories of statistics in 3 major categories: Social Statistics, Economic and Financial Statistics, and Environmental Statistics. Figure 3 shows how these major category scores have changed over time for the 165 countries that have appeared in every ODIN assessment since 2016. The Economic and Financial Statistics category has consistently received the highest overall scores in ODIN assessments, both for coverage and openness. The Social Statistics overall score fell in 2017 due to methodological changes, but scores have risen by 10 points since 2016, adding 2 points since 2020. Environmental Statistics surpassed Social Statistics in 2018 but decreased by 1.1 points in 2022. This reflects a significant challenge that countries must overcome, particularly as they work to measure risk factors and plan strategies for climate change mitigation and adaptation.
Every country is different, but countries in the same region often share a common experience. Figure 5 shows the progress by region of countries that have participated in all ODIN assessments since 2016. The largest increase was in Eastern Asia, where the median score rose by 22 points between 2016 and 2022, despite falling slightly in 2022 from 2020 (as discussed in the section, Regional Analysis). The greatest opportunities for improvement occur in regions that have lower scores and thus, more room for progress. The Caribbean region is an example, increasing its regional score by over 17 points starting from a median regional score of just 23. The smallest improvements are often found in high-income countries that already have the highest ODIN scores. North America, one of the highest performing regions in 2016, is the only region where the median score has fallen from its 2016 level, but gains were small in Northern Europe and Pacific Islands. Closing the last mile remains a significant challenge for all countries.
Eleven categories of statistics that include sex-disaggregated data or indicators unique to women, such as maternal mortality ratios and access to reproductive health care, make up the ODIN Gender Data Index (OGDI). These data are crucial for monitoring gender equality outcomes and taking action to promote the welfare of women and girls. The OGDI has consistently lagged non-gender data categories in ODIN. OGDI coverage scores, measuring the availability of gender data, have not made meaningful improvement since 2020 and remain consistently behind non-gender data categories, but in 2022 the median openness OGDI score came within 1 point of the openness of non-gender data for the first time.
Figure 7 shows changes in coverage and openness scores between the last 2 rounds -- 2020 and 2022 -- for the 185 countries assessed in both rounds. (These values differ from those reported in Figure 2 because Figure 7 includes 20 countries that did not appear in one or more earlier rounds. It does not include countries that appeared in 2022 for the first time.) The median coverage score dropped by 0.8 points, while the median openness score increased by 0.6 points.
ODIN coverage scores are calculated by assessing indicator availability, their disaggregations, the presence of subnational data (first and second administrative level), and how often the data are collected and published. ODIN openness scores are calculated by assessing machine-readability, non-proprietary data formats, download options, completeness of metadata, and the data license. Figure 8 shows the median scores for each element of coverage and openness for the 185 countries assessed in 2020 and 2022.
In 2022, the most improved coverage element was data availability at the first administrative level. Subnational data (at both the first and second administrative level) are consistently the lowest scoring coverage elements in ODIN. First and second level administrative data are more difficult for countries to collect because they require more resources. Even when these data exist, many countries are reluctant to publish them because of lack of knowledge about how to adequately anonymize the data. The slight increase in first administrative level data may indicate that some countries are learning how to anonymize their data appropriately.
Additionally, there was a very small increase in the amount of data made available in the last 10 years. An increase of scores for this element happens when countries start to publish more historical datasets or data in time series. A mistake many countries make when developing new data portals or websites for publishing data is only focusing on the most recent data, often neglecting to upload data that would allow users to see trends over time. The slight increase in ten-year data availability may indicate that some countries are reversing this mistake.
The most improved openness element in 2022 was metadata availability. ODIN does not follow any specific metadata standard but looks for three components of reference metadata that are common in all metadata standards: a definition of the indicator; name of the responsible agency for the dataset; and the date when the dataset was uploaded to the website. These reference metadata represent the minimum metadata that countries should publish alongside datasets to ensure they can be accurately interpreted by users and so that data producers can be held accountable for the timely publication of data.
Openness element scores for machine readability and download options also increased, but by less than 1 point, and non-proprietary format scores stayed the same. Some of these changes can be mostly attributed to methodology changes in 2022 that are described in the Methodology section. Non-proprietary scores are the highest scoring openness element because many governments publish data in PDF format, which is non-proprietary but not machine-readable. Ideally, countries publish data in open formats, that are both machine readable and non-proprietary. Countries that make data more available in machine-readable formats are more likely to adopt a format that is both machine readable and non-proprietary, such as CSV or XLSX (but not including DOCX files). Therefore, machine-readability scores are a better predictor of whether countries are moving to open formats.
The use of an open license remains the greatest shortcoming among all elements of data openness in 2022. Although scores for terms of use improved for 50 countries and remained unchanged from 2020 in 70 more, 65 countries saw their scores fall. The result was that the median score for all countries fell by 9.1 points, although the average score was pulled up by large improvements made by a few countries.
Figure 9 shows the changes in the median coverage and openness and overall scores of the 22 statistical categories assessed by ODIN. The overall score plateau in Environmental Statistics was driven by declines in Energy and Built Environment . Although Economic and Financial Statistics continued their improvement, only National Accounts meaningfully improved its coverage, and Social Statistics experienced a drop in the openness score of Health Facilities.
64591212e2