Data Size Notice

[Ben Webb]

We've become aware that the data published by the United States ( http://iatiregistry.org/publisher/unitedstates )  has increased in size recently from about 1GB to over 3GB - this brings the total size of the IATI dataset to over 4GB.

We just wanted to share this information so that anyone consuming data on an overnight, or automatic, process is forewarned. This may or may not affect your processes.

Some of the new files are too big for the Registry to process (e.g. http://iatiregistry.org/dataset/unitedstates-ht which is 112MB, whilst the registry has a hard limit of 50MB). Tools that rely on the Registry's update metadata (such as the Datastore), will not yet have updated the activities in such files.

Additionally, the Dashboard's automatic data download process failed today, since the server ran out of disk space, so the Dashboard website is a day out of date.

Best regards,

IATI Technical Team