pandasdmx unable to create DataFrame for some OECD data

152 views
Skip to first unread message

Petras Saduikis

unread,
Nov 9, 2017, 5:12:26 AM11/9/17
to Statistical data and metadata exchange with SDMX and Python


I’m using pandasdmx version 0.7.0, and though I’m having success with OECD data sets with a Frequency dimension, there are other data sets like the Fossil Fuel Supports data sets, which have no Frequency dimension where I’m unable to create a pandas DataFrame, like below:

from pandasdmx import Request

# http://stats.oecd.org/sdmx-json/data/FFS_BRA/all/all

oecd = Request('OECD')

try:
    data_response = oecd.data(resource_id='FFS_BRA', key='all/all')
except UnicodeDecodeError:
    pass
except KeyError:
    pass
else:
    oecd_data = data_response.data

    print(oecd_data.dim_at_obs)
    series_list = list(oecd_data.series)
    print(len(series_list))
    print(series_list[0].key)
    print(set(s.key.FUEL for s in oecd_data.series))

    fuel = (s for s in oecd_data.series if s.key.FUEL == 'HARDCOAL')
    df = data_response.write(fuel)

print("completed ...")

Output:

TIME_PERIOD
25
SeriesKeyTuple(MEA='BRA_DT_03', FUEL='HARDCOAL',  IND='CSE',INC='CONSUMPTION', STG='GENER', MEC='DT', LVL='FED')
{'NONBIOJETK', 'LIGNITE', 'HARDCOAL', 'NAPHTHA', 'NONBIODIES', 'CRUDEOIL', 'RESFUEL', 'NATGAS', 'NGL', 'LPG', 'NONBIOGASO'}

File "stackoverflow_dataframe.py", line 23, in <module>
df = data_response.write(fuel)
File "pandasdmx\api.py", line 635, in write
return self._writer.write(source=source, **kwargs)
File "pandasdmx\writer\data2pandas.py", line 109, in write
reverse_obs, fromfreq, parse_time))
File "pandasdmx\writer\data2pandas.py", line 107, in <genexpr>
series_list = list(s for s in self.iter_pd_series(
File "pandasdmx\writer\data2pandas.py", line 228, in iter_pd_series
obs_values, index=series_index, name=series.key)
UnboundLocalError: local variable 'series_index' referenced before assignment

Is this the right approach for non-frequency dimension data sets, or have I just coded it wrong?

Reply all
Reply to author
Forward
0 new messages