atomate 0.7.8+ throws json error with structure optimization

[ard...@lbl.gov]

Summary

When trying to run a structure optimization, newer atomate versions throw monty error when doing `loadfn(zpath("custodian.json"))` during the `FWAction` update.

Test code used to generate the wf:

from pymatgen import MPRester
from atomate.vasp.workflows.presets.core import wf_structure_optimization
structure = mpr.get_structure_by_material_id('mp-1')
wf = wf_structure_optimization(structure)
lpad.add_wf(wf)

Stacktrace from running

/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/pymatgen/__init__.py:89: UserWarning:
Pymatgen will drop Py2k support from v2019.1.1. Pls consult the documentation
at https://www.pymatgen.org for more details.
  at https://www.pymatgen.org for more details.""")
/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/pymatgen/io/cif.py:44: UserWarning: Please install optional dependency pybtex if youwant to extract references from CIF files.
  warnings.warn("Please install optional dependency pybtex if you"
ERROR:custodian.custodian:
{ 'actions': [ { 'action': { '_set': { 'ISYM': 0}},
                 'dict': 'INCAR'}],
  'errors': [ 'rot_matrix'],
  'handler': <custodian.vasp.handlers.VaspErrorHandler object at 0x2b80de941ac8>}
/projects/matqm/bin/vasp.std: no process killed
Traceback (most recent call last):
  File "/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/fireworks/core/rocket.py", line 262, in run
    m_action = t.run_task(my_spec)
  File "/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/atomate/vasp/firetasks/run_calc.py", line 207, in run_task
    return FWAction(stored_data=loadfn(zpath("custodian.json")))
  File "/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/monty/serialization.py", line 82, in loadfn
    return json.load(fp, *args, **kwargs)
  File "/opt/python/lib/python3.4/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/opt/python/lib/python3.4/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
INFO:rocket.launcher:Rocket finished

The same problem persists when using any structure. 

The end of this stacktrace makes it appear the problem is with the JSON library. After looking into it some more, it is a problem with 

return FWAction(stored_data=loadfn(zpath("custodian.json")))

particularly that `loadfn` doesn't seem to work on a zipped json file?

Using atomate 0.7.7 with an identical workflow and configuration works fine. It appears the problem might be related to Github issue #231.

System

• Distribution: CentOS release 6.6 (Final)
  
• Linux kernel 2.6.32-696.18.7.el6.x86_64
  
• Python 3.4.2
  

Python package versions from pip list

**Package**|**Version**

:-----:|:-----:

atomate|0.8.4

certifi|2018.11.29

chardet|3.0.4

Click|7.0

custodian|2018.8.10

cycler|0.10.0

decorator|4.3.0

FireWorks|1.8.2

Flask|1.0.2

flask-paginate|0.5.1

gunicorn|19.9.0

idna|2.7

itsdangerous|1.1.0

Jinja2|2.10

kiwisolver|1.0.1

MarkupSafe|1.1.0

matplotlib|2.2.3

monty|1.0.4

mpmath|1.0.0

networkx|2.2

numpy|1.15.4

palettable|3.1.1

pandas|0.22.0

pip|18.1

pydash|4.7.4

PyDispatcher|2.0.5

pymatgen|2018.11.30

If anyone else has had similar issues, I'll make a bug report on the repo, and we should look into why the tests are not catching this problem.

[ard...@lbl.gov]

This problem might be because of a monty (or some other package) versioning issue. Not saying it is 100% to do with the atomate code, just would be good to get another set of eyes on this.

[ard...@lbl.gov]

After trying this on Cori (and also on Alex G's setup), this issue is a monty issue specific to Python 3.4, not Python 3.6+

So for most people it will be a non issue.

[ard...@lbl.gov]

**specifically the json module of Python 3.4

[Anubhav Jain]

Hi Alex

I guess this is an upstream problem with monty. You could try asking monty to support Py3.4 if desired.

In the meantime I will update the atomate docs so it's clear that only Py3.6 and higher are supported