Example to get started with pymatgen/fireworks/custodian for HT jobs using VASP

[Ganesh]

Hello,

I am beginning to read-up on the different capabilities of the materials project, and was wondering if there is a simple example that I can follow to see how the different pieces of pymatgen/fireworks/custodian come together to do a useful calculation on a cluster? (say running vasp-relaxation-calculations on a few BCC compounds by just changing the elements submitted to a generic computer cluster)

I have all the programs installed, just need an example to get started with things. 

Thanks,
Ganesh

[Anubhav Jain]

Hi Ganesh,

We are holding a small workshop on Sunday/Monday in San Diego, in part covering this topic, and we will post the materials online afterward. Perhaps they will be helpful in answering your question.

Note that there is a https://github.com/materialsvirtuallab/fireworks-vasp repo that contains some basic tasks for writing VASP inputs, running VASP, and parsing the output.

Best,

Anubhav

[Ganesh]

That would be great, in the meanwhile I will look into your fireworks-vasp scripts. I am getting a hang of fireworks now, it is very impressive.

Sincerely,
Ganesh

[Anubhav Jain]

Hi Ganesh

You might want to see the slides here:

http://mavrl.org/software

Also there is a gist on fireworks-vasp

https://gist.github.com/computron/051eeb8c75e3e0c5b7d0

Hope it helps

Anubhav

[Panchapakesan Ganesh]

Thanks. I will give this a try.  --Ganesh

--
You received this message because you are subscribed to a topic in the Google Groups "Materials Project Development Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/matproj-develop/EyvpbhU2UVs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to matproj-devel...@googlegroups.com.
To post to this group, send email to matproj...@googlegroups.com.
Visit this group at http://groups.google.com/group/matproj-develop.

[Ganesh]

Anubhav,

Thanks for the material, I have successfully completed all example runs in your fireworks tutorial. I am also able to generate a workflow using the fireworks-vasp_demo.py to submit vasp jobs to our internal cluster from my desktop where I installed all the packages in a virtualenv. While my vasp jobs complete without errors, I ran into a few problems with the error handlers in custodian for which I would appreciate some help.

1. While my vasp job completes with no errors in my cluster, I get an error from custodian, possible during the error handling process :
FW_job.error:
/projects/pymatgen/2.9.0/lib/python2.7/site-packages/custodian/custodian.py:179: DeprecationWarning: ScratchDir is deprecated; use ScratchDir in monty.tempfile instead.
  with ScratchDir(self.scratch_dir):
Traceback (most recent call last):
  File "/projects/pymatgen/2.9.0/lib/python2.7/site-packages/fireworks/core/rocket.py", line 175, in run
    m_action = t.run_task(my_spec)
  File "/projects/pymatgen/2.9.0/lib/python2.7/site-packages/fireworks_vasp-0.1-py2.7.egg/fireworks_vasp/tasks.py", line 91, in run_task
    output = c.run()
  File "/projects/pymatgen/2.9.0/lib/python2.7/site-packages/custodian/custodian.py", line 225, in run
    skip_over_errors=self.skip_over_errors)
  File "/projects/pymatgen/2.9.0/lib/python2.7/site-packages/custodian/custodian.py", line 303, in _do_check
    if h.check():
TypeError: unbound method check() must be called with VaspErrorHandler instance as first argument (got nothing instead)
mpirun: killing job...

I have changed the handlers to "all" or specific ones such as "VaspErrorHandlers", and I still get this error. Note, I only have "fireworks_vasp" installed in my launchpad, and not in the cluster that runs vasp jobs.  The cluster has pymatgen and custodian installed in it and I can access the attribute custodian.vasp.handlers.VaspErrorHandler successfully. So I am not sure why this error is showing up? I am attaching the json file I used with this post. 

Few other questions:

2. How to do a rapidfire qlaunch in an offline mode? I can always add a task after each vasp-task such as 'lpad recover_offline' that essentially recovers the data (since it is in offline), but maybe there is a simpler way of dealing with it?

3. I noticed that typing 'lpad webgui' gives me an error.  (pasted at the end of my questions)

Thank you.

Sincerely,
P. Ganesh  

>lpad webgui
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/g5q/Codes/FW_env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/home/g5q/Codes/FW_env/local/lib/python2.7/site-packages/django/core/management/base.py", line 331, in execute
    translation.activate('en-us')
  File "/home/g5q/Codes/FW_env/local/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 145, in activate
    return _trans.activate(language)
  File "/home/g5q/Codes/FW_env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 225, in activate
    _active.value = translation(language)
  File "/home/g5q/Codes/FW_env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 209, in translation
    default_translation = _fetch(settings.LANGUAGE_CODE)
  File "/home/g5q/Codes/FW_env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 189, in _fetch
    "The translation infrastructure cannot be initialized before the "
AppRegistryNotReady: The translation infrastructure cannot be initialized before the apps registry is ready. Check that you don't make non-lazy gettext calls at import time.
(FW_env)g5q@tesla:~/Codes/Mproject_test$
(process:20825): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed

To unsubscribe from this group and all its topics, send an email to matproj-develop+unsubscribe@googlegroups.com.
To post to this group, send email to matproj-develop@googlegroups.com.

[Anubhav Jain]

Some updates to this thread that happened over email:

Anubhav,

Thanks for your response.  Please see inline below for answers to your questions: 

1a. Do you have the latest custodian installed? That one should already be using monty and you would avoid the error. In general I would make sure all codes are up to date

 yes I do. It is v0.8.1 for custodian, v0.95 for fireworks and v3.0.7 for pymatgen. 

1b.Do you have the latest fireworks-vasp?  Have you tried setting the handlers to an empty list? You might also try doing a debug print before the line that gives you an error.

yes, that gives no error and the job completes successfully even in offline mode and I can recover it back. But setting the handler gives error.  What is the next simplest choice for handler after an empty list? 

 

3. I am not sure what the problem is here and I've never seen this error; things are working for me. Are you running on a local computer or on a supercomputer? Do you have Django 1.6 installed?

I am running on my local desktop.  I have Django v1.7.1. Do I need to be in a specific directory to run this command? I have tried in the dir. that is set as 'dbpath' .  The contents of the 'dbpath' dir. are pasted below. Also, the result of the command lpad get_fws shows a completed task. 

(FW_env)g5q@tesla:~/Codes/first-database$ ls
fireworks.0  fireworks.ns  journal  local.0  local.ns  mongod.lock  _tmp
(FW_env)g5q@tesla:~/Codes/first-database$ lpad get_fws
{
    "name": "Li4 Fe4 P4 O16", 
    "fw_id": 1, 
    "state": "COMPLETED", 
    "created_on": "2014-12-11T22:21:45.451494", 
    "updated_on": "2014-12-11T22:36:41.524523"
}

Thanks,
Ganesh

[Anubhav Jain]

Hi Ganesh

>> yes, that gives no error and the job completes successfully even in offline mode and I can recover it back. But setting the handler gives error.  What is the next simplest choice for handler after an empty list?

You might try using one of the simple handler names, e.g. ['MeshSymmetryErrorHandler']. The one that has the most complex set of rules is 'VaspErrorHandler', you may want to avoid that one. Unfortunately, I cannot say for sure which one might be causing the problem on your machine, you would need to conduct some systematic tests and report any problems to the pymatgen dev list.

>> I have Django v1.7.1

If possible, you should go back to Django 1.6.5. There are known problems with FWs and Django 1.7; it's mentioned in the latest docs (http://pythonhosted.org/FireWorks/basesite_tutorial.html) as well. In the future someone on the team is working on getting everything working using Flask instead of Django anyway.

[majaoli...@gmail.com]

Hi Anubhav,

are there still slides somewhere or a tutorial/workshop material to get a first impression of how fireworks/custodian/pymatgen go together?

Best,
Maja

[Shyue Ping Ong]

The slides are available on slideshare. You can see them here: https://materialsvirtuallab.org/software/

Shyue Ping

--
You received this message because you are subscribed to the Google Groups "Materials Project Development Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to matproj-devel...@googlegroups.com.

To post to this group, send email to matproj...@googlegroups.com.

Visit this group at https://groups.google.com/group/matproj-develop.

[Donny Winston]

Hi Maja,

You may also consult the notebooks we have online at https://github.com/materialsproject/workshop-2016 from the first Materials Project hands-on workshop last August. The notebooks are viewable in your browser, and you can follow along if you install Jupyter.

Also, we are having our second hands-on workshop this August in Berkeley, and there are still a few spots left: https://discuss.materialsproject.org/t/second-annual-materials-project-workshop/502.

Best,

Donny

[Anubhav Jain]

Note that we are also working on a code called "atomate" that brings together fireworks/custodian/pymatgen. We expect to submit a paper about it soon. There are some basic slides about that here:

https://www.slideshare.net/anubhavster/opensource-tools-for-generating-and-analyzing-large-materials-data-sets

We expect to also have some better docs in the near future here (currently it is more like notes for existing developers):

https://pythonhosted.org/atomate/

Best,

Anubhav