Issue connecting to mongoDB Atlas with fireworks 1.8.9

Marnik Bercx

unread,

Apr 10, 2019, 4:16:12 PM4/10/19

to fireworkflows

First of all, thanks for the great work on developing Fireworks! I've been using the package for a while, and it's been a great tool for running my workflows.

I seem to be running into an issue when trying to connect to a mongoDB atlas server with the new version (v1.8.9). Here's the trace when trying to connect to the mongoDB database using `lpad get_fws`:

```

Traceback (most recent call last):

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 109, in get_lp

return LaunchPad.from_file(args.launchpad_file)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/utilities/fw_serializers.py", line 286, in from_file

return cls.from_format(f.read(), f_format=f_format)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/utilities/fw_serializers.py", line 254, in from_format

yaml.safe_load(f_str)))

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 258, in from_dict

authsource)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 167, in __init__

self.db = self.connection[dbname]

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1323, in __getitem__

return database.Database(self, name)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/pymongo/database.py", line 110, in __init__

_check_name(name)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/pymongo/database.py", line 51, in _check_name

"character %r" % invalid_char)

pymongo.errors.InvalidName: database names cannot contain the character '.'

Traceback (most recent call last):

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 109, in get_lp

return LaunchPad.from_file(args.launchpad_file)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/utilities/fw_serializers.py", line 286, in from_file

return cls.from_format(f.read(), f_format=f_format)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/utilities/fw_serializers.py", line 254, in from_format

yaml.safe_load(f_str)))

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 258, in from_dict

authsource)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 167, in __init__

self.db = self.connection[dbname]

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1323, in __getitem__

return database.Database(self, name)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/pymongo/database.py", line 110, in __init__

_check_name(name)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/pymongo/database.py", line 51, in _check_name

"character %r" % invalid_char)

pymongo.errors.InvalidName: database names cannot contain the character '.'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/Users/mbercx/miniconda3/envs/pmg/bin/lpad", line 10, in <module>

sys.exit(lpad())

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 1171, in lpad

args.func(args)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 216, in get_fws

lp = get_lp(args)

File "/Users/mbercx/miniconda3/envs/pmg/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 121, in get_lp

raise ValueError(err_message)

ValueError: FireWorks was not able to connect to MongoDB. Is the server running? The database file specified was /Users/mbercx/Dropbox/python/sandbox/fireworks/my_launchpad.yaml.

```

The issue can be reproduced by setting up a conda environment and installing fireworks via pip:

```

conda create -n test

conda activate test

conda install pip

pip install fireworks dnspython

```

and trying to get the fireworks from the following launchpad file using `lpad get_fws`:

```

host: mongodb+srv://tutorial-urpwq.gcp.mongodb.net

name: test

port: 27017

username: tester

password: fireworks123

ssl: true

authsource: admin

```

If I set up a new conda environment and pip install fireworks v1.8.8, everything runs smoothly, so it seems to be related to the final update. Any ideas on how to fix this?

Kind regards,

Marnik

Anubhav Jain

unread,

Apr 10, 2019, 4:34:45 PM4/10/19

to Marnik Bercx, fireworkflows

Hi Marnik

The short answer is that you're right, FWS v1.8.9 was supposed to make connecting via a Mongo URI easier but likely ended up making it more difficult / breakable for people. Thanks for reporting!

For the short term, please use FWS v1.8.8, it's the same as v1.8.9 but without the problem you mentioned.

For the medium term, I will look to release a v1.9.0 that fixes things in the next few days (e.g. Friday or so).

Best,

Anubhav

--
You received this message because you are subscribed to the Google Groups "fireworkflows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireworkflow...@googlegroups.com.
To post to this group, send email to firewo...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireworkflows.
For more options, visit https://groups.google.com/d/optout.

--

Best,
Anubhav

Marnik Bercx

unread,

Apr 10, 2019, 4:40:45 PM4/10/19

to fireworkflows

Alright, I'll stick to v1.8.8 for now. Thanks for the quick response!

Kind regards,

Marnik

Op woensdag 10 april 2019 22:34:45 UTC+2 schreef ajain:

To unsubscribe from this group and stop receiving emails from it, send an email to firewo...@googlegroups.com.

To post to this group, send email to firewo...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireworkflows.
For more options, visit https://groups.google.com/d/optout.

--
Best,
Anubhav

Anubhav Jain

unread,

Apr 12, 2019, 2:53:06 PM4/12/19

to fireworkflows

Hi Marnik

Can you see if v1.9.0 works?

Marnik Bercx

unread,

Jul 17, 2019, 4:41:21 AM7/17/19

to fireworkflows

Hi Anubhav,

Sorry for my slow reply, I somehow seem to have missed this message... I've been able to connect just fine for v1.9.0 and beyond, thanks! :)