Use of .npz files vs DB

[Maurice Waka]

Anyone with experience using large .npz files?

I find it to slow the app when calling out functions that access the (lists inside) npz files.

Would storing the lists in the DB be faster?

Regards

[Maurice Waka]

I use it like this:

import numpy as np

data = np.load(os.path.join(current.request.folder, 'modules', 'test_me.npz'), allow_pickle=True )

moa = data['x']

arr = np.asarray(moa)

for item in arr:

    ......

This I find it to be slow

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/f817bf60-7d7b-49a3-bc47-0936b4670d7cn%40googlegroups.com.

[jonatha...@whatho.net]

I'm probably wrong, but isn't this a numpy question?

[Maurice Waka]

Solved.

I had a bug in one of my modules sharing the same npz files. That's why there was a slow down in performance. 

Regards 

To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/fa4fb44b-3591-4bad-b880-7e73858b002cn%40googlegroups.com.