Use of .npz files vs DB

20 views
Skip to first unread message

Maurice Waka

unread,
Apr 30, 2022, 5:06:44 AMApr 30
to web2py-users
Anyone with experience using large .npz files?
I find it to slow the app when calling out functions that access the (lists inside) npz files.
Would storing the lists in the DB be faster?
Regards

Maurice Waka

unread,
Apr 30, 2022, 6:12:39 AMApr 30
to web2py-users
I use it like this:
import numpy as np
data = np.load(os.path.join(current.request.folder, 'modules', 'test_me.npz'), allow_pickle=True )
moa = data['x']
arr = np.asarray(moa)
for item in arr:
    ......


This I find it to be slow

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/f817bf60-7d7b-49a3-bc47-0936b4670d7cn%40googlegroups.com.

jonatha...@whatho.net

unread,
Apr 30, 2022, 5:24:01 PMApr 30
to web2py-users
I'm probably wrong, but isn't this a numpy question?

Maurice Waka

unread,
May 5, 2022, 2:55:22 PMMay 5
to web...@googlegroups.com
Solved.
I had a bug in one of my modules sharing the same npz files. That's why there was a slow down in performance. 
Regards 

Reply all
Reply to author
Forward
0 new messages