Pytables

27 views
Skip to first unread message

Bil Westerfield

unread,
May 27, 2012, 10:05:51 PM5/27/12
to PiCloud
I am trying to convert the function below to utilize Pytables instead
of pickle for reading in the array and saving the resulting array. I
have been struggling with the correct syntax for cloud.files.getf and
cloud.files.putf. Any suggestions or guidance would be greatly
appreciated.

The function calculates the cosine similarity between each row in the
array. The cosine calculation is called from a function declared
earlier in the program.

The array I would like to run this function against is very large (60
columns X 1M+ rows). My thought process is pytables may give me
better performance and enable easy querying when I download the
resulting file from the PiCloud.

Thanks,
Bil


def knn ():
neighbors=pickle.loads(cloud.files.getf('neighbors.pkl').read())
probe=neighbors
rlist=[]
for i in probe:
for j in neighbors:
rlist.append(cosine(i,j))
rarray=asarray(rlist)
cloud.files.putf(pickle.dumps(rarray),'cosine.pkl')
return

Amit Saha

unread,
May 27, 2012, 11:11:15 PM5/27/12
to pic...@googlegroups.com
Hello:
Since you are trying to use the filename/path, you wuold like to try:
put/get instead of putf/getf
(http://docs.picloud.com/moduledoc.html#module-cloud.files)

Hope it helps.

-Amit



--
http://echorand.me
Reply all
Reply to author
Forward
0 new messages