why user-based recommendation for MovieLens 100k is very slow?

338 views
Skip to first unread message

Remy Meng

unread,
Jan 16, 2013, 2:10:29 AM1/16/13
to sciki...@googlegroups.com
the benchmarks (http://www.slideshare.net/marcelcaraciolo/crab-a-python-framework-for-building-recommender-systems page-37) 
Benchmarks Pure Python w/ Python w/ Scipy Dataset dicts and NumpyMovieLens 100k 15.32 s 9.56 s http://www.grouplens.org/node/73 Old Crab New Crab


however, in my case, I need more than 30 minutes to do it, I don't know why.

my code is

model = MatrixPreferenceDataModel(recommend_data.data)
similarity = UserSimilarity(model, pearson_correlation)
recommender = UserBasedRecommender(model, similarity, with_preference=True)
recommender.recommend("6")

Raj_B

unread,
Sep 17, 2013, 3:06:04 AM9/17/13
to sciki...@googlegroups.com
I have downloaded the zip file of MovieLens 100k , after unzipping i found different files. Can you please help me which file i have to use and the required code to give that file as input?.... if i'm giving any file as,

from sys import argv
script, filename = argv
txt = open(filename)
movies = txt.read()
print movies
from scikits.crab.models import MatrixPreferenceDataModel
#Build the model
model = MatrixPreferenceDataModel(movies)


It is giving me following error....

Traceback (most recent call last):
  File "movieRecommender_UserBased_trail", line 18, in <module>
    model = MatrixPreferenceDataModel(movies)
  File "/usr/lib/python2.6/site-packages/scikits/crab/models/classes.py", line 88, in __init__
    self.build_model()
  File "/usr/lib/python2.6/site-packages/scikits/crab/models/classes.py", line 108, in build_model
    self._user_ids = np.asanyarray(self.dataset.keys())
AttributeError: 'str' object has no attribute 'keys'
[S244471@CTSINGMRVGHPC5 Crab_practice]$ vi movieRecommender_UserBased_trail

Marcel Caraciolo

unread,
Sep 17, 2013, 3:17:15 PM9/17/13
to Scikit-Crab
Hi Raj and Remy I am still working on this new crab, the old one is deprecated I will need to put it offline. This is the new release:



--
You received this message because you are subscribed to the Google Groups "Scikit-Crab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-crab...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jeongseok choi

unread,
Oct 12, 2016, 10:16:42 PM10/12/16
to Scikit-Crab

I have same problem, I use movielens100k data.
The users ares 943, movies are 1682.

And I use python 2.7 in ubuntu and crab recommender.
I need more than 3 housr to get the result " recommender.recommend(2) "
Can you give some advice?






Reply all
Reply to author
Forward
0 new messages