External Item Recommender weird behaviour

[Djoels]

I tried feeding the recommendations from the Kaggle MSD challenge winning algorithm (from Fabio Aiolli) to the External Item Recommender.

In order for this to work I had to rewrite part of the ItemRecommendation program to extract the visible (test) and training tuples and test users to seperate files.

Additionally I added a class to read a prediction file (since the original External Item Recommender code needed ratings file, while predictions files are "user\t[item1:score1,item2:score2,...itemN:scoreN]" formatted.

However, in the end, it appears that AUC = 0.5, and all other metrics (MRR, NDCG, ...) are 0. Has anybody encountered this kind of issue? 

Any idea what might be wrong or tips on how to debug this (multithreaded) code?

[Zeno Gantner]

Am 04.04.2017 12:08 nachm. schrieb "Djoels" <juv...@gmail.com>:

I tried feeding the recommendations from the Kaggle MSD challenge winning algorithm (from Fabio Aiolli) to the External Item Recommender.

In order for this to work I had to rewrite part of the ItemRecommendation program to extract the visible (test) and training tuples and test users to seperate files.

Additionally I added a class to read a prediction file (since the original External Item Recommender code needed ratings file, while predictions files are "user\t[item1:score1,item2:score2,...itemN:scoreN]" formatted.

However, in the end, it appears that AUC = 0.5, and all other metrics (MRR, NDCG, ...) are 0. Has anybody encountered this kind of issue? 

Without the code it is hard to say what is going wrong. 

Have you tried the external recommender with other data? 

The evaluation also usually outputs the number of cases - is this the number you are expecting? 

Any idea what might be wrong or tips on how to debug this (multithreaded) code? 

One way would be to start with a smaller subset of the data and a single threaded variant of the code. If that also behaves badly, then the bug is not in multithreading part. 

Cheers, 

  Z