Prediction Coverage

[Wang Dawei]

Hi 

I found that most algorithms use RMSE or MAE to show their performance. I wonder what's the coverage on the predictions? Are they 100% covered or some of the algorithms actually sacrifice some of the coverage to increase the accuracy?

Best,

Davy

[Zeno Gantner]

Hi Davy,

Sorry for the late response.

You can use the CanPredict method of IRecommender to see whether a model can give you useful (i.e. non-fallback/default) responses.

I hope that helps.

Cheers,

   Z.

--
You received this message because you are subscribed to the Google Groups "MyMediaLite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mymedialite...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Djoels]

I don't know if this is still relevant, but anyway: concerning item space coverage:

I've implemented an ItemSpaceCoverage evaluation class, that computes coverage@K and another version I call coverage@K|LongTailProportion (which removes a portion of the most popular items).

I've used this myself for expanding on the item_recommendation program, but I don't believe this adaptation production-worthy quality software, as it doesn't fit neatly with the other metrics. If I have time to clean it up I will try and set up a git and make a pull request (or whatever they are called). In the meantime, you can get inspiration from my code in the attachment to adapt it for rating data.

(ItemRecommendation.cs belongs in src/Program/ItemRecommendation and ItemSpaceCoverage.cs belongs src/MyMediaLite/Eval/Measures)

Regards,

Julien

Op maandag 23 mei 2016 09:54:22 UTC+2 schreef Wang Dawei:

[Djoels]

You will need this tile too (under src\MyMediaLite\Eval)

Op dinsdag 4 april 2017 12:49:10 UTC+2 schreef Djoels:

[Dawei Wang]

Hi Julien:

Thank you for your kind reply!

Indeed. There is no metric or benchmark that takes in the accuracy together with the coverage. Most papers only mention the predicting accuracy without mentioning their predicting coverage. If some algorithm can make a very accurate predicting but low on coverage, then it’s hard to compare across different algorithms.

Best,

D

--
You received this message because you are subscribed to a topic in the Google Groups "MyMediaLite" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mymedialite/WHsGE2qmkQE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mymedialite...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

<ItemRecommendation.cs><ItemSpaceCoverage.cs>

[Dawei Wang]

Hi Julien:

I wonder how do I call this function? Can you give me an example of the command line if I want to test the coverage of itemknnpearson on MovieLens 100k datasets?

Thank you.

Best Regards,

Dawei