Proper usage of secondary indications

[Jerard]

Hello actionml team,

First of all, let me say how great you product is! Really enjoying working with it!

And I'd like to ask you for the proper usage of secondary indicators in UR.

Let's say we're building a movie recommendation engine. So, we're operating with 2 entities user and movie. Movie has many properties such as genres, release year, writers, cast, etc. User has its own properties such as location, language, gender etc. Obviously, users watch movies and then create ratings. And primary indicator is like.

So, is it better to:

1. Track secondary indicators for each movie attribute, each time when tracking the primary indicator for this user? In other words, send primary event: like with movie id AND events for each genre "genre_like" with genre id; "writer_like" with writer id. Or, just like, secondary indicators should be sent after some other user actions?
2. In case when user rated movie and it was OK but it didn't become his favorite (no primary indicator). Is it better just track "watch" (secondary indicator) for this movie, or, also send "genre_watch", "writer_watch" etc. similar to what is described in 1)? Can tracking of secondary indicators for not favorite movies create unneeded noise for recommendations?
   
3. When and how often to track user attributes? Let's say I want to track user's language. Shall I send "user_language" event each time when the user rates a movie or one time would be enough (during user registration for example)?

Thank you in advance!

[Pat Ferrel]

indicators are actions that users take. User properties often give no “indication” of their taste in movies, only their actions do.

Think of indicators fitting into the following statement: User #n prefers some-thing. So User #n prefers “Nordic-noir” movies or “Brad Pitt Movies” etc. We record these with user-id, actor-preference, actor-id etc.

User attributes are much more difficult to use and often give no good indication for product perferences. The exception is sometime location of the person might indicate a preference for some local product. But the indicator must still fir the statement mentioned above and the indicator must record some preference based on some action the user took.

Even gender is often not much of an indicator.

I have seen only a few cases where user attributes were of much good in making rec and that would depend on your exact use case so if you want to explain more I can say what we have seen about that.

--
You received this message because you are subscribed to the Google Groups "actionml-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to actionml-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/2d185c8a-3e30-4373-aeb1-8f26481a8706n%40googlegroups.com.

[Pat Ferrel]

Are you really talking about movies, with genres and actors? We have found the strongest indicator of user preference for a movie is if they watch some % of it, more than the average. No one watches all the credits so maybe watch-95% indicates a strong preference. If you don’t have this info the watch is stronger than thumbs up, since it record the precise action that you want to encourage. Thumbs us is ok if that is all you have

dislike/thumbs down is also a good indicator but may be counter-intuitive. People who dislike the same movies also tend to like the same movies. So a dislike can help recommend what they might like.

We did experiments on the dislikes and found tthem faairly useful.

BTW free OSS support goes to the community group.

On Mar 20, 2023, at 2:19 PM, Jerard <sergiy.o...@gmail.com> wrote:

So, no user-related indications. Got you!

What about “Like Nordic-noir”, “Watch Nordic-noir”, “Dislike Nordic-noir”, “Like Brad Pitt Movies”, “Watch Brad Pitt Movies”, “Dislike Brad Pitt Movies”?

Worth to track it all or just “Like Nordic-noir” “Like Brad Pitt Movies” would be more beneficial to detect user's taste?

Thanks,

[Sergiy Onyshchuk]

Pat,

I accidentally hit wrong button that’s why this massage landed to your personal box… I will create other topics in the user group for sure!

We indeed are building the recommendation engine for movies. Also, we don’t have the data of watched % of a certain movie, but the “direct user indicators” such as like, dislike, watch, wishlist. And “curiosity user indicators” such as trailer watch - yes/no, found and navigated to movie from search page - yes/no, click on corresponding streaming service to buy it. Ofcourse, liked movie’s genres, keywords, actors and other attributes could be converted to “category-preference” and tracked too.

I will appreciate any recommendations for setting up UR with this data and for such use case.

Meanwhile, we’re going to build an engine with as mach as possible secondary indicators and then start decreasing number of them to detect the best variation and achieve best result. I can share our A/B winner if you like.

Thank,

Sergiy

[Pat Ferrel]

Before you start testing secondary indicators, try all reasonable primary indicators.

You are planomg to a/b test all these? Either you have huge traffic or you’ll need to wait for a very long time to reach significance.

If you have “watch” I’d guess that is the best. Like, dislike, and wishlist are more indirect and for business purposes you don’t care if they dislike, if they watch—or I assume so.

The best primary is important sonce all secondary indicators are tested for correlation with it. If you have a weak primary or not enough data for it, the others will suffer too.

That said, if “watch” is the best then all will add to recs being better or will have no effect. However adding more will have a dimoinishing effect.

In other cases we founs:

watch: best primary

category-pref or genre pref, taken from the genre of the watched items

dislike

search-terms: the id would be the token of the te3rm searched for after fulltxt analysis (stemming, etc)

The things like share, like, wishlist, are not much different than watch data.

Just a guess since I haven’t seen the app.

Good luck and of course share results if you can.

To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/F0D29BB4-F716-46A9-A583-EACE6DB644B2%40gmail.com.

[Jerard]

Thanks for the recommendations Pat,

One more question. When user liked a movie with Brad Pitt. Shall I send the secondary indicator "liked_actor": "Brad Pitt" (aka. category preference) next to primary indicator, so then, movies with Brad Pitt would be in more favor for this user? Or Universal Recommender will do this correlation on it's own because all items/movies have "actors" property with all the actors where Brad Pitt will appear and influence the recommendations in this way?

Thanks!

[Pat Ferrel]

No not quite. If you "liked_actor": "Brad Pitt” then other people who also "liked_actor": "Brad Pitt” will affect your recommendaations. Brad Pitt may have nothing to do with the movies you get in recs. 

THIS IS A VERY IMPORTANT POINT and tthe key to understanding collaborative filtering.

A users perferences are matched with other user’s preferences to make common taste based recs. The specific common preferences may not be reflected in the recs, only the common taste is used to make recs.

So if I like lots of movies with Brad Pit an so did you, then maybe we both like blockbuster action movies—only an example, not sure this actually makes sense. But the recs may not have Brad Pit in them at all—or they may. There is no favor given to Brad Pit, only to movies that peopkle of common taste also liked.

Collaborative filtering is about preferences FROM “birds of a feather” not the common prefs that got you into the group. 

To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/8b7c73e8-0b18-4afd-98ee-535186f0034dn%40googlegroups.com.