New plugin: fetching genres from rateyourmusic.com

[Joao Azevedo]

Hi!

I've been using beets for over an year now and decided on getting involved in improving it by trying to solve one of my ongoing problems: dealing with genres.

I use rateyourmusic.com frequently to find new music, either by peeking at ratings or by searching through genres. Genres in rateyourmusic.com are hierarchical and subject to community voting. I tend to find them fairly accurate, and usually prefer them over the last.fm tags (and therefore the genres that lastgenre fetches). Hence, I decided to create a plugin to fetch genres from rateyourmusic.com and assign them to albums and items. The plugin is available at http://github.com/jcazevedo/beets-rymgenre.

Unfortunately, rateyourmusic.com doesn't provide a webservice or an API, so it is necessary to scrape the website for the desired information. The plugin depends on lxml and requests. I don't write Python frequently, so I'm not sure if there are other dependencies that are more lightweight and serve the same purpose.

Any feedback is appreciated.

Thanks for the great software!

        Joao

[Adrian Sampson]

Very cool! This looks quite solid.

A very long time ago (back in the Google Code days) someone proposed using RYM as a full-fledged metadata source:
https://code.google.com/p/beets/issues/detail?id=144
Although nothing ever came of it.

Anyway, this looks like a great plugin already. If you’re interested, we can explore folding it into beets core—in which case we could perhaps see if there any commonalities with `lastgenre` that we can merge into a common core.

Thanks for contributing!

Adrian

> --
> You received this message because you are subscribed to the Google Groups "beets" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to beets-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Joao Azevedo]

On Wed, Oct 29, 2014 at 12:03 AM, Adrian Sampson <adrian....@gmail.com> wrote:

Anyway, this looks like a great plugin already. If you’re interested, we can explore folding it into beets core—in which case we could perhaps see if there any commonalities with `lastgenre` that we can merge into a common core.

That seems like a good idea. Shall I open a pull request and we work from there?

        Joao

[Adrian Sampson]

Yes, sounds perfect!

[Joost de Bonth]

Joao,

Wow, this is just great, thanks for making this available! I agree that the RYM genre information is by far the most relevant and accurate. I have spent a scary amount of time manually tagging my entire library in this genre format and find it absolutely indispensable!

I have managed to install beets and have your rymgenre plugin up and running. I have a few questions and would be very grateful if you could help me out.

- Is it possible to customise the genre information? Specifically what I am trying to do is to keep the genre hierarchy information as described in https://rateyourmusic.com/rgenre/. For example, instead of just 'Dark Ambient' I would like to have 'Ambient/Dark Ambient', or instead of 'Minimal Techno' I would like to have 'Electronic/Dance/Techno/Minimal'. Perhaps this could be done via a series of if-then statements which would be a perfect workable solution for me. Where would I need to set these up (within the __init__.py or genres-tree.yaml)?

- Is it possible to automatically run rymgenre from within beets during the music import process and have the rymgenre tags override those acquired by beets? I understand that in the current implementation you need to first import music and then as a second step, run rymgenre. It would be nice to get a group of newly acquired albums batch tagged in one go.

Thanks again!

Joost

[Joao Azevedo]

Hi!

On 21/04/15 14:56, Joost de Bonth wrote:
> - Is it possible to customise the genre information? Specifically what I
> am trying to do is to keep the genre hierarchy information as described
> in https://rateyourmusic.com/rgenre/. For example, instead of just 'Dark
> Ambient' I would like to have 'Ambient/Dark Ambient', or instead of
> 'Minimal Techno' I would like to have 'Electronic/Dance/Techno/Minimal'.
> Perhaps this could be done via a series of if-then statements which
> would be a perfect workable solution for me. Where would I need to set
> these up (within the __init__.py or genres-tree.yaml)?

The default plugin configuration already sets all the ancestor genres,
but doesn't repeat genres.

If I understand it correctly, you would want to have it repeated in the
genres, right? For instance, if an album would have the genres "Black
Ambient" and "Dark Jazz", you would want to have it tagged with
"Ambient/Dark Ambient/Dark Jazz, Ambient/Dark Ambient/Black Ambient". Is
that correct?

I see one issue with that, because, even though the file is called
'genres-tree', it is not actually a tree, since a given node can have
more than one parent. See "Ambient Dub" for example, which is a child of
both "Ambient" and "Electronic".

If you would somehow circumvent this issue, I think you can either
modify the 'genres-tree.yaml' for each individual genre to have the full
list of ancestors, using "depth: none" in the plugin configuration, or
modify the `_get_genres` method in '__init__.py' to build the genre
definition that way.

> - Is it possible to automatically run rymgenre from within beets during
> the music import process and have the rymgenre tags override those
> acquired by beets? I understand that in the current implementation you
> need to first import music and then as a second step, run rymgenre. It
> would be nice to get a group of newly acquired albums batch tagged in
> one go.

Currently no. The top result of rateyourmusic.com is often not the one
you want, so I decided to always force manual confirmation. This is not
compatible with beets auto tagging when timid mode is disabled.
Therefore, I decided to keep it separated.

> Thanks again!

Thank you for using it, and reminding me that I should've opened a pull
request with this to beets core a long time ago!


Joao

[Jonathan Thomas]

Just spotted this, great work.

Looks like they've had a ticket for an API since 2009, bit of a while ago http://rateyourmusic.com/rymzilla/view?id=683

[Luis Murphy]

Doing God's work here, I knew checking if someone had done it first would save me some effort haha. It's a shame that RYM still hasn't published that API after ~6 years.

As for the genres-tree dilemma, I did some quick research and found that a directed acyclic graph (https://en.wikipedia.org/wiki/Directed_acyclic_graph) would probably model the genre tree structure more accurately. Although I'm (very) new to Python, I could try to hack something together, if you're interested in the idea.

Cheers!
Luis

[Joao Azevedo]

Hi!

On 25/06/15 06:12, Luis Murphy wrote:
> As for the genres-tree dilemma, I did some quick research and found that
> a directed acyclic graph
> (https://en.wikipedia.org/wiki/Directed_acyclic_graph) would probably
> model the genre tree structure more accurately. Although I'm (very) new
> to Python, I could try to hack something together, if you're interested
> in the idea.

It's already implemented as such. For instance, the "Ambient Dub" genre
is a children of both "Electronic" and "Ambient", so you would get both
these parent genres with the `depth: all` configuration option (which is
the default).

My question was more of a presentation issue regarding Joost's question,
who, if understood correctly, wanted both explicit ancestors lists (i.e.
"Ambient/Ambient Dub, Electronic/Ambient Dub") in the genre tag.


Joao

[tom roth]

Hi

i really like your plugin, it addresses a problem i have for decades.

To elaborate a bit:

Ultimately i want to have the parent genre (oldest ancestor) in item.grouping, the primary genre in [item.genre, album.genre] and the whole set of genres in a custom attribute album.multi_genre. This does not break the genre browse tree for clients that do not support multiple genres and still allows me to search for mutliple genres. Also i will be able to use the grouping information in a browse tree like grouping/genre/artist.

I started hacking on lastgenre to write into item.grouping. 

Now lastgenre uses all lower case in genre-tree.yaml.

Would you be open to also use lower case so that a single file could be used for both extensions?

In the end i would like to call rym_genre when lastgenre failed to provide a genre (or the other way around).

Would it be possible to have one of the genre taggers fire an event hook that could be picked up by the other?

Anyways, nice work so far.