Developing an ID3tag2RDF converter using the Music Ontology
Antoine Zimmermann
I am currently developing a java application which would produce a RDF
document according to the ID3 tags in MP3 files. This raises a lot of
issues, some of them possibly connected with the development of the
Music Ontology.
This is a list of important issues:
- what URI should be used for tracks, albums, artists? All these
things are not uniquely identified by the corresponding ID3 tag. A
track can hardly be identified by its title. Many occurences of the
same title can appear on many different albums, by many different
artists. Idem for albums. Identifying an album or a track implies (at
the very least) identifying the album/track artist. Several artists
with the same name exist. How to disambiguate them?
- what can be done when the tags are incomplete (like, there is only
the track title and number)?
To implement the converter, I use an ID3 tag API (http://
javamusictag.sourceforge.net/) together with Jena (http://
jena.sourceforge.net/) to generate a RDF model, which is finally
rendered as an RDF/XML file.
The main method takes a file name or directory as an argument, look
for MP3 files in the directory and subdirectory, and for each MP3
file, produce new RDF statements, added to a common RDF model (Model
is a Jena class). Moreover, I computed a class for the Music Ontology,
using schemagen (http://jena.sourceforge.net/how-to/schemagen.html).
I make several assumptions in order to produce the RDF model:
- ID3 tag ARTIST identifies the artist: if this tag is set, then it
creates an resource for the artist, using a private base artist URI
(let's say http://myuri/) and a SHA-256 code from the artist's name.
This resource is asserted to be of type mo:MusicArtist. If no artist
is given in the tags, no resource is created. An artist has a name.
- An album is identified by the combination of an ARTIST tag and an
ALBUM tag. If the album tag is not present, no resource is created. If
album is present but not the artist, no URI is created, but an
anonymous resource (blank node) is made, with the property "rdf:label"
set to the album name. If both album and artist are given, a new URI
is made, with a base URI equal to the artist's URI, plus the SHA-256
code of the album name. Albums are of type mo:Record. An album has a
release date, a name, an artist (if specified), a number of tracks and
a number of discs.
- A track is identified by its TITLE, ALBUM and ARTIST tag (actually,
there should be another resource based on TITLE and ARTIST only, which
could appear on several albums). If the title and/or artist and/or
album is not set, then an anonymous resource is created with the
associated properties (tracknumber, album (if any), artist (if any),
genre, composer (if any), lyrics (if any)). This resource is of type
mo:Track.
Issues related to the Music Ontology:
Problems with the music ontology: there is no simple means to provide
the release date. The release date is usually given as a year in ID3
tags. In order to set the release year, I have to use the following
sequence of properties : a mo:Record is the mo:manifestation of a
mo:Signal which mo:time is a time:Instant, which has a time:inDateTime
which has a time:year equal to the corresponding ID3 tag.
There is no such property as "lyrics of a song". is the MO, lyrics are
only connected to the libretto or record in which they appear. A track
does not identify the album in which it appears. In fact, a track is
identified by an album as being a part of it. I find it more natural
the other way around, so I suggest adding the inverse property. There
is no property to indicate the number of discs of an album.
Further Work:
There are possible extensions to the application, e.g., by using the
file name and/or directory structure to identify artists, albums and
tracks more precisely. For instance, I am always using the same
structure with artists as an upper directory, in which all albums
forms a subdirectory of their own, with tracks inside. In an album
directory, I put a picture of the album folder, which could be indexed
in the RDF doc. It should also be able to use, e.g., last.fm or
discogs.org, etc URLs to identify artists, albums and songs. It could
be made interactive to resolve potential ambiguity.
Yves Raimond
> I am currently developing a java application which would produce a RDF
> document according to the ID3 tags in MP3 files. This raises a lot of
> issues, some of them possibly connected with the development of the
> Music Ontology.
Cool :-) Did you check Oscar's converter? He had a web service
outputting MO RDF out of a mp3 file. See
http://mtg100.upf.es/foafing-the-music-news/?p=18
>
> This is a list of important issues:
> - what URI should be used for tracks, albums, artists? All these
> things are not uniquely identified by the corresponding ID3 tag. A
> track can hardly be identified by its title. Many occurences of the
> same title can appear on many different albums, by many different
> artists. Idem for albums. Identifying an album or a track implies (at
> the very least) identifying the album/track artist. Several artists
> with the same name exist. How to disambiguate them?
There are quite a few methods to do so. The best is to use Musicbrainz
to achieve that. They have a pretty comprehensive set of web service,
and there is a publication of Musicbrainz as RDF data, so you can
re-use identifiers from there. For example, here is the URI for
Madonna:
http://zitgist.com/music/artist/79239441-bfd5-4981-a70c-55c3f15c1287
We also developed in my lab an algorithm for bringing this
disambiguation a bit further. It is described in
http://events.linkeddata.org/ldow2008/papers/18-raimond-sutton-automatic-interlinking.pdf
You can check out GNAT, which is a Python implementation of this
algorithm, specifically for your type of use-case. GNAT is in the
motools sourceforge project
http://sourceforge.net/projects/motools/
> - what can be done when the tags are incomplete (like, there is only
> the track title and number)?
>
Check out the Musicbrainz web service or GNAT.
> To implement the converter, I use an ID3 tag API (http://
> javamusictag.sourceforge.net/) together with Jena (http://
> jena.sourceforge.net/) to generate a RDF model, which is finally
> rendered as an RDF/XML file.
>
> The main method takes a file name or directory as an argument, look
> for MP3 files in the directory and subdirectory, and for each MP3
> file, produce new RDF statements, added to a common RDF model (Model
> is a Jena class). Moreover, I computed a class for the Music Ontology,
> using schemagen (http://jena.sourceforge.net/how-to/schemagen.html).
>
Cool :-)
GNAT just outputs RDF documents holding mo:available_as statement,
linking a particular mp3 to the corresponding track instance on
Musicbrainz.
eg.
<http://zitgist.com/music/track/d35adb91-4c88-4b61-ac68-1a3c49e831c0>
mo:available_as <myfile.mp3>.
> I make several assumptions in order to produce the RDF model:
> - ID3 tag ARTIST identifies the artist: if this tag is set, then it
> creates an resource for the artist, using a private base artist URI
> (let's say http://myuri/) and a SHA-256 code from the artist's name.
> This resource is asserted to be of type mo:MusicArtist. If no artist
> is given in the tags, no resource is created. An artist has a name.
> - An album is identified by the combination of an ARTIST tag and an
> ALBUM tag. If the album tag is not present, no resource is created. If
> album is present but not the artist, no URI is created, but an
> anonymous resource (blank node) is made, with the property "rdf:label"
> set to the album name. If both album and artist are given, a new URI
> is made, with a base URI equal to the artist's URI, plus the SHA-256
> code of the album name. Albums are of type mo:Record. An album has a
> release date, a name, an artist (if specified), a number of tracks and
> a number of discs.
> - A track is identified by its TITLE, ALBUM and ARTIST tag (actually,
> there should be another resource based on TITLE and ARTIST only, which
> could appear on several albums). If the title and/or artist and/or
> album is not set, then an anonymous resource is created with the
> associated properties (tracknumber, album (if any), artist (if any),
> genre, composer (if any), lyrics (if any)). This resource is of type
> mo:Track.
>
Hmm... I am a bit worried about the identification part. I guess it is
much better to use Musicbrainz Track URIs there, and to re-create your
own only if you can't find it in Musicbrainz? Also, a track and a
particular mp3 are not the same resource. The latter is an
mo:AudioFile. You can have multiple audio file for one track.
> Issues related to the Music Ontology:
>
> Problems with the music ontology: there is no simple means to provide
> the release date. The release date is usually given as a year in ID3
> tags. In order to set the release year, I have to use the following
> sequence of properties : a mo:Record is the mo:manifestation of a
> mo:Signal which mo:time is a time:Instant, which has a time:inDateTime
> which has a time:year equal to the corresponding ID3 tag.
>
The mo:time is not really meant to do that either. It links a
mo:Signal to a corresponding time object (eg. from 0 to 3 minutes and
24 seconds), to allow for later annotations of that interval. For the
release date, you can simply use dc:date on the track/record?
But it is true that we need a better way of handling multiple release
events. I am experimenting with some ways to do so at
http://dbtune.org/musicbrainz/, but this is still work in progress.
> There is no such property as "lyrics of a song". is the MO, lyrics are
> only connected to the libretto or record in which they appear. A track
> does not identify the album in which it appears. In fact, a track is
> identified by an album as being a part of it. I find it more natural
> the other way around, so I suggest adding the inverse property. There
> is no property to indicate the number of discs of an album.
>
Did you check the Jamendo dataset (http://dbtune.org/jamendo/)? There
are lyrics for songs described using MO there. Although it uses the
generic event:factor on the corresponding performance for that (you
could have multiple tracks for a single performance), should we coin a
mo:lyrics?
I don't really get the point about the inverse property, can you be
more precise? If you can say:
:album mo:track :track.
You can equally say
:track is mo:track of :album
True for the number of discs, I add this to the todo list [1]
> Further Work:
>
> There are possible extensions to the application, e.g., by using the
> file name and/or directory structure to identify artists, albums and
> tracks more precisely. For instance, I am always using the same
> structure with artists as an upper directory, in which all albums
> forms a subdirectory of their own, with tracks inside. In an album
> directory, I put a picture of the album folder, which could be indexed
> in the RDF doc. It should also be able to use, e.g., last.fm or
> discogs.org, etc URLs to identify artists, albums and songs. It could
> be made interactive to resolve potential ambiguity.
Yes, using directory structure would be great for that. We considered
implementing that in gnat, but no time to do so yet...
Cheers!
y
Antoine Zimmermann
Let me further explain the goal of this stuff.
I would like the application to be run (or runable) offline. It should
be able to create an RDF document out of a playlist on a computer
which is not connected to the Internet.
Moreover, I wanted to make something quick and dirty, in order to try
it fast. This is why I do not use other services like Musicbrainz etc.
But hopefully, I (or someone else) will extend it to include services
for connecting the resources to external data, like last.fm,
musicbrainz, discogs, wikipedia, etc.
On Jul 10, 12:18 pm, "Yves Raimond" <yves.raim...@gmail.com> wrote:
> Hello!
>
> > [blabla]
> Cool :-) Did you check Oscar's converter? He had a web service
> outputting MO RDF out of a mp3 file. Seehttp://mtg100.upf.es/foafing-the-music-news/?p=18
tracks-to-RDF service.
> > This is a list of important issues:
> > - what URI should be used for tracks, albums, artists? All these
> > things are not uniquely identified by the corresponding ID3 tag. A
> > track can hardly be identified by its title. Many occurences of the
> > same title can appear on many different albums, by many different
> > artists. Idem for albums. Identifying an album or a track implies (at
> > the very least) identifying the album/track artist. Several artists
> > with the same name exist. How to disambiguate them?
>
> There are quite a few methods to do so. The best is to use Musicbrainz
> to achieve that. They have a pretty comprehensive set of web service,
> and there is a publication of Musicbrainz as RDF data, so you can
> re-use identifiers from there. For example, here is the URI for
> Madonna:
>
> http://zitgist.com/music/artist/79239441-bfd5-4981-a70c-55c3f15c1287
and albums, but realised it would take some time to implement. Will do
that later (or not).
> We also developed in my lab an algorithm for bringing this
> algorithm, specifically for your type of use-case. GNAT is in the
> motools sourceforge projecthttp://sourceforge.net/projects/motools/
> > - what can be done when the tags are incomplete (like, there is only
> > the track title and number)?
>
> Check out the Musicbrainz web service or GNAT.
> > [blabla]
> Cool :-)
> GNAT just outputs RDF documents holding mo:available_as statement,
> linking a particular mp3 to the corresponding track instance on
> Musicbrainz.
>
> eg.
> <http://zitgist.com/music/track/d35adb91-4c88-4b61-ac68-1a3c49e831c0>
> mo:available_as <myfile.mp3>.
>
> Hmm... I am a bit worried about the identification part. I guess it is
> much better to use Musicbrainz Track URIs there, and to re-create your
> own only if you can't find it in Musicbrainz? Also, a track and a
> particular mp3 are not the same resource. The latter is an
> mo:AudioFile. You can have multiple audio file for one track.
file instance of a song. I want the information to be tied to a Track,
not an AudioFile, because this information can be shared more easily.
People want information about tracks, not about the specific encoding
of their neighbours. In my opinion, this is the most relevent
information. However, an extension will hopefullly store information
about the specific file (with file size, encoding, bitrate, etc).
>
> > Issues related to the Music Ontology:
>
> > Problems with the music ontology: there is no simple means to provide
> > the release date. The release date is usually given as a year in ID3
> > tags. In order to set the release year, I have to use the following
> > sequence of properties : a mo:Record is the mo:manifestation of a
> > mo:Signal which mo:time is a time:Instant, which has a time:inDateTime
> > which has a time:year equal to the corresponding ID3 tag.
>
> The mo:time is not really meant to do that either. It links a
> mo:Signal to a corresponding time object (eg. from 0 to 3 minutes and
> 24 seconds), to allow for later annotations of that interval. For the
> release date, you can simply use dc:date on the track/record?
is time:TemporalEntity. A temporal entity can be a time:Instant, which
is not an interval. But I will use dc:date instead, thanks for the
tip.
> But it is true that we need a better way of handling multiple release
> > There is no such property as "lyrics of a song". is the MO, lyrics are
> > only connected to the libretto or record in which they appear. A track
> > does not identify the album in which it appears. In fact, a track is
> > identified by an album as being a part of it. I find it more natural
> > the other way around, so I suggest adding the inverse property. There
> > is no property to indicate the number of discs of an album.
>
> Did you check the Jamendo dataset (http://dbtune.org/jamendo/)?There
> are lyrics for songs described using MO there. Although it uses the
> generic event:factor on the corresponding performance for that (you
> could have multiple tracks for a single performance), should we coin a
> mo:lyrics?
lyrics. BTW there is no such things as "song" (I mean, a "title", not
necessarily something with words), which can exist as multiple tracks.
For instance, the song "Sunday Bloody Sunday" appears as a track in
U2's album War, and as another track in Best of U2 1980-1990. There
should be two different entities for the two tracks, and another
entity for the song itself. This last entity should be linked to the
lyrics. Similarly, the tablature is connected to the "song", rather
than the track.
> I don't really get the point about the inverse property, can you be
> more precise? If you can say:
> :album mo:track :track.
> You can equally say
> :track is mo:track of :album
track "is a track of" an album, either you say that the :album
"has_track" :track or you have to make an inverse property
"is_a_track_of". This what is done with properties "maker/made",
"agent/isAgentIn", "factor/isFactorOf", "product/producedIn",
"produced_work/composed_in", "performed_in/performance_of",
"recorded_in/recording_of", "arranged_in/arrangement_of", "recorded_as/
records", "sampled_version/sampled_version_of", "published_as/
publication_of", "remixed/remixer", "produced/producer", "member_of/
member", "sampled/sampler", "compiled/compiler", "djmixed/
djmiexed_by", "published/publisher", "performer/performed", "conductor/
conducted", "listener/listened", "engineer/engineered".
So why not a property "mo:track_of" inverse of "mo:track"?
BTW, I've just noticed an error in the MO. Property http://xmlns.com/foaf/0.1/made
is described as being the inverse of http://xmlns.com/foaf/0.1/made,
while http://xmlns.com/foaf/0.1/maker is also the inverse of
http://xmlns.com/foaf/0.1/made, which implies that maker and made are
equivalent!
> True for the number of discs, I add this to the todo list [1]
>
> > Further Work:
>
> Yes, using directory structure would be great for that. We considered
> implementing that in gnat, but no time to do so yet...
functional, I will put it on my Web page so that everybody can try it
(or improve it). I must finish writing my Ph.D. thesis first.
Yves Raimond
Np :-)
>
> Let me further explain the goal of this stuff.
> I would like the application to be run (or runable) offline. It should
> be able to create an RDF document out of a playlist on a computer
> which is not connected to the Internet.
> Moreover, I wanted to make something quick and dirty, in order to try
> it fast. This is why I do not use other services like Musicbrainz etc.
> But hopefully, I (or someone else) will extend it to include services
> for connecting the resources to external data, like last.fm,
> musicbrainz, discogs, wikipedia, etc.
Fair enough :)
>>
>> Cool :-) Did you check Oscar's converter? He had a web service
>> outputting MO RDF out of a mp3 file. Seehttp://mtg100.upf.es/foafing-the-music-news/?p=18
>
> I have seen that, but it seems it is rather a last.fm-last-played-
> tracks-to-RDF service.
Did you check the second example?
http://foafing-the-music.iua.upf.edu/RDFize/track?url=http://www.archive.org/download/(…)/file.mp3
Ok, perfect, I was just a bit worried about that:)
>>
>> > Issues related to the Music Ontology:
>>
>> > Problems with the music ontology: there is no simple means to provide
>> > the release date. The release date is usually given as a year in ID3
>> > tags. In order to set the release year, I have to use the following
>> > sequence of properties : a mo:Record is the mo:manifestation of a
>> > mo:Signal which mo:time is a time:Instant, which has a time:inDateTime
>> > which has a time:year equal to the corresponding ID3 tag.
>>
>> The mo:time is not really meant to do that either. It links a
>> mo:Signal to a corresponding time object (eg. from 0 to 3 minutes and
>> 24 seconds), to allow for later annotations of that interval. For the
>> release date, you can simply use dc:date on the track/record?
>
> This is what I understood, but the property mo:time has a range which
> is time:TemporalEntity. A temporal entity can be a time:Instant, which
> is not an interval. But I will use dc:date instead, thanks for the
> tip.
>
Hmm... A temporal entity can be either an instant or an interval.
>> But it is true that we need a better way of handling multiple release
>> events. I am experimenting with some ways to do so athttp://dbtune.org/musicbrainz/, but this is still work in progress.
>>
>> > There is no such property as "lyrics of a song". is the MO, lyrics are
>> > only connected to the libretto or record in which they appear. A track
>> > does not identify the album in which it appears. In fact, a track is
>> > identified by an album as being a part of it. I find it more natural
>> > the other way around, so I suggest adding the inverse property. There
>> > is no property to indicate the number of discs of an album.
>>
>> Did you check the Jamendo dataset (http://dbtune.org/jamendo/)?There
>> are lyrics for songs described using MO there. Although it uses the
>> generic event:factor on the corresponding performance for that (you
>> could have multiple tracks for a single performance), should we coin a
>> mo:lyrics?
>
> In my opinion, there should be a mo:lyrics connecting a song to its
> lyrics. BTW there is no such things as "song" (I mean, a "title", not
> necessarily something with words), which can exist as multiple tracks.
> For instance, the song "Sunday Bloody Sunday" appears as a track in
> U2's album War, and as another track in Best of U2 1980-1990. There
> should be two different entities for the two tracks, and another
> entity for the song itself. This last entity should be linked to the
> lyrics. Similarly, the tablature is connected to the "song", rather
> than the track.
>
Indeed. That's why we have Signal, Work and Performance. In the
Jamendo dataset, the lyrics are attached with the Performance (this is
arguable, though).
>> I don't really get the point about the inverse property, can you be
>> more precise? If you can say:
>> :album mo:track :track.
>> You can equally say
>> :track is mo:track of :album
>
> There is no keyword "is" and "of" in RDF. If you want to say that a
> track "is a track of" an album, either you say that the :album
There is in N3. I just took this syntax for my examples.
> "has_track" :track or you have to make an inverse property
> "is_a_track_of". This what is done with properties "maker/made",
> "agent/isAgentIn", "factor/isFactorOf", "product/producedIn",
> "produced_work/composed_in", "performed_in/performance_of",
> "recorded_in/recording_of", "arranged_in/arrangement_of", "recorded_as/
> records", "sampled_version/sampled_version_of", "published_as/
> publication_of", "remixed/remixer", "produced/producer", "member_of/
> member", "sampled/sampler", "compiled/compiler", "djmixed/
> djmiexed_by", "published/publisher", "performer/performed", "conductor/
> conducted", "listener/listened", "engineer/engineered".
>
> So why not a property "mo:track_of" inverse of "mo:track"?
>
Hmm... We coined inverse properties for everything at the beginning.
But it now sounds more like an initial mistake, as it doubles the
number of terms for nothing. This makes MO quite hard to maintain. I
now tend to be in favor of the approach described here
http://dowhatimean.net/2006/06/an-rdf-design-pattern-inverse-property-labels
> BTW, I've just noticed an error in the MO. Property http://xmlns.com/foaf/0.1/made
> is described as being the inverse of http://xmlns.com/foaf/0.1/made,
> while http://xmlns.com/foaf/0.1/maker is also the inverse of
> http://xmlns.com/foaf/0.1/made, which implies that maker and made are
> equivalent!
>
Indeed. Fixed in the SVN. Thanks for spotting that!
>> True for the number of discs, I add this to the todo list [1]
>>
>> > Further Work:
>>
>> > [blabla]
>>
>> Yes, using directory structure would be great for that. We considered
>> implementing that in gnat, but no time to do so yet...
>
> My implementation is not yet finished, but as soon as it is
> functional, I will put it on my Web page so that everybody can try it
> (or improve it). I must finish writing my Ph.D. thesis first.
>
Heh :-) Same for me!
Cheers!
y
Antoine Zimmermann
The discussion is getting more and more technical. We are getting very
close to an agreement, but it becomes more difficult to point to the
subtleties in our slight subsisting disagreements.
It would be best if we could actually *talk* about it. I also believe
that it could make an interesting paper. Something like "Mapping ID3
tags to the Music Ontology". This an interesting problem because the
discussion can also be profitable for people who wants to map any kind
of structured metadata to RDF. And, moreover, it provides a very
practical contribution, since the mapping will be implemented and
readily usable to all would-be semwebbers.
There are ccertainly workshops (and/or conferences) which would be
very appropriate for this kind of contributions.
Here are my last thoughts about how to map ID3 tags to the M.O.
The goal is to have a mapping that:
- can be processed without an Internet connection nor external
knowledge,
- produces something sensible in case of incomplete tags,
- produces sharable and reusable data.
Here is what I suggest:
All mp3 files can be mapped to a TrackSignal (subclass of mo:Signal).
A TrackSignal is the essence of the music and sounds of the track,
rendering what has been recorded in a specific place at a specific
moment.
In absence of all tags, an anonymous instance of TrackSignal is made
(in fact, it could be identified by a signature of the file if
available) and this instance is related to the MP3 file, identified by
its location on the disk or its URL.
The format, bitrate, size, etc of the file can be assigned to the file
path or URL.
If there is an ID3:TITLE, it is attached to the TrackSignal (the title
identifies what has been recorded, not what is published on a specific
medium, IMHO) thanks to property dc:title.
If there is an ID3:ARTIST given, it identifies a mo:MusicArtist (for
now, I will assume that the artist’s name is sufficient to identify
the artist uniquely; improvement will be needed when considering
relying on external sources). The artist has a foaf:name equal to the
ID3:ARTIST tag.
If there is an ID3:ALBUM given, it identifies (together with the
ID3:ARTIST) a mo:MasterSignal. If no ID3:ARTIST is given, an anonymous
mo:MasterSignal is made, with dc:title equals to the ID3:ALBUM tag.
This is a mo:MasterSignal, not a mo:Record, because this an MP3 file,
not a track on a CD release, or Vinyl release, or cassette release.
Maybe it is ripped from a CD, maybe from a Vinyl, maybe bought as an
original MP3. But at least, there must be a Signal associated with
this ID3:ALBUM and ID3:ARTIST.
This signal is related to the TrackSignal with a relation (like
has_track).
Sometimes, there is a tag called ALBUMARTIST (not a standard ID3 tag,
but you can have whatever tag you wish with ID3v2). This is, however,
a tag present in the standard ID3 tagging interface of my audio player
software (the very excellent foobar2000). If this tag is given, then
it identifies an instance of mo:MusicArtist, the dc:author or
dc:creator of the mo:MasterSignal. This tag is usually given to albums
with various artists.
If ALBUMARTIST is not given, then the mo:MasterSignal refers to the
mo:TrackSignal’s artist name.
If a ID3:YEAR is given, then this is normally the year of the release
of an album containing the track. Therefore, a mo:Record is made,
having a releaseDate which has a time:year equal to the ID3:YEAR tag.
This mo:Record refers to the mo:MasterSignal identified above as the
underlying signal of the record. This mo:Record has a mo:Track which
refers to the TrackSignal of the file.
If a ID3:TRACKNUMBER is given, then it is attached to the mo:Track.
Idem for DISCNUMBER, if given.
If a TOTALTRACKS number is given, it is attached to the mo:Record, as
well as the TOTALDISCS number.
ID3:GENRE is attached to the TrackSignal. It makes an instance of
mo:Genre, with rdf:label equal to the ID3:GENRE tag.
If LYRICS are given, then there exists a mo:MusicalWork, attached to
the TrackSignal. This mo:MusicalWork has a property lyrics with range
mo:Lyrics. Moreover, the mo:MusicalWork can be related to the
ID3:TITLE.
BTW, I believe there should be a relation mo:expresses with domain
mo:MusicalExpression and range mo:MusicalWork.
I am preparing a picture of this to make things clearer.
Yves Raimond
>
> The discussion is getting more and more technical. We are getting very
> close to an agreement, but it becomes more difficult to point to the
> subtleties in our slight subsisting disagreements.
> It would be best if we could actually *talk* about it. I also believe
> that it could make an interesting paper. Something like "Mapping ID3
We have an IRC channel on freenode if you want: #musicontology
As I mentioned in my previous email, I don't really agree. The content
of the TITLE field in ID3 is the "as published" title.
> If there is an ID3:ARTIST given, it identifies a mo:MusicArtist (for
> now, I will assume that the artist's name is sufficient to identify
> the artist uniquely; improvement will be needed when considering
> relying on external sources). The artist has a foaf:name equal to the
> ID3:ARTIST tag.
>
Agreed
> If there is an ID3:ALBUM given, it identifies (together with the
> ID3:ARTIST) a mo:MasterSignal. If no ID3:ARTIST is given, an anonymous
> mo:MasterSignal is made, with dc:title equals to the ID3:ALBUM tag.
> This is a mo:MasterSignal, not a mo:Record, because this an MP3 file,
> not a track on a CD release, or Vinyl release, or cassette release.
> Maybe it is ripped from a CD, maybe from a Vinyl, maybe bought as an
> original MP3. But at least, there must be a Signal associated with
> this ID3:ALBUM and ID3:ARTIST.
> This signal is related to the TrackSignal with a relation (like
> has_track).
>
Same thing, I don't really think you need to get in mo:MasterSignal.
The content of the ALBUM field relates to the "as published" title
again.
> Sometimes, there is a tag called ALBUMARTIST (not a standard ID3 tag,
> but you can have whatever tag you wish with ID3v2). This is, however,
> a tag present in the standard ID3 tagging interface of my audio player
> software (the very excellent foobar2000). If this tag is given, then
> it identifies an instance of mo:MusicArtist, the dc:author or
> dc:creator of the mo:MasterSignal. This tag is usually given to albums
> with various artists.
> If ALBUMARTIST is not given, then the mo:MasterSignal refers to the
> mo:TrackSignal's artist name.
>
Also, I would relate that to the published record?
> If a ID3:YEAR is given, then this is normally the year of the release
> of an album containing the track. Therefore, a mo:Record is made,
> having a releaseDate which has a time:year equal to the ID3:YEAR tag.
> This mo:Record refers to the mo:MasterSignal identified above as the
> underlying signal of the record. This mo:Record has a mo:Track which
> refers to the TrackSignal of the file.
>
Agreed
> If a ID3:TRACKNUMBER is given, then it is attached to the mo:Track.
> Idem for DISCNUMBER, if given.
yep
>
> If a TOTALTRACKS number is given, it is attached to the mo:Record, as
> well as the TOTALDISCS number.
Do we have such a property yet?
>
> ID3:GENRE is attached to the TrackSignal. It makes an instance of
> mo:Genre, with rdf:label equal to the ID3:GENRE tag.
>
> If LYRICS are given, then there exists a mo:MusicalWork, attached to
> the TrackSignal. This mo:MusicalWork has a property lyrics with range
> mo:Lyrics. Moreover, the mo:MusicalWork can be related to the
> ID3:TITLE.
>
>
> BTW, I believe there should be a relation mo:expresses with domain
> mo:MusicalExpression and range mo:MusicalWork.
>
>
>
> I am preparing a picture of this to make things clearer.
Ok, It might indeed makes things clearer.
Again, I would really keep things simple, and I really think my
previous example would be the easiest to do. Here is another one.
:record a mo:Record;
dc:title "Longlife legendary bastards";
foaf:maker :skarface;
mo:track :track.
:track a mo:Track;
dc:title "A Ska Pirates' Life Intro";
mo:track_number 2.
Now, if you have more information to express:
* encoding etc. - attach it to the mo:AudioFile
* track length, track segmentation/synchronisation (see the BBC
proposal for inserting timestamps in Podcast, event timing or
synchronised lyrics in ID3v2.4) - attach it to mo:Signal (+
tl:TimeLine for segmentation, see examples on the Wiki).
* performer - attach it to mo:Performance
* composer - attach it through dc:creator on mo:MusicalWork
I'll try to write a small example of such a mapping this week.
Cheers
y
> >
>
Antoine Zimmermann
On Jul 20, 2:21 pm, "Yves Raimond" <yves.raim...@gmail.com> wrote:
> [blabla]
> We have an IRC channel on freenode if you want: #musicontology
> > [blabla]
> > identifies what has been recorded, not what is published on a specific
> > medium, IMHO) thanks to property dc:title.
>
> As I mentioned in my previous email, I don't really agree. The content
> of the TITLE field in ID3 is the "as published" title.
specific release, like "1997 UK CD release". Why not the "1981
cassette release"? Why not the vinyl release? And why not the online
MP3 release? How can you discriminate between these releases with only
a tracktitle and album name?
Whatever the case, it seems strange to me that a MP3 file should be
mapped to a physical release on a specific medium, while in fact, it
is a digital file that can be moved from hard discs to portable
players to memory stick.
That's why I advocate mapping a file to a signal, which can always be
done, even when it is not part of a record. With your view point, you
need a release to which you assign the tags. This is not needed with
my view point.
Moreover, the signal for the album can exist even though there is no
record (it exists before any record is sold). One can have audio files
encoding a non released album. Still, there would be a signal to which
you can map the ID3 tags.
Additionally, if you map the tags to a specific Record, you in fact
lose information, because the connection between two releases of the
same "MasterSignal" is lost (except by comparing character strings,
but this is not the idea of RDFizing metadata).
> > [blabla]
> > mo:TrackSignal's artist name.
>
> Also, I would relate that to the published record?
the album is published (nor the moment the album is released).
Therefore, it must be refered to from the MasterSignal.
> > [blabla]
> > well as the TOTALDISCS number.
>
> Do we have such a property yet?
anyway.
> Again, I would really keep things simple, and I really think my
> previous example would be the easiest to do. Here is another one.
>
> :record a mo:Record;
> dc:title "Longlife legendary bastards";
> foaf:maker :skarface;
> mo:track :track.
> :track a mo:Track;
> dc:title "A Ska Pirates' Life Intro";
> mo:track_number 2.
>
> Now, if you have more information to express:
>
> * encoding etc. - attach it to the mo:AudioFile
> * track length, track segmentation/synchronisation (see the BBC
> proposal for inserting timestamps in Podcast, event timing or
> synchronised lyrics in ID3v2.4) - attach it to mo:Signal (+
> tl:TimeLine for segmentation, see examples on the Wiki).
> * performer - attach it to mo:Performance
> * composer - attach it through dc:creator on mo:MusicalWork
Track?
I've made a picture, not the one I intended to do at first, but it
partially shows the situation. (http://exmo.inrialpes.fr/zimmer/images/
dessin.svg)
It shows that a Record publishes a MasterSignal on a specific Date on
a specific Medium. The MasterSignal is made by a MusicArtist (and not
the Record, which is released by a Label etc).
The MasterSignal is composed of several TrackSignals, which are not
necessary made by the same artists (in cas of a compilation the tracks
and the album are made by completely different persons).
The Record contains tracks, which publish the TrackSignal by putting
them in the record at a specific tracknumber. On a different Record,
the tracknumber can be different (compare a Vinyl with A side and B
side to an album). Moreover, there might be tracks which are not part
of the MasterSignal (sometimes, there are additional tracks on
reissues).
Yves Raimond
On Wed, Jul 23, 2008 at 7:03 PM, Antoine Zimmermann
<antoine.z...@gmail.com> wrote:
>
>
>
> On Jul 20, 2:21 pm, "Yves Raimond" <yves.raim...@gmail.com> wrote:
>> [blabla]
>>
>> We have an IRC channel on freenode if you want: #musicontology
>
> Good.
>
>> > [blabla]
>> > If there is an ID3:TITLE, it is attached to the TrackSignal (the title
>> > identifies what has been recorded, not what is published on a specific
>> > medium, IMHO) thanks to property dc:title.
>>
>> As I mentioned in my previous email, I don't really agree. The content
>> of the TITLE field in ID3 is the "as published" title.
>
> If I understand well, it means that the TITLE ought to be mapped to a
> specific release, like "1997 UK CD release". Why not the "1981
> cassette release"? Why not the vinyl release? And why not the online
> MP3 release? How can you discriminate between these releases with only
> a tracktitle and album name?
Well, my only point is that this distinction is not made in ID3! And
it looks a bit unachievable to translate things that don't exist in
the initial data format :-)
So I completely agree with you: with both a track title and an album
name, you can't make this distinction, so don't worry about it :-)
> Whatever the case, it seems strange to me that a MP3 file should be
> mapped to a physical release on a specific medium, while in fact, it
> is a digital file that can be moved from hard discs to portable
> players to memory stick.
There is a difference between the MP3 (mo:AudioFile) and the release
it comes from. I was talking about the latter. The two are linked
through available_as, though.
> That's why I advocate mapping a file to a signal, which can always be
> done, even when it is not part of a record. With your view point, you
> need a release to which you assign the tags. This is not needed with
> my view point.
That's indeed two different view points. I am just worried you try to
make a distinction in the target data format (MO) that doesn't exist
in the original one (ID3). I am just trying to advocate (for once :-)
) simplicity here.
You don't need to have *all* information about a particular release to
create an instance of it.
> Moreover, the signal for the album can exist even though there is no
> record (it exists before any record is sold). One can have audio files
> encoding a non released album. Still, there would be a signal to which
> you can map the ID3 tags.
The ID3 tags are detailing editorial information - you can perfectly
associate a release to any mp3 if it has such information?
If I record a concert on my cell phone, divide the resulting audio
file and put it on the web as a bunch of mp3s with id3 metadata, this
is actually a release.
The concept of mo:Record is independent from the fact it can be sold -
this is just a grouping of items, if you want (if someone else has
that same mp3, it can link to the same mo:Record, therefore inheriting
the attached editorial information)
> Additionally, if you map the tags to a specific Record, you in fact
> lose information, because the connection between two releases of the
> same "MasterSignal" is lost (except by comparing character strings,
> but this is not the idea of RDFizing metadata).
>
Same thing - you can't infer this difference solely from ID3 tags; so
why worry about it?
>> > [blabla]
>> > If ALBUMARTIST is not given, then the mo:MasterSignal refers to the
>> > mo:TrackSignal's artist name.
>>
>> Also, I would relate that to the published record?
>
> The album artist is necessarily the same whatever the medium on which
> the album is published (nor the moment the album is released).
> Therefore, it must be refered to from the MasterSignal.
Again, you can't make this distinction of "mediums on which the album
is published" with ID3 tags.
There is a small diagram on the spec page for this sort of things -
just use the concept/relationships you need within it.
http://musicontology.com/imgs/mo-workflow.jpg
>
> I've made a picture, not the one I intended to do at first, but it
> partially shows the situation. (http://exmo.inrialpes.fr/zimmer/images/
> dessin.svg)
Hmm I get a 404 on that
> It shows that a Record publishes a MasterSignal on a specific Date on
> a specific Medium. The MasterSignal is made by a MusicArtist (and not
> the Record, which is released by a Label etc).
What did the artist do? did he composed it? did he mix it?
That's why we have this "weak" foaf:made on the Record level - this is
effectively a weak relationship between a publication and a maker (the
big name on your vinyl if you want). I still think that's the same
thing you want for mapping ARTIST.
> The MasterSignal is composed of several TrackSignals, which are not
> necessary made by the same artists (in cas of a compilation the tracks
> and the album are made by completely different persons).
> The Record contains tracks, which publish the TrackSignal by putting
> them in the record at a specific tracknumber. On a different Record,
> the tracknumber can be different (compare a Vinyl with A side and B
> side to an album). Moreover, there might be tracks which are not part
> of the MasterSignal (sometimes, there are additional tracks on
> reissues).
Can you give a small real-world example of that (for example on the
two mp3s I used as example earlier on)?
To sum up my views:
* No need to make distinctions that ID3 can't make - eg. multiple
releases of a same album
* The TITLE and ARTIST are purely editorial
* Further concepts like Perfomance (PERFORMER) and MusicalWork
(COMPOSER) could be used to refine the translation
* Item -> audio file; cd; etc. Record -> hook for editorial
information ; Signal -> either the actual signal encoded into an audio
file or the result of a composing/performing/recording/mixing/etc.
process
However, I am still not sure I understood fully your views. I think
it will get clearer with an RDF example.
Cheers!
y
> >
>
Antoine Zimmermann
different releases of the same album (like, between the CD and
cassette release). If I understand well, such a specific release is an
instance of mo:Record. This is exactly why I advocate NOT mapping the
tags of an MP3 to a specific mo:Record.
Let me give an example to clarify my view.
Imagine you have an MP3 file with ID3:TITLE "Sunday Bloody Sunday",
ID3:ALBUM "War", ID3:ARTIST "U2", ID3:YEAR "1983".
Additionally, there is a Music Knowledge Base somewhere on the Web
which has information about musical releases, in RDF.
For instance, there is an instance of mo:Record with URI "wxyz1234"
which identifies the CD release of "War" by "U2" on Label "Island
Records" in the US, and there is an instance of mo:Record with URI
"abcd9876" which identifies the LP release of "War" by "U2" on Label
"Island Record".
The problem now is that you cannot map your MP3 tags to one of these
specific URIs. So you must either create a blank node, or make a new
URI.
But in this case, you cannot take advantage of this knowledge base,
because there is no way to determine that the music of your track is
the same music as track 1 of CD "War" and track "A1" of the LP (and
also track "sqdf0000" on iTunes). However, if there had been a common
entity (more precisely a Signal) which identifies the music that is
shared by the three different things (track 1 of the CD release, track
A1 of the LP, and your own MP3 file), you would be able to say that
you have a mo:AudioFile which encodes this specific TrackSignal, which
is published as mo:Track number 1 on the mo:Record "wxyz1234", and
published as mo:Track number "A1" on mo:Record "abcd9876". It doesn't
matter which exact mo:Record your own file comes from because you
don't even need to talk about such thing in your own RDF description.
If you have the artist, the album and the track name, you can identify
uniquely almost all possible "TrackSignal". However you can rarely
identify a given mo:Track on a specific mo:Record. Mapping means
associating a given entity in a set to a uniquely defined entity in
another set. Here, the entities that I would like to map are the
vectors of tags found in a given MP3 file. If you use the set of
mo:Record, there are clearly several possible correspondences, and
thus, it does not define a mapping in a strict sense. With a Signal,
it becomes quite difficult to find two different Signals having the
same author, title and MasterSignal's title.
Finally, having ID3 tags mapped to Signal does not forbid
instantiating mo:Record (for instance using a blank node) to refer to
the image of the cover, or to the booklet, or to the number of tracks/
cds/sides, and so on. But this mo:Record encapsulates the
MasterSignal.
Example:
:myMP3file is_a AudioFile
:myMP3file publishes :S-Bloody-S
:S-Bloody-S is_a TrackSignal
:S-Bloody-S title "Sunday Bloody Sunday"
:S-Bloody-S maker :U2
:U2-War has_track :S-Bloody-S
:U2-War is_a MasterSignal
:U2-War maker :U2
:U2-War title "War"
_:blankRecord is_a Record
_:blankRecord released "1983"
_:blankRecord publishes :U2-War-1983
_:blankTrack publishes :S-Bloody-S
_:blankTrack track_number "1"
_:blankRecord track _:blankTrack
Note that I assume that whenever there is a date given, it means that
it is a release date, and therefore that there is a Record somewhere
(even though in this case, the specific Record is not known). This is
exactly the RDF that I want to produce with my program. It can even
work offline. The URI of the MasterSignal is based on the album name
and artist name, the URI of the artist is based on the artist name
only, the uri of the track is based on the combination of album/artist/
title. The specific Record and Track is undetermined, so they are
blanks.
I believe it is a very good thing in the Music Ontology to have
MusicalExpression (like the MusicalSignals) made by MusicArtists on
the one hand, and MusicalManifestation (like Records) made by the
music industry, on the other hand. The Music Artists make the musical
signal (which includes the title of the tracks, of the album, who
participated in the performance and production of sounds, the date it
was produced) and then the music industry takes care of releasing it
on a Record (which means, the cover art, the medium, the date of
release, etc). Obviously, MusicArtists can participates in the making
of a Record, but they do not do it *as* MusicArtists.
BTW, here is the correct URL for the picture :
http://exmo.inrialpes.fr/~zimmer/images/dessin.svg
We can see that a mo:Record encapsulates the MasterSignal which it
publishes. The mo:Record contains mo:Tracks which publish TrackSignals
by assigning a track number to them (and assigning a specific
encoding). MusicArtists are connected to the Signals only.
I would like to add that I admit this makes things a bit more complex
and that I understand that an ontology can be used as a data specific
rather that a specific of what things *really* are. So I approve your
view as a more pragmatic one. But I will first implement the mapping
as I said, I will launch the transformation on my personal files, and
then it'll be easy to update it as needed.
Cheers,
AZ.
Yves Raimond
> As you seem to admit, it is not possible to distinguish between two
> different releases of the same album (like, between the CD and
> cassette release). If I understand well, such a specific release is an
> instance of mo:Record. This is exactly why I advocate NOT mapping the
> tags of an MP3 to a specific mo:Record.
>
> Let me give an example to clarify my view.
> Imagine you have an MP3 file with ID3:TITLE "Sunday Bloody Sunday",
> ID3:ALBUM "War", ID3:ARTIST "U2", ID3:YEAR "1983".
> Additionally, there is a Music Knowledge Base somewhere on the Web
> which has information about musical releases, in RDF.
>
> For instance, there is an instance of mo:Record with URI "wxyz1234"
> which identifies the CD release of "War" by "U2" on Label "Island
> Records" in the US, and there is an instance of mo:Record with URI
> "abcd9876" which identifies the LP release of "War" by "U2" on Label
> "Island Record".
Right, I think I begin to understand what you're coming at. But you
explicitly said you wanted to use your mapper in an "offline" mode?
Because if you're willing to use external URIs, why not just stating
something like
<http://zitgist.com/music/track/e95e5009-99b3-42d2-abdd-477967233b08>
mo:published_as <your_mp3>.
and leave the job to describe this track to Musicbrainz?
Well, OK, I guess you don't want to do that ;-)
I doubt the MP3 you have corresponds to the master signal of the
actual track, I would think it comes from one of these releases (CD,
LP, iTunes). You don't know which, but that's fine - you can still
identify it.
Then, you wnat to link explicitly to one of these releases. So either
you know where this MP3 came from (in the case of ripped cd
collection, that's quite easy) and you owl:sameAs towards the good
release. In other case, you might not know. So you can indeed create a
MasterSignal resource which joins up the two (or just
mo:other_release_of, if you want). Nothing forbids you to sameAs the
right releases later on.
>
> The problem now is that you cannot map your MP3 tags to one of these
> specific URIs. So you must either create a blank node, or make a new
> URI.
> But in this case, you cannot take advantage of this knowledge base,
> because there is no way to determine that the music of your track is
> the same music as track 1 of CD "War" and track "A1" of the LP (and
> also track "sqdf0000" on iTunes). However, if there had been a common
But the mapping I proposed doesn't forbid to have a signal resource
joining up releases? I just thought you wanted to have a "standalone"
ID3 mapper. In that case, I believe the mapping I illustrated earlier
makes sense. Then, you can *always* use any other MO resource to join
up with other databases.
> entity (more precisely a Signal) which identifies the music that is
> shared by the three different things (track 1 of the CD release, track
> A1 of the LP, and your own MP3 file), you would be able to say that
> you have a mo:AudioFile which encodes this specific TrackSignal, which
> is published as mo:Track number 1 on the mo:Record "wxyz1234", and
> published as mo:Track number "A1" on mo:Record "abcd9876". It doesn't
> matter which exact mo:Record your own file comes from because you
> don't even need to talk about such thing in your own RDF description.
>
> If you have the artist, the album and the track name, you can identify
> uniquely almost all possible "TrackSignal". However you can rarely
> identify a given mo:Track on a specific mo:Record. Mapping means
> associating a given entity in a set to a uniquely defined entity in
> another set. Here, the entities that I would like to map are the
> vectors of tags found in a given MP3 file. If you use the set of
> mo:Record, there are clearly several possible correspondences, and
> thus, it does not define a mapping in a strict sense. With a Signal,
> it becomes quite difficult to find two different Signals having the
> same author, title and MasterSignal's title.
>
I don't particularly agree with that. With the set of information in
an ID3 tag, you identify editorial information, not a signal (you
would identify the signal by using fingerprints, for example). By
using TOC information + track number etc. you can have more clues to
find a unique correspondence.
> Finally, having ID3 tags mapped to Signal does not forbid
> instantiating mo:Record (for instance using a blank node) to refer to
> the image of the cover, or to the booklet, or to the number of tracks/
> cds/sides, and so on. But this mo:Record encapsulates the
> MasterSignal.
Heh - I just made the same point, but inverted :-) Nothing forbids you
to have a Signal :-)
>
> Example:
>
> :myMP3file is_a AudioFile
> :myMP3file publishes :S-Bloody-S
> :S-Bloody-S is_a TrackSignal
> :S-Bloody-S title "Sunday Bloody Sunday"
> :S-Bloody-S maker :U2
> :U2-War has_track :S-Bloody-S
> :U2-War is_a MasterSignal
> :U2-War maker :U2
> :U2-War title "War"
> _:blankRecord is_a Record
> _:blankRecord released "1983"
> _:blankRecord publishes :U2-War-1983
> _:blankTrack publishes :S-Bloody-S
> _:blankTrack track_number "1"
> _:blankRecord track _:blankTrack
So in my view, it'd be
:myMP3file a mo:AudioFile.
:S-Bloody-S a mo:Track; dc:title "Sunday Bloody Sunday"; foaf:maker
:U2; mo:track_number "1".
:U2-War mo:track :S_Bloody-S; dc:title "War"; dc:date "1983".
(and you can create a Signal resource if you want to join up several releases).
Well, I guess we're on a good basis for a real discussion now :)
I'll take a deeper look at all that. Also, let's take some time to
consider the two proposed examples for the mapping of U2's example and
highlight the good/bad points about them - it'd be great to move that
part of the discussion to the wiki (http://wiki.musicontology.com)
Cheers!
y
>
> Cheers,
> AZ.
>
> >
>