Metadata under <metalink> and/or <file>? Comments on the specification...

[Hampus Wessman]

Metalinks can contain a lot of metadata. With metadata I mean
information like identity, description, publisher, license, tags and
version (not hashes, file name, file size and similar). Most of this
metadata can be placed either directly under <metalink> or under <file>.
The specification is currently a bit unclear about this, but most
applications seem to do it this way. This is therefore one thing that I
think should be clarified in the next revision of the metalink
specification.

In the current specification it is clear that all the metadata can be
placed under <file>, but not that it can also be placed directly beneath
<metalink> (if we don't look at the examples). I examined all kinds of
metalinks on the Internet and all of them had some kind of metadata
under <metalink>, but none had any metadata under <file>!

What I'm mainly interested in is where the metadata should be allowed.
Should all the metadata be allowed both under <file> and <metalink>?
What are the alternatives? Pros and cons?

If the metadata can be put both under <metalink> and under <file> we
have two places to put similar metadata in! The question then is how the
metadata under <metalink> should relate to the metadata under <file>.
Should the "files" inherit the metadata from the "metalink"? Especially
when we are dealing with one-file metalinks it's not completely obvious
where we should put our metadata or how a client should display it...

One can wonder if it really is that useful to be able to specify
metadata for each individual file in the metalink. The metalink format
would become a lot easier to understand and work with if there was only
one single place to put metadata - i.e. under <metalink>. The files
described in a metalink are usually related to each other (because why
would you otherwise put them in the same metalink?) and therefore it
would be sufficient to simply have one set of metadata per metalink. All
the currently available metalinks clearly indicate that this is the
case. If we only allow metadata under <metalink>, we don't need to worry
about where to put our metadata or about whether the files inherit the
metalinks metadata or not. This would also increase the focus on the
metalink, in contrast to the individual files described within it -
which I think is positive. If you choose to download a metalink it is
sufficient to get a description of the metalink!

With only one set of metadata per metalink it would also be simpler to
create metalinks and to create software that deals with metalinks. I
actually think that this has a lot more pros than cons. To keep the
format clear and simple is important if metalinks are going to conquer
the world! There would be no compatibility problems either, because
specifying metadata for the individual files is optional right now and I
haven't seen it being done yet.

What do the rest of you think? Do we really need to have metadata for
each file? Most of the time metalinks only describe one file or a
collection of files that are closely related...

[Neil M.]

I agree that some of the tags should probably be moved around, but not
all put in one place. Some tags make sense at the file level, like
language and os, but other seem to make more sense at the root level
like the ones you mentioned. Probably the easiest for creating
metalinks is to allow them both places, anything found in "file"
overrides what what is found at the "metalink" level. This is also most
flexible. A good example is for Appupdater, I've got a case where I've
got a binary installer from some company, and an autoit script to run
the installer. These may have different licenses, right now there is no
way to express that. And a metalink file of 500 files, putting a
language tag in for english 500 times isn't fun, and its going to
increase the size of the .metalink file if you can just specify that at
the root level. However, this method may make issues in the XML world,
once you start trying to merge or pull out "file" sections and put them
into other metalinks. I think we just need to ask, is it possible two
files in the same metalink file could have different values for X tag?
If yes it needs to be at the file level, otherwise it goes into the root.

Neil

[Hampus Wessman]

I agree. Allowing metadata both at root level and file level probably is
the most flexible and best solution. It becomes a little more
complicated if the files inherit metadata from the root level (ie use
the root metadata if none is specified for the file) than otherwise, but
it would surely save space and also be very flexible and compatible with
current metalinks.

It would be great if something like this was added to the specification!
The most important thing is that everyone does this the same way...

[Hampus Wessman]

In reality metadata is already allowed in both places - the
specification is just a bit unclear about it - so it is probably best
to keep it that way. This could be done in a number of other ways too,
but there's no real reason to change it now... You can simply ignore
my ideas about only allowing metadata under <metalink> (see my first e-
mail). Does everyone agree on this? Anthony and Neil seem to do.
Anyone who doesn't agree?

I am not as sure about how to treat the metadata at the two levels
(metalink and file level) though. Most current (single-file) metalinks
only have metadata at the metalink level, but at the same time this
metadata usually seem to describe the file and not the metalink...
That's not logical, is it? If it's possible to specify metadata about
the metalink and about each file in the metalink I think the most
logical way to deal with that would be to let the metalink metadata
describe the metalink and the file metadata describe the file, right?
That would mean that most single-file metalinks would have some
metadata about the file, under <file>, and probably don't need any
metadata about the metalink, under <metalink>. My Metalink Editor
currently puts metadata about the file under <metalink>, which (as
said) seems very unlogical. How are the rest of you dealing with this
right now and what do you think would be most logical?

If the files would inherit metadata from the metalink level, when none
has been specified for the individual file, it would be possible to
place the metadata about the files either under <metalink> or under
each <file>, as long as the files have the same metadata. With single-
file metalinks this would mean that the metadata could always be
placed at either level, without any difference in result. It is
currently far from clear (IMO at least) whether clients should
implement this behaviour or not. I think it would be good if the
standard specified which behaviour a client should have, so that the
creator of a metalink knows how it will be interpreted.

I'm not completely sure that I think it's a good thing that metalink
clients fill in missing file metadata from the metalink metadata. I
think it would be better to keep the two levels completely separate.
That would make it more clear what metadata you should put where (if
it's about the file it should go under <file>!). I don't think we
should assume that the metadata about the metalink is always valid for
the individual files either. We might know the publisher of the
metalink, but not of some of the files. If the files don't inherit
metadata we can specify this in a metalink, but otherwise we can't.
This would be the absolutely simplest and most flexible solution IMO.
We would lose a little space (compared to the alternative), but it
would be worth it. Note that metalink generators can fill in the
metadata at the metalink level automatically for each file if they
want, so this discussion is only about the resulting metalink files.

Some examples of metadata at the two levels:
metalink metadata: identity = "Top ten open source applications",
description = "This is a collection of our top ten favorite...",
publisher_name = "The example magazine" and so on...
file metadata: identity = "GIMP", version = "2.3", description = "The
number one image manipulation program in the open source world!",
publisher_name = "The GIMP Creators (or something)"...

Note that if one of the fields about the file wasn't specified - lets
say description, in the example above - we might not want the metalink
metadata to be filled in, if the metalink metadata really is about the
metalink...

[Bram Neijt]

Hi everybody,

I was thinking that metadata should be kept as small as possible and
use another XML namespace to insert the metadata people might want to
add. The Dublin core would be a good example.

The client doesn't have to become more specific and we can just add a
chapter to the reference on how to add metadata and advice on some XML
metadata standard already out there. This would have the benefit of
allowing things like google-bots etc. to actually start reading it.

Maybe some W3C standard can be found or used?

Bram

[Hampus Wessman]

That's a nice idea! Dublin core is a really interesting option.
Sorry for not replying sooner...
Hampus

Bram Neijt skrev:

[Peter Poeml]

On Thu, May 22, 2008 at 08:01:04 +0200, Hampus Wessman wrote:
>
> That's a nice idea! Dublin core is a really interesting option.

I have no real opinion about Dublin Core, because I'm not familiar with
it. I don't immediately see any particular value of switching to it;
however I'm ready to be convinced. How would it bring us forward?

Peter
--
"WARNING: This bug is visible to non-employees. Please be respectful!"

SUSE LINUX Products GmbH
Research & Development

[Bram Neijt]

Hi Peter,

I think this thread will stay unanswered, because nobody believes in the
DC strongly enough. The basic idea was that any metadata on the metalink
(description, title, author, origin, creation date etc) could be moved
to an already available standard like the Dublic Core.

Advantages mentioned:
- Smaller metalink spec
- Easily readable for clients that already support the DC (like some of
the search engine bots)

Problem seems to be twofold: referring to another standard for metadata
makes the metalink spec incomplete, while pointing out specific parts of
the DC as obligatory is not practical. General opinion seems to be: it
would be cool to use already existing technology, but there doesn't seem
to be a real benefit at the moment.

I'm not going to convince you, and I have been asking Anthony to veto
this off the table at this point as it does not help with the current
standardization AND we have no passionate defenders of this idea.

Greets,
Bram

[Peter Pöml]

Hi Bram,

It's a long time ago, but I always wanted to thank you for the
detailed answer!

Peter