Metalink & Homebrew?

[Anthony Bryan]

have any of you tried out Homebrew? http://mxcl.github.com/homebrew/

it's like Appupdater or apt-get for OS X, a package manager that downloads source & compiles it.

it's really cool. one issue tho is there's an army of guys (manually it seems?) updating the "Formula" (Ruby files) that describe each app when a new version comes out. so there's some lag, like any package management with maintainers that have to do manual updates.

I think this is an opportunity for metalink to automate this process.

anyone familiar w/ Ruby that might be interested? that way it could fit in w/ the existing Homebrew stuff.

anyways, one possible way would be to add metalink to the Formula (the one for curl's included at the end of msg)

  metalink 'http://curl.haxx.se/metalink.cgi?curl=tar.gz'

then the process would be:

download the metalink from similarly autoupdated addresses when new versions of (for instance) aria2 and libmetalink come out

  hb already uses curl (Library/Homebrew/download_strategy.rb)

get the app's version in the version element of the metalink XML.

if version's greater than what's in the Formula, update the url and hash

what do you think? seems relatively simple.

other metalink-related opportunities:

they'd like to support torrent downloads. -> a Formula for aria2 is already provided, I guess torrents would need a custom download strategy (see Specifying the Download Strategy section at https://github.com/mxcl/homebrew/wiki/Formula-Cookbook )

the curl metalink has a PGP signature. curl + GPGME? that would be better than the hashes.

-----

require 'formula'

class Curl < Formula

  homepage 'http://curl.haxx.se/'

  url 'http://curl.haxx.se/download/curl-7.27.0.tar.gz'

  sha256 '8cbad34e58608f0e959fe16c7c987e57f5f3dec2c92d1cebb0678f9d668a6867'

  keg_only :provided_by_osx,

            "The libcurl provided by Leopard is too old for CouchDB to use."

  option 'with-ssh', 'Build with scp and sftp support'

  option 'with-libmetalink', 'Build with Metalink support'

  depends_on 'pkg-config' => :build

  depends_on 'libssh2' if build.include? 'with-ssh'

  depends_on 'libmetalink' if build.include? 'with-libmetalink'

  def install

    args = %W[

      --disable-debug

      --disable-dependency-tracking

      --prefix=#{prefix}

    ]

    args << "--with-libssh2" if build.include? 'with-ssh'

    args << "--with-libmetalink" if build.include? 'with-libmetalink'

    system "./configure", *args

    system "make install"

  end

end

--
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads

[Bram Neijt]

Seem simple enough, however I don't have a Mac so I don't see me
testing this in any sensible manner.

Anybody with a Mac ready to give it a try? Maybe I can help with
interpreting compiler output.

Greets,

Bram

> --
> You received this message because you are subscribed to the Google Groups
> "Metalink Discussion" group.
> To post to this group, send email to metalink-...@googlegroups.com.
> To unsubscribe from this group, send email to
> metalink-discus...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/metalink-discussion?hl=en.

[Anthony Bryan]

I can test on a mac. but I was talking to Donnie Berkholz from Gentoo,
and they would be interested in a generic tool that does this (& said
that it would be helpful for many other packagers & distributions), so
maybe a proof of concept (Python?) for the generic version would be a
good start, then maybe adapt it to homebrew?

[Neil M.]

A couple of things on that.

Firstly, Zero Install seems to have an architecture already designed to do
that (compile and install) for multiple platforms. Their main website
seems to be down at the moment but Wikipedia has a good overview here:

https://en.wikipedia.org/wiki/Zero_Install

Secondly Appupdater has the capability to run arbitrary commands to
"install" a product, so this could include the compile instructions or a
ruby script (for example) to compile/install as well.

The main differences are that Zero Install is source oriented and has a
more robust XML format (but no Metalink), Appupdater is oriented to
distributing binaries and of course already has Metalink support.
Appupdater may have issues figuring out what is installed if the hashes
differ when compiled on different systems for whatever reason, but multiple
hashes for a particular compiled file is allowed.

I think really the issue with package managers is that they all have their
own formats for the data and metadata, if we can express that independently
in some type of XML we can easily generate whatever data formats are needed
to be backward compatible. ZeroInstall+Metalink+Appupdater parts should be
a good start.

Neil

On Tue, 23 Oct 2012 00:13:31 -0700, Anthony Bryan <anthon...@gmail.com>
wrote:

[Anthony Bryan]

On Tue, Oct 23, 2012 at 1:06 PM, Neil M. <nabb...@gmail.com> wrote:
> A couple of things on that.
>
> Firstly, Zero Install seems to have an architecture already designed to do
> that (compile and install) for multiple platforms. Their main website
> seems to be down at the moment but Wikipedia has a good overview here:
>
> https://en.wikipedia.org/wiki/Zero_Install
>
> Secondly Appupdater has the capability to run arbitrary commands to
> "install" a product, so this could include the compile instructions or a
> ruby script (for example) to compile/install as well.
>
> The main differences are that Zero Install is source oriented and has a
> more robust XML format (but no Metalink), Appupdater is oriented to
> distributing binaries and of course already has Metalink support.
> Appupdater may have issues figuring out what is installed if the hashes
> differ when compiled on different systems for whatever reason, but multiple
> hashes for a particular compiled file is allowed.
>
> I think really the issue with package managers is that they all have their
> own formats for the data and metadata, if we can express that independently
> in some type of XML we can easily generate whatever data formats are needed
> to be backward compatible. ZeroInstall+Metalink+Appupdater parts should be
> a good start.

the first part of what I'm talking about and what the Gentoo package
maintainer was looking for is basically a command line program which
checks for new versions of programs.

all it would do is download some Metalinks from a list of addresses,
and alert if there was a new version.

$version-check
aria2: 1.15.1 -> 1.15.2
curl: 7.27.0 -> 7.28.0
libmetalink: 0.1 -> 0.1.2


it could be run once a day, week, etc. now package maintainers would
know to update their packages. as part of the packaging process, the
next step could be specific to Appupdater, Homebrew, Fedora, Gentoo,
etc... and download/verify the source & do a test build (we won't
worry about that).

the only problem is only curl has automated metalink [1] as part of
their release process. but, we can start w/ that, add aria2 &
libmetalink as a demo & show that other programs could do the same.

[1] http://curl.haxx.se/metalink.cgi?curl=tar.gz and using
<version>7.28.0</version> from it.

[Neil M.]

On Tue, 23 Oct 2012 13:44:35 -0700, Anthony Bryan <anthon...@gmail.com>
wrote:

I see now. As part of the Appupdater "Server Tools" I built something
like this. Basically I made a "crawler" that checks download pages for
given applications and notifies me when the version number changes. It can
also automatically generate Metalinks based on that. This feeds into other
things and in many cases Appupdater packages can be completed automated in
this way.

Neil