CEAN 2.0 almost ready
Christophe Romain
Just to let you know i fixed last glitchs to the framework and CEAN 2.0
is almost done. framework is stable and extensible enough.
I'm waiting a minor contribution from bengt kleber and prepare the
public repository for initial commit. so we'll have base to start
working together.
the framework handles everything from fetching sources to building self
extractable installer, for whatever erlang version you want.
user will be able to use cean in binary install only "a la debian" or
build it's own repository and packages "a la gentoo"
temporary new site is here: http://dev4.process-one.net:8888/
some links are still broken (archive not generated) but the big picture
is ready.
so now i can focus on interraction with agner, and make sure all agner
package can be processed 'as this'. this would give us the 1st step.
Yurii, do you want to write a converter from cean's .pub to agner.config ?
may we discuss some merging convention ?
example:
etorrent.pub
{author, {"Jesper Louis Andersen", "jesper.louis.andersen...gmail.com"}}.
{packager, {"Christophe Romain", "christophe.romain...process-one.net"}}.
{category, ["net"]}.
{depends, []}.
{keywords, []}. %% used by search engine in addition to summary
{summary, "Erlang Bittorrent Client"}.
{abstract, "Etorrent is a bittorrent client implementation in Erlang focusing on fault-tolerance"}.
{home, "https://github.com/jlouis/etorrent"}.
{url, "git://github.com/jlouis/etorrent.git"}.
agner.config
{name, "etorrent"}.
{authors, ["Jesper Louis Andersen <jesper.louis.andersen...gmail.com>"]}.
{description, "Etorrent is a bittorrent client implementation in Erlang
focusing on fault-tolerance"}.
{homepage, "http://github.com/jlouis/etorrent"}.
{rebar_compatible, true}.
{license, "BSD2", "COPYING"}.
{erlang_versions, [otp_r14b, otp_r14b01, otp_r13b04]}.
{url, {git, "https://github.com/jlouis/etorrent.git", {branch, "master"}}}.
in parallel, we can discuss on binary packaging structure
you can parse what cean generates here:
http://dev4.process-one.net:8888/pub/bin/
http://dev4.process-one.net:8888/pub/src/
example:
http://dev4.process-one.net:8888/pub/bin/linux-amd64/R14/bfile/bfile.pub
{author, {"Claes Wikstrom", "klacke...hyber.org"}}.
{packager, {"Christophe Romain", "christophe.romain...process-one.net"}}.
{category, ["text"]}.
{depends, ["kernel","stdlib"]}.
{keywords, ["binary","file","io","rapid"]}.
{summary, "An interface to fast FILE I/O"}.
{abstract, ""}.
{home, "http://yaws.hyber.org"}.
{url, "http://yaws.hyber.org/download/bfile-1.2.tgz"}.
{"1.2", "2011.0606", ok}.
last line says version 1.2 has been build on 6th june and is ok
package is
http://dev4.process-one.net:8888/pub/bin/linux-amd64/R14/bfile/bfile-1.2.epkg
% tar tf bfile-1.2.epkg
bfile-1.2.src.zip
bfile-1.2.ez
depending on your local install, cean:install command will either:
- just extract bfile-1.2.ez (erlang can load beams from .ez archive)
- extract bfile-1.2.ez contents, this generates bfile-1.2/ebin
- extract bfile-1.2.ez and bfile-1.2.src.zip, this generates
bfile-1.2/ebin and bfile-1.2/src
that way, from one .epkg, user have choice of its own installation
- binary only production installation
- typical development installation
the installer generation script takes erlang minimalist VM, the lib you
want to package and all its dependencies (by extracting corresponding
.epkg) so binary and source packaging convention really matters now.
Martin, what is your opinion about this ? i just wanted to share my
vision of the big picture. again i don't know ciconia enough to see how
far we can share the packaging process. anyway there are some very cool
features in ciconia such as release management.
i think it would be terrific to merge the release management and the
self extractble installer (as it supports postinstall.sh scripts) this
would make cluster administration/deployment a child play (even
something administrator could include in .rpc/.deb packaging)
Eric Merritt
After a lot of talk Martin and I decided to drop further development
of ciconia. There are already several managers out there that have
more traction. It didn't seem like a good idea for us to spend
development time on yet a third. However, we still have a huge stake
in this game as consumers of this information. Sinan, at the very
least, will be handling its own dependencies very soon and have
plugins for agner at least.
Eric
Martin Logan
agner and cean start to merge all the better.
--
Martin Logan
Erlang & OTP in Action (Manning) http://manning.com/logan
http://twitter.com/martinjlogan
http://erlware.org
Eric Merritt
On your binary packages for linux and darwin. I don't actually think
the architecture alone is sufficient to guarantee binary
compatibility. At the very least you need the libc version number as
well as the bitwidth (x86_64) though that is probably part of the
architecture and you just don't have any x86_64 packages at the
moment.
Right now you are using the Erlang Release number (R13, R14, etc). I
personally would much rather see the erts version there, 5.6, 5.7. At
the very least I would like there to be a symlink from the Release
number to the erts version number. The erts version is a lot more
meaningful to programatic consumers then the Release number is.
These are things I can think of in a quick glance as a consumer, along
with some knowledge we have of several years of faxien support.
Eric
On Tue, Jul 12, 2011 at 4:43 PM, Christophe Romain
<christop...@process-one.net> wrote:
Eric Merritt
Erlware. It might be very worth while for us to collaborate on that. I
am not sure how you are doing it now.
Christophe Romain
To cover the glibc compatibility issue, we have two choice:
- build against many version of glibc
- build with most compatible version
for cean i choosed the second option. building under debian stable is
generally a good option for this, except for few month when major glibc
version switch.
doing so, i had compatibility issue only 2 times in 6 years, so i'm
quite happy with this simplist choice. moreover, CEAN being a build
framework, if a user is not happy with binary dependencies and needs
latest glibc linked binary, he can deploy the framework and build its
own epkg, linked to its own bleeding edge glibc.
i wanted to make cean as simple as possible. 90% users will be happy
with binary compiled with same glibc as debian stable. while i can
include glibc version is repository tree (to keep compatibility with
erlware) i won't setup a complex build farm, so i'll keep build binary
only against debian stable (libc-2.11.2 i think now)
for bitwidh, i have linux-x86 and linux-amd64. maybe should i rename
amd64 to x86_64, it's just a matter of choice here.
i thought for long about use of release number or ets number (as you are
using) and i kept simplest choice as i could not find best option here.
I finally concluded this is more a question of taste.
your proposal to use erts version, and symlink release number is good,
i can go this way for sure, so we'll have closer repository structure.
thanks for your feedback. I better understand your choices now and they
make sense, i think i'll go this way.
to build binaries, i build on differnte boxes (linux-x86 linux-amd64
osx and some xen vm) windows is generated from binary tarball.
cross compilation script is in progress for linux, i used it to
crosscompile ejabberd installers for windows and linux.
i'm also working on an cean package to .deb script converter, that way
we could have benefits of debian's compilation farm.
Eric Merritt
<christop...@process-one.net> wrote:
> Eric,
>
> To cover the glibc compatibility issue, we have two choice:
> - build against many version of glibc
> - build with most compatible version
>
> for cean i choosed the second option. building under debian stable is
> generally a good option for this, except for few month when major glibc
> version switch.
> doing so, i had compatibility issue only 2 times in 6 years, so i'm
> quite happy with this simplist choice. moreover, CEAN being a build
> framework, if a user is not happy with binary dependencies and needs
> latest glibc linked binary, he can deploy the framework and build its
> own epkg, linked to its own bleeding edge glibc.
I am completely fine with this approach. As long is its been
considered and a solution that works introduced I am happy. The
explicit solution doesn't actually matter a whole lot.
> i wanted to make cean as simple as possible. 90% users will be happy
> with binary compiled with same glibc as debian stable. while i can
> include glibc version is repository tree (to keep compatibility with
> erlware) i won't setup a complex build farm, so i'll keep build binary
> only against debian stable (libc-2.11.2 i think now)
> for bitwidh, i have linux-x86 and linux-amd64. maybe should i rename
> amd64 to x86_64, it's just a matter of choice here.
Actually, uname returns x86_64, this is the same thing that
erlang:system_info(system_architecture) info returns. It just makes it
awkward that a translation has to occur to access the packages. Minor
yes, but inelegant.
>
> i thought for long about use of release number or ets number (as you are
> using) and i kept simplest choice as i could not find best option here.
> I finally concluded this is more a question of taste.
> your proposal to use erts version, and symlink release number is good,
> i can go this way for sure, so we'll have closer repository structure.
Cool.
>
> thanks for your feedback. I better understand your choices now and they
> make sense, i think i'll go this way.
>
> to build binaries, i build on differnte boxes (linux-x86 linux-amd64
> osx and some xen vm) windows is generated from binary tarball.
> cross compilation script is in progress for linux, i used it to
> crosscompile ejabberd installers for windows and linux.
> i'm also working on an cean package to .deb script converter, that way
> we could have benefits of debian's compilation farm.
>
Do you do all this by hand?
Tristan Sloughter
Christophe Romain
>> to build binaries, i build on differnte boxes (linux-x86 linux-amd64
>> osx and some xen vm) windows is generated from binary tarball.
>> cross compilation script is in progress for linux, i used it to
>> crosscompile ejabberd installers for windows and linux.
>> i'm also working on an cean package to .deb script converter, that way
>> we could have benefits of debian's compilation farm.
>>
>
>Do you do all this by hand?
the entire build is one command, and then i rsync the generated
repository on the public one. this is a manual or cron action.
the aim of the crosscompilation script is to include it automatically
when building everything from one instance of the framework. (this will
not be ready for CEAN 2.0 release anyway)
Eric Merritt
integration, supporting at least the platforms you already support
(its my next big project). If you are interested I can open it up to
you and maybe we can both benefit.
Christophe Romain
>integration, supporting at least the platforms you already support
>(its my next big project). If you are interested I can open it up to
>you and maybe we can both benefit.
that would be terrific
by the way, i worked on an import script from archive you generate to
produce epkg as well, so we can share compilation efforts anyway.
i'll ping you shortly when i have modification we talked about in this
thread ready. our repository model will merge nicely.
Eric Merritt
Just a reminder. I am not the Agner guy thats Yuri. I am one of the
Erlware guys, we don't have a repository anymore, we are relying on
you all. Hence my interest and support :)
Eric
Christophe Romain
>you all. Hence my interest and support :)
my script uses http://repo.erlware.org/pub as input
may i consider this as obsolet then ?
Eric Merritt
ported we will be taking the repos down.