Distributing Releases to platforms
95 views
Skip to first unread message
Thomas Cioppettini
May 13, 2019, 12:39:26 AM5/13/19
to elixir-lang-core
I think we should manage the distribution of elixir to common package managers after a release has been built.
Right now we rely on other maintainers to be aware that elixir has been updated, and do their respective work to add the most recent package to their repository. This process is slow and unreliable, we should automate it so that the most recent versions of libraries are available everywhere.
José Valim
May 13, 2019, 2:35:20 AM5/13/19
to elixir-l...@googlegroups.com
Hi Thomas,
It is not possible for us to handle all of the different OSes and package managers, manage permissions, debug errors, etc. Publishing is only one part of the process.
However, we will be glad to improve our current "broadcast" infrastructure, so whenever we publish a new version, everyone can receive an event and have their automation tool publish to the OSes they consider common. Today you can get those events either via the google groups OR via the GitHub releases page (which provides a feed IIRC).
José Valim
Skype: jv.ptec
Founder and Director of R&D
On Mon, May 13, 2019 at 6:39 AM Thomas Cioppettini <t...@scalpel.com> wrote:
I think we should manage the distribution of elixir to common package managers after a release has been built.Right now we rely on other maintainers to be aware that elixir has been updated, and do their respective work to add the most recent package to their repository. This process is slow and unreliable, we should automate it so that the most recent versions of libraries are available everywhere.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/ec0791f9-8e6c-4ce4-b900-28ea262186aa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Message has been deleted
Michael Klishin
May 13, 2019, 1:54:37 PM5/13/19
to elixir-l...@googlegroups.com
As one example, team RabbitMQ produces binary builds for a range of Debian-based distributions [1] using Concourse [2].
We might extend that to RPMs at some point but currently do not produce any artifacts of our own on an RPM-based system.
Concourse uses scheduled Git resource (repository) polling, nothing particularly sophisticated.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JYGTKgnANguF0m9ZAQ5E2y1BTwtsrRd-E_SpS9%2BpdffA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
MK
Staff Software Engineer, Pivotal/RabbitMQ
Dave Cottlehuber
May 28, 2019, 6:35:30 AM5/28/19
to Thomas Cioppettini, elixir-lang-core
On Tue, 14 May 2019, at 16:49, Thomas Cioppettini wrote:
> Hmm this might be a nice side project. I think we could start with the
> most common environments and build out from there. Based on some quick
> research this may be more automatable than you may think. Based on what
> I've seen most projects just are linking to the github archive and
> hardcoding a hash of the contents. If that's it, then it should be
> fairly reasonable to just update it. I'll do some more research and get
> back to you.
This is a desirable goal & I don't want to dissuade you, but this is a surprising amount of work, often requiring manual tweaking that requires a reasonably deep understanding of how packages and package managers work on a given system.
<combinatorial explosion>
The end result is that you begin to ship a custom OTP release, including static OpenSSL, and having your own per-OS custom package repos, to try to keep up. Then you find out that popular project X doesn't yet support that version of OTP, so now you ship multiple OTP releases per Elixir release. Phoenix, RabbitMQ, & CouchDB are well-known projects with varying bits of Elixir on the inside.
</combinatorial explosion>
That said, https://repology.org/project/elixir/versions may well be a useful place to start for somebody who is interested in hunting down what's lagging behind and submitting patches. Its source code is https://github.com/repology/repology & https://github.com/repology/repology-rules, and a neat hack would be, after X days after a release, to send a gentle reminder to the maintainers listed for any lagging downstream distros. Or, to post the list to elixirforums and recommend people contribute patches for their favourite distros.
I think that having an announce@ list and encouraging maintainers to subscribe, is your biggest win for the least effort. It's certainly shortest in LoC :-)
A+
Dave
> Hmm this might be a nice side project. I think we could start with the
> most common environments and build out from there. Based on some quick
> research this may be more automatable than you may think. Based on what
> I've seen most projects just are linking to the github archive and
> hardcoding a hash of the contents. If that's it, then it should be
> fairly reasonable to just update it. I'll do some more research and get
> back to you.
>
> On Monday, May 13, 2019 at 2:35:20 AM UTC-4, José Valim wrote:
> > Hi Thomas,
> >
> > It is not possible for us to handle all of the different OSes and package managers, manage permissions, debug errors, etc. Publishing is only one part of the process.
> >
> > However, we will be glad to improve our current "broadcast" infrastructure, so whenever we publish a new version, everyone can receive an event and have their automation tool publish to the OSes they consider common. Today you can get those events either via the google groups OR via the GitHub releases page (which provides a feed IIRC).
> >
> > *José Valim*
> On Monday, May 13, 2019 at 2:35:20 AM UTC-4, José Valim wrote:
> > Hi Thomas,
> >
> > It is not possible for us to handle all of the different OSes and package managers, manage permissions, debug errors, etc. Publishing is only one part of the process.
> >
> > However, we will be glad to improve our current "broadcast" infrastructure, so whenever we publish a new version, everyone can receive an event and have their automation tool publish to the OSes they consider common. Today you can get those events either via the google groups OR via the GitHub releases page (which provides a feed IIRC).
> >
> > www.plataformatec.com.br
> > Skype: jv.ptec
> > Founder and Director of R&D
> >
> >
> > On Mon, May 13, 2019 at 6:39 AM Thomas Cioppettini <t...@scalpel.com> wrote:
> >> I think we should manage the distribution of elixir to common package managers after a release has been built.
> >>
> >> Right now we rely on other maintainers to be aware that elixir has been updated, and do their respective work to add the most recent package to their repository. This process is slow and unreliable, we should automate it so that the most recent versions of libraries are available everywhere.
Hi Thomas,
> > Skype: jv.ptec
> > Founder and Director of R&D
> >
> >
> > On Mon, May 13, 2019 at 6:39 AM Thomas Cioppettini <t...@scalpel.com> wrote:
> >> I think we should manage the distribution of elixir to common package managers after a release has been built.
> >>
> >> Right now we rely on other maintainers to be aware that elixir has been updated, and do their respective work to add the most recent package to their repository. This process is slow and unreliable, we should automate it so that the most recent versions of libraries are available everywhere.
This is a desirable goal & I don't want to dissuade you, but this is a surprising amount of work, often requiring manual tweaking that requires a reasonably deep understanding of how packages and package managers work on a given system.
<combinatorial explosion>
The end result is that you begin to ship a custom OTP release, including static OpenSSL, and having your own per-OS custom package repos, to try to keep up. Then you find out that popular project X doesn't yet support that version of OTP, so now you ship multiple OTP releases per Elixir release. Phoenix, RabbitMQ, & CouchDB are well-known projects with varying bits of Elixir on the inside.
</combinatorial explosion>
That said, https://repology.org/project/elixir/versions may well be a useful place to start for somebody who is interested in hunting down what's lagging behind and submitting patches. Its source code is https://github.com/repology/repology & https://github.com/repology/repology-rules, and a neat hack would be, after X days after a release, to send a gentle reminder to the maintainers listed for any lagging downstream distros. Or, to post the list to elixirforums and recommend people contribute patches for their favourite distros.
I think that having an announce@ list and encouraging maintainers to subscribe, is your biggest win for the least effort. It's certainly shortest in LoC :-)
A+
Dave
Rahul Sharma
Mar 30, 2020, 4:35:51 AM3/30/20
to elixir-lang-core
Hello
I am interested to see that while making a deployable elixir release, the release system can bundle in the dynamically linked binaries like openssl
Would that amount to the same effort on the elixir team's end?
It would be great to be able to create desktop apps(non gui) using elixir
Regards
R
Rahul Sharma
Mar 30, 2020, 5:07:15 AM3/30/20
to elixir-lang-core
P.S. Is it possible to provide a gist of how to do the compilation with the dynamic linking replaced with static ones.
Also, I a total noob web dev, so pardon my inaccuracies
Best regards
R
Dave Cottlehuber
Apr 9, 2020, 2:55:15 PM4/9/20
to elixir-l...@googlegroups.com
> On Monday, March 30, 2020 at 2:05:51 PM UTC+5:30, Rahul Sharma wrote:
> > Hello
> >
> > I am interested to see that while making a deployable elixir release, the release system can bundle in the dynamically linked binaries like openssl
> > Would that amount to the same effort on the elixir team's end?
If your erts (erlang runtime, aka BEAM) is built with static libraries, on a platform that supports this, then yes this is possible, but I don't think this is the Elixir team's main focus.
> > Hello
> >
> > I am interested to see that while making a deployable elixir release, the release system can bundle in the dynamically linked binaries like openssl
> > Would that amount to the same effort on the elixir team's end?
> > It would be great to be able to create desktop apps(non gui) using elixir
On Mon, 30 Mar 2020, at 09:07, Rahul Sharma wrote:
> P.S. Is it possible to provide a gist of how to do the compilation with
> the dynamic linking replaced with static ones.
http://erlang.org/doc/installation_guide/INSTALL.html
IIRC you can hypothetically embed any NIFs you like using --enable-static-nifs but it's been a while since I tried it.
Once you have that working, you could then drop your elixir .beam files in on top.
Personally, given you're just getting started, I'd leave this for later -- this will either be a Great Adventure Where you Learn A Lot or a Horrible Nightmare :-)
A+
Dave
Reply all
Reply to author
Forward
0 new messages