upload CI built binaries to nightly release

[Qian Yun]

I have done some research on how to "upload CI built
binaries to nightly release".

First, this is not directly supported by GitHub.
To have a release you need a tag, and it is not feasible
to create a tag for every commit.

What I want is to have a fixed place (URL) for the nightly
release. And the workaround I found is:

Create an empty branch and tag and do the release there,
then update the "Assets" section of this release from CI.

(More details on how to update the "Assets" sections:
use REST API:
https://docs.github.com/en/rest/releases/assets#update-a-release-asset

It is a little complicated to create a new asset from CLI,
so we can create new assets from Web interface first and use
the generated ASSET_ID to update them in CI later.)


So summary:

I'd like to upload CI built binaries to nightly
release so that people can download them without log into
GitHub account and find them on CI webpages.

To achieve this I will need to:

create a new branch (git checkout --orphan nightly-release)
add a short README and commit and tag
create "Nightly Release" from this tag
update CI scripts to upload binaries to this release


Comments are welcome. Without objections I will start
implement this next week.

- Qian

[Qian Yun]

On 8/13/22 17:36, Qian Yun wrote:
> It is a little complicated to create a new asset from CLI,
> so we can create new assets from Web interface first and use
> the generated ASSET_ID to update them in CI later.)

Fixup: still need to "upload" instead of "update".

[Ralf Hemmecke]

> I'd like to upload CI built binaries to nightly
> release so that people can download them without log into
> GitHub account and find them on CI webpages.
>
> To achieve this I will need to:
>
> create a new branch (git checkout --orphan nightly-release)
> add a short README and commit and tag
> create "Nightly Release" from this tag
> update CI scripts to upload binaries to this release

I do not quite understand. With --orphan you basically create another
commit graph inside the fricas git repo that is disconnected from the
sources that we normally commit to. Will there only be README files (one
per nightly-build) or what do you want in this nightly-release branch?

I somehow would not want to put extra stuff into the source git repo if
it is not strictly necessary. Can you explain more.

Ralf

[Qian Yun]

On 8/13/22 20:14, Ralf Hemmecke wrote:
> I do not quite understand. With --orphan you basically create another
> commit graph inside the fricas git repo that is disconnected from the
> sources that we normally commit to. Will there only be README files (one
> per nightly-build) or what do you want in this nightly-release branch?

Yes, the point is to have a disconnected single commit (tag/release) to
hold the nightly builds. In this branch there will be only one commit
-- a single README file (not one per nightly-build).

> I somehow would not want to put extra stuff into the source git repo if
> it is not strictly necessary. Can you explain more.
>

The point is to have a commit/tag/release to hold the nightly builds.
The alternative ways are:

1. The "--orphan" tree holds the commit/tag/release.
2. Select a commit from master branch to be the commit/tag/release.
This might cause confusion for users.
3. Many other projects on GitHub create a new repo to hold the
nightly builds.

- Qian

[Ralf Hemmecke]

> The point is to have a commit/tag/release to hold the nightly builds.
> The alternative ways are:
>
> 1. The "--orphan" tree holds the commit/tag/release.
> 2. Select a commit from master branch to be the commit/tag/release.
> This might cause confusion for users.
> 3. Many other projects on GitHub create a new repo to hold the
> nightly builds.

Honestly, I wouldn't care much about a single disconnected commit for
the nightly builds, but somehow I tend to favour your option 3.
We can easily create a repo fricas-nightly-releases for this.
Would that also be only one commit in that repo?

What do you think?

Ralf

[Qian Yun]

Yes.

> What do you think?
>
> Ralf
>

Both options are fine for me. I can start preparing the rest pieces
(CI script) in my repo. And I'll reply to this thread again when
I'm done with the scripts and ready to create the new repo.

- Qian

[Waldek Hebisch]

For me both options are fine too. Just if you go for separate
repo please call it nightly-builds (and not releases).

--
Waldek Hebisch

[Qian Yun]

So this is my CI scripts, comments are welcome.

Some comments:

1. You can see the testing nightly builds from
https://github.com/oldk1331/fricas/releases

2. It is hardcoded with my username right now, after setting up
repos in https://github.com/fricas, this script will be tweaked
and pushed to https://github.com/fricas/fricas.

3. I modified the binary filename to contain date so that it
is sorted by date in release page.

4. The scripts does not delete old files. It may require us
to manually clean the release page (once a month is enough I think).

5. Failure of uploading will not make the whole script fail.

@Rlaf, if you see my private email regrading repo permissions,
please reply.

- Qian

=====
- name: Upload to nightly release
if: ${{ github.event_name == 'push' && github.event.ref ==
'refs/heads/master' && github.repository == 'oldk1331/fricas' }}
run: |
export OWNER_REPO=oldk1331/fricas
export RELEASE_ID=`curl
https://api.github.com/repos/${OWNER_REPO}/releases/tags/nightly | jq .id`
curl -v -X POST -H "Accept: application/vnd.github+json" -H
"Authorization: token ${{ secrets.token }}"
"https://uploads.github.com/repos/${OWNER_REPO}/releases/${RELEASE_ID}/assets?name=FriCAS-$(date
+%Y-%m-%dT%H.%M)-linux-x86_64-$(echo ${{ github.sha }} | cut -c
1-8).tbz2" --data-binary "@FriCAS-linux-x86_64-${{ github.sha }}.tbz2"
-H "Content-Type: application/x-bzip2" || true
=====

[Qian Yun]

Done, you can see the result at:

https://github.com/fricas/fricas-nightly-builds
https://github.com/fricas/fricas-nightly-builds/releases/tag/nightly

- Qian

[Kurt Pagani]

A great convenience, +1 :)