CI pipeline to build and publish OSv artifacts "nightly"

[Waldek Kozaczuk]

Some of you may have seen my latest commit I have just pushed. It sets up and enables Travis based pipeline to nightly build and publish OSv artifacts. It builds on the initial work by Matthew Pabst and enhances it by using open source project - https://github.com/nurupo/ci-release-publisher. 

Unlike the old CI setup, we not only build kernel and test related artifacts, but also publish them to a new Github repo - https://github.com/osvunikernel/osv-nightly-releases/releases - so that hopefully one can easily assemble and run an app on latest OSv kernel using capstan without having to build from source. For more details please read comments in this commit - https://github.com/cloudius-systems/osv/commit/7305d0ff9f961ae2f983c5141801ad4aa9b0dc51. 

For now the published artifacts are focused on the unit tests and ability to build unit tests images (aka ./scripts/build image=tests) using capstan and run them. As part of the commit I have also added new Python scripts that makes it easy to navigate new OSv nightly releases repo. As you can see with this tool, the only nightly latest release has 10 artifacts:

./scripts/github_releases_util.py list-releases

NAME               TITLE                                                        PUBLISHED_AT        

ci-master-latest   Latest CI build of master branch                             2020-03-09 04:59:57 

ci-master-69       v0.54.0-134-g7305d0ff                                        2020-03-09 05:00:01 

./scripts/github_releases_util.py list-artifacts

NAME                                     SIZE       DOWNLOAD_URL (base=https://github.com/osvunikernel/osv-nightly-releases/releases/download)

index.yaml                               124        /ci-master-latest/index.yaml                      

osv-loader.qemu.gz                       3838306    /ci-master-latest/osv-loader.qemu.gz              

osv-version                              22         /ci-master-latest/osv-version                     

osv.bootstrap.mpm                        315685     /ci-master-latest/osv.bootstrap.mpm               

osv.bootstrap.yaml                       139        /ci-master-latest/osv.bootstrap.yaml              

osv.unit-tests.mpm                       4013104    /ci-master-latest/osv.unit-tests.mpm              

osv.unit-tests.yaml                      175        /ci-master-latest/osv.unit-tests.yaml             

unit_tests.manifest                      8848       /ci-master-latest/unit_tests.manifest    

In future we can enhance this pipeline and for example build various flavors of kernel - zfs-less, firecracker specific, etc or for experimental features that are on branches - one such candidate is the ipv6 branch that I am hoping to merge master to and setup similar CI pipeline as well.

Let me know what you think about it. For example right now the artifacts are published on every commit but maybe we should do it really "nightly" only if there was some commit that day. Also for now the pipeline is setup to publish up to 10 last commits and then it rolls over. We can make it higher or lower - OSv artifacts are pretty tiny - in total less than 8 MB (kernel + unit tests).

Waldek

PS. Capstan is also able to list and pull packages from github but needs some tweaks to make it friendly with the new  https://github.com/osvunikernel/osv-nightly-releases/release repo.