Hi again,
I forgot to mention that we have also improved our development
methodology based on your feedback:
We decided to move most CK modules from numerous CK repositories to
the CK framework. There are several benefits to that:
1) We can now use standard development practices to keep all the
code in one repository thus making it easier to test it as well as
update or redesign the CK kernel if necessary. One of the reasons
why I couldn't redesign the CK kernel and got stuck with old
prototype for such a long time was that I couldn't test all the
dependencies and workflows (see
these notes). All
CK functionality can be extended and tested via PRs in one place and
we can also have a faster release cycle of the CK framework with all
the main functionality included!
2) We had complaints that while the CK framework has a BSD-license
there is no guarantee that other modules pulled from numerous
repositories use the same license. For now, we indirectly solve it
by using the cTuning foundation copyright and the BSD-license in all
cTuning repositories but it is not necessarily the case for other
third-party repositories. Having all code in one place solves this
problem. That's why I collected most CK modules from numerous
cTuning foundation repositories and merged them with the CK
framework in the latest release.
The new
AI repository is used to
collect all CK components related to ML Systems workflows from the
cTuning foundation repositories in one place. Note that it has
duplicated CK modules which is needed for backwards compatibility.
If someone uses previous CK version without CK modules, they can
still use this repository (CK will pick up modules from this
repository). Otherwise, CK will pick up modules from the newer CK
framework and will ignore modules in AI repository.
Updating CK modules is relatively straightforward now: just fork the
CK framework and use it instead of the one installed via PIP. You
can then directly update any module there (you can find the path to
the "default" repository inside CK framework using the following
command "ck get_default_repo" rather than searching for them in
numerous repositories. Whenever you finish updating and testing CK
modules, just submit the PR
here. Please
mention the urgency, i.e. if you want to release this functionality
soon or it can wait until we accumulate multiple updates.
Personally, I am fine to have the fast release cycle because it will
finally allow us to start improving/redesigning the CK kernel since
it will now "follow" all CK modules! It will also make it easier for
the community to follow the CK developments and updates!
I believe that the approach we came up with this summer is sound
since it solves many issues from different users (we thought a lot
about it this summer)!
If you have any further feedback and ideas, please tell us! Thanks a
lot and happy hacking ;),
Grigori