Features for cKnowledge.io and cBench

[Patrick Hesse]

Hey everyone,

Recently I have been looking into ck to use as a tool for reproducibility as a part of the research I am doing at my university. I have mostly been working on converting an 'ad-hoc' workflow into something much more reproducible and along the way I have been able to explore some of ck's many features.

One of the features that I have stumbled upon recently is cBench and using it to push data to dashboards on cKnowledge.io. Naively, I started by pushing some test points to a test dashboard I created, to find out I cannot delete anything or update already pushed points.

I discussed a few ideas with Grigori, and since cBench is still in it's early stages, it seems there are a lot of things that are still under construction. Keeping a permanent archive while still allowing for new users like me to test out creating and pushing to a dashboard I think should be possible, what do other people think? (Grigori mentioned maybe giving the owner a remove privilege for components).

Another feature could be when you are pushing a point to a dashboard from a json file for example, maybe we could have a '--temp' flag or something so that you could see how the points look on the graph but they wouldn't be permanently there. 

On top of this, having cbench not push invalid points is another feature I think could be added, as it is pretty easy to push a json file with points that has a typo, breaking the entire graph.

These are all things I have noticed while I have just began using cbench and discussed with Grigori but I hope I can hear from others what their experience has been using cbench, and what other people would like to see added, considering it is currently a proof-of-concept.

Patrick Hesse

[Davide Del Vento]

I have not used cbench myself, but I agree with the arguments you make
to be able to remove something under those scenarios.

The '--temp' or '--test' flags for trying things around seem the best
approach to me.
For the other cases, if it's really uncomputable, it should be
automatically "deletable" and then there could be a '--retract' flag
(with mandatory explanation) which only somebody with high enough
privileges could use for cases when for example the data was tainted,
e.g. when two different experiments were mixed up for operator error.
In these cases, I think it would be good that the server preserves the
history of what happened as well as the explanation

Regards,
Davide Del Vento,
NCAR Computational & Information Services Laboratory

> --
> You received this message because you are subscribed to the Google Groups "Collective Knowledge Technology" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to collective-knowl...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/collective-knowledge/a0a4fbe7-7bee-4683-9b7c-959b71515bb0n%40googlegroups.com.

[Anton Lokhmotov]

According to Vint Cerf, "we need to find a way to distinguish between
'that which can be safely ephemeral' and 'that which should be long
lived'" :)
https://www.youtube.com/watch?v=dsMKJKTOte0&feature=youtu.be&t=182

Anton.

> To view this discussion on the web visit https://groups.google.com/d/msgid/collective-knowledge/CAMh-zaM_no4gQ%2BnY%2Bg1jPKTLrGv9rKph2DcoW7LNSgF_8WduCQ%40mail.gmail.com.

[Grigori Fursin]

Hi Patrick and David,

I answered this question on GitHub (https://github.com/ctuning/ck/issues/127) and just realized that I didn't reply here - sorry about that :( ...

Thank you for your suggestions - I agree with you and I partially implemented related features (copy/paste from the GitHub issue):

I added the functionality to delete components from the CK portal (for owner and if not permanent), and it is also possible to publish components and mark them as permanent with flag "cb publish --permanent" ....

I also fixed the problem with the graph, i.e. the platform should now ignore results in the wrong format.

To test it, you need to update cbench to v1.3.1 (I just uploaded it to Pypi): pip install cbench -U

I also updated the documentation: https://cknowledge.io/docs/guide/cbench-commands.html#cknowledge-dashboards

Thank you very much again for your feedback!!! There is still a lot to be done but your suggestions motivate me to continue improving the CK framework and the cKnowledge.io platform!

Take care and will keep in touch,

Grigori