[CCPPETMR/SIRF-SuperBuild] Number of users (#93)
Casper da Costa-Luis
We'd like to estimate the number of users. Sum:
- (SuperBuild clones) - (number of travis non-docker jobs) - 2*(number of travis docker jobs) - (a little extra for devs locally cloning?)
- (docker pulls) - (number of travis docker jobs)
- VM binary downloads
The main problem I see is (SuperBuild clones) is only accessible at https://github.com/CCPPETMR/SIRF-SuperBuild/graphs/traffic if you are logged in to github (not publicly accessible via a JSON API) so I can't get that data to generate a badge.
So... would it be OK to generate that data ourselves using cmake? By that I mean something like:
file(DOWNLOAD "http://caspersci.uk.to/cgi-bin/hits.cgi?q=sirf-superbuild" "tmpfile") file(REMOVE "tmpfile")
And then get .travis.yml to call
wget https://caspersci.uk.to/cgi-bin/hits.cgi?q=sirf-superbuild-travis
a few times... that way I can give you something like:
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
Casper da Costa-Luis
Casper da Costa-Luis
@paskino for VM binary downloads, you may want to use something link https://stackoverflow.com/questions/21982862/how-to-count-link-clicks#21982932
Edoardo Pasca
There's a way directly in Drupal to count them.
Edoardo Pasca
Accessing the clones via the GitHub API requires, as Casper said, an authorized webapp. So it can't be a simple script. I developed an authorized webapp to query GitHub API and I may modify to get the relevant info.
At any rate, data which can be collected is only for the last 14 days.
Edoardo Pasca
I managed to get drupal track the downloads of SIRF VM. ATM the info is not publicly available. Do we need live info?
Kris Thielemans
no need to be live as far as I can see
Kris Thielemans
@casperdcl I think we don't need the badge as that seems quite complicated at the moment (especially agregating it all). It'd be ok if we can keep track of that ourselves. We could then periodically update the website ourselves.
I'm not 100% sure now if we still need the file(DOWNLOAD...) trick. It's a bit ugly... If we do need it, it'd have to be done from the CCP website (and can then be counted by Drupal) I think.
Kris Thielemans
@casperdcl just merged a PR to be able to count github SIRF-Superbuild clones, subtracting travis ones. Great. @paskino counts VM downloads. Great. Are we counting docker downloads?
(In principle people can get the VM via vagrant etc, but I think this is a negligable minority, and that will create a SIRF-SuperBuild clone anyway, so we should be fine).
So are we done with this?
How do we get the relevant info? (can be manual as I said)
Casper da Costa-Luis
- count superbuild clones properly (see github-stats#viewing)
- count VM downloads (via drupal)
- count docker pulls (
)
- TODO: subtract travis' docker pulls (or just estimate this from github-stats's decrements)
Casper da Costa-Luis
@KrisThielemans we now have an automated badge! 
README on https://github.com/CCPPETMR/SIRF-SuperBuild). Just waiting for @paskino to implement a JSON API for the vm downloads (CCPPETMR/github-stats#1)
Casper da Costa-Luis
should be noted that every time we restart a job on travis (because e.g. it failed due to apt-get connections issues) then the counter will increment (because travis will have cloned the repo and then failed before we could decrement)
Casper da Costa-Luis
just noticed travis osx fails due to insecure python, vis https://travis-ci.org/CCPPETMR/SIRF-SuperBuild/jobs/348359877#L2217. Not sure if this deserves a separate issue to resolve.
urllib2.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)>
Kris Thielemans
@casperdcl this seems to be working great but the numbers are going up rather a lot. Is this realistic?
Casper da Costa-Luis
Yes, I was going to mention this. The problem is (#93 (comment)) if travis fails before it gets to the counting code, then we have no way of knowing the travis clone should not be counted.
There's a semi-solution - github also provides "unique cloners" data (presumably based on IP addresses of cloners). If we use that instead, then we accept counting up to one extra clone per day due to travis builds (even then I should be able to negate that). It might be a better approach.
Kris Thielemans
yes. I think it makes sense to call this "users" as opposed to downloads then. after all, if someone downloads it 5 times, we're not all that interested in knowing about this.
Casper da Costa-Luis
hopefully better now
Casper da Costa-Luis
it's displaying 707 now, but that may ne because of old overcounting. can easily add an offset, but let's see whether the number is still increasing rapidly first
Edoardo Pasca
The VM got 38 downloads with 20 unique IPs. (basically 20 users).
Currently we are using a drupal plugin to count the downloads. Do we really want to implement a JSON API in drupal?
Casper da Costa-Luis
you really just need to write out something like
{downloads: 20}in a file www.ccppetmr.ac.uk/.../vm.json
Presumably easy?
Edoardo Pasca
That I could do.
Casper da Costa-Luis
cool. if you can Drupal to automatically do that then that's all we need.
Edoardo Pasca
No, I can put a Json file on the website. That drupal changes it, it is a totally different ball-game. And I've currently no clue how.