run patchbot in Google Cloud?

[Ralf Stephan]

I understand SMC is already in the cloud, so what would be necessary
to have Sage with a patchbot running there? Is there some documentation
on the Sage side? Ideally I'd just pay and use an existing copy,
preferably with control over the ticket numbers.

Regards,

[William Stein]

What would be a minimal useful machine configuration to start with?

https://cloud.google.com/products/compute-engine/#pricing

I can spin up an Ubuntu 14.04 image for you try if you tell me which
machine type would work, and send me your ssh key (so you can login).
This would be "free for a few months" -- I have a grant from Google
for this sort of thing, which expires in November.

Alternatively, you could run a patchbot in a project on SMC itself.
If you make a project and send me the project id, I can increase the
disk quota to be enough for this purpose (or you could use scratch),
disable timeouts, and also peg it to only run on GCE (rather than
machines at UW). My preference would actually just to be do this as
part of an SMC project, since it is a good test of SMC.

Thoughts?

-- William

>
> Regards,
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Robert Bradshaw]

On Tue, Apr 29, 2014 at 9:30 PM, William Stein <wst...@gmail.com> wrote:
> On Tue, Apr 29, 2014 at 9:04 PM, Ralf Stephan <gtr...@gmail.com> wrote:
>> I understand SMC is already in the cloud, so what would be necessary
>> to have Sage with a patchbot running there? Is there some documentation
>> on the Sage side? Ideally I'd just pay and use an existing copy,
>> preferably with control over the ticket numbers.

As documented on the patchbot page, just run

sage -i http://sage.math.washington.edu/home/robertwb/patches/patchbot-2.1.1.spkg
sage -patchbot

> What would be a minimal useful machine configuration to start with?
>
> https://cloud.google.com/products/compute-engine/#pricing
>
> I can spin up an Ubuntu 14.04 image for you try if you tell me which
> machine type would work, and send me your ssh key (so you can login).
> This would be "free for a few months" -- I have a grant from Google
> for this sort of thing, which expires in November.
>
> Alternatively, you could run a patchbot in a project on SMC itself.
> If you make a project and send me the project id, I can increase the
> disk quota to be enough for this purpose (or you could use scratch),
> disable timeouts, and also peg it to only run on GCE (rather than
> machines at UW). My preference would actually just to be do this as
> part of an SMC project, since it is a good test of SMC.

The minimum requirements are enough RAM to compile Sage (several GB
these days for linbox, right?), enough disk for 2 installs of Sage
plus, ideally, several GB for a good ccache. There's no minimum number
of cores, the more you throw at it the faster it'll run.

- Robert

[Ralf Stephan]

Thanks that was quick. I'll try it as project, expect some stumbling. Later one 100 per cent CPU would be covered.
Of course it means that only tickets with python-only patches could be tested, right? Less buildbot, more a testbot.

You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/KTx_aqvb2yo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

[William Stein]

On Tue, Apr 29, 2014 at 9:43 PM, Ralf Stephan <gtr...@gmail.com> wrote:
> Thanks that was quick. I'll try it as project, expect some stumbling. Later
> one 100 per cent CPU would be covered.

Let me know what the project id is, so I can up some quotas (e.g.,
RAM, etc.). We can add Robert Bradshaw and a few other people as
collaborators on the project, so it's easy for them to help resolve
any issues.

> Of course it means that only tickets with python-only patches could be
> tested, right? Less buildbot, more a testbot.

No, you can test anything -- you'll just start with a build of Sage in
/scratch/.

Put things that matter in the home directory of the project and put
stuff that is ephemeral (i.e., easy to recreate) in /scratch.
Projects have a default quota of 15GB in /scratch.

Thanks.

-- William

[Ralf Stephan]

OK, after fixing the self.log patchbot problem in patchbot-2.1.1 I get:

Getting trusted author list...

WARNING: Do not use this copy of sage while the patchbot is running.

'/scratch/sage-6.1.1-x86_64-Linux'/sage -i ccache

git checkout patchbot/base

git checkout -b patchbot/base

git fetch https://github.com/sagemath/sage.git +master:patchbot/base_upstream

/scratch/sage-6.1.1-x86_64-Linux$ cat stderr

error: pathspec 'patchbot/base' did not match any file(s) known to git.

Switched to a new branch 'patchbot/base'

fatal: Unable to find remote helper for 'https'

Traceback (most recent call last):

  File "/scratch/sage-6.1.1-x86_64-Linux/local/bin/patchbot/patchbot.py", line 783, in <module>

    main(args)

  File "/scratch/sage-6.1.1-x86_64-Linux/local/bin/patchbot/patchbot.py", line 744, in main

    patchbot.check_base()

  File "/scratch/sage-6.1.1-x86_64-Linux/local/bin/patchbot/patchbot.py", line 295, in check_base

    do_or_die("git fetch %s +%s:patchbot/base_upstream" % (self.config['base_repo'], self.config['base_branch']))

  File "/scratch/sage-6.1.1-x86_64-Linux/local/bin/patchbot/util.py", line 91, in do_or_die

    raise exn_class, "%s %s" % (res, cmd)

Exception: 32768 git fetch https://github.com/sagemath/sage.git +master:patchbot/base_upstream

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/KTx_aqvb2yo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

[Robert Bradshaw]

How recent is your git?

> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[Ralf Stephan]

On Wed, Apr 30, 2014 at 10:05 AM, Robert Bradshaw <robe...@math.washington.edu> wrote:

How recent is your git?

​/scratch/sage-6.1.1-x86_64-Linux$ ./local/bin/git --version

git version 1.8.4.4

/scratch/sage-6.1.1-x86_64-Linux$ git --version

git version 1.9.1

[Ralf Stephan]

On Wed, Apr 30, 2014 at 9:31 AM, Ralf Stephan <gtr...@gmail.com> wrote:

patchbot/patchbot.py", line 295, in check_base

    do_or_die("git fetch %s +%s:patchbot/base_upstream" %  

(self.config['base_repo'], self.config['base_branch']))

​Since I couldn't get a git version that worked (reinstalled curl and ​git from source,

did I need a new libssl too?) I just changed the git fetch URL to git://

http://stackoverflow.com/questions/8329485/git-clone-fatal-unable-to-find-remote-helper-for-https

Now the binary package has built its own docs, and is self-testing. What a waste of time.

[Ralf Stephan]

It appears git changed its output with some version <=1.9.2:

sage -t --long src/sage/dev/sagedev.py

**********************************************************************

File "src/sage/dev/sagedev.py", line 897, in sage.dev.sagedev.SageDev.checkout_branch

Failed example:

    dev.git.echo.stash('apply')

Expected:

    # On branch branch2

    # Changes not staged for commit:

    #   (use "git add <file>..." to update what will be committed)

    #   (use "git checkout -- <file>..." to discard changes in working directory)

    #

    #   modified:   tracked

    #

    # Untracked files:

    #   (use "git add <file>..." to include in what will be committed)

    #

    #   untracked

    no changes added to commit (use "git add" and/or "git commit -a")

Got:

    On branch branch2

    Changes not staged for commit:

      (use "git add <file>..." to update what will be committed)

      (use "git checkout -- <file>..." to discard changes in working directory)

    <BLANKLINE>

        modified:   tracked

    <BLANKLINE>

    Untracked files:

      (use "git add <file>..." to include in what will be committed)

    <BLANKLINE>

        untracked

    <BLANKLINE>

    no changes added to commit (use "git add" and/or "git commit -a")