Cross-project executables

[Alexey Palazhchenko]

Hello,

I'm trying to use CodeSpeed for two Python projects (X and Y), running benchmarks against two executables (CPython and PyPy). In other words, my executables are not results of project compilation, they should be cross-project. Looks like it's not possible. Am I correct?

Thanks,
Alexey.

[Miquel Torres]

Hi,

from what you say I would organize things like this:

- Project X:
* exeX (cpython)
* exeX (pypy)

- Project Y
* exeY (cpython)
* exeY (pypy)

Does that make sense?

2011/9/9 Alexey Palazhchenko <alexey.pa...@gmail.com>:

[Alexey Palazhchenko]

Hi,

> from what you say I would organize things like this:
>
> - Project X:
> * exeX (cpython)
> * exeX (pypy)
>
> - Project Y
> * exeY (cpython)
> * exeY (pypy)
>
> Does that make sense?

Yes, but it is ugly. I guess changing relationship between projects and executables to many-to-many should not break too much? Going to learn some Django...

–-–
Alexey "AlekSi" Palazhchenko

[Miquel Torres]

How exactly would you rather have it organized?

2011/9/10 Alexey Palazhchenko <alexey.pa...@gmail.com>:

[Alexey Palazhchenko]

Hi,

> How exactly would you rather have it organized?

I would like to have two executables and several projects, with many-to-many relationship between them:

class Executable(models.Model):
# project = models.ForeignKey(Project, related_name="executables")
projects = models.ManyToManyField(Project, related_name="executables")

–-–
Alexey "AlekSi" Palazhchenko

[Miquel Torres]

I think I did not properly clarify: I am asking to the user, not to
the programmer. What is your use case that you want to have an
executable associated with two projects? I don't quite understand
that.

An executable always belongs to *one* project. Always. Another thing
is that you execute it in different ways, under different conditions.
From what you say, you run your benchmarks once using cpython and once
using PyPy. That doesn't change the fact that you are testing the same
the same program. You are just changing how it is executed. It is
analogous to compiling a programm with gcc using different
optimization levels (O2 or O3 for example). And in that case you
wouldn't have two projects. You would have two differently compiled
executables associated to the same project.

Or am I missing something?

2011/9/10 Alexey Palazhchenko <alexey.pa...@gmail.com>:

[Alexey Palazhchenko]

Hi,

> An executable always belongs to *one* project. Always.

Consider CodeSpeed instance running benchmarks for Django, Twisted and other projects. Does CPython and PyPy belong to them? I don't think so - they are just installed. Projects are changing, not executables.

I have those Python interpreters, running benchmarks for several projects in Python. Projects are not related and do not have shared benchmarks. In this case relationships between projects and executables are very loose.

> - Project X:
> * exeX (cpython)
> * exeX (pypy)
>
> - Project Y
> * exeY (cpython)
> * exeY (pypy)

This will work for small number of projects only. This way I'm having N*2 options instead of just 2. Having cross-project executables makes UI and results sending easier.

Started to work on this feature there: https://github.com/AlekSi/codespeed/commits/cross-project-executables

–-–
Alexey "AlekSi" Palazhchenko

[Miquel Torres]

> Consider CodeSpeed instance running benchmarks for Django, Twisted and other projects.
> Does CPython and PyPy belong to them? I don't think so - they are just installed. Projects are changing, not executables.

In that case, you won't be benchmarking cpython or PyPy at all. You
are benchmarking the projects Django, Twisted, etc, which will have
one or more executables each. That you want to test those with
different runtime options and optimizations, doesn't mean that you now
want to test cpython or PyPy. They are just a means to benchmark
Django or twisted. Why would you associate Django with PyPy? They are
completely orthogonal to me. The Django project itself is not changing
just because you use pypy-c.

To put it another way. Imagine you have program "numbercruncher". You
compile and test with gcc and the intel compiler. Would you say that
it is associated with the gcc project? You wouldn't even have a gcc
project. You would rather have the "numbercruncher gcc" and
"numbercruncher intel-x64" executables or whatever.

> This will work for small number of projects only. This way I'm having N*2 options instead of just 2. Having cross-project executables makes UI and results sending easier.

I can see that it will get cluttered for lots of projects. How would
your cross-project exes make the UI better. Maybe I am failing to
visualize that.

2011/9/10 Alexey Palazhchenko <alexey.pa...@gmail.com>:

[Miquel Torres]

Thinking about this a bit more, I think for a site in the form "lots
of python projects which will get benchmarked by different
interpreters", a solution could be to use environments for that (which
you can also use for 32/64 bits instead of several exes).

Could you imagine that? that way you avoid the N*2 problem

2011/9/11 Miquel Torres <tob...@googlemail.com>: