Coexistence of py2 and py3

[Simon King]

Hi!

At #28414, I'm trying to upgrade my group cohomology package so that it
works both on sage-with-py2 and sage-with-py3.

On the laptop where I work on the new package version (I didn't publish
the code yet), I have a py2 and a py3 installation of Sage. To build the
package without the hassle of recompiling everything, I build the
development version of my package by opening a Sage shell in the
package's src folder, and then run
sage -python setup.py build
sage -pip install . --upgrade

However, when I do the above in both versions of Sage (py2 and py3),
then in the py2 version strange things happen. Namely: In some Cython
modules of my package, one has str==unicode, which means that an
explicit cast to str results in a unicode, which results in errors when
passing it to functions from other modules expecting a str.

Dima said on the ticket that he met similar confusions about str and
unicode in the past, and he had the impression that it had to do with
both Sage versions sharing the same $DOT_SAGE folder. However, the
problem persists when I remove $DOT_SAGE.

Although `sage -python setup.py build` does create the .so files in
different folders for different python versions (namely
build/temp.linux-x86_64-2.7 and build/temp.linux-x86_64-3.7), it creates
the .c files in THE SAME directory (namely where it finds the corresponding
.pyx files).

My theory is that the confusion is caused by having the .c files in the
same directory. But a duckduckgo search didn't tell me how to prescribe
a different directory for the .c files (resp. the solutions I found,
such as "python setup.py build --build-lib <build directory>" didn't
seem to work).

Can you tell me how to prescribe a folder for the .c files?

Best regards,
Simon

[Dima Pasechnik]

Shouldn't coexisting installations like this have different --prefix ?

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qmd97s%2425hp%241%40blaine.gmane.org.

[Vincent Delecroix]

Are you sure that the problem comes from the C files? If you erase
them between the two installations what do you get?

[Simon King]

Hi Vincent,

On 2019-09-24, Vincent Delecroix <20100.d...@gmail.com> wrote:
> Are you sure that the problem comes from the C files? If you erase
> them between the two installations what do you get?

That's what I'm testing now.

BTW, meanwhile I found that I can prescribe in my setup.py where to
create the .c files (the question in my post was how to do it):
cythonize(...,
build_dir=os.path.join("build","c_files-{}.{}".format(version_info.major,
version_info.minor)))

With this in setup.py, the .c files will be in build/c_files-2.7 resp.
in build/c_files-3.7, whereas (as before) the .so files are in
build/lib.linux-x86_64-2.7 resp. ... -3.7.

I will see if this is enough to solve the problem.

Best regards,
Simon

[Simon King]

Hi Dima,

On 2019-09-24, Dima Pasechnik <dim...@gmail.com> wrote:
> Shouldn't coexisting installations like this have different --prefix ?

Of course. But if I understand correctly, this only affects where stuff
is installed, not where it is built. The situation is as follows:

- the .c files are created in the source directory, i.e., where the
corresponding .pyx files live. Therefore the .c files are only
re-generated when the .pyx file is touched, but not when the language
level changes.
- the .so files are created in build/lib.linux-x86_64-2.7 resp. -3.7,
hence, the paths do take into account the language level. This is of
course a good idea, but not enough (because of the .c files).
- eventually the .so files are installed in --prefix.

The .c file can very well depend on the language level. E.g., in the Sage
library, compile time variables are used to tell Cython what code to
use for py2 and what code for py3.

Hence, IMHO it is clearly a bug that the .c files are not by default
created in a path that depends on the language level.

But I don't know where that bug should be reported.

Best regards,
Simon

[Dima Pasechnik]

I'm surprised that *.c files created by Cython for pyX may be used for
py(5-X) without many problems.
I'd expect this to break down totally...

Dima

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qmdiei%2479ou%241%40blaine.gmane.org.

[Simon King]

Hi Dima,

On 2019-09-24, Dima Pasechnik <dim...@gmail.com> wrote:
> I'm surprised that *.c files created by Cython for pyX may be used for
> py(5-X) without many problems.
> I'd expect this to break down totally...

Indeed. It is surprising that I only got two kinds of spurious errors:
In one place, a pickle of a dictionary was unpickled apparently as a
string, and sometimes <str> was identical to <unicode>. But not much
else.

Anyway, first tests seem to indicate that the problems vanish when
changing setup.py so that creation of .c files takes place in different
folders for different python versions.

Best regards,
Simon