regression between 8.7 and 8.8.rc1 with operations on polygons with vertices in a quadratic field
152 views
Skip to first unread message
Sébastien Labbé
Jun 20, 2019, 10:59:01 AM6/20/19
to sage-devel
Dear sage-devel,
The proof of the main result of my last preprint [https://arxiv.org/abs/1906.01104] can be reproduced in Sage-8.7 + my optional slabbe package. The code is provided in appendix and is also available at https://github.com/seblabbe/slabbe/blob/develop/demos/arXiv_1906_01104.rst.
With sage-8.7 + slabbe-0.5.1, I can reproduce the computations and it takes less than 15 seconds:
$ ~/GitBox/sage-8.7/sage -t arXiv_1906_01104.rst
Using --optional=dochtml,memlimit,mpir,python2,sageDoctesting 1 file.
sage -t arXiv_1906_01104.rst
[41 tests, 14.66 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 14.7 seconds
cpu time: 14.7 seconds
cumulative wall time: 14.7 seconds
With sage-8.8.rc1 + slabbe-0.5.1, the first operations takes much longer and eventually dies with
sage -t arXiv_1906_01104.rst # Timed out
----------------------------------------------------------------------
Total time for all tests: 304.9 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds
----------------------------------------------------------------------
Total time for all tests: 304.9 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds
The operations that I perform are essentially translations and intersections of polygons with vertices in the Number Field in phi with defining polynomial z^2 - z - 1. I am using the default backend 'field'. I know that some progress has been done with respect to polyhedron recently. Therefore, I was expecting to gain some efficiency by defining explicitly the polyhedron backend (like normaliz), but I was not expecting such a regression for the default with no backend provided.
I also noticed the following modification for the base ring (nb field *with* embedding) I am using (see below). Can this be an explanation for operations to be slower?
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.7, Release Date: 2019-03-23 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
sage: z = polygen(QQ, 'z')
sage: K = NumberField(z**2-z-1, 'phi', embedding=RR(1.6))
sage: K
Number Field in phi with defining polynomial z^2 - z - 1
│ SageMath version 8.7, Release Date: 2019-03-23 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
sage: z = polygen(QQ, 'z')
sage: K = NumberField(z**2-z-1, 'phi', embedding=RR(1.6))
sage: K
Number Field in phi with defining polynomial z^2 - z - 1
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.8.rc1, Release Date: 2019-06-13 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
sage: z = polygen(QQ, 'z')
sage: K = NumberField(z**2-z-1, 'phi', embedding=RR(1.6))
sage: K
Number Field in phi with defining polynomial z^2 - z - 1 with phi = 1.618033988749895?
│ SageMath version 8.8.rc1, Release Date: 2019-06-13 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
sage: z = polygen(QQ, 'z')
sage: K = NumberField(z**2-z-1, 'phi', embedding=RR(1.6))
sage: K
Number Field in phi with defining polynomial z^2 - z - 1 with phi = 1.618033988749895?
Thanks for any pointers,
Sébastien
Vincent Delecroix
Jun 20, 2019, 11:13:57 AM6/20/19
to sage-...@googlegroups.com
Salut Sébastien,
- Is it with or without PyNormaliz installed?
- You should run a profiler on both versions and give a more complete
report than a timing difference.
Best
Vincent
- Is it with or without PyNormaliz installed?
- You should run a profiler on both versions and give a more complete
report than a timing difference.
Best
Vincent
Frédéric Chapoton
Jun 20, 2019, 11:16:14 AM6/20/19
to sage-devel
This could come from
Sébastien Labbé
Jun 20, 2019, 11:56:20 AM6/20/19
to sage-devel
My current sage-8.8.rc1 has normaliz installed:
slabbe@miami ~ $ sage -optional | grep normaliz
normaliz................................3.7.2 (3.7.2)
pynormaliz..............................2.7 (2.7)
normaliz................................3.7.2 (3.7.2)
pynormaliz..............................2.7 (2.7)
and my current sage-8.7 has no optional packages installed:
slabbe@miami ~ $ GitBox/sage-8.7/sage -optional | grep normaliz
normaliz................................3.6.3 (not_installed)
pynormaliz..............................1.19 (not_installed)
normaliz................................3.6.3 (not_installed)
pynormaliz..............................1.19 (not_installed)
If I do %prun on the first command which takes much longer (90s instead of 5s), I get on both systems, the following:
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.7, Release Date: 2019-03-23 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
sage: from slabbe import PolyhedronPartition
sage: from slabbe import PolyhedronExchangeTransformation as PET
sage: from slabbe import Substitution2d
sage: from slabbe import PolyhedronExchangeTransformation as PET
sage: from slabbe import Substitution2d
sage: z = polygen(QQ, 'z')
sage: K = NumberField(z**2-z-1, 'phi', embedding=RR(1.6))
sage: phi = K.gen()
sage: P0 = PolyhedronPartition.jeandel_rao_tilings_partition()
sage: Gamma0 = matrix.column([(phi,0), (1,phi+3)])
sage: fundamental_domain = polytopes.parallelotope([(phi,0), (0,phi+3)])
sage: R0e2 = PET.toral_translation(Gamma0, vector((0,1)), fundamental_domain)
sage: y_le_1 = [1, 0, -1] # syntax for the inequality y <= 1
sage:
sage: %time P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
CPU times: user 4.6 s, sys: 84 ms, total: 4.68 s
Wall time: 4.64 s
sage: %prun P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
sage: P0 = PolyhedronPartition.jeandel_rao_tilings_partition()
sage: Gamma0 = matrix.column([(phi,0), (1,phi+3)])
sage: fundamental_domain = polytopes.parallelotope([(phi,0), (0,phi+3)])
sage: R0e2 = PET.toral_translation(Gamma0, vector((0,1)), fundamental_domain)
sage: y_le_1 = [1, 0, -1] # syntax for the inequality y <= 1
sage:
sage: %time P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
CPU times: user 4.6 s, sys: 84 ms, total: 4.68 s
Wall time: 4.64 s
sage: %prun P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
3059637 function calls (3058959 primitive calls) in 5.373 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
4878 0.619 0.000 1.713 0.000 {method 'right_kernel' of 'sage.matrix.matrix2.Matrix' objects}
4184 0.392 0.000 0.489 0.000 {method 'solve_right' of 'sage.matrix.matrix2.Matrix' objects}
43084 0.253 0.000 0.280 0.000 matrix_space.py:720(__call__)
9756 0.209 0.000 0.294 0.000 number_field.py:3081(_repr_)
30698 0.191 0.000 0.253 0.000 matrix_space.py:103(get_matrix_class)
4184 0.166 0.000 0.182 0.000 {method 'pivots' of 'sage.matrix.matrix0.Matrix' objects}
4184 0.166 0.000 0.205 0.000 {method 'pivot_rows' of 'sage.matrix.matrix2.Matrix' objects}
5243 0.152 0.000 0.171 0.000 {method 'rank' of 'sage.matrix.matrix0.Matrix' objects}
5005 0.146 0.000 0.325 0.000 free_module.py:662(__init__)
54383 0.140 0.000 0.200 0.000 free_module.py:973(_element_constructor_)
14168 0.138 0.000 0.197 0.000 sequence.py:80(Sequence)
435687 0.122 0.000 0.122 0.000 {isinstance}
35194 0.115 0.000 0.127 0.000 representation.py:1181(evaluated_on)
30698 0.113 0.000 0.367 0.000 matrix_space.py:432(__classcall__)
4184 0.106 0.000 1.284 0.000 double_description.py:641(initial_pair)
54383 0.102 0.000 0.304 0.000 free_module.py:5569(_element_constructor_)
9637 0.101 0.000 0.312 0.000 {sage.modules.free_module_element.vector}
8368 0.080 0.000 0.169 0.000 {method 'rows' of 'sage.matrix.matrix1.Matrix' objects}
41111 0.072 0.000 0.149 0.000 misc.py:295(cputime)
10231 0.070 0.000 0.072 0.000 double_description.py:313(R_by_sign)
35194 0.066 0.000 0.247 0.000 representation.py:694(contains)
41111 0.057 0.000 0.057 0.000 {resource.getrusage}
52510 0.051 0.000 0.051 0.000 free_module.py:328(create_key)
4593 0.050 0.000 0.112 0.000 {method 'columns' of 'sage.matrix.matrix1.Matrix' objects}
4878 0.045 0.000 0.156 0.000 free_module.py:1593(basis_matrix)
2092 0.043 0.000 0.408 0.000 double_description_inhomogeneous.py:273(_extract_Vrep)
4878 0.042 0.000 0.443 0.000 free_module.py:5643(__init__)
19512 0.038 0.000 0.038 0.000 {method 'sub' of '_sre.SRE_Pattern' objects}
14168 0.036 0.000 0.039 0.000 sequence.py:405(__init__)
2965 0.036 0.000 0.057 0.000 representation.py:330(_set_data)
2092 0.036 0.000 2.732 0.001 double_description_inhomogeneous.py:154(__init__)
43834 0.032 0.000 0.281 0.000 base.py:587(<genexpr>)
1882 0.031 0.000 0.063 0.000 number_field.py:1554(_element_constructor_)
5005 0.030 0.000 0.042 0.000 free_module.py:7218(element_class)
6983 0.028 0.000 0.030 0.000 {method 'inner_product' of 'sage.modules.free_module_element.FreeModuleElement' objects}
35194 0.028 0.000 0.028 0.000 backend_field.py:92(_is_nonneg)
2965 0.028 0.000 0.084 0.000 representation.py:311(__init__)
10231 0.024 0.000 0.193 0.000 double_description.py:689(add_inequality)
41111 0.023 0.000 0.173 0.000 misc.py:525(verbose)
19512 0.023 0.000 0.023 0.000 re.py:230(_compile)
1882 0.023 0.000 0.025 0.000 polynomial_ring.py:320(_element_constructor_)
5010 0.022 0.000 0.085 0.000 modules.py:112(__classcall_private__)
347 0.022 0.000 0.328 0.001 point_configuration.py:1827(contained_simplex)
31 0.021 0.001 4.147 0.134 polyhedron_partition.py:1204(refinement)
47132 0.020 0.000 0.020 0.000 number_field.py:3792(pari_polynomial)
1745 0.019 0.000 3.565 0.002 backend_field.py:214(_init_from_Hrepresentation)
6276 0.019 0.000 0.020 0.000 double_description.py:112(__init__)
409 0.018 0.000 0.027 0.000 {method 'inverse' of 'sage.matrix.matrix2.Matrix' objects}
7979 0.018 0.000 0.037 0.000 fields.py:63(__contains__)
2092 0.018 0.000 0.747 0.000 double_description_inhomogeneous.py:451(_init_Vrep)
93 0.018 0.000 0.041 0.000 point_configuration.py:332(__init__)
4878 0.017 0.000 0.481 0.000 free_module.py:3794(span)
5629 0.017 0.000 0.017 0.000 {hasattr}
9675 0.017 0.000 0.018 0.000 free_module.py:2371(zero_vector)
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.8.rc1, Release Date: 2019-06-13 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
sage: from slabbe import PolyhedronPartition
sage: from slabbe import PolyhedronExchangeTransformation as PET
sage: from slabbe import Substitution2d
sage: from slabbe import PolyhedronExchangeTransformation as PET
sage: from slabbe import Substitution2d
sage: z = polygen(QQ, 'z')
sage: K = NumberField(z**2-z-1, 'phi', embedding=RR(1.6))
sage: phi = K.gen()
sage: P0 = PolyhedronPartition.jeandel_rao_tilings_partition()
sage: Gamma0 = matrix.column([(phi,0), (1,phi+3)])
sage: fundamental_domain = polytopes.parallelotope([(phi,0), (0,phi+3)])
sage: R0e2 = PET.toral_translation(Gamma0, vector((0,1)), fundamental_domain)
sage: y_le_1 = [1, 0, -1] # syntax for the inequality y <= 1
sage:
sage: %time P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
CPU times: user 12.9 s, sys: 2.62 s, total: 15.6 s
Wall time: 1min 27s
sage: %prun P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
sage: P0 = PolyhedronPartition.jeandel_rao_tilings_partition()
sage: Gamma0 = matrix.column([(phi,0), (1,phi+3)])
sage: fundamental_domain = polytopes.parallelotope([(phi,0), (0,phi+3)])
sage: R0e2 = PET.toral_translation(Gamma0, vector((0,1)), fundamental_domain)
sage: y_le_1 = [1, 0, -1] # syntax for the inequality y <= 1
sage:
sage: %time P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
CPU times: user 12.9 s, sys: 2.62 s, total: 15.6 s
Wall time: 1min 27s
sage: %prun P1,beta0 = R0e2.induced_partition(y_le_1, P0, substitution_type='column')
4286879 function calls (4284765 primitive calls) in 89.043 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1388 69.712 0.050 69.712 0.050 {time.sleep}
1041 1.724 0.002 1.724 0.002 {posix.read}
4878 1.427 0.000 7.762 0.002 {method 'right_kernel' of 'sage.matrix.matrix2.Matrix' objects}
347 1.270 0.004 1.270 0.004 {posix.forkpty}
9756 0.998 0.000 2.509 0.000 number_field.py:2955(gen_embedding)
19512 0.998 0.000 1.740 0.000 {method 'format' of 'str' objects}
4184 0.671 0.000 0.845 0.000 {method 'solve_right' of 'sage.matrix.matrix2.Matrix' objects}
43035 0.554 0.000 0.631 0.000 matrix_space.py:720(__call__)
111653 0.521 0.000 0.906 0.000 copy.py:66(copy)
1388 0.447 0.000 0.447 0.000 {select.select}
30669 0.424 0.000 0.551 0.000 matrix_space.py:103(get_matrix_class)
4951 0.387 0.000 0.835 0.000 free_module.py:662(__init__)
4184 0.320 0.000 0.390 0.000 {method 'pivot_rows' of 'sage.matrix.matrix2.Matrix' objects}
4184 0.312 0.000 0.345 0.000 {method 'pivots' of 'sage.matrix.matrix0.Matrix' objects}
5243 0.302 0.000 0.334 0.000 {method 'rank' of 'sage.matrix.matrix0.Matrix' objects}
530292 0.279 0.000 0.279 0.000 {isinstance}
54201 0.270 0.000 0.373 0.000 free_module.py:973(_element_constructor_)
14168 0.258 0.000 0.365 0.000 sequence.py:80(Sequence)
30669 0.250 0.000 0.801 0.000 matrix_space.py:432(__classcall__)
13844 0.223 0.000 0.231 0.000 polynomial_ring.py:320(_element_constructor_)
19512 0.214 0.000 0.436 0.000 qqbar.py:4023(interval)
11962 0.212 0.000 0.617 0.000 rings.py:845(__getitem__)
11962 0.195 0.000 0.338 0.000 polynomial_ring_constructor.py:52(PolynomialRing)
4184 0.193 0.000 2.282 0.001 double_description.py:640(initial_pair)
54201 0.186 0.000 0.562 0.000 free_module.py:5584(_element_constructor_)
9637 0.178 0.000 0.520 0.000 {sage.modules.free_module_element.vector}
39121 0.172 0.000 0.359 0.000 homset.py:84(Hom)
41104 0.147 0.000 0.303 0.000 misc.py:295(cputime)
347 0.142 0.000 3.001 0.009 point_configuration.py:1829(contained_simplex)
8368 0.140 0.000 0.308 0.000 {method 'rows' of 'sage.matrix.matrix1.Matrix' objects}
48780 0.137 0.000 0.137 0.000 sets_cat.py:974(_element_constructor_from_element_class)
35194 0.130 0.000 0.218 0.000 representation.py:1200(evaluated_on)
10231 0.130 0.000 0.133 0.000 double_description.py:313(R_by_sign)
41104 0.110 0.000 0.110 0.000 {resource.getrusage}
1882 0.108 0.000 0.210 0.000 number_field.py:1554(_element_constructor_)
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1388 69.712 0.050 69.712 0.050 {time.sleep}
1041 1.724 0.002 1.724 0.002 {posix.read}
4878 1.427 0.000 7.762 0.002 {method 'right_kernel' of 'sage.matrix.matrix2.Matrix' objects}
347 1.270 0.004 1.270 0.004 {posix.forkpty}
9756 0.998 0.000 2.509 0.000 number_field.py:2955(gen_embedding)
19512 0.998 0.000 1.740 0.000 {method 'format' of 'str' objects}
4184 0.671 0.000 0.845 0.000 {method 'solve_right' of 'sage.matrix.matrix2.Matrix' objects}
43035 0.554 0.000 0.631 0.000 matrix_space.py:720(__call__)
111653 0.521 0.000 0.906 0.000 copy.py:66(copy)
1388 0.447 0.000 0.447 0.000 {select.select}
30669 0.424 0.000 0.551 0.000 matrix_space.py:103(get_matrix_class)
4951 0.387 0.000 0.835 0.000 free_module.py:662(__init__)
4184 0.320 0.000 0.390 0.000 {method 'pivot_rows' of 'sage.matrix.matrix2.Matrix' objects}
4184 0.312 0.000 0.345 0.000 {method 'pivots' of 'sage.matrix.matrix0.Matrix' objects}
5243 0.302 0.000 0.334 0.000 {method 'rank' of 'sage.matrix.matrix0.Matrix' objects}
530292 0.279 0.000 0.279 0.000 {isinstance}
54201 0.270 0.000 0.373 0.000 free_module.py:973(_element_constructor_)
14168 0.258 0.000 0.365 0.000 sequence.py:80(Sequence)
30669 0.250 0.000 0.801 0.000 matrix_space.py:432(__classcall__)
13844 0.223 0.000 0.231 0.000 polynomial_ring.py:320(_element_constructor_)
19512 0.214 0.000 0.436 0.000 qqbar.py:4023(interval)
11962 0.212 0.000 0.617 0.000 rings.py:845(__getitem__)
11962 0.195 0.000 0.338 0.000 polynomial_ring_constructor.py:52(PolynomialRing)
4184 0.193 0.000 2.282 0.001 double_description.py:640(initial_pair)
54201 0.186 0.000 0.562 0.000 free_module.py:5584(_element_constructor_)
9637 0.178 0.000 0.520 0.000 {sage.modules.free_module_element.vector}
39121 0.172 0.000 0.359 0.000 homset.py:84(Hom)
41104 0.147 0.000 0.303 0.000 misc.py:295(cputime)
347 0.142 0.000 3.001 0.009 point_configuration.py:1829(contained_simplex)
8368 0.140 0.000 0.308 0.000 {method 'rows' of 'sage.matrix.matrix1.Matrix' objects}
48780 0.137 0.000 0.137 0.000 sets_cat.py:974(_element_constructor_from_element_class)
35194 0.130 0.000 0.218 0.000 representation.py:1200(evaluated_on)
10231 0.130 0.000 0.133 0.000 double_description.py:313(R_by_sign)
41104 0.110 0.000 0.110 0.000 {resource.getrusage}
1882 0.108 0.000 0.210 0.000 number_field.py:1554(_element_constructor_)
I do not know where these sleep calls come from??
Dima Pasechnik
Jun 20, 2019, 12:38:07 PM6/20/19
to sage-devel
some other way.
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/618fa213-bef5-4b57-88e3-8a79c784d090%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Dima Pasechnik
Jun 20, 2019, 12:54:15 PM6/20/19
to sage-devel
Anyhow, I don't get a timeout on sage -t arXiv_1906_01104.rst (with
basically sage 8.8.rc2), it gets done in 27 sec on my
medium-speed-range Gentoo linux laptop.
basically sage 8.8.rc2), it gets done in 27 sec on my
medium-speed-range Gentoo linux laptop.
Dima Pasechnik
Jun 20, 2019, 1:05:38 PM6/20/19
to sage-devel
And if I try your %prun test, I don't see any "sleep" or "posix.*"
stuff in the output.
I guess your 8.8.rc1 installation is a bit broken.
stuff in the output.
I guess your 8.8.rc1 installation is a bit broken.
Matthias Koeppe
Jun 20, 2019, 1:15:40 PM6/20/19
to sage-devel
Just a quick note: The 'normaliz' backend is currently not used by default. It needs to be requested explicitly using backend='normaliz'.
Sébastien Labbé
Jun 20, 2019, 2:01:00 PM6/20/19
to sage-devel
On Thursday, June 20, 2019 at 7:05:38 PM UTC+2, Dima Pasechnik wrote:
And if I try your %prun test, I don't see any "sleep" or "posix.*"
stuff in the output.
I guess your 8.8.rc1 installation is a bit broken.
interesting. It may be because of optional packages also.
do you have optional packages installed on your 8.8.rc2?
Dima Pasechnik
Jun 20, 2019, 2:18:59 PM6/20/19
to sage-devel
This is what I have, and no external pip modules, IIRC.
It might be that you have installed some stuff via pip
that makes this test fail...
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
Sébastien Labbé
Jun 20, 2019, 2:21:55 PM6/20/19
to sage-devel
On another computer running 8.8.beta6, I get no problem.
$ sage --version
SageMath version 8.8.beta6, Release Date: 2019-05-22 $ sage -t arXiv_1906_01104.rst
Running doctests with ID 2019-06-20-20-18-43-d5a5294b.Git branch: 26552
Using --optional=build,cbc,ccache,cryptominisat,dochtml,dot2tex,e_antic,glucose,memlimit,normaliz,notedown,openssl,pandoc_attributes,pycosat,python2,rst2ipynb,sage
Doctesting 1 file.
sage -t --warn-long 70.5 arXiv_1906_01104.rst
[41 tests, 15.12 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Will try with a fresh installation of 8.8.rc2 on both of my computer now.
Sébastien Labbé
Jun 20, 2019, 2:26:21 PM6/20/19
to sage-devel
>
> do you have optional packages installed on your 8.8.rc2?
Using -- optional=bliss,build,dochtml,e_antic,gap_packages,igraph,jupymake,lidia,lrslib,memlimit,normaliz,python2,python_igraph,sage
This is what I have, and no external pip modules, IIRC.
It might be that you have installed some stuff via pip
that makes this test fail...
As a comparison here are the optional packages I have on my "broken" 8.8.rc1 :
Using --optional=4ti2,awali,bliss,build,cbc,ccache,cmake,cryptominisat,dochtml,dot2tex,e_antic,glucose,latte_int,lidia,lrslib,memlimit,mpir,ninja_build,normaliz,
notedown,openssl,pandoc_attributes,pycosat,pynormaliz,python2,qhull,rst2ipynb,sage,topcom
Sébastien Labbé
Jun 20, 2019, 3:02:51 PM6/20/19
to sage-devel
On the computer with broken 8.8.rc1, with a fresh 8.8.rc2 installation with no optional package at all, it takes 5 seconds more than with sage-8.7 but it works:
$ ~/GitBox/sage-8.7/sage -t arXiv_1906_01104.rst
too many failed tests, not using stored timingsRunning doctests with ID 2019-06-20-20-55-07-02c9a7a7.
Git branch: develop
Using --optional=dochtml,memlimit,mpir,python2,sage
Doctesting 1 file.
sage -t arXiv_1906_01104.rst
[41 tests, 14.82 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 14.9 seconds
cpu time: 14.8 seconds
cumulative wall time: 14.8 seconds
cpu time: 14.8 seconds
cumulative wall time: 14.8 seconds
$ ~/GitBox/sage-8.8.rc2/sage -t arXiv_1906_01104.rst too many failed tests, not using stored timings
Running doctests with ID 2019-06-20-20-54-08-4be2ab14.
Git branch: develop
Using --optional=build,dochtml,memlimit,mpir,python2,sage
Running doctests with ID 2019-06-20-20-54-08-4be2ab14.
Git branch: develop
Using --optional=build,dochtml,memlimit,mpir,python2,sage
Doctesting 1 file.
sage -t arXiv_1906_01104.rst
[41 tests, 20.17 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 20.2 seconds
cpu time: 20.2 seconds
cumulative wall time: 20.2 seconds
cpu time: 20.2 seconds
cumulative wall time: 20.2 seconds
Running the %prun on the same command as above with my fresh 8.8.rc2 installation, I do not get any sleep calls. So, I still do not understand what is happening with my broken 8.8.rc1.
Sébastien Labbé
Jun 20, 2019, 4:29:05 PM6/20/19
to sage-devel
I am now able to reproduce the problem. It seems to be related with the presence of the topcom optional package. I am able to reproduce the issue with sage-8.7 + topcom + slabbe-0.5.1 and sage-8.8.rc2 + topcom + slabbe-0.5.1 :
$ ./sage -version
SageMath version 8.7, Release Date: 2019-03-23
$ ./sage -t ~/arXiv_1906_01104.rst
Using --optional=cmake,dochtml,memlimit,mpir,python2,qhull,sage,topcom
Doctesting 1 file.
Doctesting 1 file.
....
----------------------------------------------------------------------
sage -t /home/slabbe/arXiv_1906_01104.rst # Timed out
----------------------------------------------------------------------
sage -t /home/slabbe/arXiv_1906_01104.rst # Timed out
----------------------------------------------------------------------
Total time for all tests: 304.2 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds
Do you know why and where topcom becomes automatically involved in my computations?
Is there an easy command to uninstall topcom?
Dima Pasechnik
Jun 20, 2019, 4:41:14 PM6/20/19
to sage-...@googlegroups.com
make topcom-clean
should work...
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/b8dcf2ed-f542-4844-8312-bb53928119a6%40googlegroups.com.
Sébastien Labbé
Jun 20, 2019, 4:53:51 PM6/20/19
to sage-devel
Is there an easy command to uninstall topcom?make topcom-cleanshould work...
it seems to uninstall properly (topcom does not appear in the list of optional packages), but the issue persists after uninstallation
Dima Pasechnik
Jun 20, 2019, 5:34:04 PM6/20/19
to sage-...@googlegroups.com
perhaps a dodgy pip-installed module, installed as a dependency, or some other thing that turned that weird multiprocessing (mostly sleeping though) thing on.
I would also check if there is not something in ~/.sage/ that causes this issue.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/908c6a80-62b6-4f47-8e5b-9e12d0158335%40googlegroups.com.
Sébastien Labbé
Jun 21, 2019, 4:32:47 AM6/21/19
to sage-devel
it seems to uninstall properly (topcom does not appear in the list of optional packages), but the issue persists after uninstallationperhaps a dodgy pip-installed module, installed as a dependency, or some other thing that turned that weird multiprocessing (mostly sleeping though) thing on.
On a fresh sage-8.8.rc2 clone, the following reproduces the problem without any dodgy pip-installed module:
sage -i topcom
sage -t arXiv_1906_01104.rst
topcom has only one dependency being cddlib which is a standard package. So the only other thing that turns that weird multiprocessing thing has to be caused by the installation of TOPCOM itself.
I would also check if there is not something in ~/.sage/ that causes this issue.
I am able to reproduce the issue even after doing:
mv .sage .sage-moved
So I do not think something in .sage causes the issue.
Sébastien
Sébastien Labbé
Jun 21, 2019, 4:36:38 AM6/21/19
to sage-devel
perhaps a dodgy pip-installed module, installed as a dependency, or some other thing that turned that weird multiprocessing (mostly sleeping though) thing on.On a fresh sage-8.8.rc2 clone, the following reproduces the problem without any dodgy pip-installed module:sage -i topcomsage -t arXiv_1906_01104.rst
oups, you are right, I forgot there is one dodgy pip module involved to reproduce the issue:
sage -pip install slabbe
Sébastien Labbé
Jun 21, 2019, 5:39:11 AM6/21/19
to sage-devel
Ok so here is an easy way to reproduce the sleep problem:
With topcom installed, I get a bunch of sleep calls when I compute the volume of a triangle:
sage: p = Polyhedron([(0,1/3),(0,1),(1/3,1)])
sage: %prun p.volume()
sage: %prun p.volume()
750 function calls (749 primitive calls) in 0.216 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
5 0.201 0.040 0.201 0.040 {time.sleep}
3 0.006 0.002 0.006 0.002 {posix.read}
1 0.004 0.004 0.004 0.004 {posix.forkpty}
4 0.001 0.000 0.001 0.000 {select.select}
1 0.000 0.000 0.001 0.001 point_configuration.py:1291(volume)
2 0.000 0.000 0.215 0.107 point_configuration.py:671(_TOPCOM_communicate)
1 0.000 0.000 0.216 0.216 base.py:5201(volume)
21 0.000 0.000 0.000 0.000 {posix.lstat}
1 0.000 0.000 0.010 0.010 ptyprocess.py:172(spawn)
1 0.000 0.000 0.000 0.000 matrix_space.py:720(__call__)
1 0.000 0.000 0.000 0.000 ptyprocess.py:152(__init__)
1 0.000 0.000 0.000 0.000 matrix_space.py:432(__classcall__)
2 0.000 0.000 0.214 0.107 point_configuration.py:597(_TOPCOM_exec)
1 0.000 0.000 0.216 0.216 <string>:1(<module>)
6 0.000 0.000 0.000 0.000 expect.py:13(new_data)
4 0.000 0.000 0.002 0.001 expect.py:91(expect_loop)
4 0.000 0.000 0.000 0.000 contextlib.py:21(__exit__)
1 0.000 0.000 0.000 0.000 {method 'close' of '_io.FileIO' objects}
4 0.000 0.000 0.000 0.000 ptyprocess.py:670(isalive)
1 0.000 0.000 0.000 0.000 matrix_space.py:103(get_matrix_class)
4 0.000 0.000 0.000 0.000 spawnbase.py:192(compile_pattern_list)
2 0.000 0.000 0.101 0.050 pty_spawn.py:526(send)
21 0.000 0.000 0.000 0.000 posixpath.py:132(islink)
3 0.000 0.000 0.000 0.000 base.py:2062(dim)
3 0.006 0.002 0.006 0.002 {posix.read}
1 0.004 0.004 0.004 0.004 {posix.forkpty}
4 0.001 0.000 0.001 0.000 {select.select}
1 0.000 0.000 0.001 0.001 point_configuration.py:1291(volume)
2 0.000 0.000 0.215 0.107 point_configuration.py:671(_TOPCOM_communicate)
1 0.000 0.000 0.216 0.216 base.py:5201(volume)
21 0.000 0.000 0.000 0.000 {posix.lstat}
1 0.000 0.000 0.010 0.010 ptyprocess.py:172(spawn)
1 0.000 0.000 0.000 0.000 matrix_space.py:720(__call__)
1 0.000 0.000 0.000 0.000 ptyprocess.py:152(__init__)
1 0.000 0.000 0.000 0.000 matrix_space.py:432(__classcall__)
2 0.000 0.000 0.214 0.107 point_configuration.py:597(_TOPCOM_exec)
1 0.000 0.000 0.216 0.216 <string>:1(<module>)
6 0.000 0.000 0.000 0.000 expect.py:13(new_data)
4 0.000 0.000 0.002 0.001 expect.py:91(expect_loop)
4 0.000 0.000 0.000 0.000 contextlib.py:21(__exit__)
1 0.000 0.000 0.000 0.000 {method 'close' of '_io.FileIO' objects}
4 0.000 0.000 0.000 0.000 ptyprocess.py:670(isalive)
1 0.000 0.000 0.000 0.000 matrix_space.py:103(get_matrix_class)
4 0.000 0.000 0.000 0.000 spawnbase.py:192(compile_pattern_list)
2 0.000 0.000 0.101 0.050 pty_spawn.py:526(send)
21 0.000 0.000 0.000 0.000 posixpath.py:132(islink)
3 0.000 0.000 0.000 0.000 base.py:2062(dim)
Dima Pasechnik
Jun 21, 2019, 6:11:51 AM6/21/19
to sage-devel
On Fri, Jun 21, 2019 at 10:39 AM Sébastien Labbé <sla...@gmail.com> wrote:
>
> Ok so here is an easy way to reproduce the sleep problem:
>
> With topcom installed, I get a bunch of sleep calls when I compute the volume of a triangle:
>
> sage: p = Polyhedron([(0,1/3),(0,1),(1/3,1)])
> sage: %prun p.volume()
>
> 750 function calls (749 primitive calls) in 0.216 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 5 0.201 0.040 0.201 0.040 {time.sleep}
> 3 0.006 0.002 0.006 0.002 {posix.read}
yes, I can reproduce this.
>
> Ok so here is an easy way to reproduce the sleep problem:
>
> With topcom installed, I get a bunch of sleep calls when I compute the volume of a triangle:
>
> sage: p = Polyhedron([(0,1/3),(0,1),(1/3,1)])
> sage: %prun p.volume()
>
> 750 function calls (749 primitive calls) in 0.216 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 5 0.201 0.040 0.201 0.040 {time.sleep}
> 3 0.006 0.002 0.006 0.002 {posix.read}
I think it's stuff in src/sage/geometry/triangulation/point_configuration.py
that plays fast and loose with "engine" parameter.
Basically, this is a great illustration that pexpect interface (what's
topcom using)
is slow...
Let us fix this https://trac.sagemath.org/ticket/28037
> 1 0.004 0.004 0.004 0.004 {posix.forkpty}
> 4 0.001 0.000 0.001 0.000 {select.select}
> 1 0.000 0.000 0.001 0.001 point_configuration.py:1291(volume)
> 2 0.000 0.000 0.215 0.107 point_configuration.py:671(_TOPCOM_communicate)
> 1 0.000 0.000 0.216 0.216 base.py:5201(volume)
> 21 0.000 0.000 0.000 0.000 {posix.lstat}
> 1 0.000 0.000 0.010 0.010 ptyprocess.py:172(spawn)
> 1 0.000 0.000 0.000 0.000 matrix_space.py:720(__call__)
> 1 0.000 0.000 0.000 0.000 ptyprocess.py:152(__init__)
> 1 0.000 0.000 0.000 0.000 matrix_space.py:432(__classcall__)
> 2 0.000 0.000 0.214 0.107 point_configuration.py:597(_TOPCOM_exec)
> 1 0.000 0.000 0.216 0.216 <string>:1(<module>)
> 6 0.000 0.000 0.000 0.000 expect.py:13(new_data)
> 4 0.000 0.000 0.002 0.001 expect.py:91(expect_loop)
> 4 0.000 0.000 0.000 0.000 contextlib.py:21(__exit__)
> 1 0.000 0.000 0.000 0.000 {method 'close' of '_io.FileIO' objects}
> 4 0.000 0.000 0.000 0.000 ptyprocess.py:670(isalive)
> 1 0.000 0.000 0.000 0.000 matrix_space.py:103(get_matrix_class)
> 4 0.000 0.000 0.000 0.000 spawnbase.py:192(compile_pattern_list)
> 2 0.000 0.000 0.101 0.050 pty_spawn.py:526(send)
> 21 0.000 0.000 0.000 0.000 posixpath.py:132(islink)
> 3 0.000 0.000 0.000 0.000 base.py:2062(dim)
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/2b4c27bd-ee66-4814-8502-bcf9889dd888%40googlegroups.com.
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
Dima Pasechnik
Jun 21, 2019, 6:59:18 AM6/21/19
to sage-devel
(2nd part of the description). One must remove executables.
(all these $SAGE_LOCAL/bin/point2*)
if I remove them manually, everything gets back to normal.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
Sébastien Labbé
Jun 21, 2019, 7:14:04 AM6/21/19
to sage-devel
no, it does not, please see https://trac.sagemath.org/ticket/28037
(2nd part of the description). One must remove executables.
(all these $SAGE_LOCAL/bin/point2*)
if I remove them manually, everything gets back to normal.
Thanks for creating the ticket and for that clean trick.
Simon King
Jun 21, 2019, 8:46:52 AM6/21/19
to sage-...@googlegroups.com
Hi Dima,
On 2019-06-21, Dima Pasechnik <dim...@gmail.com> wrote:
> Basically, this is a great illustration that pexpect interface (what's
> topcom using)
> is slow...
Question: Is that slowness OS dependent? I recall that when I was using
openSuse on my laptop, a couple of years ago, I needed to patch the OS
because otherwise the pexpect interface became extremely slow.
Best regards,
Simon
On 2019-06-21, Dima Pasechnik <dim...@gmail.com> wrote:
> Basically, this is a great illustration that pexpect interface (what's
> topcom using)
> is slow...
openSuse on my laptop, a couple of years ago, I needed to patch the OS
because otherwise the pexpect interface became extremely slow.
Best regards,
Simon
Dima Pasechnik
Jun 22, 2019, 4:15:00 AM6/22/19
to sage-devel
I don't know, it might be. I tested this on Gentoo Linux. Obviously we cannot rely on Sage users patching their kernels.
Dima
Best regards,
Simon
Reply all
Reply to author
Forward
0 new messages