The high memory cost of creating many Polyhedron objects

[Nathann Cohen]

Hello everybody,

I am trying to compute a couple of things on polyhedra, and for that I
need to generate and test a lot of them, each at a time. While I never
store them in any way my code stops after a while because of lack of
memory: would somebody know if this is caused by same reason as for
Posets ?

Have fun,

Nathann

[jplab]

Hi Nathann,

Have a look at the thread Memory leak in Polyhedron object.

Would that be the problem for you? In this case, the problem is the ticket #14356.

Best,
JP

[Nathann Cohen]

Hello,

Have a look at the thread Memory leak in Polyhedron object.

Would that be the problem for you? In this case, the problem is the ticket #14356.

I had not noticed that Polyhedron created a Poset, and if so that 'makes sense'. I had the same problem with both classes: it makes it very hard to create a lot of Polyhedron objects, or many posets (and this, in turns, makes it hard to write a routine that enumerates posets up to isomorphism) :-/

Nathann

[jplab]

Hi,

I had not noticed that Polyhedron created a Poset, and if so that 'makes sense'. I had the same problem with both classes: it makes it very hard to create a lot of Polyhedron objects, or many posets (and this, in turns, makes it hard to write a routine that enumerates posets up to isomorphism) :-/

Yes, this is really bad. At the time, I had to create loads of relatively small polyhedral cones and discard them but the computation was hitting the RAM limit way too fast.

Is the related poset problem really difficult to fix?

I believe that trying to have a workaround for polyhedron to avoid the usage of the face lattice is

1) a lot of work
2) not clear how fast it could be

Of course, once the face lattice is computed many combinatorial data is stored there, so it's logic to use this data structure. So a workaround is not really ideal anyway.

JP

[jplab]

One thing could be to see how much of the information of the face lattice is required for some specific methods...

Perhaps some of them could be implemented without using the face lattice in certain cases and be just as fast...

[Nils Bruin]

On Wednesday, April 29, 2015 at 1:11:22 AM UTC-7, jplab wrote:

Hi,

I had not noticed that Polyhedron created a Poset, and if so that 'makes sense'. I had the same problem with both classes: it makes it very hard to create a lot of Polyhedron objects, or many posets (and this, in turns, makes it hard to write a routine that enumerates posets up to isomorphism) :-/

Yes, this is really bad. At the time, I had to create loads of relatively small polyhedral cones and discard them but the computation was hitting the RAM limit way too fast.

Is the related poset problem really difficult to fix?

As is pointed out on #14356, the solution is to have a lighter weight PoSet. The analysis there indicates that the memory leaks happens via the UniqueRepresentation cache. Parents only need to be UniqueRepresentation if they need to participate seriously in the coercion framework, which I suspect isn't so interesting for Posets it general. So if this problem keeps creeping up, someone should bite the bullet and implement non-unique representation posets and use those (in nearly all situations?)
 

I believe that trying to have a workaround for polyhedron to avoid the usage of the face lattice is

1) a lot of work
2) not clear how fast it could be

Instantiating parents is inherently expensive. If you're concerned with speed and you make a lot of posets, make sure that the object representing those posets is lightweight, so not a "parent".

[Dima Pasechnik]

well, what kind of things are you computing that need the full face facet?
There are several extremely inefficient things about Polyhedron, e.g. things like
adjacency_matrix() surely do not need the full face lattice, etc...

Dima

 

Have fun,

Nathann