Reaching stack limit: what solutions ?

[Kenneth V]

Hello,

Since it's my first post here: thanks for this nice piece of software ! Really helpful.

I ran into the problem of Eigen failing on OBJECT_ALLOCATED_ON_STACK_IS_TOO_BIG at compile time.
Responsible for that is that I'm setting up an AutoDiffCostFunction of P parameters and R residuals with P=131K and R=4K.
I suppose Ceres allocates things on the stack in proportion to P and that Eigen complains about that.

I see three possibilities: increase the stack, use the heap, or do static memory allocation:
1) Increase the hard stack limit Eigen complains about, but I suppose that won't scale very well (I'm planning to increase P).

2) Have Ceres allocate on the heap instead of the stack. The latter can be achieved with DynamicAutoDiffCostFunction if I'm right. But is that the right thing to do here or is there a better way ? This has been built for when the size is not known at compile time (cf. documentation), not for my case. So how does it deal with dynamically allocating big amounts of memory ?
I suppose I'll find out, but if anyone here has a recommendation/suggestion about the right way to go, I'd be happy to benefit from it.

3) Have Ceres use static memory allocation. In theory, this might be possible, as everything the amount of required memory is known at compile-time when using the non-dynamic version. But I don't know if this feature has been built in (and I suppose it may be an implementation choice not to propose this).

Thanks you very much for your comments on this!

Best,

Kenneth

PS: the problem is dense, and not separable into several ResidualBlocks (or actually it would be, but at a dramatic overload in computation).

[Sameer Agarwal]

Kenneth,

My replies are inline.

On Tue, Dec 22, 2015 at 9:44 AM Kenneth V <kenneth...@gmail.com> wrote:

Hello,

Since it's my first post here: thanks for this nice piece of software ! Really helpful.

I ran into the problem of Eigen failing on OBJECT_ALLOCATED_ON_STACK_IS_TOO_BIG at compile time.
Responsible for that is that I'm setting up an AutoDiffCostFunction of P parameters and R residuals with P=131K and R=4K.
I suppose Ceres allocates things on the stack in proportion to P and that Eigen complains about that. 

I see three possibilities: increase the stack, use the heap, or do static memory allocation:
1) Increase the hard stack limit Eigen complains about, but I suppose that won't scale very well (I'm planning to increase P).

Allocating on the heap has a significant cost in runtime for AutoDiffCostFunction.

Though to be honest with P = 131K and R = 4K, I wonder if you are better off computing your derivatives analytically.

 

2) Have Ceres allocate on the heap instead of the stack. The latter can be achieved with DynamicAutoDiffCostFunction if I'm right. But is that the right thing to do here or is there a better way ? This has been built for when the size is not known at compile time (cf. documentation), not for my case. So how does it deal with dynamically allocating big amounts of memory ?

It may work for you, because what it does is, it uses small chunks of the parameter space to do the computation in multiple passes. 

 

I suppose I'll find out, but if anyone here has a recommendation/suggestion about the right way to go, I'd be happy to benefit from it.

3) Have Ceres use static memory allocation. In theory, this might be possible, as everything the amount of required memory is known at compile-time when using the non-dynamic version. But I don't know if this feature has been built in (and I suppose it may be an implementation choice not to propose this).

This I believe has to do with the stack inside your autodiff functor, internally ceres does allocate on the stack and/or the heap as the size of these object grows. 

Sameer

 

Thanks you very much for your comments on this!

Best,

Kenneth

PS: the problem is dense, and not separable into several ResidualBlocks (or actually it would be, but at a dramatic overload in computation).

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/883fbed7-6f07-424c-b1d1-de155553e590%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kenneth V]

Hi Sameer,

Thanks a lot for the quick reply. Some comments inline.

On Tue, 22 Dec 2015 at 19:15 'Sameer Agarwal' via Ceres Solver <ceres-...@googlegroups.com> wrote:

Kenneth,

My replies are inline.

On Tue, Dec 22, 2015 at 9:44 AM Kenneth V <kenneth...@gmail.com> wrote:

Hello,

Since it's my first post here: thanks for this nice piece of software ! Really helpful.

I ran into the problem of Eigen failing on OBJECT_ALLOCATED_ON_STACK_IS_TOO_BIG at compile time.
Responsible for that is that I'm setting up an AutoDiffCostFunction of P parameters and R residuals with P=131K and R=4K.
I suppose Ceres allocates things on the stack in proportion to P and that Eigen complains about that. 

I see three possibilities: increase the stack, use the heap, or do static memory allocation:
1) Increase the hard stack limit Eigen complains about, but I suppose that won't scale very well (I'm planning to increase P).

Allocating on the heap has a significant cost in runtime for AutoDiffCostFunction.

That would vote in favor of solution 1 (increase stack) and against solution 2 (use heap) isn't it ?

Though to be honest with P = 131K and R = 4K, I wonder if you are better off computing your derivatives analytically.

I would have if it were possible :).

 

2) Have Ceres allocate on the heap instead of the stack. The latter can be achieved with DynamicAutoDiffCostFunction if I'm right. But is that the right thing to do here or is there a better way ? This has been built for when the size is not known at compile time (cf. documentation), not for my case. So how does it deal with dynamically allocating big amounts of memory ?

It may work for you, because what it does is, it uses small chunks of the parameter space to do the computation in multiple passes. 

Indeed that would do I suppose. But still, there's the significant cost of heap allocation you mentioned above that will slow things down ...

 

I suppose I'll find out, but if anyone here has a recommendation/suggestion about the right way to go, I'd be happy to benefit from it.

3) Have Ceres use static memory allocation. In theory, this might be possible, as everything the amount of required memory is known at compile-time when using the non-dynamic version. But I don't know if this feature has been built in (and I suppose it may be an implementation choice not to propose this).

This I believe has to do with the stack inside your autodiff functor, internally ceres does allocate on the stack and/or the heap as the size of these object grows. 

Well it does call a few static functions, each having a few parameters, but that's it and I don't think that is big ...

So, if I understood well, the stack is more efficient than the heap, so I should try to use solution 1 as long as I can (as long as overriding Eigen's stack limit does not make the OS complain).

Thanks again.

Kenneth

Sameer

 

Thanks you very much for your comments on this!

Best,

Kenneth

PS: the problem is dense, and not separable into several ResidualBlocks (or actually it would be, but at a dramatic overload in computation).

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/883fbed7-6f07-424c-b1d1-de155553e590%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/e54ghmebPeQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUCSH11aF6hNnorA04SbqwTqAwoVAgUuf-3iNpBw1-UbqQ%40mail.gmail.com.

[Sameer Agarwal]

Try using dynamicautodiffcostfunction, it should require a very tiny change to your code.

I think that is your fastest option.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CA%2B%2Bq7pxASezUKygKjWXf3USdKpxGGmedZM6j%2BD68Q-NtaTDipQ%40mail.gmail.com.

[Kenneth V]

Hi,

So, I have both tried to extend the heap size (solution 1) and use dynamicautodiffcostfunction (solution 2).
1) Made C++ complain instead (std::bad_alloc). I suppose the OS limits allocations at some point.
2) That works (in the sens: it does something and doesn't crash). I haven't evaluated correctness / performance yet because that requires some significative work on my code.

Thanks for the useful suggestions.

Best,

Kenneth

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAK0oyEoqKRxqS1EM6s06uS005zEbqiRXos6Ow%2B8P4pdzpaEqpA%40mail.gmail.com.

[Yinghao Huang]

Hi Keeneth,

I encountered a similar issue as yours. Can you please deliberate on how you solve it? If you have some free time, can you please look at this post created by me? The link is: https://groups.google.com/forum/#!topic/ceres-solver/_1VWa9lY5Zs.

Thanks!

Yinghao

[Kenneth V]

Hi Yinghao,

I see that you collected quite some answers from Sameer already, hopefully to your satisfaction. It's been a while I haven't touched the code you mention, so I don't remember the solution by heart. Still, if there are more questions, I can do my best to answer in a reasonable time effort.

Best,

Kenneth

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/b5a09c09-e06a-4dc8-aa3c-a6822e82372b%40googlegroups.com.