Multi-threading in ceres

[Weipeng Xu]

Hi all,

I'm trying to run some multi-thread code inside a overload function Evaluate(double const* const* parameters, double* residuals, double** jacobians) of my own cost function. But it seems that it doesn't work in parallel. Is that due to that ceres disabled multi-thread while computing the residuals and jacobians? The code is like this:

bool MyCostFunctor::Evaluate(double const* const* parameters, double* residuals, double** jacobians) const

{

#pragma omp parallel for

for (int i = 0; i < 100; i++)

{

std::cout << i << std::endl;

}

      

...

}

The output is 0~99 in successive order. But the same code in anywhere else gives 0~99 shuffled.

Best,

Weipeng

[Sameer Agarwal]

What compiler and platform? When you disable threading in ceres it its threaded loops which are disabled it should have no impact on your own code.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/b93e8aba-0b43-4394-b9ce-20225d824556%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sameer Agarwal]

Weipeng,

I have no experience with Visual Studio and its openmp threading abilities, but if it supports OpenMP I see no reason why it should not work.
Sameer

On Wed, Oct 12, 2016 at 6:03 AM Weipeng Xu <xuweip...@gmail.com> wrote:

Hi Sameer,

It's visual studio 2013, on x64.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/7996a32c-c800-42a8-8a4f-4f34a26f927a%40googlegroups.com.

[Weipeng Xu]

Hi Sameer,

It's quite wired. I wrote a function to print 0-99 in parallel. If I call it in the constructor of the cost function, it's in parallel. But if I call it in Evaluation(), it's not in parallel. I suppose there must be something that shuts down the openmp when calling the Evaluation().

Weipeng

[Weipeng Xu]

Hi Sameer,

I think I've found the reason. When ceres calls Evaluation(), it uses parallel for loop, i.e. it already assigned a thread to compute each residual block. Therefore, in Evaluation() function of each residual block, parallel for loop is not allowed.

Is there any way to fix this issue?

Best,

Weipeng

[Sameer Agarwal]

but if you have turned threading off in ceres, is this still a problem?

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/e54e357b-f47a-47ae-abb9-34a30e0e0d1d%40googlegroups.com.

[Weipeng Xu]

If you mean setting options.num_threads = 1, yes, it's still a problem. By doing that, ceres creates a single thread with #pragma omp parallel for num_threads(options.num_threads) to evaluate the residual blocks.

[Sameer Agarwal]

do you mean to say that the same then applies to any further nested openmp calls?

On Wed, Oct 12, 2016 at 7:31 AM Weipeng Xu <xuweip...@gmail.com> wrote:

If you mean setting options.num_threads = 1, yes, it's still a problem. By doing that, ceres creates a single thread with #pragma omp parallel for num_threads(options.num_threads) to evaluate the residual blocks.

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/d885e5b3-f2d5-47f6-8c5b-9bd3c3f62ea4%40googlegroups.com.

[Weipeng Xu]

Yes. When calling Evaluation(), ceres assigned a thread to each residual block, and in each thread, you cannot create more threads.

[Sameer Agarwal]

Hmm, I am not sure what the solution is here, I will look into it. Can you file a bug on github please. It may take me a bit though.

On Wed, Oct 12, 2016 at 7:40 AM Weipeng Xu <xuweip...@gmail.com> wrote:

Yes. When calling Evaluation(), ceres assigned a thread to each residual block, and in each thread, you cannot create more threads.

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/5443c01f-21ec-419a-9dd5-2863af54b0f7%40googlegroups.com.

[Weipeng Xu]

I think one possible solution is not to use #pragma omp parallel for num_threads(num_threads) before Evaluate(), when num_threads is set to 1.

Yes, I will file the bug.

[Weipeng Xu]

Hi Sameer,

I have tried my solution by commenting all the #pragma omp bla in ceres::internal::ProgramEvaluator::Evaluate(..) function. The parallel stuff works now.

Best,

Weipeng

[Peter Karasev]

Hi, 

  You may want to consider breaking up your problem at a higher level; for me that tends to do more for performance than parallelizing a for-loop. 

_______________
Peter  Karasev
kara...@gmail.com

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/4d2cf1c6-44fa-4334-8a80-84c780fda57d%40googlegroups.com.

[Sameer Agarwal]

I am with Peter. If you are having to thread stuff inside your CostFunction then it is likely the case that you are creating a huge jacobian, in which case it is worth exposing the construction of the jacobian to the solver, it will parallelize it for you, and reduce memory usage.

Sameer

On Wed, Oct 12, 2016 at 11:01 AM Peter Karasev <kara...@gmail.com> wrote:

Hi, 

  You may want to consider breaking up your problem at a higher level; for me that tends to do more for performance than parallelizing a for-loop. 

_______________
Peter  Karasev
kara...@gmail.com

On Wed, Oct 12, 2016 at 11:43 AM, Weipeng Xu <xuweip...@gmail.com> wrote:

Hi Sameer,

I have tried my solution by commenting all the #pragma omp bla in ceres::internal::ProgramEvaluator::Evaluate(..) function. The parallel stuff works now.

Best,

Weipeng

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/4d2cf1c6-44fa-4334-8a80-84c780fda57d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAA7K0yOSyO2n7QepENNi0XumR32LaOoV14ryGdKUEs%2BBgU8%3DPg%40mail.gmail.com.

[Simme]

Hi Weipeng,

I think with Visual Compiler openmp, nested parallel regions are disabled by default.
Try turning it on with omp_set_nested(true) or with the OMP_NESTED environment variable.
This should enable your parallel loop inside the Evaluate loop.

Simme

[Weipeng Xu]

Hi Simme,

It works. Thanks a lot.

Weipeng