Output when initial values close to exact values

[Aditya Gupta]

Hello all,

I am using LM algorithm for my problem, where I have around 120 equations and need to determine the values of parameters a,b,c (depending on the experiment I have three to eight variables). The equations are non linear with a,b,c in the powers of 1 to cubic. The below code works well when the initial values are approximately in the range of the exact values. Now I am facing a strange problem that when the initial values are very close or equal to the exact values then the algorithm gives outputs that are way off from the exact values and incorrect. Any ideas why is this the case and any hints where I can start to fix this?

DEFINE_string(minimizer, "trust_region",
   "Minimizer type to use, choices are: line_search & trust_region");
Solver::Options options_smart;
   options_smart.trust_region_strategy_type = ceres::LEVENBERG_MARQUARDT;
   options_smart.max_num_iterations = 500;
   options_smart.linear_solver_type = ceres::DENSE_QR;
   options_smart.minimizer_progress_to_stdout = true;

   Problem problem;
   double a, b, c;
    for (int i = 0; i < logf.size(); ++i) {
         eqn_abc_t *eqn_abc_1 = new eqn_abc_t(logf[i], Zr[i]);
         problem.AddResidualBlock(new AutoDiffCostFunction<eqn_abc_t, 1, 1, 1, 1>(eqn_abc_1), NULL, &a, &b, &c);
         eqn_C2_abc_t *eqn_C2_abc_1 = new eqn_C2_abc_t(logf[i],ZC2[i]);        
         problem.AddResidualBlock(new AutoDiffCostFunction<eqn_C2_abc_t, 1, 1, 1, 1>(eqn_C2_abc_1), NULL, &a, &b, &c);
      }

      Solver::Summary summary;
      Solve(options_smart, &problem, &summary);

Thank you.

Regards,
AGupta

[Chris Sweeney]

It would help if you could provide the two cost functions as well.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/e66052f1-a989-42e2-b0f0-fcf58c6c54a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aditya Gupta]

Here are the cost functions:

class eqn_abc_t{
public:
   eqn_abc_t(double x, double y)
      : x_(x), y_(y) {}
   template <typename T>
   bool operator()(const T* const a, const T* const b, const T* const c, T* residual) const {
      residual[0] = (T(y_) - a[0])*(1.0 + (4.0 * Pisq * pow(10, T(x_)) * pow(10, T(x_)) * b[0] * b[0] * c[0] * c[0]))
         - b[0];
      return true;
   } 
private:
   const double x_;
   const double y_;
};

class eqn_C2_abc_t{
public:
   eqn_C2_abc_t(double x, double y)
      : x_(x), y_(y) {}
   template <typename T>
   bool operator()(const T* const a, const T* const b, const T* const c, T* residual) const {
      residual[0] = T(y_) + (T(y_) * (4.0 * Pisq * pow(10, T(x_)) * pow(10, T(x_)) * b[0] * b[0] * c[0] * c[0]))
         - (b[0] * b[0] * 2.0 * Pi * c[0] * pow(10, T(x_)));
      return true;
   }
private:
   const double x_;
   const double y_;
};

Thanks.

[Sameer Agarwal]

When you put in the exact answers, do you get a zero or near zero residual?

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/5676efac-c706-47f0-91c9-3ad0d4de978a%40googlegroups.com.

[Aditya Gupta]

Yes, I get a near zero residual. For example: the exact values of a,b,c for a particular experiment are 10,90, 1E-6. If I enter these exact values, a,b,c evaluate to 68, 0.001 and 1E-12 and my residual using these values is 1E-12 to 1E-18.

[Sameer Agarwal]

Can you share an execution log?

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/d2ce89c4-6b6c-4156-8dae-aa6ec3eb81ec%40googlegroups.com.

[Aditya Gupta]

Here is the execution log below and see attached image (R1, R2 and C1 are my a,b,c and the initial values are the exact values):

Initial R1 = 10Initial R2 = 90, Initial C1 = 1e-006

Solver Summary (v 1.11.0-eigen-(3.2.8)-no_lapack-openmp)

                                     Original                  Reduced
Parameter blocks                            3                        3
Parameters                                  3                        3
Residual blocks                           322                      322
Residual                                  322                      322

Minimizer                        TRUST_REGION

Dense linear algebra library            EIGEN
Trust region strategy     LEVENBERG_MARQUARDT

                                        Given                     Used
Linear solver                        DENSE_QR                 DENSE_QR
Threads                                     1                        1
Linear solver threads                       1                        1

Cost:
Initial                         6.284755e+009
Final                           1.321121e+005
Change                          6.284623e+009

Minimizer iterations                        5
Successful steps                            5
Unsuccessful steps                          0

Time (in seconds):
Preprocessor                           0.0006

  Residual evaluation                  0.0019
    Line search cost evaluation        0.0000
  Jacobian evaluation                  0.1182
    Line search gradient evaluation    0.0569
  Linear solver                        0.0042
  Line search polynomial minimization  0.0000
Minimizer                              0.1713

Postprocessor                          0.0001
Total                                  0.1719

Termination:                      CONVERGENCE (Function tolerance reached. |cost_change|/cost: 2.956158e-012 <= 1.000000e-006)

Final R1 = 9.95906, C1 = 8.33924e-013, R2 = 25.4487

Thanks.

-Aditya

[Sameer Agarwal]

You said earlier your residuals were near zero but this execution log shows they start at e9 ?

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/58130d7f-0524-4cd9-87aa-f78e286b55bc%40googlegroups.com.

[Aditya Gupta]

Interesting, when I substitute the initial values in my cost functions, they all evaluate to zeros (or values e-18). Rechecked this. Not sure why the log shows initial cost as e9. Here is my code again, am I missing something?

struct Zreal {
   Zreal(double x, double y)
      : x_(x), y_(y) {}
   template <typename T> bool operator()(const T* const R1, const T* const R2, const T* const C1, T* residual) const {
      residual[0] = T(y_) + (T(y_) * (4.0 * Pisq * pow(10,T(x_)) * pow(10,T(x_)) * R2[0] * R2[0] * C1[0] * C1[0])) - R1[0] -
         (R1[0] * (4.0 * Pisq * pow(10, T(x_)) * pow(10, T(x_)) * R2[0] * R2[0] * C1[0] * C1[0])) - R2[0];

      return true;
   }
private:
   const double x_; const double y_;
};

struct Zimag {
   Zimag(double x, double y)
      : x_(x), y_(y) {}
   template <typename T> bool operator()(const T* const R1, const T* const R2, const T* const C1, T* residual) const {
      residual[0] = T(y_) + (T(y_) * (4.0 * Pisq * pow(10, T(x_)) * pow(10, T(x_)) * R2[0] * R2[0] * C1[0] * C1[0]))
         - (R2[0] * R2[0] * 2.0 * Pi * C1[0] * pow(10, T(x_)));

      return true;
   }
private:
   const double x_;  const double y_;
};

DEFINE_string(minimizer, "trust_region",
              "Minimizer type to use, choices are: line_search & trust_region");

int main(int argc, char** argv) {
  CERES_GFLAGS_NAMESPACE::ParseCommandLineFlags(&argc, &argv, true);
  google::InitGoogleLogging(argv[0]);

  //Initial Values
  double initialR1 = 10.0;
  double initialR2 = 90.0;
  double initialC1 = 1E-06;
  double R1 = initialR1;
  double R2 = initialR2;
  double C1 = initialC1;

  //reading the input data file1
  readinputFile(logf_val, Zr_val, Zi_val);

  Problem problem;
  for (int i = 0; i < logf_val.size(); ++i) {
     problem.AddResidualBlock(new AutoDiffCostFunction<Zreal, 1, 1, 1, 1>(new  Zreal(logf_val[i], Zr_val[i])),
        NULL,
        &R1, &R2, &C1);
     problem.AddResidualBlock(new AutoDiffCostFunction<Zimag, 1, 1, 1, 1>(new Zimag(logf_val[i], Zi_val[i])),
        NULL,
        &R1, &R2, &C1);
  }
  problem.SetParameterLowerBound(&R1, 0, 0.001);
  problem.SetParameterUpperBound(&R1, 0, 1E+6);
  problem.SetParameterLowerBound(&C1, 0, 10E-15);
  problem.SetParameterUpperBound(&C1, 0, 10E-6);
  problem.SetParameterLowerBound(&R2, 0, 0.001);
  problem.SetParameterUpperBound(&R2, 0, 1E+6);

  Solver::Options options;
  LOG_IF(FATAL, !ceres::StringToMinimizerType(FLAGS_minimizer,
                                              &options.minimizer_type))
      << "Invalid minimizer: " << FLAGS_minimizer
      << ", valid options are: trust_region and line_search.";

  options.trust_region_strategy_type = ceres::LEVENBERG_MARQUARDT;
  options.max_num_iterations = 500;
  options.linear_solver_type = ceres::DENSE_QR;
  options.minimizer_progress_to_stdout = true;

Thank you.

[Sameer Agarwal]

were your initial values that you substituted in the parameter bounds?

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/4cbe635d-0610-4c27-89ad-2a913cbd2b25%40googlegroups.com.

[Aditya Gupta]

Yes they are. Also I modified the parameter bounds to increase the range and similar result. If I take out the parameter bounds, I obtain another result, which is again off from the initial/exact values. With the initial values equal to the exact values, the cost functions evaluate to zero, should the algorithm not just converge and output the initial values as the best fit? I guess the algorithm starts searching in the trust region near the initial values and does not evaluate the initial values. Is this the right understanding? If yes, how I can change this? Thank you.

[Sameer Agarwal]

I think there is a mistake in the way you are setting up the problem. perhaps you are scrambling the order of the parameters which is why you are getting these large residuals. The bottom line is that the problem you think you are setting up is not the problem that the solver is seeing, which is why it is taking you elsewhere. The thing to figure out, by adding additional logging to the cost function would be to see what are the initial set of parameters that the cost function is getting in the very first iteration and do they match the initial parameters you are giving to the solver.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/295c37b2-5af4-4300-9794-1b07cdb0ed2e%40googlegroups.com.