Gradient norm always zero
305 views
Skip to first unread message
Matthias Bühlmann
Jun 8, 2021, 8:44:26 PM6/8/21
to Ceres Solver
Hello there,
It seems that would indicate that my residuals are not differentiable, but I don't quite see why that's the case. The following is my CostFunctor:
template<int ColorIndex>
struct CostFunctor {
template <typename T>
bool operator()(const T* const v1_coeffs,
const T* const v2_coeffs,
const T* const v3_coeffs,
const T* const weight_px_00,
const T* const weight_px_10,
const T* const weight_px_01,
const T* const weight_px_11,
const T* const coeff0_px_00,
const T* const coeff0_px_10,
const T* const coeff0_px_01,
const T* const coeff0_px_11,
T* residual) const {
const Observation& observation = container_->observations_[observation_index_];
const double width = container_->resolution_[0];
const double height = container_->resolution_[1];
const double frame_u = observation.GetU();
const double frame_v = observation.GetV();
const double texel_u = observation.GetU() * (width - 1) + 0.5;
const double texel_v = observation.GetV() * (height - 1) + 0.5;
double u_fraction = texel_u - std::floor(texel_u);
if (u_fraction >= 0.5) {
u_fraction -= 0.5;
} else {
u_fraction += 0.5;
}
double v_fraction = texel_v - std::floor(texel_v);
if (v_fraction >= 0.5) {
v_fraction -= 0.5;
} else {
v_fraction += 0.5;
}
double v1_w, v2_w, v3_w;
if(frame_u >= frame_v) {
// triangle 0
//frame_u == v1_w * 0 + v2_w * 1 + v3_w * 1;
//frame_v == v1_w * 0 + v2_w * 0 + v3_w * 1;
v3_w = frame_v;
v2_w = frame_u - v3_w;
v1_w = 1.0 - v2_w - v3_w;
} else {
// traingle 1
// frame_u == v1_w * 0 + v2_w * 1 + v3_w * 0;
// frame_v == v1_w * 0 + v2_w * 1 + v3_w * 1
v2_w = frame_u;
v3_w = frame_v - v2_w;
v1_w = 1.0 - v2_w - v3_w;
}
std::array<double, kNumSHCoeffs> coeffs;
observation.GetCoeffs(&coeffs);
const double sample_color = observation.GetColor(ColorIndex);
// bilinear interpolation
const T weight = (weight_px_00[0] * (1.0 - u_fraction) + weight_px_10[0] * u_fraction) * (1.0 - v_fraction)
+ (weight_px_01[0] * (1.0 - u_fraction) + weight_px_11[0] * u_fraction) * (v_fraction);
const T coeff0 = (coeff0_px_00[0] * (1.0 - u_fraction) + coeff0_px_10[0] * u_fraction) * (1.0 - v_fraction)
+ (coeff0_px_01[0] * (1.0 - u_fraction) + coeff0_px_11[0] * u_fraction) * (v_fraction);
T color = coeff0;
for(int i=0; i < kNumSHCoeffs-1; i++) {
const double coeff = coeffs[i + 1];
color += v1_coeffs[i] * v1_w * coeff
+ v2_coeffs[i] * v2_w * coeff
+ v3_coeffs[i] * v3_w * coeff;
}
residual[0] = sample_color - (color * weight);
return true;
}
//Observation
Container* container_;
size_t observation_index_;
};
The solver summaries all look something like that (single iteration and gradient of 0):
Solver Summary (v 2.0.0-eigen-(3.3.8)-no_lapack-eigensparse-no_openmp)
Original Reduced
Parameter blocks 438 438
Parameters 648 648
Residual blocks 135 135
Residuals 135 135
Minimizer TRUST_REGION
Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver ordering AUTOMATIC 438
Cost:
Initial 2.649840e-02
Final 2.649840e-02
Change 0.000000e+00
Minimizer iterations 1
Successful steps 1
Unsuccessful steps 0
Line search steps 0
Time (in seconds):
Preprocessor 0.004548
Residual only evaluation 0.000000 (0)
Line search cost evaluation 0.000000
Jacobian & residual evaluation 0.002872 (1)
Line search gradient evaluation 0.000000
Linear solver 0.000000 (0)
Line search polynomial minimization 0.000000
Minimizer 0.010438
Postprocessor 0.000142
Total 0.015128
Termination: CONVERGENCE (Gradient tolerance reached. Gradient max norm: 0.000000e+00 <= 1.000000e-10)
These are my solver options:
ceres::Solver::Options solver_options;
solver_options.linear_solver_type = ceres::DENSE_QR;
solver_options.num_threads = 1;
solver_options.logging_type = ceres::LoggingType::SILENT;
how come my residual isn't differentiable? I'm using only multiplications and additions on the Jets
Sameer Agarwal
Jun 8, 2021, 10:55:57 PM6/8/21
to ceres-...@googlegroups.com
The short answer is that your autodiff code is broken. Your bilinear interpolation code is creating zero valued derivatives. This is why the BiCubicInterpolator was written. Please use it to replace your blinear interpolation code.
--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/758aba5c-56cd-4036-a227-0a133e417c28n%40googlegroups.com.
Matthias Bühlmann
Jun 9, 2021, 8:11:03 AM6/9/21
to Ceres Solver
Thank you for your prompt reply!
Might I ask you to maybe give me the long answer? I don't quite see how it produces zero valued derivatives, as the u_fraction and v_fraction values are constant at each observation. Preferably I'd like to use bilinear interpolation of the four values since this is the same interpolation the model is later evaluated with (by hardware).
Also, If I'm going to use BiCubicInterpolator, how can I use it within my AutoDiff cost functor?
Thank you!
Matthias Bühlmann
Jun 9, 2021, 8:15:12 AM6/9/21
to Ceres Solver
To me it looks like BiCubicInterpolator is used to interpolate (constant) observation values by a parameter-dependent value, but I do the opposite, I interpolate parameter values by a (constant) observation.
Sameer Agarwal
Jun 9, 2021, 1:06:01 PM6/9/21
to ceres-...@googlegroups.com
you are right, I am wrong. I was too hasty in my reply. sorry about that.
a simple thing to check is to see if you log weight and coeff0 do they have non-zero jet parts?
you should just be able to add LOG(INFO) << weight; to your code and see whats happening.
and then log residual[0] before exiting.
Sameer
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/61377105-2a8b-4a02-97ac-2ace0390c838n%40googlegroups.com.
Matthias Bühlmann
Jun 10, 2021, 1:42:36 AM6/10/21
to Ceres Solver
Thank you Sameer!
Great to see that it's so easy to log the jets :)
Doing so shows that the weight and coeff0 have zero parts everywhere except at the positions of the parameter blocks they depend on (the first three blocks have a length of 8 each and the remaining blocks have all a length of 1). I assume that's what's to be expected? But then, how can I resolve that?
sample_color: 0.0288391
weight: [0 ; 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.047448, 0.711337, 0.0150835, 0.226131, 0, 0, 0, 0]
coeff0: [0 ; 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.047448, 0.711337, 0.0150835, 0.226131]
color: [0 ; 0.0236095, -0.0207792, 0.0860466, 0.049584, -0.0119739, -0.0500102, -0.0436399, 0.0835539, 0.0537573, -0.0473129, 0.195923, 0.1129, -0.0272639, -0.11387, -0.0993653, 0.190247, 0.0485485, -0.0427286, 0.176939, 0.10196, -0.0246221, -0.102837, -0.0897373, 0.171813, 0, 0, 0, 0, 0.047448, 0.711337, 0.0150835, 0.226131]
residual[0]: [0.0288391 ; -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0]
Sameer Agarwal
Jun 10, 2021, 11:34:58 AM6/10/21
to ceres-...@googlegroups.com
the problem is not the derivative part but the fact that your weight has value zero (even though its derivative is non-zero).
so when you evaluate residual = sample_color - color * weight
it reduces to just
residual = sample_color
which is a constant with zero derivative. So the problem seems to be weight_px_*.
Sameer
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/fb5b3821-fb5d-4e4f-bc21-bc966871b9edn%40googlegroups.com.
Matthias Bühlmann
Jun 14, 2021, 7:29:52 AM6/14/21
to Ceres Solver
Perfect, after initializing the weight_px values to something non-zero, the problem is gone.
Thank you Sameer :)
Reply all
Reply to author
Forward
0 new messages