Best way to use L1 regression instead of L2?

[Matthias Bühlmann]

I use AutoDiffCostFunction and I'd like to use L1 norm instead of L2 for my regression. The reason I'd like to try L1 norm is that my observations contain Monte Carlo noise (which is meant to be averaged to produce the actual output), but I'd like to fit my model before resampling (and averaging) the observations, and (in my mind at least) it seems L1 norm would be more appropriate to model this 'averaging'.

I guess I could just do 

residual[0] = ceres::sqrt(ceres::abs(residual[0]));

But maybe there is a better way to do this?

should I wrap all my AutoDiffCostFunctions in ConditionedCostFunctions? (if yes, in what way would that be better than taking the abs and square root before outputing the residual from my AutoDiffCostFunction?

[Dmitriy Korchemkin]

> residual[0] = ceres::sqrt(ceres::abs(residual[0]));

This will mess up derivatives near 0, if you're using autodiff.

Huber loss function and SoftLOne loss functions are essentially L1 loss as residual goes to infinity (and are smooth near zero), so you might try ceres::HuberLoss / ceres::SoftLOneLoss.

If you have some assumptions on the noise distribution - it might be more beneficial to optimize log-likelihood of this particular distribution (rather than solving some NLLS problem) though.