Fitting piece wise model with knot points not being updated during optimization

[Kassym Dorsel]

Hi,

I'm having troubles with a piecewise curve fitting problem where the knot points are also part of the optimization. The optimization always returns the same initial value provided for the knot points. It looks like they do not get optimized in any way. From the screenshot it is easy to see the location of the knots where the curve is discontinuous. These locations are the initial values provided during initialization. I have also drawn green lines across the curves which estimates the best positions for the knots to be.

This would reduce the error and also make the returned function closer to being continuous. The piecewise uses three segments, the center one being an arc while the two outside ones being parabolas.

Why does the model not optimize the knot points and what can I do to change this?

Is there a way to put a constraint on the model where the segments need to be continuous at the knot points?

Thanks!

[Matt Newville]

Hi Kassym,

Yes, this is a common problem.  Basically, the knot points (your `lb` and `ub`) are discrete points, while the variables (your `a` and `b`) are continuous variables.  In the optimization, the fit is going to first evaluate the function with `a=-80` and then with `a=-79.999999` to try to figure out which way and by how much to change `a`.  But your function is not sensitive to such a variation in `a` that is smaller than 1 -- you only use it for `lb = np.argmax(x > a)`.   You cannot use lmfit to fit discrete values.

I think you might be able to do what you want (switch between your 'arc' function and your 'para' function) if you made the transition not discrete but more gradual, say with a logistic function that was centered at your `a` and `b` values but had a width of a few X units.    Does that make sense?

--Matt

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To post to this group, send email to lmfi...@googlegroups.com.
Visit this group at https://groups.google.com/group/lmfit-py.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/62b9443d-84cb-4b57-b99d-ee6ae6619f3a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

--Matt Newville <newville at cars.uchicago.edu> 630-252-0431

[Kassym Dorsel]

Yes, that makes perfect sense. I will make the transition continuous.

Looking through the documentation I see a step size when using the brute force method. Is there a way to define a minimum step size when using other fitting methods? My X data is in increments of 0.05 so I would like to define a minimum step size of ~0.06-0.1.

On Friday, May 3, 2019 at 11:23:31 AM UTC-4, Matt Newville wrote:

Hi Kassym,

Yes, this is a common problem.  Basically, the knot points (your `lb` and `ub`) are discrete points, while the variables (your `a` and `b`) are continuous variables.  In the optimization, the fit is going to first evaluate the function with `a=-80` and then with `a=-79.999999` to try to figure out which way and by how much to change `a`.  But your function is not sensitive to such a variation in `a` that is smaller than 1 -- you only use it for `lb = np.argmax(x > a)`.   You cannot use lmfit to fit discrete values.

I think you might be able to do what you want (switch between your 'arc' function and your 'para' function) if you made the transition not discrete but more gradual, say with a logistic function that was centered at your `a` and `b` values but had a width of a few X units.    Does that make sense?

--Matt

On Fri, May 3, 2019 at 9:56 AM Kassym Dorsel <k.d...@gmail.com> wrote:

Hi,

I'm having troubles with a piecewise curve fitting problem where the knot points are also part of the optimization. The optimization always returns the same initial value provided for the knot points. It looks like they do not get optimized in any way. From the screenshot it is easy to see the location of the knots where the curve is discontinuous. These locations are the initial values provided during initialization. I have also drawn green lines across the curves which estimates the best positions for the knots to be.

This would reduce the error and also make the returned function closer to being continuous. The piecewise uses three segments, the center one being an arc while the two outside ones being parabolas.

Why does the model not optimize the knot points and what can I do to change this?

Is there a way to put a constraint on the model where the segments need to be continuous at the knot points?

Thanks!

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lmfi...@googlegroups.com.

To post to this group, send email to lmfi...@googlegroups.com.
Visit this group at https://groups.google.com/group/lmfit-py.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/62b9443d-84cb-4b57-b99d-ee6ae6619f3a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Newville]

Hi Kassym, 

On Fri, May 3, 2019 at 11:03 AM Kassym Dorsel <k.do...@gmail.com> wrote:

Yes, that makes perfect sense. I will make the transition continuous.

Looking through the documentation I see a step size when using the brute force method. Is there a way to define a minimum step size when using other fitting methods? My X data is in increments of 0.05 so I would like to define a minimum step size of ~0.06-0.1.

This varies with optimization method and some methods don't really think in terms of "step size".   For the default Levenberg-Marquardt solver in the most common case of not having an analytic Jacobian function, the Jabobian is estimated with a relative change in each parameter value x of sqrt(epsfcn)*x.  The default value of epsfcn is 2e-16, but this can be changed.   With `lmfit.Model`, you can set this with

     result = mymodel.fit(y, params, ..., fit_kws={'epsfcn': 1.e-10})

for example.   Unfortunately, there is not a way to set this differently per variable.   In my experience, the results (or number of function evaluations used) does not depend strongly (or really, predictably!) with changing epsfcn, but maybe someone has different experience.    Anyway, that isn't really "smallest step", it's "value used to do the partial derivative to determine the next step size". 

So, you could try using mod.fit(..., fit_kws={'epsfcn': 0.01})  or some other huge value.  That might be able to cause your `a` and `b` parameters to change from their initial values, But I still don't think it will really solve the problem of using continuous variables used to set discrete boundaries. 

--Matt

[Kassym Dorsel]

Changing the epsfcn gave much better results, but I found for this data the best combination was using the logistic function and changing the epsfcn value. I've included plots and the updated code.

Is there a way to add a tangential requirement between the segments? The transition points between the segments need to be continuous. Would I be better off moving to using the minimize() function and writing my own loss/residual function with large negative weights for step transition points?

Thanks again!

[Matt Newville]

Hi Kassym,

On Sat, May 4, 2019 at 2:35 PM Kassym Dorsel <k.do...@gmail.com> wrote:

Changing the epsfcn gave much better results, but I found for this data the best combination was using the logistic function and changing the epsfcn value. I've included plots and the updated code.

Yeah, I would expect that using a logistic function to be the more important strategy, and that changing epsfcn should have a less dramatic effect on the result.  But, it's fine to do both...

Is there a way to add a tangential requirement between the segments?

That would be something you would have to do in your model.  Lmfit has no idea that there are segments.

The transition points between the segments need to be continuous.

Yes, this is why a logistic (or similar smoothed step) function is useful. 

Note that using broadened step functions, you can replace 

   ymodel = np.concatenate(ymodel1[:nx] , ymodel2[nx+1:]))

with

   ymodel = ymodel1 * smoothed_step_up . + ymodel2 * smoothed_step_down + ymodel3 * smoothed_window_function.

where all those pieces span the full data range.   That seems less fussy and easier to maintain/modify.  And it should help with smoothness at transitions.

Would I be better off moving to using the minimize() function and writing my own loss/residual function with large negative weights for step transition points?

Not necessarily.  The main difference between the Model approach and using minimize() is that Model assumes there is data to modeled, and that there are independent variables that are clearly defined.  OTOH, minimize() can be used for non-curve-fitting problems.  There are some things that are sort of conceptually challenging when using Model, especially regarding the independent variables (like, if you want to fit parameters that re-calibrate the "X" axis).  

But I believe your problem really is a curve-fitting problem with a clear X axis as the common independent variable for Model and Data.  You want to build your Model function as sum of segments -- that should be fine.    Again I would encourage you to think about your model as a sum of "model_componet*smoothed_window_function".

 

--Matt