DiffFunction with custom Type

[Alex Minnaar]

Apologies if this has been asked before but I could not find the answer anywhere.  I am trying to create a DiffFunction that I can optimize via LBFGS, but my particular use-case differs from the example shown in the quickstart tutorial in that my function requires several different input parameters that are not known apriori (so hardcoding them in the calculate function is not an option).  I am thinking that one option might be to create a case class that contains all of the required parameters and use this as the DiffFunction type.  The problem with this though is that the gradient in the return tuple must be the same type so this probably won't work (the gradient is a DenseVector).  Another option is to just concatenate all of the input parameters into one really long DenseVector which I suppose would work since the gradient should be a DenseVector as well - However this will be somewhat messy since I have a lot of parameters that are both vectors and matrices.  Is there a more elegant solution to this problem?

Thanks

  

[David Hall]

I usually just make a class that takes the parameters in the ctor:

case class MyFun(params: Params) extends DiffFunction[DenseVector[Double]] {

 // ...

}

Does that not work here?

On Thu, Dec 29, 2016 at 4:06 PM, Alex Minnaar <minna...@gmail.com> wrote:

Apologies if this has been asked before but I could not find the answer anywhere.  I am trying to create a DiffFunction that I can optimize via LBFGS, but my particular use-case differs from the example shown in the quickstart tutorial in that my function requires several different input parameters that are not known apriori (so hardcoding them in the calculate function is not an option).  I am thinking that one option might be to create a case class that contains all of the required parameters and use this as the DiffFunction type.  The problem with this though is that the gradient in the return tuple must be the same type so this probably won't work (the gradient is a DenseVector).  Another option is to just concatenate all of the input parameters into one really long DenseVector which I suppose would work since the gradient should be a DenseVector as well - However this will be somewhat messy since I have a lot of parameters that are both vectors and matrices.  Is there a more elegant solution to this problem?

Thanks

  

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze+unsubscribe@googlegroups.com.
To post to this group, send email to scala-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/0891d95c-3141-4634-b71f-e899312fb105%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Minnaar]

Great idea! This works.  The only minor cosmetic issue now is that since all my parameters are passed into the class constructor I don't necessarily need to pass anything into the `calculate` method - but I can just pass in an empty DenseVector so not a big deal.  Thanks!

On Friday, December 30, 2016 at 3:23:41 PM UTC-5, David Hall wrote:

I usually just make a class that takes the parameters in the ctor:

case class MyFun(params: Params) extends DiffFunction[DenseVector[Double]] {

 // ...

}

Does that not work here?

On Thu, Dec 29, 2016 at 4:06 PM, Alex Minnaar <minna...@gmail.com> wrote:

Apologies if this has been asked before but I could not find the answer anywhere.  I am trying to create a DiffFunction that I can optimize via LBFGS, but my particular use-case differs from the example shown in the quickstart tutorial in that my function requires several different input parameters that are not known apriori (so hardcoding them in the calculate function is not an option).  I am thinking that one option might be to create a case class that contains all of the required parameters and use this as the DiffFunction type.  The problem with this though is that the gradient in the return tuple must be the same type so this probably won't work (the gradient is a DenseVector).  Another option is to just concatenate all of the input parameters into one really long DenseVector which I suppose would work since the gradient should be a DenseVector as well - However this will be somewhat messy since I have a lot of parameters that are both vectors and matrices.  Is there a more elegant solution to this problem?

Thanks

  

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze...@googlegroups.com.

[David Hall]

wait, i'm very confused... if you don't have any parameters you're calculating the gradient on, then why do you need to implement the DiffFunction interface at all?

To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze+unsubscribe@googlegroups.com.

To post to this group, send email to scala-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/dd263847-b2f7-4d2c-8b76-efab819a374a%40googlegroups.com.

[Alex Minnaar]

To clarify, this is what its looking like now

  case class myFunc(params: Params) extends DiffFunction[DenseVector[Double]] {

    def calculate(x: DenseVector[Double]) = {

      val value =...  //uses params to calculate value

      val gradient =...  //uses params to calculate gradient

      (value, gradient)

    }

  }

 So the gradient calculation uses `params` but it doesn't use `x`.  But I think `x` still needs to be there because it needs to be the same type as the gradient (i.e. DenseVector[Double]) unless I am misunderstanding something. 

On Friday, December 30, 2016 at 5:40:13 PM UTC-5, David Hall wrote:

wait, i'm very confused... if you don't have any parameters you're calculating the gradient on, then why do you need to implement the DiffFunction interface at all?

On Fri, Dec 30, 2016 at 4:02 PM, Alex Minnaar <minna...@gmail.com> wrote:

Great idea! This works.  The only minor cosmetic issue now is that since all my parameters are passed into the class constructor I don't necessarily need to pass anything into the `calculate` method - but I can just pass in an empty DenseVector so not a big deal.  Thanks!

On Friday, December 30, 2016 at 3:23:41 PM UTC-5, David Hall wrote:

I usually just make a class that takes the parameters in the ctor:

case class MyFun(params: Params) extends DiffFunction[DenseVector[Double]] {

 // ...

}

Does that not work here?

On Thu, Dec 29, 2016 at 4:06 PM, Alex Minnaar <minna...@gmail.com> wrote:

Apologies if this has been asked before but I could not find the answer anywhere.  I am trying to create a DiffFunction that I can optimize via LBFGS, but my particular use-case differs from the example shown in the quickstart tutorial in that my function requires several different input parameters that are not known apriori (so hardcoding them in the calculate function is not an option).  I am thinking that one option might be to create a case class that contains all of the required parameters and use this as the DiffFunction type.  The problem with this though is that the gradient in the return tuple must be the same type so this probably won't work (the gradient is a DenseVector).  Another option is to just concatenate all of the input parameters into one really long DenseVector which I suppose would work since the gradient should be a DenseVector as well - However this will be somewhat messy since I have a lot of parameters that are both vectors and matrices.  Is there a more elegant solution to this problem?

Thanks

  

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze...@googlegroups.com.
To post to this group, send email to scala-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/0891d95c-3141-4634-b71f-e899312fb105%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Scala Breeze" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze...@googlegroups.com.
To post to this group, send email to scala-...@googlegroups.com.

[Debasish Das]

Hi Alex,

x for you the current assignment of the variables by the solver. Say we started with a random initialization point x = [0.2, 0.3]

You are solving for f(x) = xy - 2x

Based on this you have to find out the function value and next gradient

At given point x = [0.2, 0.3], value is a function of param and 0.2*0.3 - 2*0.2

Similarly the gradient will be df/dx = y df/dy = x At this point the gradient will be [0.3, 0.2]

Based on these, solver will find the next step: x_next = [0.2 + delta_x, 0.3 + delta_y]

So basically you will write the logic to calculate function value and gradient using x at any given solver iteration..

[Alex Minnaar]

Thanks Debasish,  it looks like you are describing the standard case where x is a DenseVector.  My Question is what if x is something more complicated like a case class with a DenseVector and a DenseMatrix?  So basically I would want to do something like

case class Param(z: DenseVector[Double],y:DenseMatrix[Double])


val f= new DiffFunction[Param] {
  
  def calculate(x: Param) = {
    
    val v:Double = ... //some function of x that returns double
    val g:DenseVector[Double] = ...//some function of x that returns DenseVector[Double]

    (v, g)
  }

}

However this will not work since x is of type Param and the gradient is of type DenseVector[Double].  So basically I am trying to figure out how to use the optimizer in this case.  Any help would be appreciated.  Thanks!

[David Hall]

Ah, now I think I understand. generally I pack/unpack things into a DenseVector, sometimes with a wrapping class.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-breeze+unsubscribe@googlegroups.com.

To post to this group, send email to scala-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-breeze/dd93e4ce-e34b-47ad-ba5f-ccabe3bc5d2b%40googlegroups.com.

[Alex Minnaar]

Thanks that sounds like a good solution