curve fitting

[Seb Binet]

hi there,

one of the requests from [1] is for gonum to provide curve fitting routines.

I'd be also interested in giving it a stab.

any pointers?

-s

[1]: https://www.reddit.com/r/golang/comments/4i4gwl/gonum_3_years_and_a_bit_old/

[Seb Binet]

hi,

On Mon, May 9, 2016 at 9:35 AM, Seb Binet <seb....@gmail.com> wrote:

hi there,

one of the requests from [1] is for gonum to provide curve fitting routines.

I'd be also interested in giving it a stab.

any pointers?

so... I have been following Andrew Ng's lectures on Coursera about Machine Learning and one chapter is about linear regression using gradient descent.

I came up with this kind of code:

 func gradDesc(x, y, theta *mat64.Dense, alpha float64, n int) (*mat64.Dense, []float64) {

history := make([]float64, n)

m, _ := x.Dims()

coeff := alpha / float64(m)

var (

preds mat64.Dense

delta mat64.Dense

errs  mat64.Dense

)

history[0] = cost(x, y, theta)

for i := 0; i < n; i++ {

preds.Reset()

delta.Reset()

errs.Reset()

preds.Mul(x, theta)

delta.Sub(&preds, y)

errs.Mul(delta.T(), x)

errs.Scale(coeff, &errs)

theta.Sub(theta, errs.T())

history[i] = cost(x, y, theta)

}

return theta, history

}

this can be used to fit a data set of points:

func work(xdata, ydata []float64) {

x, y := dataAsDense(xdata, ydata)

//fmt.Printf("x=\n%v\n", mat64.Formatted(x))

//fmt.Printf("y=\n%v\n", mat64.Formatted(y))

theta := mat64.NewDense(2, 1, []float64{0, 0})

fmt.Printf("theta=\n%v\n", mat64.Formatted(theta))

fmt.Printf("J=%v\n", cost(x, y, theta))

alpha := 0.01

n := 1500

theta, hcost := gradDesc(x, y, theta, alpha, n)

fmt.Printf("theta\n%v\n", mat64.Formatted(theta))

fmt.Printf("cost: %v\n", hcost[:20])

}

func dataAsDense(xdata, ydata []float64) (*mat64.Dense, *mat64.Dense) {

var x mat64.Dense

x.Augment(

mat64.NewDense(len(ydata), 1, gen(len(ydata), 1)),

mat64.NewDense(len(ydata), 1, xdata),

)

y := mat64.NewDense(len(ydata), 1, ydata)

return &x, y

}

func gen(n int, v float64) []float64 {

o := make([]float64, n)

for i := range o {

o[i] = v

}

return o

}

now... that's a first step towards a least-squared fitting function.

but how should this be integrated with the framework provided by gonum/optimize.{Method,Problem} ?

-s

[Brendan Tracey]

If you want to do least-squares, the code is

x := mat64.NewDense(len(xdata), 1, xdata)
y := mat64.NewVector(len(ydata), 1, ydata)
var theta mat64.Vector
err := theta.SolveVec(x, y)
// handle err

We also have gradient-descent in optimize if you're going to do batch-learning (as in your code) https://godoc.org/github.com/gonum/optimize#GradientDescent.

If you are doing batch learning, gradient descent is one of the least efficient ways to solve least-squares. We do need to provide a way to do stochastic gradient descent for fitting algorithms. It doesn't belong in the base optimize, but it may in a subpackage, or just in the fitting repositiory.