Gradient Descent Algorithm

[Rajeev S]

The following link gives an implementation of the gradient descent algorithm in C.
Gradient Descent for optimization of linear regression.
Data set used in the code.(Binary file,not in human readable form)

Dataset:

size | cost |
+------+------+
| 1700 |   50 |
| 1000 |   32 |
| 1280 |   30 |
| 1324 |   34 |
| 1324 |   33 |
| 1550 |   56 |
| 3500 | 250 |
|  982  |   35 |
| 2500 |   95 |
| 2600 |   85 |
| 3108 | 100 |
| 4000 | 200 |
| 1500 |   24 |
| 1900 |   42 |
| 1713 |   45 |
| 2000 |   45 |
| 2200 |   55 |
-------------------
The algorithm is as per the Stanford University Machine Learning course, but it fails to fetch proper outputs because the parameters become infinity (nan) on execution.(both in c and python).

Is the problem with the implementation of algorithm or choice of language or the dataset?

--
Regards,
Rajeev S

B Tech CSE Student

Government Engineering College,Thrissur

http://rajeevs.tk

[Pramode C.E]

On Fri, Dec 16, 2011 at 11:24 PM, Rajeev S <rajee...@gmail.com> wrote:

The following link gives an implementation of the gradient descent algorithm in C.
Gradient Descent for optimization of linear regression.
Data set used in the code.(Binary file,not in human readable form)

I have not looked at your code ... but just one hint. You have to be extra cautious when doing

floating point arithmetic; it is inherently tricky because of the inexact nature of the computations

involved:

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Sometimes, the problem may not be with your logic - but with the way errors accumulate during

float arithmetic.

-pramode

http://pramode.net