Re: Digest for stanford-classes@googlegroups.com - 2 Messages in 1 Topic

[Sumod K Mohan]

Hi Rajeev,

Did you get this fixed. If not, could you give me the pseudo-code. I haven't seen the new lectures, but have implemented gradient descent and linear regression some time back in the past. It will take me longer to actually go thru your code and make the pseudo-code to check if the math is right. I am hoping you haven't done some obvious programming mistakes. I want to help if you have some problems with the algorithm or the math, because I am sure you can fix the other bugs yourselves, if any :)

Cheers,
-Sumod

On Sat, Dec 17, 2011 at 4:29 AM, <stanford...@googlegroups.com> wrote:

  Today's Topic Summary

Group: http://groups.google.com/group/stanford-classes/topics

• Gradient Descent Algorithm [2 Updates]

 Gradient Descent Algorithm

Rajeev S <rajee...@gmail.com> Dec 16 11:24PM +0530  

The following link gives an implementation of the gradient descent
algorithm in C.
Gradient Descent for optimization of linear
regression.<https://github.com/rajeevs1992/myCodes/blob/master/gradientDescent.c>
Data set used in the
code.<https://github.com/rajeevs1992/myCodes/blob/master/data>(Binary
file,not in human readable form)
 
Dataset:
 
size | cost |
+------+------+
| 1700 | 50 |
| 1000 | 32 |
| 1280 | 30 |
| 1324 | 34 |
| 1324 | 33 |
| 1550 | 56 |
| 3500 | 250 |
| 982 | 35 |
| 2500 | 95 |
| 2600 | 85 |
| 3108 | 100 |
| 4000 | 200 |
| 1500 | 24 |
| 1900 | 42 |
| 1713 | 45 |
| 2000 | 45 |
| 2200 | 55 |
-------------------
The algorithm is as per the Stanford University Machine Learning course,
but it fails to fetch proper outputs because the parameters become infinity
(nan) on execution.(both in c and python).
 
Is the problem with the implementation of algorithm or choice of language
or the dataset?
 
--
*Regards,
Rajeev S*
*B Tech CSE Student*
*Government Engineering College,Thrissur*
*http://rajeevs.tk* <http://rajeevs.tk>


 

"Pramode C.E" <ma...@pramode.net> Dec 17 07:52AM +0530  

> Gradient Descent for optimization of linear regression.<https://github.com/rajeevs1992/myCodes/blob/master/gradientDescent.c>
> Data set used in the code.<https://github.com/rajeevs1992/myCodes/blob/master/data>(Binary
> file,not in human readable form)
 
I have not looked at your code ... but just one hint. You have to be extra
cautious when doing
floating point arithmetic; it is inherently tricky because of the inexact
nature of the computations
involved:
 
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
 
Sometimes, the problem may not be with your logic - but with the way errors
accumulate during
float arithmetic.
 
-pramode
http://pramode.net


 

You received this message because you are subscribed to the Google Group stanford-classes.
You can post via email.
To unsubscribe from this group, send an empty message.
For more options, visit this group.

[Rajeev S]

Thanks for replying.I have not fixed the issue.

Here is the pseudo code

Gradient descent for linear regression:

1.procedure gradientDescent(float x1,x2):

2.    temp0=x0-alpha*(derivative of J(x0,x1) wrt x0) 

3.    temp1=x1-alpha*(derivative of J(x0,x1) wrt x1)

4.    if |temp0 - x0|<.001 and |temp1-x1|<.001 :

5.        declare convergence and print temp0,temp1

6.        exit

7.    else

8.         gradientDescent(temp0,temp1)

This is a bit different from the code in the previous mail,but the result is same.

--
Regards,
Rajeev S

B Tech CSE Student

Government Engineering College,Thrissur

http://rajeevs.tk