NICE! Pitt email group?

[CrazyCarl]

Anyone else out there getting this?

Im going to be around Tuesday Wednesday and Thursday

im up to nerd it up if anyone else is interested!

[Kamal Gupta]

Its just me for now! I am waiting for more material to be available online. Next week might be better.

--

Kamal

[carl crott]

sweet

so whats your story man? I'm all kinds of cool w meeting new people .. especially when we can talk MEANINGFUL shit

are you also taking the machine learning class?  I'm getting the operations ... in a manner of do 1...2....3 

but say matrix transposition as used to get the minimal cost function .. im not so sure as to the WHY


howd you like the homework??

-Carl

--
Carl Crott

Web Applications Engineer

www.black-glass.com

412-610-0600

[Kamal Gupta]

Hey

I am interning in CMU for a year and taking up both the AI and ML classes to improve my weak maths. I am working in robotics, so I have a bit of background in both the subjects :). 

I'd say they kept the first hw easy so as not to freak out students. I haven't started off with ML yet, waiting for the right motivation.

P.S. I am new in US (~4 months), and I have a very little idea of working of a Study Group. So let me know how do you wanna go about it. My contact number is 412-951-1887

--

Kamal

[Kamal Gupta]

hey hey

I guess we both got too much busy :) I am kinda doing okay with both ai and ml. still trying to keep up with hw assignments along with the work. I have a packed schedule for this month as well. Leaving for India next week and coming back in the beginning of December. 

How about you? Still following the courses right? Thought I should say hi once before the course is finished :)

(P.S. You might receive 2/3 copies of this email. My machine has gone crazy with some internet error)

- Kamal

[carl crott]

Kamal,

thanks for the email man!  

I've ignored the AI class as the ML is more what im after .. but currently matrices aren't making too much sense ... so I'm posting up in the forums to try and get a simple working transformation as to how gradient descent is implemented in a linear regression.

This is homework 1 =/

Do you happen to have a working octave solution for H1 ex 2.2.4 the gradient descent?

I keep asking in #ml-class but everyone is trying to teach me their method and thats a pain in the ass!

-Carl 

[Kamal Gupta]

There you go. (Hope you won't email Andrew Ng for me breaking the honor code :) ) I am sorry I haven't documented the code or anything. But I have been using matlab for past 3 years and also did know the fair amount of what had been taught in the class till now. So I can help you debug the code/matrix problems any time on the email but I am not sure if I would have much of a time till 28th of Nov.

- Kamal

[carl crott]

haha so turns out i THINK i just solved it !

and actually there are a few otehr HW1s which are online 

but yeah lemme check how you ran this.

if Its not too personal .. can I ask what you highest math has been?

-Carl

[carl crott]

Oh wait haha this is hw 2 ... im after hw1 !

[Kamal Gupta]

Ohh my bad. and I am not sure what do you mean by "highest maths"? I did study maths in high school and college. (I am in Senior Year of my bachelors right now)

- Kamal

[carl crott]

as in like what was your most difficult math class ... 

I've done all this stuff but just more rusty than I'd like to be... time to revisit more of the linear algebra reviews!

[carl crott]

so this makes sense ... save for the fact there is no "sum" operation

yours:

theta = theta - alpha/m*((theta'*X'-y')*X)';

mine ( non functional ):

theta = theta - alpha * (1/m) * sum( ( X*theta) -  y   ) *   X

why / how does yours not have a "sum" operation?

[Kamal Gupta]

Closer look at the update equation. Lets call ( X*theta) -  y   )  as the error vector with size m*2 (if we have m examples and say 2 dimensions). The summation part of equation 

• takes jth dimension i.e. X(:,j) since our X is m*2
• multiply each of error vector with X(:,j). 
• sum the product (for that particular dimension). Equivalent to taking dot product

Only bug in your code is that you take the sum of error vectors and then multiply this scalar with X (which is not the gradient).

Lets see. Most of my courses revolve around mathematics. I did Linear Algebra in the freshman year and Probability in sophomore. My statistics sucks and I learnt most of it from a course called Signal Theory.

- Kamal

[carl crott]

So I understand the specifying I and j but I dont see that in your equation

[Kamal Gupta]

error = (theta'*X'-y') gives me 1*m error vector. When I do a matrix multiplication by X vector (which is m*2), I'll get 2 elements 1st of which is sum (error.*X(:,1)) and second is sum(error.*(X(:,2))). So here I make use of vectorization in matlab(/octave)

- Kamal

[carl crott]

Damn I was up in pitt this weekend wanted to get coffee!

So one question on sum being equal to the dot product.. Could you give an example.. And dot product..you mean multiplication?

Also when this stuff starts to click I'd love to do a google hangout for anyone who wants to get a really good understanding and maybe do some face to face chats

thoughts?

[Kamal Gupta]

Not sure if I get your question right

So if A = [1 2 3] is a 1x3 matix and B = [4 7;5 8;6 9] is a 3x2 matrix

B(:,1) = [4;5;6] and B(:,2) = [7;8;9]

A*B = [(1.4+2.3+3.6) (1.7+2.8+3.9)] = [28 50]

dot product = sum of product of elements of two vectors of same dimensions. If C is 3x1 and D is 3x1 matrices, then dot(C,D) in matlab gives me the dot product. dot(C,D) = C'*D = sum(C.*D)

I am sorry I made a typo earlier, error vector ( X*theta) -  y   ) is m*1 vector. error' is 1*m. So if X is m*2

error'*X = [error'*(X:,1)  error'*X(:,2)]

Does that help? I can meet you up if you are around CMU or U.Pitt after Dec 5 when I come back from India.

- Kamal

[carl crott]

I think I've come up with a workaround which lets me understand the operations which were made to the matrices ...

basically when working with non square matrices .. they can only be multiplied in a single direction .. thus you can transpose them all around but the product will still be the same

ALSO i just finished the normalizing on ex1...

i was talking in #ml-class and THIS is SUPPOSED to work:

X_norm = ( X(1:end,:) - mu ) ./ sigma

which was the original implementation I used but since its only active on dev .. well I had to settle for this:

  X_norm = bsxfun(@minus,X,mu)

  X_norm = bsxfun(@rdivide,X_norm,sigma)

but yours is quite elegant ... good solution =]

-Carl

PS yeah lets plan a meetup!

[carl crott]

kamal

How are things?? I believe CMU has completed classes for the semester?? I went back through out emails and you had mentioned robotics.

I'm not sure if I had mentioned that I've been working on some of my own =]

Let me know if you're still in town I'd love to pitch the idea and get some feedback on it and see what you're up to.

-Carl

[Kamal Gupta]

Hey

I certainly am in Pittsburgh but just for 1 more month. I wasn't taking any classes at CMU :). I am just on a 1 year intern in Robotics Institute. It would be great to learn about your work. I will be around campus this weekend. I am good to go anywhere around Oakland/Squirrel Hill/Shadyside. Do you have some place in mind?

- Kamal

[carl crott]

Oakland is a fav spot of mine to geek out and get coffee... but! I think there is a coffee shop at CMU in the gates building 

I say we geek out at CMU

good times today?

-Carl

[Kamal Gupta]

I would be free sometime tomorrow evening. How about 5pm tomorrow Gates 3rd floor (at the coffee shop). My phone number is 412-951-1887

- Kamal

[carl crott]

Ahhh I need to be in greensburg @ 5:30 ... Anytime earlier in the day?

-Carl

[Kamal Gupta]

I'm sorry I'll be busy with a couple of project meetings and a seminar before 5. How about next week?

- Kamal

[carl crott]

Sure thing kamal. Lets talk next week.