What are the commons and differences among Density Estimation, Gaussian Process Regression, KNN?

[Yitong Zhou]

I kind of feel like these methods seem to have common and even I am wondering they are actually talking about the same thing. But when to use what? And what are the pros and cons of each method? For example, I am wondering whether it is possible to replace kNN with GPR while try to introduce more advanced kernel at the same time.

Thanks,

Yitong Zhou

[Barnabas Poczos]

Density estimation is an example for unsupervised learning (you have inputs x_1,...,x_n and want to estimate the density p(x) at some point x). Regression (e.g. Gaussian Process regression) is an example for supervised learning: from (x_i, y_i) i=1...n input-output pairs you try to learn a function f such that f(x)=y (or to be more precise f(x)=E(y|x)).

kNN, Parzen-kernels, and kernels in RKHS are tools for these two problems. You can use them both for density estimation (unsupervised learning) and regression/classification (supervised learning).

It is a difficult question when to use what. It depends on what is important for you: computational complexity, accuracy, robustness, simplicity, easy implementation, etc.


 

--
http://alex.smola.org/teaching/cmu2013-10-701 (course website)
http://www.youtube.com/playlist?list=PLZSO_6-bSqHQmMKwWVvYwKreGu4b4kMU9 (YouTube playlist)
---
You received this message because you are subscribed to the Google Groups "10-701 Spring 2013 CMU" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 10-701-spring-201...@googlegroups.com.
To post to this group, send email to 10-701-spri...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.