Archive | October, 2012

SPC (self-powered commute)

27 Oct

Week before last week: commuted 18 miles by feet, 12 miles by bike
Last week: commuted 24 miles by feet, 6 miles by bike
This week: commuted 30 miles by feet, saw beautiful rainbow one day, enjoyed blue sky, snow like cloud, fresh breeze(in the morning), and wonderful sunset.

Two good courses in coursera

27 Oct

1. Probabilistic Graphical Models by Daphne Koller

2. Neutral networks by Geoffrey Hinton


16 Oct


Best way to learn GLM

14 Oct

Question: What is the best way to learn GLM (or logistic regression …)?
Answer: If you know R, and have used lm for linear regression, then the best way to learn logistic regression is to derive its formula/algorithm and implement it using lm.

Use neutral network for unsupervised learning

14 Oct


Main idea:

Neural net can be used for predictive modeling: train a model with
(x_1, y_1), ... , (x_n , y_n )
, then we can use the model to predict y for a new x.

Now if in our training data, we have
x_1 = y_1, ... , x_n = y_n
, for example, if x is an image, y is the same image, then the trained neural network is simply to predict an identity function. Seems pretty boring, right? Not necessary, if our neutral net is a sparse model (i.e., most parameters are zero or most nodes are inactivated), then the neutral net actually output a sparse representation (i.e coding) of x.

This tutorial gives interesting examples in image and audio coding.

For this sparse coding to work well, it is expected that there must be some structures in the training data, which is true for natural images and human voices.