Question on the Netflix example

[DeanS]

So far in the course, we have always assumed that a human could determine the various factors which go into the solution. Eg. each movie has the dimensions for comedy, action, lead actor, etc. From these factors, we formed X and based on X, the data gave Ys and ultimately the weights.

However, there are areas where human experts cannot agree on what are the relevant factors. I wonder if there are algorithms where the computer determines the relevant factors from the data. Eg., the only data we have on movie rentals is the names of the customers, what movies they actually rented (possibly with a rating from 0 - 10 of their own like or dislike of the movie), and possibly what movies the declined to rent after having it suggested.

As an example, if I have a friend who I know generally likes the same movies I like, then if he/she rented a new movie and liked it, I would probably rent it regardless of any external classifications of factors.

Thanks.

[yaser]

Originally Posted by DeanS

So far in the course, we have always assumed that a human could determine the various factors which go into the solution. Eg. each movie has the dimensions for comedy, action, lead actor, etc. From these factors, we formed X and based on X, the data gave Ys and ultimately the weights.

However, there are areas where human experts cannot agree on what are the relevant factors. I wonder if there are algorithms where the computer determines the relevant factors from the data. Eg., the only data we have on movie rentals is the names of the customers, what movies they actually rented (possibly with a rating from 0 - 10 of their own like or dislike of the movie), and possibly what movies the declined to rent after having it suggested.

As an example, if I have a friend who I know generally likes the same movies I like, then if he/she rented a new movie and liked it, I would probably rent it regardless of any external classifications of factors.

What you are describing is precisely the algorithm given in the lecture. The meaning of the factors is not predetermined by humans. The factors in the algorithm are generic and initialized randomly. They only assume a "meaning" after training on ratings data. That meaning will not necessarily be along any coordinate that we humans understand. The comedy and other factors were only examples to make us understand how factors translate into ratings, but none of these meaningful factors is introduced in the algorithm.