blank

Hit my pre-Xmas target. Now 7th on the leaderboard - the 4th team. Stopping for Xmas.

Came across this great gadget for the Google homepage:

By Rob Kodey
Watch the Netflix Prize Leaderboard, right from your Personal Homepage! NEW: Wide Mode option. NEW: See how team scores are improving. See how teams are moving up and down the Leaderboard. User Preferences control the display. Comments, feedback, and suggestions welcome.

I can now watch my progress without leaving Google. Great.

Also started trialling Mathematica (so that I can hide my mathematical ignorance from my family!). It looks very powerful, but its going to take some time to figure out how to work it.

Presented latest model to family for mathematical analysis. Roundly condemned for its inadequacy. Back to the drawing board...

I've been wrestling with a deceptively formula for about a week and a half now. I cannot get it to yield up anything exciting (and that's before I try and code it). This project can absorb a lot of time!

Gave my daughter some more differentiation. Tested the results with some numbers in Excel - she'd got it right. It gave her great joy to apply her school maths course to a real problem and get a result. Perhaps we should figure out a way of getting all kids to solv real problems in the maths class. It might encourage maths takeup.

I think we have a new partnership. I come up with the ideas and she figures out the maths and I program the maths. It should speed things up.

Enlisted my daughter and a friend of hers to produce a mathematical function for me. Pleased to see that the A level maths course is proving useful.

Complaints about the noise of the computers at night. It looks like the proverbial garage might become a literal garage. Have toyed with the idea of putting the computer in the shed - its good and cold, but unfortunately I think it would get stolen.

One of the (perhaps) unintended consequences of the internet is the democratization of knowledge. Vast amounts (especially of academic) information is available at the click of a button. Google makes access to the information easy.

This may not be a good think. Sharing information promotes groupthink. The collaborative filtering literature maybe a case in point. There appear to be two underlying psychological models being used.

1. If you rate something similarly to someone else then you can use their predictions on unrated movies as an estimate of your rating for that movie plus some noise.
2. Your preference for a movie consists of a set of (orthogonal?) preferences for different factors within the movie multiplied by the amount of that factor within the movie plus some noise - (if I've understood the matrix factorization stuff correctly).

The mathematics (and there is vast reams of it) seems to revolve mainly around calcuating similarity in the first instance or working out how best to tackle the noise in the second approach.

Staggeringly, and please correct me someone if I'm wrong, I can find absolutely no discussion whatsover (not even a single paper) that discusses the merits of the two psychological models underlined in 1 and 2. Yet, at best, they are very crude and, imho wrong. One must be able to do better.

Even the competition's organisers seem to think that only the mathematical sciences are important for this problem. I came across this quote from the CEO of Netflix in the New York Times.

"Mr. Hastings said he thought it was important to make the ratings database widely available. “Unless you work at Microsoft research or Yahoo research or for Jim Bennett here at Netflix, you won’t have access to a large data set,” he said. “The beauty of the Netflix prize is you can be a mathematician in Romania or a statistician in Taiwan, and you could be the winner.”"

No mention of psychologists, or even economics only mathematicans or statisticans. Has the computer science world got itself into one big groupthink on the problem?

The new kittens have found their way to the computer. Its a nice warm spot for them to lie on. I'm not sure its very good for the ventilation. Will I be the only contestant with a kitten guard on the computer? Hmmm...

There is a fair amount of data. I can see I will have to use the games machine. Its a good job the kids are at school. I hope visual basic is upto it.

Had another look at the forums - plenty to read. Interesting, especially for what is left out - nothing about Tversky and Kahneman, anchoring, biases, choice theory, JND etc. Plenty on maths especially SVD, but all (or at least all the items I read) ignore the fact that there are humans actually making the ratings. Surely, this piece of information should be informing the way people go about trying to build their models?

The Netflix prize

Decided to take the netflix prize seriously. Looks kind of fun. Not sure where I will get to as I am not an academic or a mathematican. However, being an unemployed psychologist I do have a bit of time.