tag:blogger.com,1999:blog-24594173062680016482024-03-23T03:16:20.979-07:00Just a guy in a garageJust a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.comBlogger31125tag:blogger.com,1999:blog-2459417306268001648.post-81173850571831313752011-07-14T03:22:00.001-07:002011-07-14T04:37:12.561-07:00Supercomputing made super easyI don't want to harp on about the same subject twice but I think what Hybrid DSP have achieved is something very significant. There are two basic problems when analysing very large datasets:<div><ol><li>Writing the code correctly.</li><li>The performance (i.e. speed at which it runs). </li></ol><div>The solution to the first problem is to use an easy to understand language that you are familiar with that enables you to write code at a high level. The solution to the second problem is to use the graphics card on your pc (or buy one for a couple of hundred pounds/dollars). Up until now, however, you couldn't do the two together - either you used a high level language and lost out on performance or you used a much more complicated and harder to understand language (Nvidia C) and got the performance gains from the graphics card.</div><div><br /></div><div>Hybrid DSP's solution neatly solves this conundrum. By taking your code and compiling it into Nvidia C automatically we can have the best of both worlds. This opens up huge opportunities across a wide range of different problems especially in the field of data analysis. </div></div><div><br /></div><div>For example, based on the results of the data mining competitions that I have entered, it seems inevitable that if you really want to squeeze the last ounce of accuracy from a forecasting technique you have to use machine learning approaches. These, however, have the very significant disadvantage that they almost always take a very long time to run. There isn't, though, a ready supply of GPGPU programmers available (at least not at a reasonable price point) who can convert accurately and fast working code into code for a GPGPU. </div><div><br /></div><div>Now by using Hybrid DSP's product you can make a few minor tweaks to your existing code and get the benefits. To see just how simple it is you can read my post which contains so a description of how I converted some Kmeans code to run on the GPU. <a href="http://www.codeproject.com/KB/Parallel_Programming/UsingCudafyGPGPUProg.aspx">http://www.codeproject.com/KB/Parallel_Programming/UsingCudafyGPGPUProg.aspx</a></div><div><br /></div><div>So what this seems to give us (according to those who do these kind of comparisons) is a computer equivalent to the world's fastest supercomputer in 1996 sitting under our desks for around $300 that can be programmed in any of the Microsoft .Net languages (used by millions of programmers around the world). Thats got to introduce some signifcant new ways of doing things and problems that can be solved.</div><div><br /></div><div>(Just for the avoidance of any sort of doubt - I have no financial or any other relationship with Hybrid DSP except that I use their product.)</div><div><br /></div><div><br /></div>Gavinhttp://www.blogger.com/profile/17163367164018928050noreply@blogger.com0tag:blogger.com,1999:blog-2459417306268001648.post-36254015415704686042011-07-08T00:30:00.000-07:002011-07-08T14:32:54.979-07:00Massively parallel programming and the GPGPUI don't often delve into the deep technicalities of programming, but I was running a model that took 4 days to finish and decided there must be a better way. The model was already optimised, running in parallel etc, so there was nothing for it but to have a go and using a graphics card to do the computation.<div><br /></div><div>A quick Google suggested mixed views. Clearly in the best cases speedups of 40 or more possible, but that was after some pretty complex optimization (exactly what memory you use on the card and how you access seemed to play a big part in the gains that you get). However, 4 days was not really satisfactory.</div><div><br /></div><div>Being a die hard .net user and not really willing to learn anything new a bit more Googling revealed a number of potential ways of automatically converting .net code to work with the graphics card. I had a look at 3 or 4 different methods and the one I chose is Cuda running on a Nvidia card (as it is by far the best supported GPGPU as far as I can tell) and Cudafy (available from <a href="http://www.hybriddsp.com/Products/CUDAfyNET.aspx">http://www.hybriddsp.com/Products/CUDAfyNET.aspx</a> (free) as it seemed the most elegant approach.</div><div><br /></div><div>The effort wasn't too bad - Cudafy automatically translates .net code into something that can run on the graphics card so all you need to do is to figure out how to get Cudafy to work. It took about a day (I'd like to see a lot more documentation) but it worked pretty much as it claimed. - various little issues, like checking for integer overflows is switched on in the Visual Studio compiler and needs to be switched off when generating the graphics code took a bit of time to uncover but now they are sorted they will remain sorted. Hats off to Hybrid for producing a very easy to use product.</div><div><br /></div><div>Converting your code is also pretty trivial - Provided that its written to be analysed in parallel in the first place that is. You add a variable that tells the "kernel" (sub routine in more traditional language) which thread is running the kernel and what you want each thread to do and let rip. Its possible to generate millions of threads and the graphics card will schedule them all over the multiple processors that it has available (240 in my case).</div><div><br /></div><div>The results are amazing - my out of the box conversion (no memory optimizations or anything fancy just a default translation of my code) produced about a 20 fold increase in speed over the .net version. I reckon that by being a little bit fancy you could probably double the speed on the graphics card. - a day well spent. It will definitely become my method of choice for running large models and datasets.</div>Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com21tag:blogger.com,1999:blog-2459417306268001648.post-74317332813224219732011-06-29T05:48:00.000-07:002011-06-29T06:22:10.748-07:00Recommendation system competitionsHaving met (physically) in the last month 3 people who were kind enough to say that they read my blog - I reckon I ought to spend a little more time keeping it up to date.<div><br /></div><div>This entry describes my efforts in the Kddcup 2011 competition in which Yahoo kindly supplied some 300m music ratings. No Netflix $1m at the end of it, but considerable kudos if you can win.</div><div><br /></div><div>From the outset I decided to limit myself to a single method (stochastic gradient descent of "real world" models of human rating behaviour since you ask). As the only psychologist (at least who has come out so far) entering these types of competition I'm determined to try and drive through approaches based on new models of human behaviour rather than try and improve a computer science approach to the problem. In addition, I set myself the target to try and create a single model that would perform close to the ensemble methods that won the Netflix prize. (mainly I must admit because I didn't have time to try out a variety of approaches).</div><div><br /></div><div>I guess my biggest discovery - which was enough to propel me to 5th place at one stage of the competition was that ratings are not made in isolation but are made relative to the other items being rated at the time and the ratings that they receive. So one simple effect is the anchoring effect well known to behavioural (behavioral I guess for the US) economics. i.e. if you give a high value to one rating, you are likely to give a high value to the next one regardless of how much you actually like it and vice versa. This effect can plainly be seen in the Yahoo dataset as there is a very high correlation between ratings - even when adjusting for all other effects. So unless Yahoo were feeding items to be recommended in a particular order, this would seem to provide significant support for the existence of such an effect, In fact, without knowing anything at all about preferences for different items you can achieve a score of around 24.5 rmse on the validation set simply by exploiting this fact. </div><div><br /></div><div>In addition, there are other such anchoring type effects. As a thought experiment try rating anything on (say) 5 different criteria. The question to ponder - is did you at any stage consider the relative ratings that you gave to the different criteria rather than the absolute values. This relative effect led to the development of a series of moving average models. </div><div><br /></div><div>Rather than trying to predict preferences in isolation of previous items, I developed a series of models that compared the moving average score and the predicted moving average of the preferences to an individual items preferences and score. So if, for example, the last 5 items had an average score of 75 and calculated preferences of 70 and the item I was trying to predict had a calculated preference of 90 then I simply multipled 75 by 90/70 (or depending on the model tried) added (90-70) to 75. The advantage of this approach is that it eliminated many of the issues to do with how preferences change over time (do they change over time? - or is it just the score you attribute to a preference that changes - it still seems an open question to me) and led to a single model score of 21.5 rmse on the validation set. - alas not translated to the same score on the test set. I think, however, that is probably better than most single models based on SVD which was the preferred approach during the Netflix competition - It was certainly better than the tests that I ran using some of the standard Netflix approaches.</div><div><br /></div><div>Anyway with the time available I realised I couldn't catch the leader - hats off to them for a very good score almost from the outset - it will be interesting to hear what they did. So that's where I left it.</div><div><br /></div><div><br /></div>Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com3tag:blogger.com,1999:blog-2459417306268001648.post-80614845088559073672010-08-29T22:38:00.000-07:002010-08-29T23:31:18.632-07:00The Singularity has arrivedThere is increasing talk of the possibility of a singularity where super human intelligence is created. One of the first people to suggest this was Vernon Vinge who stated in 1993 <br />"Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended." These themes have been taken up by other futurologists such as Kurzweil in his recent book "The singularity is near". Now this may seem the stuff of science fiction, but I'm not so sure.<br /><br />I was struck the other day at just how "intelligent" Google was. I asked it two questions "Can I burn Leylandii (a type of wood)" and what was "Mary Hopkin's greatest hit". For both these questions the answer was contained within one of the top 3 web pages that was returned. It struck me that the ability to answer questions and return pages containing the answers is indeed a form of intelligence.<br /><br />Lest you need further convincing, let me ask you a question. If you were playing "Who wants to be a millionaire", would you rather a lifeline that enabled you to "ask a friend" or a lifeline that gave you 45 seconds on Google to find the answer. I'd wager that most people would prefer to ask Google. <br /><br />If the knowledge of facts is important to man (what did we do at school for all those years if facts aren't important) and the definition of intelligence, then a simple test of man vs machine would be to ask a series of factual questions to an individual and to a search engine and see whether the individual can answer more questions correctly or the search engine produce the answer in (say) the top 3 web pages it returns.<br /><br />I agree this isn't the Turing Test (a test to see if you can distinguish between man and machine), but it seems indisputable that Google can provide the answers to a wider range of questions than any living human being, and in one sense, therefore, is superhuman. We need to recognise that it is not necessary to ape human behaviour and human methods to create something superhuman.<br /><br />Interestingly, this has not - as these authors would predict, come about as a result of some form of artificial intelligence but as the direct result of millions of individuals contributing their knowledge to a common data source (the internet) and some clever (but not that clever) data search and retrieval mechanisms. It is the record volumes of data and its accessability that has created this 'intelligence' not a fundemental breakthrough in artificial intelligence or indeed any other scientific discipline.<br /><br />It also suggests that even minor improvements in data search, analysis, mining and retrieval could be of immense value when multiplied by the vast volume of data accessible through the internet. If we can improve the value of each piece of data stored on the internet even slightly by making it more relevant and accessible to the appropriate queries, when multiplied by the number of data items that exist, huge amounts of intelligence will be created.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com2tag:blogger.com,1999:blog-2459417306268001648.post-66427197967135471932009-09-28T09:33:00.000-07:002009-09-28T09:43:31.134-07:00Presented at idate 09 last week. A fascinating conference. Even a relatively new industry like online dating is in the process of being heavily challenged by new ideas. Dating sites are going to have to choose what to do next from amongst others: integrating with social networks, implementing mobile offerings, implementing personalisation techniques (the basis of my talk), implementing location based offerings etc. Constant change - standing still is not an option, prioritization is a nightmare.<br /><br />It was also interesting for one other insight (at least to me). The biggest company in the online dating scene is undoubtly Google - who take a huge percentage of the total value from the industry value chain. The cost of acquisition is by far the largest cost and Google advertising is the dominant method of acquisition. I'd love for someone to crunch the numbers - my guess is that they would be astonishing.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com52tag:blogger.com,1999:blog-2459417306268001648.post-63345705954458654802009-09-14T00:55:00.000-07:002009-09-14T01:13:07.729-07:00Recommendation systems for datingI'm very excited to announce a new spin-off from the Netflix competition. Online dating. For the last few months, I've been working with a dating expert (Nick Tsonis) to see if we can improve the way in which people find dates.<br /><br />Well, our first dating recommendation system went live last month. <a href="http://www.yesnomayb.com/">www.yesnomayb.com</a> Its early days to calculate whether its adding to the sum of human happiness, but first results are very, very promising. Its already taken over as the main method of finding potential dates on the site. Even with its relatively rudimentary implementation, it's preferred roughly 60/40 over the more traditional search mechanism (I'm a boy looking for a girl aged between 25-30, non-smoker etc etc).<br /><br />What this suggests, in the first instance, is that when searching for hedonic items (i.e. those chosen on the basis of the pleasure they might bring (books, music, dates etc )), its very difficult to describe to a search engine exactly what you are looking for. Discovery processes based on analysing yours, and everyone elses, actual behaviour provide a better method of getting you to your desired target.<br /><br />With further launches on other dating sites planned for September, October and January - we should be able to start to collect even more data on what works best in helping people find their ideal dates and, hopefully, make a sea change to the way online dating works.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com6tag:blogger.com,1999:blog-2459417306268001648.post-78413658745826199732009-07-27T01:35:00.000-07:002009-07-27T02:22:50.734-07:00Reflections on the Netflix Competition<span style="font-weight: bold;">Thanks and Congratulations</span><br /><br /><span style="font-style: italic;">1. First and foremost to Netflix</span> for organising such a well designed competition. It was run in an exemplary fashion throughout and should, I believe, become the model for other competitions that people might choose to run. Some of the key features that made it such a success are:<br /><br />a. A clear, unambiguous target and challenging target. How a 10% target was chosen, will I suspect, remain forever a mystery but it was almost perfect - seemingly unattainable at the beginning and difficult enough so that it took almost 3 years to crack - but not so difficult as to be impossible.<br />b. Continuous feedback provided so one could identify whether the approaches you were investigating were going in the right direction.<br />c. A forum so that the competitors could share ideas and help each other (more about that later).<br />d. Conference sessions so competitors could meet and discuss ideas.<br />e. Zero entry cost (apart, of course, from the contestant's time). <br />f. A clear set of rules.<br /><br /><br /><span style="font-style: italic;">2. Brandyn Webb a.k.a. Simon Funk</span> For early on giving away in complete detail one of (at the time) leading approaches to the problem, thereby opening up a spirit of co-operation between the contestants.<br /><br /><span style="font-style: italic;">3. The contestants</span> Despite the prize of a $1million dollars, the competition was conducted in a spirit of openeness and co-operation throughout with contestants sharing hints, tips and ideas in the forum, through academic papers and at the conference sessions setup to discuss approaches. This undoubtly helped us all progress, and made the process a whole lot more enjoyable.<br /><br /><span style="font-style: italic;">4. And of course, the winners</span> for driving us all forwards and keeping us targeted on trying to improve and getting to the target first. As all of us who tried, we know it wasn't easy.<br /><br /><span style="font-weight: bold;">Was the competition worth it?</span><br /><br />There will, undoubtly be, some discussion about whether the science generated was worth the $1million plus untold researcher and other time trying to achieve the goal. I think the answer to this is unambiguously yes because:<br /><br />a. The competition has trained several hundred, if not more, people how to properly implement machine learning algorithms on a real world, large scale dataset. I'm not sure how many people already have these skills, but I would be prepared to bet that the total pool of such ability has widened considerably. This can only be a good thing.<br /><br />b. It has widened the awareness of machine learning techniques and recommender systems within the broader business community. I have had many,many requests from<br />businesses asking how to implement recommender systems as a result of the competition and I guess other competitors have too. The wider non machine learning community is definitely looking for new applications (see my previous posts for some examples) and this can only be good for the field as a whole.<br /><br />c. It has improved the science - I leave it to the academics to argue by how much, but it is certainly true that matrix factorization techniques have been the runaway success of this competition- Marrying such techniques with real-world understanding of the problem (incorporation, for example, of date and day effects) have provided by the far the most effective single technique - Such techniques, it seems to me, now need to be applied to a much wider set of problems to test their general applicability.<br /><br />d. It has gifted the research community with a huge dataset for analysis as computer scientests, statisticians and I hope, from a personal perspective, as psychologists and behavioural economists too. It was a disappointment to me that I'm still the only contestant as far as I'm aware from a social sciences background. This is, almost undoubtly, the world's largest set of data on repeated decision making and ripe for analysis. The analysis may not win the competition, but it sure should provide some insights into the way that humans make decisions.<br /><br />e. It was a lot of fun. I certainly enjoyed it, and I get the impression that most of the other contestants did too.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com4tag:blogger.com,1999:blog-2459417306268001648.post-5095581797498856762009-07-26T22:31:00.000-07:002009-07-26T22:52:12.231-07:00The Netflix prize winnerI'll post some reflections on the Netflix prize at a later date, but as someone who has been with the competition since the beginning I thought it might be useful to explain why the second place team on the leaderboard Bellkor's Pragmatic Chaos are almost definitely the winners.<br /><br />The reason is that there are two datasets against which every competitor is judged. The first is the Quiz dataset - the results of which are reported back to the competitors and appear on the leaderboard and a second dataset which is called the Test dataset which is actually used to determine the winner. The purpose of this is to stop what is called "overfitting", i.e. using the results you achieve on the Quiz dataset every time you make a submission to figure out the actual values. Now with 1.5 million datapoints its impossible to figure out each value, but I'm sure both teams used some of the information from the results on the quiz dataset to work out the optimal combination of numbers to contribute - Given that the teams are separated by less than a one point difference in the fourth decimal place only a very small amount of overfitting could cause the positions to switch and in this case it looks like Bellkor's Pragmatic Chaos overfitted slightly less than "The Ensemble" and hence according to the posts on the Netflix forum are the ones to be validated.<br /><br />There is a final stage that needs to be gone through and that is validation where the top team have to demonstrate how they achieved the results and to publish how they did it - but given that Bellkor have been through this twice before on the progress prize it should be a formality.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com3tag:blogger.com,1999:blog-2459417306268001648.post-11554965583140820782009-06-29T03:00:00.000-07:002009-06-29T03:17:59.911-07:00After NetflixWell - a (combined) team has finally managed to get to the finishing line - many,many congratulations to them. I must admit I feel a mix of regret not to be slightly further up the leaderboard and relief that I can now (bar a few desperate throws of the dice) concentrate on taking the learnings from Netflix elsewhere.<br /><br />The competition has been very good to me, and I'm now engaged on a variety of projects trying to leverage the skills learnt including: <br /><br /><ul><li>Producing a film and television recommendation system http://marketingfeeds.nl/TechCrunch/2009/06/03/beeTV_Raises_$8_Million_For_Stunning_Personal_TV_Recommendation_System<br /></li><li> Working for a number of dating agencies http://www.onlinepersonalswatch.com/news/2009/04/gavin-potter-and-nick-tsinonis-founders-of-intro-analytics.html trying to help them identify compatible people - the interesting twist here is that as well as the person having to like the movie the movie has to like the person as well - if you see what I mean)<br /></li><li>Identifying who might have to go to the accident and emergency department of a hospital so that careplans can be put in place to reduce the likelihood of an emergency admission thereby reducing costs and improving patient satisfaction. (the movie equivalent here is the treatments they received in the last year).</li><li>Working on a project to predict the prices of ... (I'm afraid I can't talk about this one just yet).<br /></li></ul>The interesting thing is, that in all these cases, the application of the Netflix algorithms makes a substantial improvement over the status quo - I think the learnings from the Netflix competition have enormous applications both within recommendation systems and elsewhere. Hats off to Netflix for producing such a valuable advance in both the science (and probably more importantly) the number of people who can now tackle these kinds of problem.<br /><br />If any of the Netflix contestants are interested in working on "real problems" please don't hesitate to get in touch. I've more work than I can handle at the moment.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com4tag:blogger.com,1999:blog-2459417306268001648.post-85416545893120483102009-06-19T09:13:00.000-07:002009-06-19T09:38:16.075-07:00How Netflix predicts the price of wine<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfz-kFy_ESd3iqAgBvubnRzTTTqs8Oo7SeYyGANkxGIq1n9rTG7Mq-tDl6q1rXFxVLi_-hgNa4cXAWbPFPiS-t9ECbY19b4_1QcOcCDuxR1d7toCx6L8wADr_dN_xbWs_kB16WGengq4eP/s1600-h/wineprices.jpg"><img style="cursor: pointer; width: 320px; height: 259px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfz-kFy_ESd3iqAgBvubnRzTTTqs8Oo7SeYyGANkxGIq1n9rTG7Mq-tDl6q1rXFxVLi_-hgNa4cXAWbPFPiS-t9ECbY19b4_1QcOcCDuxR1d7toCx6L8wADr_dN_xbWs_kB16WGengq4eP/s320/wineprices.jpg" alt="" id="BLOGGER_PHOTO_ID_5349075242792800418" border="0" /></a><br />I, and I know, a number of others are beginning to be sidetracked into other things that we might do with the knowledge that we have garned from the Netflix prize whilst we let our betters (go for it Pragmatic Theory) battle to get that last little of rmse that will land them the $1million prize.<br /><br />I'll publish a number of ideas that I've been involved in over the last year. One that has surprised me is the ability to predict the price of a wine from comments collected from the web. Its early days yet, but a project that I've been involved in is looking to see whether we can predict the price of clarets (ranging from $3,000 a case to $300 a case) based solely on wine reviews.<br /><br />Slightly surprisingly this is working very well. The picture above shows the fit (in £(UK)) of the price of around 100 wines to their actual values. In Netflix terms the rmse of the prices is around £370 a case (once the mean price is subtracted), once you include the contributions from the words the rmse falls to around £140 a case, so slightly over half the variance can be accounted for.<br /><br />What is also interesting is some of the key words that indicate a high price. These words are in order of importance with the words at the bottom of the list being negative indicators of price.<br /><br />woody <br />pencil <br />hard <br />fat <br />complex <br />spicy <br />tannin <br />cherry <br />smoky <br />fragrant <br />green <br />elegant <br />soft<br />balanced <br />tobacco <br />fruit <br />oak <br />blackberries <br />fruity <br />lingering <br />flabby <br />expressive <br />aromatic <br />Jammy <br />smooth <br />thin <br />rounded <br /><br />So woody and pencil are the words to look for when choosing expensive wines from Bordeaux. Try it when you next purchase a wine, its already changed what I look for on a wine description.<br /><br />Why do it. Well its a little bit of a labour of love, to see if we can produce a system that can identify underpriced wines to buy. However, the success so far has suggested that if we can find a more liquid market (no pun intended), then there might be the potential to make some money by identifying underpriced opportunities, and we are currently exploring a few other ideas that are looking promising.<br /><br /><img src="file:///C:/DOCUME%7E1/GAVINP%7E1/LOCALS%7E1/Temp/moz-screenshot.jpg" alt="" />Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com3tag:blogger.com,1999:blog-2459417306268001648.post-13418866743176622962009-06-09T09:10:00.000-07:002009-06-09T09:32:32.052-07:00The psychological meaning of billions of parametersThe leaders in the Neflix competition have made great strides since my last post. <br /><br />Essentially my understanding is that they have done this by modelling thousands of factors on a daily basis. i.e for each person they model (say 2000) factors on an individual and individual day basis. The set of ratings provided for the competition gives enough information so that you can work out that a particular person had a particular preference of a particular strength on a particular day to watch something funny (or given that there are 2000 factors or so) something rather more obscure (maybe watch something in sepia or something). The ratings also enable you to calculate how much a film meets those requirements (again on a particular day - what seemed funny at one time period may not seem funny at another).<br /><br />By combining the two sets of factors you can then work out how a person will rate a particular movie and improve your score in the competition. This is an undoubtedly impressive feat from a statistical / machine learning viewpoint.<br /><br />It strikes me that this is also interesting from a psychological viewpoint - do we really believe that people have such nuanced preferences across such a large number of dimensions. I have an open mind about this - apriori I would have thought people would use many fewer factors in arriving at a rating decision - certainly 2000 factors (or even 20) can't all be combined consciously - the subconscious must be heavily involved. Maybe, on the other hand, there are only a few factors that we take into account - but they are different per person and the only way in which they can be explained is by taking a mix of the 2000 or so factors that are modelled. <br /><br />It strikes me that depending on your view on the above your choice of research direction on the Netflix competition, recommendation systems and indeed psychological processes in general will vary.<br /><br />I'd welcome views.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com5tag:blogger.com,1999:blog-2459417306268001648.post-50069873753920780732008-03-03T08:07:00.000-08:002008-03-03T08:18:02.108-08:00Signalling and sequencesOne of my daughter's friends suggested that sequels would, on average, recieve lower scores than the original movies - as, at least in her experience, they were invariably worse. I thought I'd just confirm her suspicions so that I could let her know that she was thinking about the problem in a good way.<br /><br />However, to my surprise the opposite appears to be true. Here is the mean score - the 0.5879992 number (adjusted for various things) for each episode of Sex in the City.<br /><br /><br />Sex and the City: Season 1 0.5879992 41138<br />Sex and the City: Season 2 0.5824835 43795<br />Sex and the City: Season 3 0.6523933 38983<br />Sex and the City: Season 4 0.7066851 34616<br />Sex and the City: Season 5 0.7359862 33380<br />Sex and the City: Season 6: Part 1 0.8097552 33532<br />Sex and the City: Season 6: Part 2 0.8241694 27914<br /><br />As you can see the later the sequel the better the result. This seems, at least to me, counter intuitive - However the answer may lie in the second number which is the number of people who rated the movie. It seems that once there have been (in this case two episodes), then people who don't like the movie drop out and don't watch any further. So although less people watch it, they give a higher average rating.<br /><br />This might be interpreted as some form of signalling. If a movie can accurately 'signal' to its potential audience that it is worth watching then the average rating will be higher. Interestingly this is - potentially - in conflict with the aim of the movie companies who might want to maximise the number of people watching irrespective of what they think of the movie (at least in the short term).Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com20tag:blogger.com,1999:blog-2459417306268001648.post-72277731778351203672008-02-25T09:30:00.000-08:002008-02-25T09:48:36.501-08:00Wired ArticleThank you all who have sent suggestions as to how to improve my score on the Netflix prize as a result of reading about my attempts in the article in the latest edition of Wired. I'm very grateful and will incorporate any that I can figure out a way of converting into a computer program.<br /><br />Just to say a little bit more about me. - I'm fascinated by the use of computers to understand how the mind works and how we can then use this knowledge to help predict human behaviour. The Netflix prize provides probably the largest dataset collected on human decision making that has been made publicly available. My attempts at the prize are based around a desire to understand how we can use such a dataset to better understand human decision making (and, of course, the outside chance of winning $1million).<br /><br />My progress to date suggests that there is at least something in this approach and I'm open to offers to work on other datasets that incorporate a human decision making element - I currently have time available. As, an example, have a look at this company that I have helped setup recently, <a href="http://www.customerfusion.co.uk/">www.customerfusion.co.uk</a> which aims to harness some of the learning from the Netflix competition to fuse market research data with customer information. Better still, if you have any market research data that you want to extract more value from - drop us an email.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com5tag:blogger.com,1999:blog-2459417306268001648.post-74300643682802686852008-02-25T00:59:00.000-08:002008-02-25T01:09:39.534-08:00Simple Heuristics that make us smartJust back from holiday. Managed to finish Gigerenzer and Todd's "Simple Heuristics That make us smart", an interesting idea in a long book. Basiclly they list a number of simple ways in which people make decisions and demonstrate that the simple methods can be as accurate, or in some cases, more accurate than sophisticated statistical techniques.<br /><br />Now if one could work out which people use when rating videos...<br /><br />As an aside - In the UK a woman called Sally Clark was convicted of killing her children based on a completely false understanding of probabilities and none of the Judge, defending lawyers, prosecution lawyers or expert witnesses picked up on the gross misunderstandings that occured. She is now dead - undoubtly as a result of the miscarriage of justice. - A 'must read' is Gigerenzer's "Reckoning with Risk". Gigerenzer provides an extremely clear introduction to the mistakes that people make (primarily doctors and lawyers) and suggests ways of presenting evidence to make sure errors don't occur. If I'm ever in front of a doctor or a judge I'm going to make sure I assess my own probabilities or give them a copy of his book.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com1tag:blogger.com,1999:blog-2459417306268001648.post-64815720399971068022008-02-06T02:15:00.000-08:002008-02-06T02:20:15.633-08:00The Korbell papersDecided to try and implement one of the Korbell algorithms. After much angst managed to get their IncFctr algorithm working (although not producing quite such good results). It doesn't seem to lead to much better results than the Funkian gradient approach but it sure is a lot faster. It must be at least 10* faster than the gradient approach. The Korbell team are to be congratulated.<br /><br />I wish I'd tried implementing it earlier - it would haved saved considerable time.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com1tag:blogger.com,1999:blog-2459417306268001648.post-44907677883563139532008-02-03T07:41:00.001-08:002008-02-03T07:42:35.829-08:00Book recommendationCame across a great book on collaborative filtering. Its called Collective Intelligence by Toby Segaran and is an excellent introduction to the field - very clear, especially for the non-mathematican.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com3tag:blogger.com,1999:blog-2459417306268001648.post-21579338340160166132008-01-14T20:02:00.002-08:002008-01-14T20:16:32.534-08:00Cult movies and TrekkiesBeen doing some work on the Netflix prize to identify whether cult followings of different movies provide more accurate estimates. The Trekkies win hands down. These are the top most similar scores produced by the model to Star Trek : Voyager: Season 1 (in order).<br /><br />Star Trek : Voyager: Season 2<br />Star Trek : Voyager: Season 3<br />Star Trek : Voyager: Season 4<br />Star Trek : Voyager: Season 5<br />Star Trek : Voyager: Season 6<br />Star Trek : Voyager: Season 7<br /><br />Such reliability - even down to the order of the seasons! Quite incredible. Even more amazing, the 26 most similar scored films are all versions of Star Trek and the 37 most similar films are all Star Trek or Stargate films.<br /><br />This is a phenomena definitely worth pursuing.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com1tag:blogger.com,1999:blog-2459417306268001648.post-85076456286957043842008-01-14T20:02:00.001-08:002008-01-14T20:02:31.378-08:00BlankJust a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com0tag:blogger.com,1999:blog-2459417306268001648.post-58887588897152616412008-01-10T05:29:00.000-08:002008-01-10T05:31:26.270-08:00Back from the holidays. Found a great new way to calculate global effects - giving me a score of .9507, better than the original Netflix collaborative filtering method at the beginning of the contest. Now can I use it to improve my overall score?Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com1tag:blogger.com,1999:blog-2459417306268001648.post-26730610248324332012007-12-30T02:27:00.000-08:002008-01-02T04:20:51.192-08:00blankJust a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com0tag:blogger.com,1999:blog-2459417306268001648.post-48576249791203230022007-12-20T05:24:00.001-08:002008-01-14T20:24:33.073-08:00Hit my pre-Xmas target. Now 7th on the leaderboard - the 4th team. Stopping for Xmas.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com0tag:blogger.com,1999:blog-2459417306268001648.post-66805380287490502492007-11-30T01:52:00.000-08:002007-11-30T01:54:43.654-08:00Came across this great gadget for the Google homepage:<br /><br />By <a href="http://www.google.co.uk/ig/authors?hl=en&author=rob.kodey%2Bnetflix.prize.module%40gmail.com">Rob Kodey</a><br />Watch the Netflix Prize Leaderboard, right from your Personal Homepage! NEW: Wide Mode option. NEW: See how team scores are improving. See how teams are moving up and down the Leaderboard. User Preferences control the display. Comments, feedback, and suggestions welcome.<br /><br />I can now watch my progress without leaving Google. Great.<br /><br />Also started trialling Mathematica (so that I can hide my mathematical ignorance from my family!). It looks very powerful, but its going to take some time to figure out how to work it.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com1tag:blogger.com,1999:blog-2459417306268001648.post-34494947923967806662007-11-28T03:45:00.000-08:002007-11-28T03:46:29.496-08:00Presented latest model to family for mathematical analysis. Roundly condemned for its inadequacy. Back to the drawing board...Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com0tag:blogger.com,1999:blog-2459417306268001648.post-34249037237254698712007-11-26T12:36:00.000-08:002007-11-26T12:37:43.189-08:00I've been wrestling with a deceptively formula for about a week and a half now. I cannot get it to yield up anything exciting (and that's before I try and code it). This project can absorb a lot of time!Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com0tag:blogger.com,1999:blog-2459417306268001648.post-64858601804764209142007-11-16T07:18:00.000-08:002007-11-16T07:21:44.437-08:00Gave my daughter some more differentiation. Tested the results with some numbers in Excel - she'd got it right. It gave her great joy to apply her school maths course to a real problem and get a result. Perhaps we should figure out a way of getting all kids to solv real problems in the maths class. It might encourage maths takeup.<br /><br />I think we have a new partnership. I come up with the ideas and she figures out the maths and I program the maths. It should speed things up.Just a guy in a garagehttp://www.blogger.com/profile/06988408157706909722noreply@blogger.com0