neuronwave

yesterday's technology tomorrow

Saturday, October 04, 2003

Search engines, information retrieval and recommendations
Since 1996, I've been working, to various degrees, on recommendation systems; first at Accenture's research labs with the team who create Bargain Finder and since then at AgentArts working on Music and other entertainment recommendation systems. Most recently I built Blog Change Bot which is a little IM Bot which sends you a message when a site you are interested in is updated. Aside - The AOL network has troubles with large lists of buddies (in my case subscribers) so a rewrite of the app is ahead of me this weekend.

In 1998 I attended a conference on Intelligent Agents and did a workshop on Information Retrieval and related topics. Its been interesting how this theme has reappeared consistently over the years. Last week we got called by a company interested in us doing Movie Recommendations for them. This got me thinking about how we could create a database of movie recommendations. I remembered an available database of around 80,000 movie reviews and plot overviews. Traditionally, we have built recommendation databases by data mining people's usage patterns e.g songs downloaded, search terms, movies hired etc. but for this client none of this data was available. So I wondered if we could take the reviews and plots and automatically extract the most distinctive words from the text and match this against other movie's reviews and plots to generate a measure of similarity. So for the last 3 days I've been beaverishly coding what is basically a fully fledged search engine that processes and parses 80,000 XML files, creates and inverted index of them and then matches a query to specific movies. Next week with a few tweaks, we should have a comprehensive database of relationships. As an example, I searched for "nuclear bomb comedy", thinking of Dr Strangelove, and got Space Cowboys which was very accurate.

I'm pretty pleased with the effort so far and the results...not bad for 3 days work. I'll have a demo available in a week or so and point people to it then and hopefully it won't be long before you wander into the local video/dvd store and no longer have to hunt around to find something that you might like.

0 comments | 8:09 AM
Playground Finder Flink Labs - Data Visualisation

About
neuronwave is the personal site of Ben Hosken. I am the founder of Flink Labs, a data visualisation studio based in Melbourne. Previously I founded AgentArts, where I developed a recommendations system that we ultimately sold to MSFT.I have two patents, one in Recommendation Systems and one for a datamining algorithm I invented. I have a strong interest in physical computing, emergence, self organising systems, immersive games, bluetooth ad hoc networks and other fun things.
A while ago I built and ran Blog Change Bot - an AOL IM Bot which notified you when blogs you like are updated. I also built IcyPole, a bluetooth based P2P mobile phone app in 2004. With Suzette, I've built Playground Finder - a community contributed site for parents to find playgrounds nearby and while traveling for their kids to play in. Its a "playground" for my social, location, community and mobile interests.
I live in Melbourne with my wife Suzette, son Oskar Tex and my little girl Jasmine

Random Pile of Ben
Bodypump, Flink Labs, autumn weather, balance.

Downloads etc
The Best Bagel Recipe ever
iTunes XML structure
Photo of some of my best Panetones

Neuron EcoSystem
Flink Labs
oskartex.comOskar Tex
JasmineJasmine Aurora
Suzette
Playground Finder

Archives

ben at neuronwave dot com