Tuesday, October 9, 2012

Predictive Modeling Competition

If a person or an organization would know what would happen in the future a lot of money can be made or a lot of problems can be solved before it occurs. Yet this is not possible. One of the best tools around to come close to knowing the future is through predictive modeling.

There are a lot of data scientists who want access to real world data to develop and refine their technique.  There are a lot of organizations who have tons of data but don’t have access to advanced statistical techniques that will help them make the most of their data.  Anthony Goldbloom has come up with an innovative way of bridging these two worlds.

Goldbloom founded Kaggle and is the company CEO. What the company provides is a platform for predictive modeling competitions.  From around the world companies, governments, and researchers present datasets and problems. A competition is then held within a given time frame and the best data scientists in the world compete to come up with the best solutions. When the competition ends, the competition host pays prize money and in return gets the intellectual property behind the winning model.

So far Kaggle has never failed to outperform pre-existing accuracy benchmark. This is because there is no one approach to solving predictive modeling problems.  A single participant isn’t capable of trying them all. With many participants trying different ways the boundaries of what can be achieved expands. Results are given real time which pushes everyone else to try harder until the full extent is reached.

Kaggle has around 17,000 PhDs, data analyst, and scientists from around the world who want to compete in building predictive models.  Aside from public competition there are also private, invitation-only competitions with data subject to non-disclosure agreements.

Goldbloom who is in his late twenties is from Australia. He and business partner, chief data scientist Jeremy Howard started the company in Melbourne, Australia and relocated it to San Francisco where they find the environment very conducive for startups.  Prior to Kaggle Goldbloom worked for the Australian Treasury, a short stint with the Economist, and for the Australian Reserve Bank.