Warmest 100 even hotter than forecast

 

How data analysis spoiled one of the world's largest song contests.

Nick Drewe, an online marketer from Brisbane, got the last laugh tonight after he successfully predicted the Top 10 songs voted in Triple J's Hottest 100 by analysing data from 35,080 votes published on social media.

Two weeks ago, Drewe announced that he felt confident he could predict the outcome of the Hottest 100, which attracts a whopping 1.5 million votes from Australian music lovers. His Warmest 100 website became a much-discussed hit, attracting 75,000 unique visitors within 24 hours of launch.

Drewe's predictions rattled broadcasters and bookmakers alike - the latter halted all betting on the Hottest 100.

Tonight, Nick Drewe's Warmest 100 correctly predicted the top three Triple J Hottest 100 songs, in order. 

All 10 of the top 10 songs were predicted, five at the correct ranking.

And of all the songs in the Hottest 100, 92 of them were predicted by the Warmest 100.

 

The first few results announced had caused some doubts, but by late this evening it became clear the analysis had been wildly successful.

“It could barely have gone better,” Drewe said after the number one song was announced.

“Earlier in the day, as the first few songs were announced, people were saying things on Twitter like ‘Warmest 100, more like Coldest 100’, but up by the time we got to number 30, the feedback online was mostly positive,” he said. “People got more excited as the day went on.”

How did he do it?

Radio broadcaster Triple J used social media to promote the annual music countdown. Participants could share their votes by posting to sites such as Twitter, Facebook and Pinterest using a button on the Triple J voting site. The resulting post contained a link to that user’s unique vote page, showing all ten of their votes.

Drewe first had the idea for The Warmest 100 when he noticed the URL of shared votes contained a unique identifier. He then used the search mechanisms of the various social platforms to see how many votes he could find. “Facebook Graph Search wasn’t that useful,” he said, “I got 80 percent of the votes from Twitter.”

It was then just a matter of collecting as many links as he could and visiting the resulting page on the Triple J website to scrape off the votes.

“I used a plugin called iMacros for Firefox,” Drewe said. “People have since said I could have used a script in Perl or something, but I just used the tools I use every day.”

(Drewe’s day job is as Search Engine Marketing Manager at Search Factory in Brisbane.)

The votes were saved into a simple spreadsheet, and then tallied up. Nothing more sophisticated than a simple count was used to analyse the results.

It was Drewe’s colleague Tom Knox, previously of music discovery site We Are Hunted, who had the idea of putting the list up on the web. The punningly named Warmest 100 site was then built using Wordpress hosted at HostGator. Drewe used a simple theme with each song as a post, adding widgets provided by YouTube or Soundcloud to allow visitors to listen to each song.

Friends Jack Murphy (designer) and Andy Thelander (developer) chipped in with free time to come up with the second round design for the site, which went live just prior to the official countdown. The replacement design made use of a new user-interface to improve the page-loading experience for users.

It was only after the story broke on music website TheVine that others waded in with more sophisticated analyses. Professor You-Gan Wang, Chair of Applied Statistics at the University of Queensland, confirmed that the sample size was large enough to predict perhaps 90 of the predicted songs that would ultimately prove to be in the official list.

Then David Quach, an Australian economist and MATLAB developer with Supported Intelligence in Chicago,also performed some statistical analysis on the data, which Drewe shared with him. Quach used a statistical technique called “bootstrapping” to analyse the potential accuracy of the sample data. He predicted that the number one song was 83 percent likely to match The Warmest 100 prediction.

The Warmest 100 team now expect their success will be a one-off - it will undoubtedly impact what access  Australia's public broadcaster will provide to voting information.

The team speculate Triple J may take votes using custom ‘walled garden’ apps on Facebook, or use captchas to foil scripted vote downloads.

“There’s some pressure on us to back it up with something," Drewe said. "But we have no specific plans.”

Copyright © iTnews.com.au . All rights reserved.


Warmest 100 even hotter than forecast
 
 
 
Top Stories
Frugality as a service: the Amazon story
Behind the scenes, Amazon Web Services is one lean machine.
 
Negotiating with the cloud email megavendors
[Blog post] Lessons from Woolworths’ mammoth migration.
 
Qld govt to move up to 149k staff onto Office 365
Australia's largest deployment, outside of the universities.
 
 
Sign up to receive iTnews email bulletins
   FOLLOW US...

Latest VideosSee all videos »

The great data centre opportunity on Australia's doorstep
The great data centre opportunity on Australia's doorstep
Scott Noteboom, CEO of LitBit speaking at The Australian Data Centre Strategy Summit 2014 in the Gold Coast, Queensland, Australia. http://bit.ly/1qpxVfV Scott Noteboom is a data centre engineer who led builds for Apple and Yahoo in the earliest days of the cloud, and who now eyes Asia as the next big opportunity. Read more: http://www.itnews.com.au/News/372482,how-do-we-serve-three-billion-new-internet-users.aspx#ixzz2yNLmMG5C
Interview: Karl Maftoum, CIO, ACMA
Interview: Karl Maftoum, CIO, ACMA
To COTS or not to COTS? iTnews asks Karl Maftoum, CIO of the ACMA, at the CIO Strategy Summit.
Susan Sly: What is the Role of the CIO?
Susan Sly: What is the Role of the CIO?
AEMO chief information officer Susan Sly calls for more collaboration among Australia's technology leaders at the CIO Strategy Summit.
Meet the 2014 Finance CIO of the Year
Meet the 2014 Finance CIO of the Year
Credit Union Australia's David Gee awarded Finance CIO of the Year at the iTnews Benchmark Awards.
Meet the 2014 Retail CIO of the Year
Meet the 2014 Retail CIO of the Year
Damon Rees named Retail CIO of the Year at the iTnews Benchmark Awards for his work at Woolworths.
Robyn Elliott named the 2014 Utilities CIO of the Year
Robyn Elliott named the 2014 Utilities CIO of the Year
Acting Foxtel CIO David Marks accepts an iTnews Benchmark Award on behalf of Robyn Elliott.
Meet the 2014 Industrial CIO of the Year
Meet the 2014 Industrial CIO of the Year
Sanjay Mehta named Industrial CIO of the Year at the iTnews Benchmark Awards for his work at ConocoPhillips.
Meet the 2014 Healthcare CIO of the Year
Meet the 2014 Healthcare CIO of the Year
Greg Wells named Healthcare CIO of the Year at the iTnews Benchmark Awards for his work at NSW Health.
Meet the 2014 Education CIO of the Year
Meet the 2014 Education CIO of the Year
William Confalonieri named Healthcare CIO of the Year at the iTnews Benchmark Awards for his work at Deakin University.
Meet the 2014 Government CIO of the Year
Meet the 2014 Government CIO of the Year
David Johnson named Government CIO of the Year at the iTnews Benchmark Awards for his work at the Queensland Police Service.
Q and A: Coalition Broadband Policy
Q and A: Coalition Broadband Policy
Malcolm Turnbull and Tony Abbott discuss the Coalition's broadband policy with the press.
AFP scalps hacker 'leader' inside Australia's IT ranks.
AFP scalps hacker 'leader' inside Australia's IT ranks.
The Australian Federal Police have arrested a Sydney-based IT security professional for hacking a government website.
NBN Petition Delivered To Turnbull's Office
NBN Petition Delivered To Turnbull's Office
UTS CIO: IT teams of the future
UTS CIO: IT teams of the future
UTS CIO Chrissy Burns talks data.
New UTS Building: the IT within
New UTS Building: the IT within
The IT behind tomorrow's universities.
iTnews' NBN Panel
iTnews' NBN Panel
Is your enterprise NBN-ready?
Introducing iTnews Labs
Introducing iTnews Labs
See a timelapse of the iTnews labs being unboxed, set up and switched on! iTnews will produce independent testing of the latest enterprise software to hit the market after installing a purpose-built test lab in Sydney. Watch the installation of two DL380p servers, two HP StoreVirtual 4330 storage arrays and two HP ProCurve 2920 switches.
The True Cost of BYOD
The True Cost of BYOD
iTnews' Brett Winterford gives attendees of the first 'Touch Tomorrow' event in Brisbane a brief look at his research into enterprise mobility. What are the use cases and how can they be quantified? What price should you expect to pay for securing mobile access to corporate applications? What's coming around the corner?
Ghost clouds
Ghost clouds
ACMA chair Chris Chapman says there is uncertainty over whether certain classes of cloud service providers are caught by regulations.
Was the Snowden leak inevitable?
Was the Snowden leak inevitable?
Privacy experts David Vaile (UNSW Cyberspace Law and Policy Centre) and Craig Scroggie (CEO, NextDC) claim they were not surprised by the Snowden leaks about the NSA's PRISM program.
Latest Comments
Polls
Which bank is most likely to suffer an RBS-style meltdown?





   |   View results
ANZ
  20%
 
Bankwest
  9%
 
CommBank
  11%
 
National Australia Bank
  17%
 
Suncorp
  24%
 
Westpac
  19%
TOTAL VOTES: 1498

Vote