Agile development key to eBay analytics

Ebay has pinned the effectiveness of its data analytics operations on an agile software development framework called Scrum.

According to Patrick Firouzian, senior director of eBay's Asia Pacific IT division, the iterative, collaborative approach allowed analysts to deliver solutions at a competitive pace.

"We want to out-think and out-execute the competition," said Firouzian, who was based in the company's information management and data warehouse in China.

Speaking at the Teradata Universe 2010 conference in Sydney this week, Firouzian described "very, very aggressive" competitors, alluding to Amazon -- "the big A".

Technologies in the online retail market were quickly replicated, he said, adding that new, disruptive innovations were scarce.

Because the Scrum framework involved agile prototyping, it allowed eBay to respond quickly to changing demands, and deliver useful information in days instead of months.

The technique, which involved small teams of two to nine analysts, was initially used to develop solutions for eBay's Trust and Safety (TnS) division.

"We had to be responding to [fraud] as quickly as possible," Firouzian explained.

Scrum was later used in other areas like shipping and billing. Firouzian said that 70 percent of eBay's analytics projects were now based on the Scrum framework.

Meanwhile, eBay's team of analysts had grown from 15 in 2000 to around 400 currently. 90 percent of its data warehouse staff had Scrum master certifications and 96 percent had Teradata master certifications.

Besides detecting and preventing fraud, eBay's data analysts aimed predominantly to "optimise the distance between searching and bidding", based on behavioural and transactional data.

Firouzian noted that while some Web site improvements may only affect one or two percent of transactions, "half a percent is millions of dollars".

He described attempts to determine how best to build customer loyalty with coupons, and how to profile customers based on their search terms.

Ebay also had developed software that would automatically bid on relevant Google AdWords search terms to ensure it received the best value for money from search engine marketing.

The various projects were supported by a decentralised IT environment, and a culture that encouraged and rewarded innovations with contests and patents, Firouzian said.