Monash Uni splunks server logs

  • Email a Friend
  • Print Page
Monash Uni splunks server logs
Aug 19, 2009 2:45 PM
Tags: monash | splunk | search | server | logs | network | san | storage | telstra

Search firm also pursues additional Telstra projects.

Monash University has licensed the next version of a tool called Splunk that it uses to index system log data from 600 physical and 100 virtual servers.

The tool, which is available in free and paid versions, enables companies to search across system, security incident and audit logs to troubleshoot IT issues and investigate issues.

Senior systems and storage administrator at Monash, Joshua Edmonds, told iTnews the University trialled the free version of the tool for a year before upgrading.

Versions are priced on the amount of log data that is indexed and searched each day. The free license included a 500MB limit, which Splunk said would cover "a handful of servers generating a modest amount of [log] data".

Edmonds said the University's license enabled it to index and search up to 1GB of data per day.

"At the moment we're only indexing our syslog data from the servers," he said. "We're not capturing log data from network equipment or storage devices.

"On a typical day we're looking at about 300 to 400MB but when there are problems that can peak at up to 1.2GB," he said.

The tool provided some leeway if data limits were breached. Edmonds said they had not been prevented from examining occasional data spikes.

Edmonds said Splunk had been deployed on a central logging server that captured log data from across its entire server environment.

He said it was possible the University would examine a more distributed architectural approach to Splunk's deployment.

Splunk said one of the advantages of version four of the tool was that the processing required for indexing and searching the log data could be conducted in a distributed fashion.

Splunk co-founder Michael Baum said rather than collect logging data and aggregating it in a single point, Yahoo! had installed Splunk in each of its 32 data centres to enable processing to occur locally.

"We then federate search across all those different locations in real time," he said.

Baum said the company had a "couple of hundred thousand" free users and 1100 paid customers. The paid version starts at US$9000 (AU$10,860) in Australia.

Existing customers in Australia included government agencies and Telstra, which had deployed the tool to support multimedia delivery on mobile phones.

Splunk said it was "talking to Telstra for more projects."

Baum believed there was a need for tools like Splunk - in part because vendors were "notoriously bad at giving [customers] good tools to analyse log data.

"Companies like Telstra can no longer afford to leave these logs out on end network devices where they typically get overwritten every couple of hours because there's not enough on-board memory," Baum said.

"[IT departments] really need a few days of data to be able to get a baseline and determine trends."


 
Comments

Be the first to comment on this article.
Thoughts on this article? Add a comment below.
Comment:
Want to participate in the discussion?
Or log in now to comment
 
 
 
Top Stories
TIO website hit by malware
Weekend malware runs one new process per target machine.
 
Microsoft announces Azure launch date
Australia in second wave of country releases.
 
CBA embarks on "database-as-a-service"
Analysis: How the bank intends to save megabucks.
 

Spotlightthe topics we're following

Latest Comments

"It never fails to astound me at the greed of corporate executives and politicians, and this ..."
by BernieG Feb 10, 2010 7:55 AM
 
"Hahahah...What a joke!! "Conroy had said that it was not possible to apply ISP-level filtering ..."
by gerson Feb 9, 2010 10:39 PM
 
"@@Comments, yes, and history keeps repeating itself. Remember the earlier pr-and-media-fuelled ..."
by anonymous Feb 9, 2010 6:40 PM
 
"I would have paid good money to be in court when that clanger dropped. Could you imagine, the ..."
by Private Citizen Feb 9, 2010 6:23 PM
 
"He is not yet listed on NBN Co. website as part of their team of executives (http://www.nbnco.com..."
by Private Citizen Feb 9, 2010 6:07 PM
1) HTC Magic16 plans 2%
2) Nokia N9743 plans 9%
3) Nokia E7149 plans 1%
4) Apple iPhone 3GS 16GB30 plans 11%
5) Apple iPhone 8GB42 plans 5%
1) iiNet32 plans 5%
2) Netspace36 plans 11%
3) TPG Internet19 plans 14%
4) Optus33 plans 1%
5) Telstra BigPond30 plans 2%

Mobiles | Broadband | Credit Cards

iTnews

Polls

What is the sweet spot for Apple's entry 16GB Wi-Fi iPad?




   |   View results
$549
  78%
 
$579
  10%
 
$619
  4%
 
$649
  3%
 
$699
  5%
TOTAL VOTES: 382

Vote