A guide to better logging

Powered by SC Magazine
 

Logs are more important than many think.

To build a good security monitoring program, logs are critical.

They feed almost everything we do in monitoring from event correlation to auditing.  We need logs from things such as security tools, network devices, servers, databases and applications in order effectively monitoring networks.

Any SIEM, analyst, or auditor is only as good as the data available for analysis. Think garbage in and garbage out.

Networks and systems today are growing ever larger and more complex.  With that, almost all devices can generate logs and there are many different levels of logging that can be configured for each device. 

It is crucial to ensure you get the right logs from the right devices and at the right level.  Without this you can and will miss events of interest.

However, getting logs from all devices and systems can lead into thousands of devices sending logs and requiring storage of those logs. 

Here are some key points to consider when setting up or evaluating your logging infrastructure:

1.  Know your budget.
The more important the data, the more money companies are generally going to spend on it.  I know money is tight and sadly security is often times the first place for cuts.

However, you can only work within the means of the funding you have available.  This is important because the more logs you take in requires a more robust logging infrastructure as well as storage for those logs. 

You may have to choose smartly about the logs you can process from which devices.  Know the budget you have to work with when starting up.  For those doing an evaluation, its a good time to see if you need to grow and how much that will cost.

2.  Determine the devices you should have logging.  
There is no simple answer to that question.  Ideally its everything, but realistically that won't work.

Ask yourself how much storage do you have available? How many messages per second can your infrastructure support?  How big are the different logs from the different systems you'll be receiving? 

You also need to know what you are trying to protect and where it lives. Once you answer those questions, it becomes easier to look at your infrastructure for the key devices/systems that you will need logs from. You want logs coming in that will allow you to paint as complete of a picture as possible.

3.  Determine what level of logging is needed and document it.
This should be a group decision.  What groups in your organization use the logs to support their various jobs? 

It is those groups who need to have a say in what that level would be be. It is equally important to document this and the logging level for the different device types for future reference.  

For example, a Cisco router has eight different logging levels and each of these will provide more granular information resulting in more log entries.  If you set the level to debug, you can fill your central log storage up pretty fast if you have alot of devices logging.

If you set it to alerts then you may not get the information you want.  You can even mix and match the logging levels by having devices that are forward facing have more detailed logging levels and those devices that are more protected farther back in the network having less detailed logging.

Remember, that this can be changed as necessary.

4.  Know the log retention policy.
Not only do you have to have enough log storage capacity on your logging infrastructure, but you may also face the requirement of retaining those logs for a specified length of time.

If you have a log retention policy, you need to know what it is to ensure you have enough SAN or offline storage available to retain the logs you generate.

Just because your logging infrastructure may be able to capture the logs you are generating, doesn't mean that you have the necessary long term storage capabilities to meet the retention capabilities.  

5.  Monitor your log submissions.
How do you know are still getting the logs you asked for, at the level you want and nothing has changed? 

This is probably one of the toughest areas and the one most often overlooked  My experience has been that people hand folks a SOP with how to send their logs, confirm they are getting logs and nothing more.

This is tough, especially in a large organisation where you can have thousands of devices sending you logs.  How do you know if anything has changed?  You can't afford not to know when so much is riding on the logs you recieve.

6.  Plan for the future.
If your network is going to grow, you need to ensure your logging infrastructure can grow with it.  This can include many areas such as log servers, appliances, SAN storage, offline storage, capacity of tools that are going to ingest these logs, additional personnel, and so on.  You need to plan well in advance of when you are going to need to expand.

Your logging team has to have a pulse on everything going on with the network.  A logging infrastructure takes a lot of work, but the benefit is worth it. 

It provides the foundation of your security monitoring and deserves more time than it is often given.

If your auditors check logs for a specific issue and report that all is normal, ask yourself if it is because the log information is lacking.

This blog appeared at SANS ICS.

Copyright © SC Magazine, Australia


A guide to better logging
 
 
 
Top Stories
Hockey flags billion-dollar Centrelink mainframe replacement
Claims 30 year-old tech is holding Govt back.
 
Ombudsman wants to monitor warrantless metadata access
Requests ability to report publicly.
 
Frugality as a service: the Amazon story
Behind the scenes, Amazon Web Services is one lean machine.
 
 
Sign up to receive iTnews email bulletins
   FOLLOW US...

Latest VideosSee all videos »

The great data centre opportunity on Australia's doorstep
The great data centre opportunity on Australia's doorstep
Scott Noteboom, CEO of LitBit speaking at The Australian Data Centre Strategy Summit 2014 in the Gold Coast, Queensland, Australia. http://bit.ly/1qpxVfV Scott Noteboom is a data centre engineer who led builds for Apple and Yahoo in the earliest days of the cloud, and who now eyes Asia as the next big opportunity. Read more: http://www.itnews.com.au/News/372482,how-do-we-serve-three-billion-new-internet-users.aspx#ixzz2yNLmMG5C
Interview: Karl Maftoum, CIO, ACMA
Interview: Karl Maftoum, CIO, ACMA
To COTS or not to COTS? iTnews asks Karl Maftoum, CIO of the ACMA, at the CIO Strategy Summit.
Susan Sly: What is the Role of the CIO?
Susan Sly: What is the Role of the CIO?
AEMO chief information officer Susan Sly calls for more collaboration among Australia's technology leaders at the CIO Strategy Summit.
Meet the 2014 Finance CIO of the Year
Meet the 2014 Finance CIO of the Year
Credit Union Australia's David Gee awarded Finance CIO of the Year at the iTnews Benchmark Awards.
Meet the 2014 Retail CIO of the Year
Meet the 2014 Retail CIO of the Year
Damon Rees named Retail CIO of the Year at the iTnews Benchmark Awards for his work at Woolworths.
Robyn Elliott named the 2014 Utilities CIO of the Year
Robyn Elliott named the 2014 Utilities CIO of the Year
Acting Foxtel CIO David Marks accepts an iTnews Benchmark Award on behalf of Robyn Elliott.
Meet the 2014 Industrial CIO of the Year
Meet the 2014 Industrial CIO of the Year
Sanjay Mehta named Industrial CIO of the Year at the iTnews Benchmark Awards for his work at ConocoPhillips.
Meet the 2014 Healthcare CIO of the Year
Meet the 2014 Healthcare CIO of the Year
Greg Wells named Healthcare CIO of the Year at the iTnews Benchmark Awards for his work at NSW Health.
Meet the 2014 Education CIO of the Year
Meet the 2014 Education CIO of the Year
William Confalonieri named Healthcare CIO of the Year at the iTnews Benchmark Awards for his work at Deakin University.
Meet the 2014 Government CIO of the Year
Meet the 2014 Government CIO of the Year
David Johnson named Government CIO of the Year at the iTnews Benchmark Awards for his work at the Queensland Police Service.
Q and A: Coalition Broadband Policy
Q and A: Coalition Broadband Policy
Malcolm Turnbull and Tony Abbott discuss the Coalition's broadband policy with the press.
AFP scalps hacker 'leader' inside Australia's IT ranks.
AFP scalps hacker 'leader' inside Australia's IT ranks.
The Australian Federal Police have arrested a Sydney-based IT security professional for hacking a government website.
NBN Petition Delivered To Turnbull's Office
NBN Petition Delivered To Turnbull's Office
UTS CIO: IT teams of the future
UTS CIO: IT teams of the future
UTS CIO Chrissy Burns talks data.
New UTS Building: the IT within
New UTS Building: the IT within
The IT behind tomorrow's universities.
iTnews' NBN Panel
iTnews' NBN Panel
Is your enterprise NBN-ready?
Introducing iTnews Labs
Introducing iTnews Labs
See a timelapse of the iTnews labs being unboxed, set up and switched on! iTnews will produce independent testing of the latest enterprise software to hit the market after installing a purpose-built test lab in Sydney. Watch the installation of two DL380p servers, two HP StoreVirtual 4330 storage arrays and two HP ProCurve 2920 switches.
The True Cost of BYOD
The True Cost of BYOD
iTnews' Brett Winterford gives attendees of the first 'Touch Tomorrow' event in Brisbane a brief look at his research into enterprise mobility. What are the use cases and how can they be quantified? What price should you expect to pay for securing mobile access to corporate applications? What's coming around the corner?
Ghost clouds
Ghost clouds
ACMA chair Chris Chapman says there is uncertainty over whether certain classes of cloud service providers are caught by regulations.
Was the Snowden leak inevitable?
Was the Snowden leak inevitable?
Privacy experts David Vaile (UNSW Cyberspace Law and Policy Centre) and Craig Scroggie (CEO, NextDC) claim they were not surprised by the Snowden leaks about the NSA's PRISM program.
Latest Comments
Polls
Which bank is most likely to suffer an RBS-style meltdown?





   |   View results
ANZ
  20%
 
Bankwest
  9%
 
CommBank
  12%
 
National Australia Bank
  17%
 
Suncorp
  23%
 
Westpac
  19%
TOTAL VOTES: 1516

Vote