How to select a SIEM solution

'SIEM' and 'SIM' are terms I first came across close to nine years ago when I was looking at setting up my company, earthwave - an MSSP (Managed Security Service Provider) that focuses on delivering real-time threat analysis and incident response services. Back then, the technology was in its infancy and there were no vendors with any local presence, so I was forced to spend three months in the U.S. evaluating products before investing my life savings into this technology.

At the time we were certainly an early adopter of what would have been version one of almost all of the six products we evaluated, of which only two remain 'pure play' vendors in that space, while others have been acquired by larger vendors.

Growth in SIEM solutions in recent years has been almost meteoric, with some vendors experiencing growth rates of 30 to 60 per cent over the past three years. In the meantime, Australian companies are pouring millions of dollars into these SIEM projects without the return that they anticipated.

The aim of this piece is to use my experience in the SIEM space to provide potential SIEM buyers with a guide for defining requirements, evaluating SIEM solutions, the technology options available, and deployment considerations, together with real world examples. 

Before I start it is important to make a clear distinction of the terminology as it relates to functionality and respective products in the market. The products I cover here are those that I have had either project or lab experience with as a result of earthwave's Australian customer deployments. 

Security Information Management (SIM) - These products provide reporting and analysis of data to support regulatory compliance initiatives (such as privileged user and access monitoring and compliance reporting), internal threat management, and security policy compliance management.

They provide strong log management capabilities and have the capacity to store multi-terabyte logs over very long periods of time. The reason for this is that they either use a flat file system for storing the logs and as a result they generally can compress logs on a 10:1 ratio. Searching through the logs and reporting is also super fast when compared to SEM platforms, again as a result of storing logs on a flat file system and using indexes.

Most tier-1 MSSPs have developed their platforms internally based on this architecture but with more 'fancy' portal interfaces, and as a result they do not have any advanced correlation, threat analysis or visualisation capabilities.

I consider the following products to be in the SIM category: Splunk, ArcSight Logger, Log Logic, RSA envision, NetIQ Security Manager, IBM TCIM and the eIQnetworks range.

SEM - Security Event Management - These products provide strong event management, real-time threat analysis, visualisation, ticketing, incident response, and security operations. They are typically based on enterprise SQL databases such as Oracle.

SEM products are ideal for running security operations such as an MSSP. Unlike SIM products, SEM-based products are not ideal for log management and long-term storage of excessive amounts of logs as they are poor at log compression. They are slow when producing reports and rely on a massive index to allow for database queries.

I consider the following products to be in the SEM category: ArcSight ESM, netForensics, Novell Sentinel, Intelitactics, Cisco MARS and IBM TSOM.

SIEM - Security Information & Event Management - These products combine SIM and SEM capabilities, however, they generally excel in only one of these categories. SIM products are simple to deploy and operate while SEM products are more complex. It's a bit like comparing financial applications MYOB to SAP.

The following list of best practices will assist organisations in making the right SIEM choice for their environment.

Take Project Ownership
A significant number of SIEM deployments fail due to a lack of project involvement and ownership from the organisation. Too many organisations throw money at the problem thinking that it will provide the solution.

Every failed SIEM project that I have witnessed was due to a lack of ownership from the client. Too many organisations depend on the vendor and say that "once the product is installed I will figure out how to make use of it" and don't bother defining their requirements upfront and take control of the project outcome.

Unfortunately, I have seen a number of Australian organisations spend in excess of five million dollars on their SIEM deployment without any return on their investment. But you don't need to spend too much money if you know what you want to get out of the project.

Getting key stakeholders and business groups involved in helping define the requirements also plays a big part in ensuring the success of the project. At a minimum you need executive sponsorship and internal audit, compliance, IT security and IT operations involved.

Clearly define your operational requirements
Too often the vendor is depended upon to guess what the client's requirements are and the end result is a vanilla install of the product that doesn't meet any of the specific business requirements. The value of any SIEM product is directly related to the data that is put into the system, the understanding of what the system can derive from the data, and the actions the system can be programmed to take.

You need to clearly define why you're deploying a SIEM solution and look past the vendor's marketing messages. Is it to support your 24/7 security operations centre? Is it to provide real-time and historic views into external attacks? Is it to meet your audit and compliance mandates? Is it to manage insider threats? Is it to manage fraud? Once you meet your key requirement you then have all the time to extend your SIEM deployment into other areas. Develop a two to three year road-map for all functions that will influence buying decisions for the initial implementation.

Think outside the square
Where is your important data? On the routers, switches and firewalls, or is it on the back-end databases and mainframes?

Too often, organisations collect terabytes of meaningless data while their core assets and data are ignored. Too often a SIEM is used for simple log management, while its real power is never utilised and the value that can be gained from the internal assets ignored. No auditor cares about which servers are secure unless they relate to a business application. When considering log analysis for security governance, make sure you understand which physical and logical assets support the business application and don't get caught up in technology silos. List these supporting components and ignore log output which is not relevant to the business application. Some of the better deployments I have seen include:

• Mobile device security - a mobile operator used a SIEM to detect malware-like behavior among subscribers based on connectivity patterns, as well as preventing DDoS attacks by monitoring increases in Call Detail Records (CDR) averages.

• Fraud management - a financial institution detected known, unknown, and potential instances of fraud by comparing disparate data sources, finding irregularities amongst a baseline of legitimate transactions, and identifying repeated events that represent actions taken by fraudsters.

• Management and monitoring of the convergence of physical and logical security - a government agency collected events from their door-access system and used their SIEM to send a command to their camera system to start recording anytime someone entered their data centre cage and stop recording when they exited. The same agency also combined physical door-access events to Windows Active Directory network access events to look for staff that were logged into the network but that had not swiped into the front door's card access system.

• ITIL Compliance - a service provider used a SIEM to reconcile their ITIL processes such as change, configuration and incident management.

• Insider threat management - a software company that held substantial amounts of intellectual property used their SIEM as an early warning system, detecting suspicious activity such as printing large numbers of files outside of business hours, emailing large attachments to personal email accounts, employee communication with competitors, and the clearing of system audit logs. In addition to the early warning system, the insider threat management includes information leak and IT sabotage-specific detection capabilities such as real-time rules designed to identify inappropriate access or transmission of sensitive data, or internal use and presence of hacking tools.

Don't underestimate the resourcing requirements
Things to watch out for include: the time it takes to define your requirements from all the key business stakeholders, disk space requirements, server hardware requirements, training requirements, and ongoing support requirements.

Security log volumes can be enormous and grow with explosive speed. Ensure that each log message that is being stored and retained supports some outcome. Don't store meaningless data. I have witnessed organisations invest significantly in SIEM licensing only to realise that they didn't budget enough for server hardware and storage.

Some of the SIEM products are complex and require extensive training and regular access to vendor support. Too many SIEM vendors still operate their consulting and support capability from overseas and are unable to offer support within our local time zone, and consequently any on-site support and training can end up quickly eating into any IT budget you have left. For instance, if you need a new report, will your vendor work with you to develop this? What if you need a custom correlation rule or if you have a new data source that the vendor does not support? Or if you have a looming audit and need your vendor's guidance through the process?

Ask the vendor these questions. A trusted provider will be willing to give you value-added services without breaking your budget - this can often mean the difference between a good and a great project. I know a number of organisations that continue flying consultants from the vendor's head office to assist them with their ongoing SIEM operation.

Demand a flexible and agile solution
Security is about staying ahead of the game, so your SIEM solution needs to be very flexible in order to grow with your organisation. A universal reason for purchasing a SIEM solution is to tame the data overload problem. But as organisations gain a consolidated view of their logs, true management-use cases emerge. These include the ability to customise multiple areas in the product - such as data collection policies, correlation rules and associated actions, workflow, dashboards, reporting and investigation tools - to drive maximum efficiency.

Demand a proof of concept
As a customer, it's your right to ask for a trial with your live data across your security infrastructure. Typically, this takes only a week and is well worth the effort. This is your chance to see if the vendor's marketing message is consistent with the product capability (in my experience it never is, but some vendors come close). A live trial - one with real data representing extensive volumes of transactions with unique source devices and systems, and actual testing of real threats - helps organisations avoid the serious pitfalls that can arise from relying on canned product demonstrations.

Not all SIEMs are the same
Don't assume that all apparently-similar feature sets provide the same functionality. Vendors approach the same problem in their own unique way and as a result some are far superior to others. For example, I have seen some vendors only offer very basic correlation while others offer multiple advanced correlation techniques.

Another example is with asset integration, where many vendors claim to have asset integration but only allow you to manually load asset data, which is unacceptable for all but the smallest of implementations. Some SIEM vendors will only leverage asset information for historical reporting, unable to correlate asset information in real time for either security or compliance purposes. This is a major limitation and results in an inability to perform effective risk management. True asset integration, accessible by the real-time correlation engine, enables the SIEM solution to automatically prioritise workload by focusing only on the logs related to assets critical for PCI compliance, for example.

Make sure you get serious correlation
Like many SIEM terms, the word 'correlation' lacks standard agreement in principle and practice. SIEM technologies that claim to perform asset, vulnerability and threat correlation achieve this with varying degrees of success.

Without going into the details, the following are some of the questions you should be asking the SIEM vendors about their correlation capabilities: How does the product perform cross-device correlation? How does the product derive priorities? How does the product process identities and roles? How does the product incorporate asset value? How does the product track and escalate threat levels? Can the product perform correlation on both real-time and historical data? Can the product automatically discover unknown threats? How does the product deal with time in the correlation process? How easy is it to alter, tune and author new rules?

Appliance vs. Software
Generally, appliances are easier to set up and operate, while software-based solutions are far more functional and scalable. SIM platforms generally run on an appliance while SEM platforms are software-based.

Some organisations simply do not have the resources to operate a software-based solution. Appliance-based solutions generally are not very extensible and do not offer all of the enterprise feature sets such as workflow, customisation and extensive threat identification capabilities.

Does it meet your compliance initiative?
By leveraging a SIEM, you can build more automation into your compliance program, increase your productivity in monitoring, responding and maintaining compliance, and substantially reduce the operational costs of building and supporting a comprehensive compliance methodology. Some SIEMs have built-in compliance features that are available as part of the base product, while others charge for it as an additional module.

There is also a considerable difference between the compliance features of the different SIEM products on the market. Some simply add some basic compliance-specific reports and call it 'compliance ready', while others invest substantial development resources in defining compliance-specific correlation rules, dashboards, notifications, reports and much more. If this is the reason why you're deploying a SIEM, then be certain to ask the vendor to demonstrate their full compliance features and include these in the POC.

In summary, decisions should be driven by organisation-specific requirements in areas such as the relative importance of SIM vs. SEM capabilities, ease and speed of deployment, cost, the IT organisation's support capabilities, and integration with system and application infrastructures.

Carlo Minassian is the founder and CEO of earthwave, an Australian provider of security services.