Many SOC analysts are starting to shut off high-alert features to keep pace with the volume, new study shows.
Yet another red flag that the security operations center is burning: Most SOC analysts now say their primary role is to close alert investigations or the number of alerts as quickly as possible rather than to study and remediate security threats, according to a new survey.
Last year, some 70% of SOC analysts surveyed by CriticalStart actually said the reverse: They saw their main role as analyzing and fixing security threats and issues. But in the managed detection response firm’s newest survey — published this week — only 41% say that’s their priority. And most (more than 70%) handle 10 or more security alerts per day, an increase over last year, when 45% were operating at that workload.
SOCs also get hit with false-positive security alerts at a rate of 50% or higher, according to the report, and nearly 40% say when SOC analysts can’t keep up with alert, they either turn off some alerting functions or hire more analysts.
“Security teams are just trying to survive without the required resources and head count. They often filter alerts using email rules to send notifications to an alert folder, where they are ignored. Some teams even turn off security-system alerting so there is no record of security events not being monitored,” says Rob Davis, CEO of CriticalStart, which recently surveyed 50 SOC professionals in enterprise organizations, managed security services providers, and managed detection and response providers. “It is very common for analysts to increase the thresholds for creating security events to reduce volume.”
Turning down the volume on alerts to keep up with them puts organizations at risk of a real attack, says Chris Calvert, co-founder and vice president of product strategy at Respond Software, a security automation vendor. “If I am being attacked constantly and I have real vulnerabilities to manage but only a small team, how do I prioritize? In today’s environment, detection and remediation is just as important as prevention and we often don’t have the budget to cover everything,” he says.
This latest study is yet another in a string of recent SOC reports this year underscoring the growing problem of an overwhelming and impossible volume of security alerts to sift through for that needle in the haystack, and the lack of people to fill the seats in the SOC. CriticalStart’s report shows how these stresses are leading to heavy turnover, with more than 75% seeing SOC turnover raters of more than 10%, and close to half seeing turnover rates of 10% to 25%.
A Ponemon Institute study last month revealed that more than half of IT and security pros consider their SOC inadequate to thwart security threats and some 65% were thinking of leaving their positions due to alert overload, long hours, and incomplete visibility into their IT infrastructures.
“Turnover continues to increase and retention is a major issue for companies,” notes CriticalStart’s Davis. “With virtually no unemployment, the best analysts are constantly recruited. Executives fear the expertise from completed security projects will walk out the door and ruin their investment in tools.”
It’s a vicious cycle: Much of the stress in the SOC comes from analysts surrounded by too many security tools that don’t work well together or that they don’t have time or resources to fully master, as more alerts bombard their screens every day. They just don’t have the time or expertise to master the tools, or stay on top of the alerts these systems pump out.
“More security sensors and log sources containing more signatures of potentially malicious activity combined with exponential IT growth — and a dramatic increase in malicious attacks,” Calvert explains. He says SOCs should measure the time and effort spent on false positives and automate the process where they can.
The noise and overload of tools and alerts can escalate quickly, according to Larry Ponemon, president of the Ponemon Institute. “A lot of research studies find the whole issue of interoperability and scalability is largely ignored and as result, the technologies don’t actually work together, and you have more [tools] than you need,” Ponemon says.
An overwhelmed SOC can result in dangerously long times to resolve and remediate an attack. Some 42% of the SOC analysts in Ponemon’s report, sponsored by Devo Technology, say it takes months or years on average to resolve a hack. That mean time to resolution, as it is called, occurs at 22% of organizations in a matter of hours or days.
CriticalStart’s report found that most SOC analysts (78%) take 10 or more minutes to investigate each alert they see.
Security experts recommend outsourcing some or most SOC operations to managed or cloud-based providers, training up existing SOC analysts, and automating low-level tasks.
“The SOC of the future will look very different and provide the advantages of an in-house SOC with the lower cost of outsourcing,” Davis notes. “A SOC must quickly detect attacks and respond before a breach occurs. This requires resolving every alert instead of filtering or ignoring security events. If you aren’t resolving every alert, then you can’t detect every attack.”
He says that because most alerts are common across organizations, an outsourced SOC model can tackle “known good” false positives more efficiently and cheaply.
The challenges of building a skilled in-house SOC that has the manpower and skill to adequately protect your organization are well known. Building a SOC is a big investment and not cost effective if your team is overburdened, creating gaps in coverage and diminishing depth of investigations. MSPs thus fill a need based on their ability to provide experienced analysts who see a broad range of attacks and work closely with technological and intelligence partners to fine tune detection methods and methodologies to provide increased value. The other major benefit of MSPs is their ability to provide more coverage at a cost savings since they’re watching over multiple environments and a bigger staff translates to more efficient approaches to sharing workloads. MSPs, of course, are not all created equally and standards of alerts per analyst per day or analyst hours per client will vary across providers and service model. Finding the right fit that is flexible enough to meet your needs and established enough to provide battle tested detection and response methodologies and processes can provide significant ROI for your security dollar while alleviating the workload on your trusted in-house security resources.
Secure Sense remains one of Canada’s fastest growing IT Security service providers offering a variety of tiered service options for our suite of managed security services. If you have any questions or want to find out more about our services, contact firstname.lastname@example.org.
For more blogs like this, check out:
Secure Sense is the security provider that cares. We are a team of experts with a passion for IT and protecting your organization is what motivates us daily. If you have questions or want to learn more about how we can improve your organization’s security, our services or just want to chat security please give us a shout. If you’re looking to guest blog, please send an email here.