9 ways to eliminate SIEM false positives

If you have a SIEM or are about to implement one, then you're probably struggling with one of the biggest challenges in cybersecurity—false positives. 

According to Orca Security's 2022 Cloud Security Alert Fatigue Report, cybersecurity teams are inundated with cloud security alerts—59% of respondents receive more than 500 of these alerts per day. And because of the sheer volume of alerts needing to be addressed, 55% of respondents said that very real, critical alerts are being missed, often on a daily or weekly basis. But the problem doesn't come from a lack of people—it comes from false positives.

Useless alerts often take the same amount of time to investigate as real ones. The traditional approach—which a lot of MSSPs still use today—is to hire a huge team of people to attempt to review every alert. Given the survey results and recent cybersecurity headlines, how well do you think this works?

If you want to catch cybersecurity threats in your environment, you have to focus on eliminating false positives so that the security experts you do have can focus on remediating real problems. As we've seen, this is a process and technology issue. Simply adding more people is not the solution. Cybersecurity threats and attacks need to be dealt with efficiently, so today, we're going to go through our top nine tips for eliminating false positives in your SIEM environment.

1. Properly define false positives

An accurate alert or notification should be defined as anything that requires immediate action—and that's it. Anything else alerting you is a false positive. Not because it didn't happen but because there is no real action to take. Using this definition rather than just "what is an accurate alert regardless of criticality" will dramatically help you streamline your IT resources as it pertains to alert management.

This is one of the hardest concepts for security operations managers to accept. To help, ask yourself (or your  /provider) this question for every possible alert: "When the team gets this alert, what action will they take?" If the answer is "uh…" or "none, but….", then that alert would be a false positive.

Don't worry—we're not saying you will never see this information. But to help avoid alert fatigue, it should show up on a report your team regularly reviews, not as an alert that opens a ticket. 

2. Get rid of rules you don't need

This sounds obvious, but you would be amazed how many people install a SIEM and leave every default rule turned on. Many rules are designed specifically for a particular network device or IT system. If you don't have that system or device in your network, disable the rule! Leaving it enabled will only create false positives and lead to alert fatigue. While you're at it, make sure the rules that remain active actually detect what you think they do. Many default rules in a SIEM are often mislabeled or have other errors, so check carefully!

3. Tune the rules to your specific environment thresholds

Rules are really nothing more than, "This thing happened this many times over this period of time…." or a combination of such things. The appropriate "counts" and thresholds in your environment are very different from other environments. These thresholds need to be adjusted exactly between what is "normal" traffic in your environment and what is abnormal traffic. This requires setting up a network baseline by running the system for several weeks and analyzing the traffic to know the appropriate thresholds for each rule. Believe it or not, very few companies take the time to tune their SIEM to their actual environment! The reality is many good IT folks don't know how to do this accurately, and it may require a SIEM expert.

4. Context is king

Most SIEMs don't have this capability, and it's key in eliminating false positives, so I hope you are reading this before you purchase your SIEM. Here's an example to help illustrate this:

You get an alert from your SIEM stating it has detected a SQL injection attack against one of your servers. That is serious, right?! Well, it's really only serious if you have SQL on that server. Otherwise, it is just another false positive. A good SIEM has the ability to look at the configuration of your systems to determine if an attack can be successful. Configuration management data included within the SIEM gives you an enormous advantage to eliminate some of the peskiest false positives. Ask your SIEM provider if their solution incorporates change management information and has a change management database (CMDB). This eliminates the worst kind of false positives, the sleep-stealing alerts that wake you up at 3:00am. No one wants that, and you need to understand the context of the network systems to eliminate these false positives.

By the way, if your SIEM doesn't have detailed configuration and asset information for critical context, you may want to view a live demo to see how ConnectWise SIEM™ can work for you, and contact us for a new SIEM!

5. Adjust the criticality to your environment

Remember we said that only events that require action now should be alerts and that low-level alerts and most medium-level events don't need immediate action—therefore, they should not be alerts? These should get rolled up into a report that is delivered to the right person at an appropriate frequency, perhaps weekly. With that in mind, many SIEM vendors set their default criticality to a level that's way too high for most environments. Something that is critical in someone else's environment may only be medium level in yours. Do not trust the default criticality setting. You must review this in the context of "What will we do when this alert is sent,".

6.   Use a threat feed and geolocation data

Most SIEM technologies allow you to blend outside data into the system to get higher accuracy. A threat feed can be used to increase the accuracy of events through cross-correlation and context. For example, if an IP range in a threat feed is from a known hacker cell, it can increase the criticality of that event to high. Geolocation data can also be used to increase or decrease criticality based on the source or destination of your network traffic. With this, your SIEM can automatically detect the difference between inter-office traffic, remote traffic, and foreign traffic.

A word of caution on threat feeds—A low-quality threat feed (usually free ones) can actually increase your false positives tremendously! If you're going to use a threat feed, use a high-quality one that updates regularly, is constantly cleaned of stale information, and is specific in its threat data rather than generically blocking huge network segments, such as the ConnectWise CRU threat feed. 

7. Trust your security devices

Most organizations have cybersecurity devices such as a firewall or an intrusion prevention system that block malicious traffic. Many people configure their SIEM to alert them for an event that was already stopped. If your firewall is blocking that attacker, why would you want a ticket on that? Report this somewhere, sure, but don't open a ticket only to make someone close it later. Remember, if it doesn't require action right now, you shouldn't be getting an alert.

8. Ignore low level alerts

Most low level alerts can be turned off entirely. But if there are low level alerts that you do want to track, do that with a report periodically. I'm sure you're getting tired of hearing this by now, but if it doesn't require action, you shouldn't be getting an alert.

9. Tuning is not a one-time event

Anyone who thinks that they can set up their SIEM and it will remain highly tuned is sorely mistaken. Security information and event management systems, by their very nature, require a lot of ongoing care and feeding—daily. Adjustments will need to be made when network devices are added, removed, or updated. Tuning will be needed when firmware updates occur or software is upgraded. Even if nothing changes in your environment, the threat landscape changes, which requires changes to your SIEM, not to mention your SIEM should be getting updates with new rules and rule updates that need to be applied and maintained. A properly tuned SIEM will be your greatest cybersecurity asset. A neglected SIEM or a SIEM maintained by untrained staff will be a nightmare, a huge waste of money, and raise your risk exposures. Most importantly, when you get a false positive, use it as a feedback loop to adjust the SIEM so that the same false positive doesn't show up again. If you just clear the alert and don't make a change, it will happen again and again and again. Get in the habit of adjusting the SIEM right away so you save scores of hours in the future.

It's worth it to work with a SIEM expert

While this may sound easy to do, SIEMs are very complicated. I mean, would you trust a guy to do surgery on you that had only learned from an online video? No! When you run the math, it's far less expensive to outsource management and tuning your SIEM to an expert than to do it yourself—If you get the right partner.

The good news is that this is what ConnectWise does best! Depending upon the SIEM technology you're using, we can manage and tune your SIEM initially and ongoing to ensure you practically eliminate all your false positives.

Haven't selected a SIEM yet? Download this eBook for tips on what to look for in a SIEM and how to add it as a service.

We have some amazing options for you, including our SIEM hosted by us or an on-premise managed SIEM that you host at your data center or in your cloud environment. Using ConnectWise for your SIEM needs means getting the most out of your SIEM investment and getting the best possible cybersecurity and compliance available at a fraction of the cost of doing it yourself.