The managed security service provider market is crowded with 5,000+ companies worldwide offering some degree of MSS. That’s good news and bad news for customers. Competition should drive quality of service up across the board. But it also presents a paradox of choice, and many customers find themselves with buyer’s remorse.
A good MSSP, however, is invaluable. The variety and volume of security technologies available create a web that becomes too complex and costly for the enterprise to manage and maintain itself. Complexity, after all, is the enemy of security.
What Is an MSSP, and What Is It Not?
Before we begin, it’s important to understand what you should expect from your MSSP.
At its most basic, a managed security services provider is an outsourced partner who monitors and manages security technology on behalf of the enterprise to aid in incident detection and response. MSS, however, can be much more than that. The right provider will understand the bigger security picture for the enterprise and be able to contextualize the threat, reduce time to detect the breach, and, ultimately, lessen its impact.
Most traditional MSSPs aren’t set up to achieve those outcomes, however. Instead, they’re comprised of bolted-on services primarily driven by sales opportunities. A customer purchases a large amount of technology and asks if they might also be able to manage that technology. Then another technology is purchased, and more services are created and sold.
The problem with this approach is it simply shifts the complexity to a different team. That team may have more technical knowledge, but the services are still siloed and independent from the total security strategy and ecosystem.
Understanding the differences in approaches is critical to the long-term success of your MSSP relationship. So how do you tell a good MSSP from a bad one? Here are seven red flags to look for.
Listen to our Fire Your MSSP Webcast here to learn more.
7 Signs It’s Time to Move on from Your MSSP
#1 Their portal has an ugly interface.
Forgive us for being vain, but the usability of the MSSP portal absolutely matters. The portal should be beautiful, easy-to-use, and, most importantly, provide value and context from the very first screen. Many portals today are outdated and not user friendly. If your MSSP has a portal, and you never log into it, that’s a problem.
#2 They are just an alert factory.
Is your MSSP simply “alerting” you to alerts? You deserve more! Your MSSP should be able to provide insight and context as to why that alert is or is not relevant to you. If the alerts you receive are generic and templatized, you’re essentially paying your MSSP to manage escalations.
#3 They can’t give you a unified view of incidents across environments.
In this day and age, providing management and visibility across environments—IT, OT, Cloud, etc.—is table stakes. If your MSSP can’t give you a unified view of incidents across environments, or if they can’t provide security visibility regardless of where your data resides, it’s time to move on.
#4 They say they do threat hunting, but can’t prove it.
Threat hunting has become a buzzword that MSSPs use to lure in prospective clients. But can they actually back it up? Threat hunting should not be abstract. In our case, we show clients exactly which threats we’ve detected and relevant incidents right in their portal. This should be the norm, not the exception.
#5 They have restrictive SLAs and a nickel-and-dime attitude.
This one is pretty simple. If your MSSP is holding you to an SLA, or if their own SLAs are prohibitive, they do not have your best interest at heart. Similarly, if they charge for every extra hour or request outside your retainer, they’re loyalties lie with their bottom line, rather than your security wellbeing.
#6 They can’t give you real-time visibility into the service you’re paying for.
Do you know if the services you were promised are being delivered? If service was interrupted, would you be able to tell? If not, it’s time to look for a better provider.
#7 You’re only with them because they were easy to get through procurement.
Would you believe that often customers don’t actually choose their number one MSSP? It’s true! Customers often end up choosing the MSSP that’s best from a budget, procurement or MSA perspective, rather than the one that offers the best services. With a service that you’ll interact with nearly every day, it’s important not to fall into the “procurement trap.”
Kudelski Security Recognized as Leader in The Forrester Wave™: Midsize MSSPs, Q3 2020
3 MSSP Requirements You Shouldn’t Compromise On
If you decide it’s time to let your MSSP go, it’s important not to repeat the same mistakes you have in the past. Here are three criteria to add to your checklist when selecting an MSSP.
Modern interfaces and collaboration tools. Today’s security engineers have been raised on mobile devices and chat apps. Streamlining the user experience and offering more real-time collaboration will ultimately lead to better client satisfaction.
Tailored, strategic service. An MSSP that customizes its services to your specific environment and is committed to your long-term success will ultimately be more successful than one that relies on a more transactional approach.
Honesty and transparency. Your MSSP will likely not be able to “do it all.” There may be areas where your team is stronger or where a technology vendor may be able to provide better service. Your MSSP should work with you to define and shape requirements rather than claim they can check all the boxes.
Automated detection will fail. This is not a FUD (Fear, Uncertainty, Doubt) statement designed to strike fear into the hearts of CISO’s, it’s a fundamental problem that’s unlikely to be solved in my lifetime. This problem is not limited to technology alone, sometimes it’s a failure related to process or people, and sometimes it’s a murky mixture. Add any sort of complexity to the mix and the odds become greatly stacked against us.
Regardless of the reason, these factors can result in a failure to notice something bad happening in our environment and puts us in an awkward position. The investment we made to protect ourselves works as intended, but only most of the time.
As security professionals, is it time to admit that we can’t spend our way out of being vulnerable to a breach; as security vendors and service providers, is it time to admit that we can’t actually stop every breach?
IFTTT (If This Then That) or what?
This doesn’t mean we shouldn’t have great technology, people, and processes helping us to make decisions about the activity going on around us. Air disasters have dramatically and steadily declined over the past couple of decades. This is mostly due to advances in pilot training, the design of the planes themselves and fly-by-wire automation technology that most come equipped with today. However, accidents still happen; airspeed indicators freeze over sending instruments into chaos prompting pilots to chase down problems and react in ways that aren’t necessary to resolve the actual problem thereby making the overall situation worse.
We are in a similar situation, great technology that keeps us safe, well-trained operators following a solid process, and automatic detection of most threats.
At this point our conversation can go in many directions, perhaps we’d talk about Risk Mitigation, Security Control Frameworks, the future of AI and Machine Learning, blockchain, next-gen, virtual reality, etc. but you already hear enough about those. I want to talk about this problem from a Managed Security Services Provider perspective.
Does MSS drive value to its clients and are consumers of Managed Security Services expecting enough of their MSSP?
MSSP’s, in general, are not delivering on their promises. “We are an extension of your team”: hardly, as nearly every time you talk with your MSSP it involves explaining something you’ve already explained many times in the past. “You can take advantage of our wide visibility into a large client base to realize improvements in our detection capabilities for you”: doubtful, most MSSPs don’t have the infrastructure or process in place to ensure this actually happens. “We don’t just throw alerts over the fence to our clients”: no comment necessary here, I imagine.
Truth is that MSSPs struggle to provide value. The majority of MSSPs were created when a client opportunity came up to manage and monitor a technology, and due to this, most are only built to monitor security technology and the alerts it generates. This continues throughout the life of the provider. Got a new technology you need managed? MSS will take it on!
On the other hand, consumers of MSSP services have been conditioned to expect that the value of these services is in the expansion of their security device management and monitoring to 24×7 by a larger set of eyes. This is a great expectation, but what some may not realize is that an MSSP will have the same struggle to contain technology sprawl as any enterprise. The more technology an MSS manages and monitors the harder it is to be effective and efficient at doing so. The complexity of it all becomes overwhelming and service delivery suffers as economies of scale disappear. MSSP’s compete in the same job market as everyone else, so this complexity leads to stress and job dissatisfaction which inevitably leads to analyst turnover, only exacerbating the problem. It might be interesting to note that clients tend to overlook blips in service during the duration of the contract because the value is in the coverage, not the actual outcome of the service. At renewal time, however, the realization that little value was delivered is exposed and many organizations look elsewhere (or internally) for a SOC.
These are just some of the problems with legacy MSSPs (yes, there’s more) and with over a decade of experience working for some of the biggest and best, I consider them lessons learned. When we came to Kudelski Security in 2016 we asked for and were granted the opportunity to stop selling our MSS and take a hard look at our service model and at the MSSP vertical in general. With the lessons learned in mind, we went about the process of rebuilding everything on top of our Cyber Fusion strategy. Sitting together in many (many!) meetings a fundamental and critical objective bubbled up. We need to deliver value to our clients, not just the perceived value based on extending coverage of internal teams but real value based on business outcomes that reduce overall risk. To do this we needed to understand how to contextualize the modern threat, detect a breach quickly, and limit the impact.
Automated detection will fail and we should assume breach, this is the genesis of our strategy to tackle delivering those business outcomes. When we started to work on our infrastructure, our goal was to have the top Threat Monitoring Service in the world. We built in the capability to ingest business context just as easily as we could ingest curated threat intelligence. Luckily Kudelski Security provided us with a team of 30 DevOps engineers dedicated to MSS.
If an organization is monitoring junk, sending that junk to an MSSP doesn’t make it better so we created a set of standard Use Cases which we could deploy regardless of technology as well as the capability to customize Use Cases as needed so our clients could consume alerting with consistency across their environment. We see the network perimeter as deteriorated, so we placed extra focus on the endpoint by developing Managed EDR and Attacker Deception Services, which landed us in the 2017 Gartner MDR Market Guide. By the way, we do have a select set of great technologies we manage as well. This list is kept intentionally small for the reasons we covered above.
If we had stopped there, Kudelski Security would be a great MSSP; we wanted to be greater.
Challenge the MSSP vertical to change.
Fundamentally I want to see all MSSP’s better protect their clients. To induce this market change we provide Threat Hunting as part of our Threat Monitoring Service at no extra cost.
We believe this is what every MSS, every SOC, and every security team should do regularly because automated detection will fail and we must assume breach.
Threat Hunting is an integral part of Threat Monitoring and as such should not be separated on a pricing sheet.
Our hunting is not just marketing lip service either, it comes in 3 flavors and they are all included with our Threat Monitoring.
- We have a set of Threat Hunting use cases which we monitor for anomalies 24/7/365
- We meet Monday – Friday every week to identify noteworthy threats to hunt. It could be based on input from our clients, from what we’ve seen in the intel community, or what we’re seeing with fast-breaking threat events such as notpetya, wannacry, etc.
- We enable every analyst regardless of level to hunt, at any time, based on their hunches and intuition. If you see something interesting, hunt for it.
Our threat hunting is performed by our own MSS Analysts and not a separate professional services team who mostly do point in time projects. We are always hunting, searching for that clue, that breadcrumb, that something is amiss. We’ve found hidden threats otherwise missed by monitoring. Hunting also allows us to continually improve as many of our hunts have resulted in new monitoring techniques. Allowing everyone to hunt has also increased the job satisfaction of our analysts, virtually eliminating turnover.
If it works for us, it can work for everyone and it should be a normal part of your threat monitoring program.
Francisco Donoso, our lead MSS Architect is writing a follow up to this post titled “SIEM is dead, long live SIEM”. He’s got some great content that emphasizes the work we’ve put into the some of the technical ideas behind what we are all about as an MSSP.
Automated detection will still fail, and breaches will still occur, but with our approach, we can contextualize the threat, reduce the time it takes to detect a breach and limit its impact.
MSSPs out in the marketplace, consider this a challenge. We hope you will accept?
One year ago, we sat around a big table at The House of Blues Foundation Room in Mandalay Bay, Las Vegas, meeting with potential clients and partners and telling them the Kudelski Security story. In the United States, it’s quite a short story, but reality is that we’re just a new chapter in a decades-long saga which is Kudelski Group in Switzerland. Founded 65+ years ago by a Polish inventor named Stefan Kudelski, Nagra (which means “record” in Polish) would go on to set the defacto standard in analog sound recording. Inventing one of the world’s first high-fidelity recording devices was not enough for Stefan, it had to be the most precise, true-to-sound and most reliable recording device on the market. His hard work, alongside that of his team, led to numerous industry awards including winning three Academy Awards and two Emmy Awards. Yes, our trophy case has three Oscars and two Emmy’s in it.
Kudelski Group has a knack for recognizing shifts in the market. Understanding that digital was rapidly overtaking analog, the Group shifted its business model accordingly. Digital content created new challenges for producers and distributors, one of which is how to protect it from piracy and theft. Kudelski began to create technology and converged systems that provided security and encryption to content and media. While shifting from analog was a major step, the Kudelski Group core remained the same: an Engineering company.
We’ve added physical access security and lots of cool engineering and encryption technology since then and in 2012, leveraging decades of experience and expertise gained from defending, monitoring, and protecting nearly 400 million devices against digital piracy, Kudelski Security was born.
Kudelski Security and the Cyber Fusion Center
Our Cyber Fusion Center (CFC) is at the heart of our cybersecurity offering. The CFC takes business intelligence, threat intelligence, and security content and merges them to produce interesting, relevant, and contextualized threat information to our clients.
The next step in our evolution was to take on the largest security market in the world, the United States. In early 2016, we started planning our new approach and how to organize our services. We looked at the state of the MSSP vertical and realized after more than 10 years it hadn’t changed much. MSSPs were still content just trying to prevent breaches and while a noble goal, it wasn’t working. Organizations were still getting breached and the rate was accelerating; something had to be done differently and with a new perspective.
A New Perspective to Deliver Different Results
We built our services around the way an attacker does what they do. We organized into pre-breach, breach, and post-breach pillars, each with its own set of services. We recognized that with a strong post-breach offering we may just be able to become predictive in our analysis, strengthening our pre-breach and breach detections including our Threat Monitoring and Hunting which lies across all three pillars of our strategy. We included Threat Hunting in our Monitoring at no additional cost as it’s our perspective (and the Kudelski mantra) that a shift is necessary. The MSSP world needs to be prepared to handle the new challenges presented by advanced adversaries.
We pay special attention to the post-breach pillar because that is where attackers spend most of their time. We added Endpoint Detection and Response and Attacker Deception to complement our business and threat intelligence from our clients and it’s working. We’ve been recognized by Gartner in their recent Market Guide for Managed Detection and Response. You might be saying “So what?” but you should give us a look as we are one of the only pure-play MSSPs included, and the only representative vendor that provides hunting, deception, prevention, detection, and response as part of an overall post-breach strategy. We assume breach, which can be a tough pill to swallow, but necessary if we want to reduce the time it takes to contextualize the threat, detect a breach, and limit its impact.
For More of the Story…
There’s much more to this story that I’d love to share but let’s do that at BlackHat, DefCon, and BSides in Las Vegas. We’re back at the House of Blues Mandalay Bay again. I’m bringing some of the best minds in modern security with me, including Francisco Donoso, and we would enjoy talking with you about what our Cyber Fusion Center is all about and how our EDR strategy and partners can prepare your organization to face today’s most difficult threats. So, let’s meet up either for a one-on-one meeting, or at our debrief session at the Four Seasons Hotel, Monday 24th July.
See you there!
- Microsoft Windows 10 Enterprise includes a feature called “Credential Guard”. This feature can prevent certain attacker tools from compromising administrative credentials using well known techniques such as a Pass the Hash attack. Having this feature enabled would have prevented NotPeya from harvesting local credentials to spread within a local network (one of the methods used by the worm component). More Information: below:
- Microsoft is also releasing a new feature for Windows 10 in September/October which enables certain files and folders and should provide end users and enterprises another tool to protect against ransomware. This feature is being called “Controlled Folder Access”. More Information:
- The malcode used to create the installation ID which would presumably then be used to create a customized decryption key for each victim was randomly generated and useless. Kudelski Security reiterates: DO NOT PAY THE RANSOM.
UPDATE: 5:30 P.M. EST
As we often see in these global outbreak and response scenarios, information can change quickly. The following are a few updates based on what we’ve learned since our initial advisory.
- The ransomware is not actually petya.a. It does use some its components but the malcode used in today’s attacks was built to look like petya instead
- There does appear to be a kill switch in this first variant that stops the local encryption. The malcode looks for a copy of itself in C:\windows. The file name has been identified as perfc.dat. Unfortunately, it still appears to attempt its spread across the network.
- There are reports that “patient zero” is a finance technology company based in Ukraine
- We have seen reports of thousands of devices compromised within a just a few minutes at several different organizations
- CVE-2017-0199 is not part of this malcode. It was mentioned early on as related but was likely a misattribution due to near simultaneous detections of different attacks
- General steps of the infection
- ARP Scan
- Check/Get credentials (mimikatz or similar)
- Psexec to execute WMI
- If psexec fails use eternalblue
- Reboot to encrypt
- If clients can catch the reboot before it completes, it has been reported that files can be saved by not turning on the computer and recovering files offline.
- We urge caution when looking for some the common IOC’s that have been released so far. Some of them will generate high volumes of false positive alerts, in particular those related to CVE-2017-0199 (see #5)
- The malcode used a fake MS certificate and XOR to avoid most of the current AV detection routines.
- DO NOT PAY the ransom. The email associated with the bitcoin wallet is not valid.
- This attack and the code associated with it is far more professional and dangerous than what we saw WannaCry.
- Expect to see new and creative ways that attackers can automate propagation of malcode through an environment.
wCry2 Ransomware spreading via EternalBlue (MS17-010)
Update May 13
Data was coming in very quickly on Friday and while we worked to provide timely and reasonable information we know now more about what happened and how the Wana Decrypt0r 2.0 ransomware outbreak managed to escalate so quickly.
First some good news: The malware, once executed checked for the existence of a randomly generated domain. If the domain did not exist or could not be reached, the execution of malicious code continued. If the domain existed and was accessible, a kill switch was activated and the infection was halted. A malware blogger and reverse engineer from the U.K registered the domain which effectively slowed the malware spread in the U.S. Unfortunately, many anti-virus vendors began to block the domain, unintentionally allowing the installation to continue, realizing the error some of the anti-virus vendors have removed the block and now sinkhole the domain instead.
More information here:
The unfortunate news is that there are now samples emerging that no longer contain the domain based “kill switch”.
An example of this new variant is available here:
Additionally, after further review of the malicious binaries, we’ve identified that all RF1918 (private) netblocks as well as randomly generated internet netblocks are also scanned looking for further propagation avenues. This means that organizations could also potentially be affected by way of site-to-site VPN connection with business partners or vendors. The ransomware has also spread via guest wifi, thus users should be cautious as it is possible they could be affected while connected to an open wifi hotspot.
Researchers have noted that WannaCry 2.0 is not the actual worm. The worm is the MS17-010 “spreader”. WannaCry 2.0 is dropped by the “spreader” which can also be used to drop other binaries and files. Thus, it is extremely critical that organizations apply the MS17-010 patches as quickly as possible.
Mac OS and Linux users running Windows VMs or Wine are also affected if not patched.
Along with the ETERNALBLUE components, the dropper also calls out and downloads DOUBLEPULSAR. Organizations affected will want to check for the existence of DOUBLEPULSAR once the initial attack is remediated. There is a free script available to check for this located here:
The Wana Decrypt0r 2.0 ransomware campaign utilized 3 Bitcoin wallets and as of today they show modest returns. Note: there is no indication that paying the ransom actually provided the user with the keys to decrypt their data and some researchers reported that users had to interact with a human via phone or web chat to negotiate. In the ransom note, the attackers mention that if someone is “too poor” to pay that their files will automatically decrypt in 6 months.
The following Bitcoin wallets have been linked to this ransomware campaign:
The Global response to this campaign has been swift and effective, unfortunately, too late for a large number of European organizations. Microsoft released updates to its malware protection engine to block the malware. Additionally, Microsoft has unexpectedly released security patches for EternalBlue and MS17-010 vulnerabilities for the unsupported Windows XP, Vista, Windows 8, and Windows server 2013 operating systems.
When unfortunate events like this take place, it’s easy for information security practitioners to point fingers and assign blame but the global information security community would be better served by helping organizations understand and avoid these situations in the future.
Moving forward, Kudelski Security expects to see most if not all ransomware and malware families using similar techniques to spread quickly and infect large numbers of users and organizations.
This global ransomware outbreak is a stark reminder that organizations must have the basics covered. Organizations must review and evaluate their vulnerability and patch management programs to ensure confidence, comprehensiveness, and effectiveness. Security patches are a fundamental and critical foundation of any organizations security program and should be tested and applied quickly. Organizations should also perform a “health checkup” and review backup strategies, test backups regularly, and ensure backups are easily accessible while also being protected from encryption and deletion. Also, organizations should review and reevaluate what traffic is allowed to and from the internet.
Once the basics are covered, now is the time to start looking at some of the newer endpoint protection platforms that rely on behavioral indicators that executables could be malicious instead of solely relying on signatures.
Now is the time to take a look at security, review and apply the basics, and then pragmatically strengthen its effectiveness.
On May 12 2017, a widespread cyber-attack utilizing the WCry2 ransomware, also known as Wana Decrypt0r 2.0, began spreading across the globe. At the time of this writing, the Ransomware has currently impacted organizations in 99 countries and continues to spread. Wana Decrypt0r 2.0 uses the EternalBlue exploit (MS17-010), released by the Shadow Brokers in March 2017. This SMB exploit is used to attempt to infect other machines within the same network and to scan for, and infect, potentially vulnerable Windows machines on the internet.
Wana Decrypt0r 2.0 is a highly effective ransomware variant that encrypts several file types, making them inaccessible to the user, and demands a payment of $300 U.S dollars in Bitcoin to decrypt the files.
Additional details on Wana Decrypt0r 2.0 and EternalBlue (MS17-010)
Wana Decrypt0r 2.0 is a variant of the WannaCrypt ransomware family that is currently being spread by exploiting EternalBlue (MS17-010). Wana Decrypt0r 2.0 encrypts several file types on an infected computer demands a ransom of $300 USD in Bitcoin to decrypt the inaccessible files.
ExternalBlue is an exploit that takes advantage of previous vulnerabilities in SMB, a critical protocol for Windows Systems. The exploit allows for the remote execution of malicious code on vulnerable systems without requiring any use interaction. The ExternalBlue exploit requires that the systems be vulnerable and expose the SMB service (enabled by default on Windows systems) to successfully compromise a system and replicate across network infrastructure to other vulnerable Windows systems.
At the time of this writing, this cyber-attack has quickly spread to 99 countries across multiple regions of the world. This global threat arrives in the form of a phishing email with a malicious attachment, once the malicious attachment is opened a dropper begins to download and unpack the actual ransomware code. The ransomware encrypts the user’s files, scans the networks to which the machine is connected, and uses the EternalBlue exploit to spread across organizations with unpatched Windows systems.
Kudelski Security has observed several industries and regions being specifically targeted by this ransomware campaign. Kudelski Security has intelligence that indicates that other ramsomware campaigns are activity integrating more of the Fuzzbunch framework exploits into their code.
As of this writing, according to internet scanning tool Shodan, there are approximately 2.4 million internet exposed systems which may be vulnerable to this exploit.
Mitigation and Response
Microsoft released a patch for the EternalBlue and other critical remote code execution vulnerabilities in March 2017 as part of Microsoft Security Bulletin MS17-010.
Kudelski Security recommends that clients immediately apply the patch for MS17-010. For organizations unable to quickly apply the Microsoft patches, potential mitigations include using a GPO to apply Windows Firewall rules to block inbound SMB connections on all unpatched endpoint systems and limiting SMB connections between servers.
Kudelski Security also recommends limiting all inbound and outbound communication on UDP ports 137 & 138 and TCP ports 139 & 445 on internet firewalls in order to reduce exposure and the slow the propagation of this ransomware.
Kudelski Security recommends backing up all files, including systems already affected by the ransomware in case future decryption tools become available.
Additionally, Kudelski Security recommends that organizations evaluate their vulnerability management programs to ensure that updates and patches are tested and applied quickly once they are released.
The Kudelski Security Cyber Fusion Center has ensured all managed and monitored security devices are updated with detection signatures and methodology to detect the uses of the Wana DeCrypt0r 2.0 ransomware and exploitation with ExternalBlue and other recent Windows exploits.
VirusTotal analysis of malicious PDF
Learn More about Kudelski Security’s Managed Security Services (powered by our Cyber Fusion Center)