What is threat hunting in cyber security?
In cyber security, threat hunting is the act of proactively searching and monitoring networks, systems, endpoints, datasets etc. to identify any malicious behaviours or patterns that are not detected by existing security tools.
In threat hunting, threat intelligence and data such as Indicators of Compromise (IOCs), Indicators of Attack (IOAs), attack tactics, techniques and procedures (TTP) are monitored and observed within the organisation by automated security tools and human effort to stop potential threats before the breach actually happens.
Threat indicators such as unknown or strange IP addresses, malware, phishing emails, unusual network traffic, unusual resource access etc. all help the threat hunters to determine whether an adversary is trying to attack their IT environment or if the environment is already compromised.
Threat hunting and threat intelligence – the difference
When we talk about threat intelligence, it refers to the data collected about successful and unsuccessful as well as attempted cyber attacks. These data sets are usually collected using a number of different security tools and solutions with the help of machine learning and artificial intelligence (AI).
Threat hunting utilises this threat intelligence feed or threat intelligence indicator searches to conduct system and network lookups for malicious behaviours (this can also include entity behaviour analytics). In other words, threat hunting is based on the threat intelligence gathered.
How does threat hunting works?
The first and foremost step into threat hunting for any organisation is to have enterprise security systems in place that are collecting data and threat intelligence. Only when such information is available can the organisation perform an effective threat hunting activity.
Along with automated security systems in place for cyber hunting, human threat hunters are employed by an organisation to search, log, monitor and identify cyber threats before the threat materialises and causes harm.
These threat hunters are cyber security professionals who understand the company’s operations and comb through security data to protect the company. They search for hidden malware attacks, backdoors and malicious actors as well as look for suspicious patterns and activities within the company’s daily operations to identify any and all forms of threats. Once a threat is identified, the threat hunters also help in patching the systems to prevent the cyber attacks from reoccurring.
Threat hunting playbook activities for those interested
Here are examples of some of the key indicators of compromise (IOC) / indicators of attack (IOA) that businesses can monitor for finding any unusual and suspicious activities:
- Unusual outbound network traffic
- Unusual geographical location based activities
- Use of privileged accounts during out of business hours
- Registry or file-system changes
- HTML response sizes, large number of requests targeted at same page, script or endpoint
- DNS requests such as DDoS activity, request anomalies
- Anomalies in the privileged user account activities
- Database read volume size upticks
Different threat hunting methodologies
As a rule of thumb, threat hunters work under the assumption that the adversary or malicious activity is already taking place within the systems, by doing so they initiate the investigations and try to look for unusual behaviours indicating a compromise.
Threat hunting usually falls into three main categories as defined below. These approaches involve a mixture of human effort, threat intelligence gathering, crowdsourced information as well as a touch of data analysis and machine learning.
Hypothesis-driven investigation
Threat intelligence is usually shared amongst organisations to help better protect cyberspace as a whole. Once a new threat has been identified by any organisation, information related to that threat is made available through crowdsourced data. This new threat intelligence information includes the attacker’s tactics, techniques and procedures (TTP), them the threat hunters from different organisations look for that specific TTP in their environments to ensure if they are compromised or not. This type of threat hunting is known as hypothesis-driven investigations.
Investigation based on Indicators of Compromise (IOCs)
In this approach, threat hunters use the existing Indicators of Compromise (IOCs) or Indicators of Attack (IOAs) and check them against the organisation’s current activities and operations to uncover any malicious behaviours or hidden attacks taking place.
Advanced analytics and machine learning
Since it is not humanly possible to go through gigabytes of information in a short time, this third approach utilises machine learning and data analysis to comb through a huge amount of collected information in order to identify malicious irregularities and events. These anomalies then become hunting leads for threat hunters to start their investigations.
Types of threat hunting
When a threat hunter identifies a trigger, they can go about their further investigation in a deeper manner by following any of the below types of threat hunting, i.e. structures, unstructured or even driven.
Structure hunting
A structured approach to threat hunting is based on the analysis of Indicators of Attack (IOAs) and tactics, techniques and procedures (TTPs) of an attacker. By using this approach a threat hunter is usually able to identify threats in the environment before the actual attack takes place. This hunting technique utilises the MITRE ATT&CK framework.
Unstructured hunting
An unstructured hunt is initiated on the basis of a trigger, this trigger can be an Indicator of Compromise (IOC). When a trigger is notified, the threat hunter usually looks for and analyses pre and post-detection patterns to identify the attack.
Situational or entity driven hunting
When an organisation conducts an internal risk assessment activity or a vulnerability analysis, the resulting trends are called a situational hypothesis. On the other hand if TTPs are gathered from crowdsourced data then these are known as event-driven leads.
Data collected by both of these situations can be used as triggers for a threat hunter to carry out an investigation within the organisation.
Why is it important to utilise cyber threat hunting
Organisations invest a great amount into the latest and top-notch automated security solutions. Although these tools and solutions immensely help Security Operations Center (SOC) analysts in identifying the majority of the threats, roughly up to 80%, however, the remaining 20% can still go undetected.
This 20% are advanced threats and can materialise into much more adversely impacted threats than the rest of the 80% combined, as these threats go unnoticed and can cause significant damage over time. Given enough time and resources to such threat actors, they can go unnoticed for up to 280 days on average.
Such cyber adversaries usually sit in the organisation’s systems, for weeks and months waiting for the opportunity to cause maximum financial or reputational damage. In such scenarios threat hunting comes into play, and threat hunters monitor and analyse all the information within the organisation’s environment for any malicious patterns and behaviours.
Using threat hunting capabilities can save organisations millions of pounds by stopping such security incidents. Therefore, it is a wise move for established security teams to provide threat hunting training to their teams including development training for improving their threat hunting service.
The threat hunting steps
Cyber threat hunting can be broken down into five basic steps:
1. Hypothesis
The threat hunting begins by a hypothesis or statement that a specific threat might exist in the organisation’s environment. The threat hunter then uses his experience and knowledge to decide how to go about identifying this threat and building a logical path to detection using the IOCs, IOAs and TTPs etc.
2. Collection and processing of data and intelligence
Before the actual threat hunting process begins, it is important to collect quality intelligence and data regarding a threat. The threat hunter must develop a plan to collect, centralise and process the required data.
Security Information and Event Management (SIEM) solutions may be used here to provide valuable insights into activities carried throughout the organisation’s network environment.
3. The trigger
Once the threat detection tools discover an anomaly, the hypothesis is converted to a trigger. This is when a threat hunter starts the investigation against a system, or specific area of the network which is suspected to be compromised.
4. Investigation
The threat hunter then starts the investigation, trying to identify the affected system, the entry point of the cyber attack and the impact the attack could have. Security technology such as Endpoint Detection and Response (EDR) can be of use in this step to analyse systems in depth.
5. Response and resolution
After data is gathered that confirms a malicious activity exists place a proper response mechanism is initiated. Here actions to stop the attack from executing and propagating are taken place, such as removing malware files, isolating the affected systems, restoring the systems to a known secure state, updating firewall and IPS/IDS rules, and installing security patches, fine-tuning security configurations etc.
The threat hunting maturity model
A threat hunting maturity model defines the quality, state and effectiveness of an organisation’s threat hunting practices and capabilities. It helps an organisation to see their current capabilities and the improvements and skills they should further enhance.
SANS institute identifies the maturity model as such:
- Initial – Level 0
The organisation is completely reliant on automated tools and does not have manual threat hunting capabilities. They also do not routinely collect data.
- Minimal – Level 1
The organisation incorporates threat intelligence and indicators. They have a moderate level of routine data collection.
- Procedural – Level 2
The organisation uses human threat hunting as well as automated tools, along with following successful data analysis procedures created by other entities. They have a high level of routine data collection.
- Innovative – Level 3
The organisation creates new data analysis procedures and contributes to threat intelligence feeds. They have a very high level of routine data collection.
- Leading – Level 4
The organisation has automated the majority of the successful data analysis procedure and has multiple capable threat hunters. They have an extremely high level of routine data collection.
Prior requirements before starting threat hunting
Before an organisation starts to actively hunt threats, it is important to meet a few prior requirements to achieve a successful threat hunting program, as described below:
Human capabilities:
For effective, efficient and proactive threat hunting, human element is a necessary requirement. Organisations should employ cyber security professionals having the required capability and skills to discover and resolve threats with accuracy in a timely manner.
A threat hunter must have a sound understanding of the organisation’s IT environment along with substantial knowledge of the ever growing threat landscape. They should be creative and intuitive to grasp new methods of attacks that may occur.
Organisation model:
The organisation model varies from company to company and is dependent on what the management chooses. The organisation size along with budgets and the availability and capabilities of security analysts determine how the model would look like.
According to SANS “Threat hunting entails a more mature organisation with a defensible network architecture, advanced incident response capabilities, and security monitoring/security operations team.”.
Tools and technology:
Apart from the human element in threat hunting, it is necessary to have the appropriate tools and technology deployed to help threat hunters in their tasks. Generally, enterprise threat hunters use comprehensive endpoint security solutions along with other security monitoring management tools.
These can include tools such as SIEM solutions, statistical analysis tools such as SAS programs, threat intelligence providers (TIPS), threat intelligence feeds and data banks, known bad IP addresses, publicly disclosed vulnerabilities, EDR solutions etc. Threat hunting solutions are available in the market from mature teams who have a history of offering stand-alone or managed threat hunting services.
These tools and security technologies are generally siloed and require human expertise to manually comb through the data and come to a conclusive decision.
Threat intelligence and data:
For an organisation to manage and develop threat hunting capabilities it is essential to routinely collect data from the entire IT environment for processing. The threat hunters must establish a baseline or normal behaviour for the organisation and anything that varies from that baseline would be considered an anomaly and must be analysed further.
Threat intelligence feeds from crowdsourcing and gathering IOCs, IOAs, TTP from the public also greatly benefits an organisation, as they can use this information and check whether their own organisation is affected by similar malicious activities or not.
Tools and platforms for threat hunting
Cyber threat hunters use a combination of tools to help them in discovering hidden threats, these tools include SIEM, MDR and EDR solutions, security analytical tools and other security monitoring tools.
- Security monitoring tools
Monitoring tools include firewalls, antiviruses, endpoint detection and response tools as well as managed detection and response tools. These tools monitor the IT environment and help to collect security activities and other data which the threat hunters use in processing.
- SIEM solutions
SIEM solutions help to parse the raw data collected from the monitoring solutions into meaningful information, they provide real time analysis of security threats. Using a SIEM solution, threat hunters can uncover anomalies and other irregularities for a deeper investigation.
- Security analytical tools
These are statistical and intelligence analysis software that provide the threat hunters with a visual report using charts and graphs, which make it easier for the threat hunters to correlate data and patterns.
Wrapping up
Threat hunting is a great innovation skill set and activity to add to organisations existing security process, as it can help organisations identify and stop a data breach before it even happens. Although highly skilled and the best threat hunters are needed for the job, with the right threat hunting approach business can keep themselves secure from the latest Advanced Persistent Threats (APTs).
Get in touch for an independent digital attack surface assessment for your organisation before it’s too late.
