Artificial Intelligence for Cybersecurity — Notes

Photo by Markus Winkler on Unsplash

Today’s article is a collection of my notes while attending the LinkedIn course entitled “Artificial Intelligence for Cybersecurity”. It is the 10th of 12 courses in the “Become an IT Security Specialist” learning path. The course is divided into 5 sections which are:

• Demystifying Artificial Intelligence for Security.
• Security Objectives and approaches.
• Leveraging AI to solve Complex Security Problems.
• Machine Learning for Security.
• Practical Considerations, Risks and Limitations.

Demystifying Artificial Intelligence for Security

Artificial intelligence is the intelligence demonstrated by machines while mimicking the cognitive behavior of human beings. An intelligent agent is a machine, software or system that maximizes the probability of goal completion, by sensing, learning and acting.

The emergence of cloud computing, big data and internet of things caused the organizations to fail to keep up with the huge amount of data to protect. The traditional approaches are no longer effective and IA should be leveraged to add an extra layer of protection.

The intelligent agent will be trained through a feed of information to solve a particular security problem. It can enhance the security by one of 3 mechanisms:

1. Predictive insights: uncovers data patterns that might be difficult to detect by humans.
2. Actionable recommendations: makes recommendations to the security professional on what actions to take.
3. Autonomous mitigation: takes a corrective action on behalf of the security professional to mitigate a security issue.

Artificial intelligence augments the current people, processes and tools of an organization. Further, it has the potential to do so at the entire continuum of security from prevention, detection, to response.

• Prevention: scan source code and components, threat modelling …
• Detection: malware, network and endpoint intrusion …
• Response: security operations, automatic mitigation …

For a machine to be considered as an intelligent agent it has to have at least 6 foundational capabilities:

1. Ability to understand natural language.
2. Ability to store information.
3. Ability to reason.
4. Ability to learn from new information.
5. Ability to see and perceive surrounding objects.
6. Ability to manipulate and move physical objects.

For security purposes, the intelligent agent doesn’t require all capabilities. The intelligent agent doesn’t need to see or manipulate physical objects. However, it needs to:

• Store and process large data (like logs).
• Identify patterns.
• Adapt.
• Make recommendations or take actions.

The learning discipline is the most important discipline when it comes to security. It relieves the security system from the limitations of programming and rule based approaches.

Security Objectives and approaches

Every organization, regardless of the industry and nature of business, share some common security goals. They follow the CIA model (Confidentiality, Integrity and Availability). Whether the attacks materialize into a security breach or not, you want to detect (or predict using AI) these attacks and quickly mitigate against them. At every stage of the security process there are lessons to be learned and there are opportunities for feedback so you can correct or fine-tune your security controls. In addition, that feedback loop is a great opportunity for retraining your machine learning models with new threat intelligence, newly-discovered behavior patterns, and attack vectors.

You prevent a security incident by applying layers of administrative, technical, and physical controls. Some of these controls are:

• Educate employees.
• Define security policies.
• Establish authentication and authorization access controls.
• Establish a secure development lifecycle.

You apply defense and depth controls when it comes to detecting suspicious activity in your network. Some of these controls are:

• Intrusion detection.
• Monitoring data.
• Malware detection.
• Monitoring social engineering attacks.

You want to have a playbook ready to execute when a compromise is detected. Some of the actions in your playbook are:

• Alert the right responders.
• Have meaningful context for triage and decision.
• Contain the spillover from the attack to unaffected assets.
• Document the incident.

There are 4 main characteristics of security problems:

1. Problem of scale.
2. Problem of context.
3. Problem of precision and accuracy.
4. Problem of speed.

Leveraging AI to solve Complex Security Problems

One of problems that face a security professional is the huge amount of data to go through while studying a system for intrusions or failures. You can rely on a number of different tools defined signs of suspicious activity, but such tools have limitations, their programmatic, rule-based approach does not scale to handle the massive amount of data being generated in a enterprise.

an IDS is used to monitor the traffic on a network. Typical IDS systems rely of a database of already known malware signatures to determine whether the traffic is considered malicious or not. If IDS fails to find a signature, or even worse, the signature doesn’t even exist in the database, it will fail to catch intrusion and their impending attack will go undetected. When you apply an IDS tool that leverages artificial intelligence, instead of searching for patterns, you are effectively creating a predictor model behind the scenes. You then deploy the trained predictor model and let the incoming logs of traffic data continuously pass through the model and protection. And finally, the deployed model determines if a new event is indeed an intrusion, or business as usual.

Data Loss Prevention (DLP) systems are used to ensure the confidentiality of business information. A traditional DLP uses a text matching approach to look for fingerprints or patterns against a set of predetermined words or phrases. An AI powered DLP is trained and then used to identify sensitive data based on the context. The machine learning model is fed multiple sets of training data. The model is fed information about semantic relationships among the words using a technique known as word embedding. The model is then trained using a variety of learning algorithms.

While being extremely cautious is a valuable trait in the security field, reporting and acting on false alarms is inefficient and dangerous. Acting on false positives not only drains the organizational resources but also distracts the rest of the security team from a real issue that may be hidden behind the scenes. We can use a phishing attack to understand the problem. The traditional approach to catch such fake websites used for phishing attacks is to compare the URL against block lists. A block list is usually maintained using webcrawlers built using heuristics approach. A website that is genuine and trustworthy is going to exhibit a pattern of attributes along three domains its reputation in the form of incoming links, certificate provider, and Whois records. These attributes can be used to build a training model and then later used by the web server to detect if the website is legitimate or not.

Security teams, not only they have to be right every time, but they must also act fast. AI helps security teams be proactive with predictive analysis. It prepares you to be one step ahead of the adversary by giving you the ability to predict a future incident and act beforehand.

Machine Learning for Security

The choice of machine learning algorithm to use depends on the type of security to be solved and the type of training data available. The data can be either labeled (the model uses supervised learning to analyze it) or unlabeled (the model uses unsupervised learning to analyze it). Types of problems vary:

• Predict a future event from past data.
• Split data into known categories.
• Discover new patterns.
• Generate synthetic data.

These 2 factors lead to a choice of a machine learning algorithm to use.

ML algorithms for

A quite common need in security space is to predict a future event based on the known data at hand. Such prediction problems fall under the broader category of so called regression problems. Some of the commonly used algorithms for solving regression problems are:

• Linear regression.
• Support vector regression.
• Random forest.
• Decision trees.
• Regression ANN.

You want to segregate events into known categories, such problems fall into a broader category of so called classification problems. Classification is the process of segregating new data into previously known categories. Some of the classification algorithms are:

• Logistic regression.
• K-near neighbors (KNN).
• Support vector machines (SVM).
• Naive Bayes.
• Decision tree classification.
• Random forest classification.
• Classification ANN.

Clustering algorithms gathers unlabeled data into groups without prior knowledge which makes it different from classification. Classification uses supervised learning while clustering uses unsupervised learning. You can break clustering problems into 2 categories pattern recognition and anomalies detection. Algorithms used for clustering are:

• K-means.
• Self-organizing maps.
• DBSCAN.
• Bayesian Gaussian mixture models.

For security purposes you might need to generate new data which is known as synthetic data. You use synthetic data for many use cases, such as cracking passwords, fuzz testing, vulnerability testing of your application, steganography, … Synthetic data can come handy when you don’t want to share the original data, but still share a representative sample with someone for testing. Some of the algorithms used to generate data are:

• Markov chains.
• Boltzmann machines.
• GANs.

Practical Considerations, Risks and Limitations

There are 3 ways AI can impact your organization:

1. Limitations and poor implementation of AI.
2. Attacks against your implementation of AI.
3. Use of AI by attackers.

The algorithms, especially deep learning algorithms, do not have domain knowledge. They don’t understand the underlying network topologies, the physics, or your business logic. These algorithms only look at input and output data and identify relationships between them without any meaning attached to that relationship. Also, machine learning has a dependency on large data sets and in many cases on the availability of labeled data. You may not have such data available in your organization to train the machine learning models.

AI opened a new attack surface. Not only AI can be a target to attacks, it can also expose the system and makes it prone to infiltration through AI. On the other hand, AI is empowering cyber criminals to carry out security attacks with greater speed, scope and stealth. Attackers use AI to automate common tasks such as scanning the network and discovering services. By applying Machine Learning, an attacker can bypass capture control, or even crack a password, or furthermore, utilize the data and API offered by a tool such as VirusTotal, to test a fully undetectable malware. As part of reconnaissance, attackers gather data about their potential targets.