Going for a Data Deep Dive in the AppSec Wild- Part II

A data analyst takes a deep dive into defect datasets. What does she learn about AppSec on the way? Welcome to part 2 of Enso Security’s three-part AppSec research series comparing defect data, the technologies involved in collection, severity levels, and CVE alignment over time from the perspective of a data analyst. 

In the previous article, we discussed some of the trends and challenges facing AppSec professionals, including:

• The emergence of new vulnerabilities over time, and the challenge of AppSec teams to manage massive amounts of data.
• How the overall rise in use of open-source tools in software development contributes to the rise in disclosing new third-party code vulnerabilities.
• The struggle of AppSec teams in finding an effective remediation plan. 

These factors, along with an ever-expanding attack surface, have produced an array of application security testing tools that aim to help organizations address potential security risks.

Common practice dictates that when planning vulnerability coverage of an organization’s most valuable assets, SAST and SCA scanning tools are two main methods of application security testing. As a data team member, I believe it would be beneficial to take a comparative deep-dive into these two scanning technologies to understand their scope and effectiveness.

Using extensive data collection, in this blog, we will try to find the best approach a software organization should take for building and scaling their AppSec program tools.  In our research, we analyzed metadata of ~500K SAST and SCA defects from our production data collection.

‍

How to choose between SAST and SCA?
Comparing SAST and SCA tools is like comparing apples to oranges. To address software vulnerabilities effectively, AppSec teams should not only decide how to combine these two methods but must also manage massive amounts of data from these two types of scans.

‍

SAST and SCA have different approaches to scanning code assets:

SAST (Static Application Security Testing) tools analyze an application’s source code before it is built in order to identify security vulnerabilities and coding errors that could be exploited by attackers.
SCA (Software Composition Analysis) tools, on the other hand, can analyze the source-code and build applications to identify libraries, frameworks, and third-party components that the application depends on and check those for any known security vulnerabilities. 

‍

Severity coverage:

Many AppSec teams create a triage policy for issues according to their severity, focusing on high severity issues (CVSS 7-10). In the first blog of this series, we demonstrated how high severity issues are less likely to be identified, and indeed both SAST and SCA tools mainly detect low severity issues.

In addition, CVE feeds focus on the disclosure of critical issues, which is a limitation of automated tools like SAST tools. While SCA tools are able to detect certain types of vulnerabilities, such as those related to the deployment of the software, SAST tools excel in more general types of vulnerabilities such as missing input validation.

‍

‍

Technology coverage:

When discussing the technological coverage of these tools, we should mention that SCA and IaC are both specific types/scopes of SAST and there are other unique tools that address these scans. 

Looking into our defects dataset, we examined common software development technologies including:

1. Well known open-source dependencies-oriented programming languages: Python, Javascript, and Ruby (interpreted programming languages);
2. Java to reference a compiled programming language;
3. Docker and Terraform configuration files as a reference for IaC (Infrastructure as  Code) technologies.

SAST tools are known for their ability to offer complete code coverage when analyzing every single line of code and are able to point out the exact location of vulnerabilities in the source code.

‍

According to our data, SAST tools may be missing higher severity issues, as we see a significant amount of medium severity issues in all examined programming languages. Since SAST tools are scoped to scan only code written by development teams, they skip all open-source components that are widely used in web applications. In addition, improperly configured source code might lead SAST tools to detect a high number of false positives.

However, this can be an advantage when running IaC scanning tools. Secrets, infrastructure misconfigurations, and compliance issues are often overlooked. Using SAST tools for IaC can yield a high return on investment due to their ability to provide broader coverage on high-severity issues, as shown above.

‍
We can expect to have better results on SCA coverage of high and critical severity issues when comparing popular high-level programming languages.

In our last blog post, we showed that CVE-associated issues in medium to high severity are more likely to be exploited. As SCA tools can detect open source issues from NDV’s list, they may have a better chance of detecting exploitable issues. In addition, SCA tools can detect license compliance issues associated with open-source software. 

Although they provide a relatively quick and simple remediation plan (simply upgrading packages, for example), SCA tools can also generate massive amounts of potential issues, including negligible risks and false positives. 

When considering what tools to use in your organization, use the following suggestions to inform your decision-making: 

• Applying SCA tools into the development environment is relatively easy, especially for small organizations that are new to AppSec. 
• SCA tools can bring value both in detecting high-severity issues and suggesting quick remediation plans.
• SAST tools are a great solution when considering IaC security coverage, as they focus on detecting high-severity issues in your infrastructure. Large enterprises can benefit from using IaC-dedicated scanning tools to cover compliance issues in a complex infrastructure environment.
• It is important to extract the key requirements of your organization before choosing cutting-edge cybersecurity tools, as they may produce massive amounts of data that will need to be managed.

‍

In the next blog post, we will discuss the importance of categorizing vulnerabilities and continue reviewing the differences between SAST and SCA tools in different categories.

‍

About the author

Alina Nikitin is a Data Analyst at Enso Security, the first Application Security Posture Management (ASPM) tool used daily by AppSec teams to enforce, manage and scale a robust AppSec program, all without interfering with development. Questions and suggestions are welcome! Please reach out to Alina at alina@enso.security to get in touch.