Vulnerability scanning vs penetration testing

What is the difference between a vulnerability scan and a penetration test?

A scan reports what might be wrong; a pentest proves what is wrong and how far it goes. NIST SP 800-115 puts it in one sentence: vulnerability scanners check only for the possible existence of a vulnerability, while the attack phase of a penetration test exploits the vulnerability to confirm its existence.

A scanner sends known probes, compares the responses to signatures, and reports matches. It is fast, cheap to repeat, and consistent. A penetration tester uses scanners too, then reads the application, tries each lead by hand, combines weak findings into stronger ones, and follows access wherever it goes. The two are complements. Buying one when you need the other is the common mistake.

How do they compare?

What does "validated" mean?

A validated finding is one someone has confirmed exists in your system, with evidence. Vendors use the word loosely, so it helps to separate three levels:

1. Detected. A tool matched a signature, often a version string. "OpenSSH version X is vulnerable to CVE-Y" is detection.
2. Verified. A non-destructive check confirmed the vulnerable condition is present and reachable, for example a request that triggers the flawed behavior without causing harm.
3. Exploited. The tester used the weakness to gain something: read another user's data, run a command, escalate a role. The report shows the request and the response.

Most scanners report at level 1, some at level 2. A penetration test should report at level 3 wherever the rules of engagement allow, and say plainly when it stopped at level 2 and why. When a vendor calls findings "validated," ask which level and ask for the evidence on one example.

Why do scanners produce false positives?

Because they infer from indirect signals. NIST SP 800-115 notes that application scanners, like other signature-based tools, typically have high false positive rates, and that findings need validation, ideally by manual examination. Common causes:

• Version matching. The banner shows an old version, but the vendor backported the fix.
• Status codes taken at face value. The scanner reads a 200 response as success.
• Scanner-specific severity. Each tool uses its own scale, which NIST says makes findings hard to compare and may not reflect real risk.

Here is the second case in practice. A scanner flags an unauthenticated admin export endpoint because it returned 200:

GET /admin/export HTTP/1.1
Host: app.example

HTTP/1.1 200 OK
Content-Type: application/json

{"error": "login_required", "redirect": "/login"}

The body shows the request was refused. The application returns errors with a 200 status, so every protected path looks open to a naive check. A tester rules this out in a minute. A team handed hundreds of such lines without validation spends days on them, and learns to distrust the next report.

The reverse also happens. A false negative is a real flaw the scanner cannot see, and it is the more expensive error. Cross-tenant access usually returns a normal 200 with valid data, which looks correct to any tool that does not know which user should see what. The PCI SSC's guidance gives another example: credentials exposed through a web application error message that an ASV scan did not flag because the issue had a low CVSS base score. NIST adds that scanners cannot detect vulnerabilities that only appear in combination, so several low-risk findings may hide one high-risk path.

How do teams use both?

Scanning covers breadth and frequency; penetration testing covers depth. A common arrangement:

1. Scan on a schedule, internal and external, authenticated where possible. PCI DSS sets a floor of every three months and requires authenticated internal scans (11.3.1.2).
2. Triage scan output by exploitability, using CVSS for severity, EPSS for likelihood of exploitation, and the KEV catalog for vulnerabilities known to be exploited.
3. Give the latest scan results to the pentester. The PCI SSC's guidance lists reviewing current scan results as preparation for a test. The tester skips what the scanner already found and spends time on what it cannot.
4. Run a penetration test at least annually and after significant change, gray-box and authenticated for applications.
5. Retest pentest findings, rescan scan findings. Each has its own verification loop.

Application scanners (DAST) sit between the two: they send attacks to a running app but still work from known patterns. See penetration testing for what the manual side involves, and what AI pentests find for one dataset comparing exploit-driven automated testing with signature scanners.

How are PCI ASV scans different from penetration tests?

They are separate requirements with separate rules, and neither substitutes for the other.

• External vulnerability scans (11.3.2) run at least once every three months, by a PCI SSC Approved Scanning Vendor (ASV), and must meet the ASV Program Guide requirements for a passing scan, with rescans as needed. ASVs are companies whose scan solutions have been tested and approved by the PCI SSC.
• Scans after significant change (11.3.2.1) must resolve vulnerabilities scored 4.0 or higher by CVSS, and can be run by qualified personnel with organizational independence. They do not require an ASV.
• Internal vulnerability scans (11.3.1) run at least once every three months by qualified personnel, and high-risk and critical vulnerabilities must be resolved and rescanned.
• Penetration tests (11.4.2 and 11.4.3) run at least once every 12 months and after significant change, internal and external, by a qualified internal resource or qualified external third party with organizational independence. An ASV is not required.

A passing ASV scan says no known vulnerabilities above the threshold were detected on your external IPs that quarter. It says nothing about access control or logic inside your application. That is what requirement 11.4 is for.

Common mistakes

• Buying a "penetration test" that is a scan with a cover page. Ask for a finding with the exploit request and response.
• Sending raw scanner output to engineers without validation, which teaches them to ignore security tickets.
• Scanning without credentials and concluding the app behind the login is clean.
• Treating a clean quarterly ASV scan as evidence of application security.

Written by Parameter · Last reviewed September 27, 2026