Pentest, vulnerability scanning, red team, purple team, and TLPT
These terms describe different forms of assessment.
Vulnerability scanning is largely automated detection of known weaknesses and configuration errors. It is repeatable and can run frequently, but it does not replace manual analysis of business logic, chaining of weaknesses, or assessment of business impact.
A pentest has a defined objective and scope, such as a web application, external infrastructure, Active Directory, an internal network, or a mobile application. The tester uses automated tools, but the key decisions about attack hypotheses, validation, exploitation, and further movement are made manually. OWASP WSTG describes web security testing as active analysis for weaknesses, technical flaws, and vulnerabilities, with identified issues reported to the system owner together with impact and mitigation guidance.[8]
A red team exercise evaluates the broader ability of an organization to detect and respond to an adversary. It may include techniques beyond a conventional pentest, such as social engineering, cloud abuse, trust relationships, or physical access, but only when those activities are explicitly authorized. The defensive team may not know the details of the exercise because one of the objectives is to measure detection and response.
Purple teaming is not simply a "lighter red team." It is a collaborative model in which offensive and defensive teams execute techniques and immediately compare them with telemetry, detection logic, and response. The current TIBER-EU framework includes replay and purple teaming during the closure phase.[5]
TLPT, or threat-led penetration testing, is a regulated form of testing driven by threat intelligence specific to the tested institution. In the financial sector, the legal basis is DORA, while Commission Delegated Regulation (EU) 2025/1190 specifies selection criteria, testing phases, tester requirements, scope, reporting, and cooperation with authorities.[3][4] In 2025, the ECB updated TIBER-EU to align it with DORA and the delegated regulation. Relying only on the 2018 TIBER-EU framework is therefore outdated in 2026.[5]
There is no universal sequence such as "annual pentest, red team every three years" that fits every organization. Frequency and depth should follow risk, change velocity, service criticality, legal requirements, and monitoring maturity. A red team exercise provides limited value when the organization lacks telemetry and response processes that can actually be tested.
Start with scope and rules of engagement
A professional test begins before the first packet is sent to the target. The parties should agree at least on:
- the objective of the test and the decisions it is intended to support;
- the exact technical scope, including domains, addresses, applications, network segments, cloud tenants, and test accounts;
- exclusions;
- the access model: black box, gray box, or white box;
- permitted and prohibited actions, especially phishing, persistence, lateral movement, controlled exfiltration, DoS testing, and physical access;
- permitted hours and load limits;
- emergency contacts and immediate stop conditions;
- handling of data, secrets, tokens, copied files, and evidence;
- removal of test artifacts;
- escalation of critical findings during the engagement;
- retest scope and closure criteria.
NIST SP 800-115 and CREST treat planning and agreement as part of the testing process, not as administrative overhead.[6][7] Regulated tests require even more formalization. The current TIBER-EU framework requires, among other things, identification of critical or important functions, the systems and services in scope, and management-body sign-off of that scope.[5]
Black box, gray box, and white box
Black-box testing gives the tester minimal prior knowledge. It is useful for evaluating an external attack surface and public exposure, but some paid testing time is consumed by rediscovering information the client already possesses. It also does not automatically produce a more complete view of security.
Gray-box testing provides limited information and controlled user accounts, for example accounts with several roles. In business applications, it usually enables much better testing of authorization, tenant isolation, and process logic. It is often the best compromise between realism and coverage.
White-box testing may include architecture documentation, configuration, source code, and privileged accounts. The objective is not to reproduce the attacker's initial position but to maximize the probability of finding weaknesses within limited time. For custom applications or deep authorization testing, it may be more valuable than a pure black-box exercise.
There is no sound basis for claiming that one model is always "more realistic" or "better." The access model should match the question being asked. Testing resistance to an unknown external adversary and performing a deep security assessment of a custom application are different objectives.
Methodologies and reference documents in 2026
There is no single global standard that exhaustively defines every type of penetration test. In practice, several documents with different status are used.
NIST SP 800-115 dates from 2008, but it is still listed by NIST as a current final publication. It structures technical security testing planning, technique selection, analysis, and remediation.[6] It is not Polish law.
OWASP Web Security Testing Guide v4.2 remains the stable WSTG release as of 29 August 2026. OWASP is developing version 5.0 in parallel, but its "latest" content is subject to change. Contracts and reports should therefore identify a concrete version instead of saying only "according to the latest OWASP".[8]
For mobile applications, an important change occurred on 4 July 2026 when OWASP released stable MASTG v2.0.0. MASTG should be used together with MASVS because one provides testing techniques and the other defines expected security properties.[9]
MITRE ATT&CK is not a penetration-testing methodology. It is a knowledge base of adversary behaviors that is useful for describing techniques, building red-team scenarios, and comparing those scenarios with detections. As of 29 August 2026, the current release is ATT&CK v19.2.[10] Mapping every finding to ATT&CK is not useful when the issue does not represent adversary behavior.
PTES can still be useful as a community process model, especially for pre-engagement activities, but it is neither a formal standard nor a regulation and should not be presented as a source of legal obligations.[20]
CREST Defensible Penetration Test and CREST procurement guidance are useful when buying a service because they emphasize three elements: the maturity of the provider organization, the competence of the individual testers, and an agreed test specification.[7] Accreditation can be a useful quality signal, but it does not by itself prove the quality of a specific engagement.
How a defensible test proceeds
A strong test does not begin by launching every scanner available. The tester first builds a model of the target environment and attack hypotheses.
Reconnaissance establishes what is reachable, which dependencies exist, and which components may lead to the test objective. In an application, this may mean mapping functions, roles, APIs, and data flows. In an internal network, it may mean trust relationships, authentication services, administrative paths, and segmentation.
Vulnerability validation separates a tool signal from an actual weakness. Identifying a software version is not enough when a distribution has backported a fix without changing the visible version number, the service is not reachable through the tested vector, or the vulnerable component is not used.
Exploitation should be proportional to the objective. Sometimes a minimal proof that a control can be bypassed is sufficient. In other engagements, the objective is to demonstrate a complete chain from initial access through privilege escalation to an agreed target asset. The greater the possible operational impact, the more explicit the limits and stop conditions should be.
Post-exploitation must not turn a pentest into an uncontrolled incident. Persistence, additional credential acquisition, lateral movement, controlled exfiltration, and command-and-control activity are appropriate only when the scope allows them. Every artifact should be identifiable and removable.
Cleanup is part of the engagement. The tester should remove created accounts, files, scheduled tasks, keys, rules, tunnels, tokens, and other artifacts, and the client should be able to verify the resulting state. Missing cleanup procedures are particularly dangerous in red-team and TLPT engagements.
A retest should verify not only that the original exploit no longer works, but also that the fix has been deployed across all affected components and has not created another attack path. A ticket marked "closed" is not evidence of remediation.
The report: evidence, not an alert dump
A penetration-test report should allow a competent recipient to reproduce the most important findings. That does not mean distributing secrets widely. Detailed technical evidence can be placed in a restricted appendix.
A good finding contains:
- the criterion or expected security property;
- the observed condition and the circumstances under which it occurred;
- evidence;
- the exploitation path;
- technical and business impact;
- limitations of the conclusion;
- remediation guidance;
- the retest method.
CVSS can help express the technical severity of a vulnerability, but it should not replace organizational context. Attack chains are particularly important in penetration testing because several individually moderate weaknesses may combine into a critical outcome.
The management summary should answer different questions from the technical appendix. It should state which attacker objectives were achievable, which layers of defense failed, which decisions are required, and where findings depend on one another. It should not imply that failure to discover an attack path proves that no such path exists.
Pentesting, SOC, and detection
An offensive test provides additional value when its timeline can be compared with SOC telemetry. The organization then gets an answer not only to "could the attacker get in?" but also to "when should we have noticed?"
ATT&CK v19.2 can help describe executed behaviors and compare them with existing detections.[10] That mapping should not become a detached percentage score for "ATT&CK coverage." A rule mapped to a technique but lacking the required telemetry, or never having been tested, does not provide a real detection capability.
After a red-team exercise, a joint replay of the attack path by offensive and defensive teams can be especially valuable. The current TIBER-EU framework formalizes replay and purple teaming to connect red-team actions with blue-team response and remediation.[5]
At 4crypto, penetration testing is treated as a separate assessment method that complements continuous vulnerability scanning, SOC monitoring, hardening, and security auditing. Combining these activities is useful only when their objectives, evidence, and ownership of conclusions remain distinct.
OT and ICS: active testing can itself become a hazard
Operational technology requires a different degree of caution than a conventional office network. NIST SP 800-82 Rev. 3 states that penetration testing on OT networks should be used carefully so that OT functions are not adversely affected; examples of compensating measures include testing replicated, virtualized, or simulated systems and scheduling tests during planned outages.[11]
This does not create a universal rule that production OT must only be tested passively. Active production testing may sometimes be possible, but it requires process knowledge, operator involvement, constrained techniques, and an explicit risk decision. Testing PLCs, RTUs, or safety-related devices without understanding the process can create a larger hazard than the vulnerability being investigated.
The IEC 62443 family is an important reference for industrial automation and control system security, but a formal criterion should identify the specific part and edition. A generic reference to "IEC 62443" does not define a single penetration-testing methodology.
Testing AI and LLM systems
Security testing of systems that use large language models extends conventional application testing with model-specific and integration-specific failure modes. Common test classes include direct and indirect prompt injection, leakage of contextual data, misuse of tools exposed to an agent, tenant-separation failures, policy bypasses, and the use of untrusted content to manipulate model behavior.
It would be incorrect, however, to state that from 2 August 2026 all high-risk AI systems already have to comply with Article 15 of the AI Act on the originally planned schedule. Regulation (EU) 2026/1744 postponed the application of Sections 1-3 of Chapter III for high-risk systems: to 2 December 2027 for systems classified under Article 6(2) and Annex III, and to 2 August 2028 for systems classified under Article 6(1) and Annex I.[14][15]
The situation is different for general-purpose AI models with systemic risk. Article 55 of the AI Act requires their providers, among other things, to evaluate the model using appropriate state-of-the-art protocols and tools, including conducting and documenting adversarial testing to identify and mitigate systemic risks.[14] The scope of that obligation is much narrower than the casual statement "the AI Act requires AI pentests".
Legal requirements in Poland and the EU
KSC and NIS2. NIS2 requires covered entities to apply proportionate cybersecurity risk-management measures and policies or procedures for assessing their effectiveness.[2] In Poland, concrete obligations arise primarily from the current Act on the National Cybersecurity System. Following the 2026 amendment, Article 8 requires essential and important entities, among other things, to address security in acquisition, development, maintenance, and operation of information systems, including testing of those systems, and to maintain policies and procedures for evaluating the effectiveness of controls.[1] The Act does not create a universal requirement for every organization to perform an annual pentest.
GDPR. Article 32(1)(d) requires, where appropriate to risk, a process for regularly testing, assessing, and evaluating the effectiveness of technical and organizational measures.[12] It does not say that penetration testing is the only acceptable method, nor does it prescribe one frequency for all controllers.
DORA. Article 24 requires a risk-based digital operational resilience testing program. TLPT is its advanced component. Under Article 26, selected financial entities, subject to the Regulation's exclusions, perform TLPT at least every three years, while the competent authority may adjust the frequency based on risk profile and operational circumstances.[3] Selection criteria and detailed procedures are specified in Delegated Regulation (EU) 2025/1190.[4] Not every bank, insurer, or other DORA-regulated entity automatically performs TLPT solely because DORA applies to it.
PCI DSS. For entities to which PCI DSS applies, the current version is v4.0.1. Requirement 11.4 covers regular internal and external penetration testing, while 11.4.2 and 11.4.3 require testing at least once every 12 months and after significant infrastructure or application changes.[13] PCI DSS is an industry standard, not generally applicable Polish legislation.
ISO/IEC 27001. The standard can serve as a criterion for an information security management system and as a reference point for a broader control-effectiveness program, but certification does not prove that a specific application has undergone a particular penetration test. As of 29 August 2026, ISO/IEC 27001:2022 with Amd 1:2024 remains current.[16]
How to select a provider
There is no single certification whose presence automatically guarantees a good pentest. When selecting a provider, verify:
- the experience of the actual testers with the technology in scope;
- the proposed report structure and evidence practices;
- quality-review procedures;
- handling of client data and secrets;
- insurance and contractual liability where relevant;
- escalation of critical findings during the engagement;
- conflict-of-interest handling;
- retest terms and deletion of data after the project.
Independence should be assessed in context. There is no universal rule that an IT supplier can never perform a penetration test. If the objective is an independent assessment of that supplier's own implementation, the conflict of interest strongly favors another provider. TLPT under DORA has additional explicit requirements for external and internal testers under DORA and Delegated Regulation 2025/1190.[3][4]
Common procurement mistakes
- Buying vulnerability scanning under the name of a pentest. A tool-generated report may be useful for vulnerability management, but it does not demonstrate manual validation, business-logic analysis, chaining of weaknesses, or impact verification.
- An overly broad promise paired with an underspecified budget. "Pentest the entire infrastructure" without the number of applications, segments, hosts, roles, interfaces, and exclusions is a slogan, not a scope.
- Expecting a predetermined number of findings. Zero critical vulnerabilities proves neither strong security nor a weak tester. What matters is scope coverage, tested hypotheses, evidence quality, limitations, and reproducibility.
- Omitting the retest. A report describes the state during the engagement. If remediation has not been verified, the organization only knows that someone said the fix was implemented.
- Failing to connect the exercise with detection. If the organization operates a SOC or SIEM, known tester actions are unusually valuable data for verifying whether telemetry and detection rules actually work.
What real breach data tells us
The Verizon DBIR 2026 reports that vulnerability exploitation accounted for 31% of known initial access vectors in its dataset, while credential abuse accounted for 13%.[19] This is not evidence that every penetration test should focus primarily on public CVEs. It is, however, strong evidence against limiting an assessment program only to phishing or identity configuration.
ENISA Threat Landscape 2025 and the CERT Polska 2025 annual report also show that the threat landscape depends on sector, region, and service type.[17][18] In threat-led testing, these sources are inputs for scenario design, not a ready-made attack checklist to be executed without context.
Conclusion
A good penetration test is not a contest in CVE counts or a demonstration of tools. Its value comes from a well-defined question, properly bounded scope, tester competence, high-quality manual validation, reproducible evidence, and the organization's ability to act on the result.
A mature program does not ask only "did we run a pentest this year?" It asks which scenarios were actually tested, what remained out of scope, which defensive layers worked, which failed, and whether retesting demonstrates that the most important attack paths have been closed.
10 questions before signing the contract
- What exact problem is the test supposed to resolve?
- What is in scope and what is excluded?
- Which access model will be used, and why?
- Are social engineering, persistence, lateral movement, C2, DoS testing, and physical access explicitly allowed or prohibited?
- Who can stop the test, and how quickly can that person be reached?
- How will data, secrets, and evidence copies be protected?
- How does the provider document manual validation and attack chains?
- Does the report distinguish technical impact, business impact, and assessment limitations?
- Is retesting included, and what exactly does it cover?
- How will the results feed into hardening, detection improvement, and risk management?
Frequently asked questions
- Can a pentest damage production?
-
Yes. Professional preparation reduces the likelihood of disruption but cannot eliminate it. Older equipment, OT systems, load testing, unknown dependencies, and actions that modify state carry particular risk. The engagement therefore needs limits, a stop procedure, and an explicit decision about which techniques are allowed in production.
- How long does a web application pentest take?
-
There is no defensible universal person-day figure for a category such as "medium application." Duration depends on the number of functions, roles, APIs, tenants, technologies, code, integrations, access model, and required depth. Pricing should be based on explicit scope assumptions. Comparing providers only by days is risky when they assume different scopes.
- Does zero critical findings mean the pentest was poor?
-
No. Likewise, finding many critical vulnerabilities does not automatically prove test quality. Review scope coverage, hypotheses, evidence, manual validation, and limitations. The engagement should not be judged against an expected number of vulnerabilities.
- Must the pentest be independent from the IT supplier?
-
Not always as a matter of law. Independence becomes important when the purpose is an independent evaluation of that supplier's own work. Some regimes, including DORA TLPT and PCI DSS, impose additional requirements concerning tester qualifications and independence.
- Does red teaming always include phishing and physical entry?
-
No. It includes only techniques accepted in the scope and rules of engagement. Social engineering and physical access increase realism but also legal, operational, and reputational risk, so they require separate and explicit authorization.
- Can a tester install an implant or C2 beacon?
-
Yes, when the client is entitled to authorize the activity and the scope precisely defines allowed systems, communications, duration, stored data, emergency procedures, and cleanup. It is unsafe to make the blanket statement that every implant becomes "legal" after one document is signed. Responsibility depends on the rights of the parties, the systems and data involved, third-party providers, and the actual conduct of the test.
- Is black-box testing enough for a web application?
-
Sometimes, if the objective is external exposure. For a complex business application, accounts with several roles and a gray-box component are usually valuable because authorization and business-logic testing require understanding the intended access model. WSTG v4.2 separately covers areas including identity management, authentication, authorization, session management, and business logic.
- How is a mobile application pentest different?
-
Beyond the backend and API, it evaluates security properties of the application running on the device: data storage, cryptography, network communication, platform integration, resistance to tampering, and runtime behavior. Since July 2026, OWASP's current stable mobile testing guide is MASTG v2.0.0. There is no evidence-based universal rule that mobile testing always requires a fixed multiple of web-testing effort.
- What exactly is TLPT under DORA?
-
It is an advanced threat-led test for selected financial entities. DORA sets the obligation and high-level framework, Delegated Regulation 2025/1190 defines detailed requirements, and the updated 2025 TIBER-EU framework provides a coherent implementation process. It covers preparation and scoping, targeted threat intelligence, red-team testing, red- and blue-team reporting, replay, purple teaming, a test summary, and remediation planning.
- Does a penetration test satisfy NIS2 or the Polish KSC?
-
Not by itself. It may provide evidence for parts of the requirements concerning system testing and control-effectiveness evaluation, but KSC covers a much broader management system involving risk, incidents, continuity, supply chain, monitoring, identity, training, and other controls. A pentest is an assessment technique, not a complete compliance program.
Need consulting in this area?
A free 30-60 minute consultation. No obligations. We discuss needs, scale and a high-level timeline.
Related content
Other competence areas
- IT security audit
- Vulnerability scanning
- Device and system hardening
- Email security audit
- KRI compliance audit
- KSC and NIS2 audit
- GDPR compliance audit
- Information security policy
- ISMS - information security management system
- Security awareness - onsite and online
- SOC 24/7 - monitoring and response
Compliance and regulation
Bibliography and sources
Source status verified on 29 August 2026. Legal acts link to ELI or EUR-Lex; methodological documents link to official publisher pages.
- [1] regulationParliament of the Republic of Poland (2018). Act of 5 July 2018 on the National Cybersecurity System. Consolidated text Journal of Laws 2026 item 20, with amendments in force on 29 August 2026, in particular items 252 and 815. · Dz.U. 2026 poz. 20 · poz. 252 · poz. 815
- [2] regulationEuropean Parliament and Council (2022). Directive (EU) 2022/2555 (NIS2). In particular Article 21. · EUR-Lex
- [3] regulationEuropean Parliament and Council (2022). Regulation (EU) 2022/2554 (DORA). In particular Articles 24-27. · EUR-Lex
- [4] regulationEuropean Commission (2025). Commission Delegated Regulation (EU) 2025/1190 of 13 February 2025 specifying regulatory technical standards for TLPT. · EUR-Lex
- [5] guidelineEuropean Central Bank (2025). TIBER-EU Framework. Edition updated to align with DORA. · ecb.europa.eu
- [6] guidelineScarfone, K., Souppaya, M., Cody, A., Orebaugh, A. (2008). NIST SP 800-115: Technical Guide to Information Security Testing and Assessment. NIST. · DOI: 10.6028/NIST.SP.800-115
- [7] guidelineCREST (2022). CREST Defensible Penetration Test and Guide to Penetration Testing. CREST. · crest-approved.org
- [8] standardOWASP Foundation (2020). Web Security Testing Guide v4.2. OWASP. Stable release as of 29 August 2026; version 5.0 in development. · owasp.org
- [9] standardOWASP Foundation (2026). Mobile Application Security Testing Guide v2.0.0 and MASVS. OWASP. MASTG v2.0.0 was released on 4 July 2026. · mas.owasp.org
- [10] reportMITRE (2026). MITRE ATT&CK v19.2. MITRE. Status as of 29 August 2026. · attack.mitre.org
- [11] guidelineStouffer, K. et al. (2023). NIST SP 800-82 Rev. 3: Guide to Operational Technology (OT) Security. NIST. · csrc.nist.gov
- [12] regulationEuropean Parliament and Council (2016). Regulation (EU) 2016/679 (GDPR). In particular Article 32. · EUR-Lex
- [13] standardPCI Security Standards Council (2024). PCI DSS v4.0.1. PCI SSC. In particular Requirement 11.4. · pcisecuritystandards.org
- [14] regulationEuropean Parliament and Council (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act). In particular Articles 15 and 55, consolidated version. · EUR-Lex
- [15] regulationEuropean Parliament and Council (2026). Regulation (EU) 2026/1744 amending the application dates of certain high-risk AI Act requirements. · EUR-Lex
- [16] standardInternational Organization for Standardization (2022). ISO/IEC 27001:2022 - Information security, cybersecurity and privacy protection - Information security management systems - Requirements. ISO/IEC. Together with Amd 1:2024. · iso.org
- [17] reportEuropean Union Agency for Cybersecurity (2026). ENISA Threat Landscape 2025. ENISA. Version 1.2 dated 9 January 2026. · enisa.europa.eu
- [18] reportCERT Polska / NASK PIB (2026). Annual report on CERT Polska activities in 2025. CERT Polska. · cert.pl
- [19] reportVerizon Business (2026). 2026 Data Breach Investigations Report. 19th edition. · verizon.com
- [20] guidelinePTES Team (2014). Penetration Testing Execution Standard. In particular Pre-engagement Interactions. A community model, not a standard or regulation. · pentest-standard.org