The Limitations of Microsoft 365 Cybersecurity Scanning Tools

Automation is limited

Automated scanning tools are powerful—but they’re not perfect. Many organizations rely entirely on them to evaluate their Microsoft 365 security posture, only to discover later that “100% compliant” didn’t mean what they thought it did.

Here’s the reality: even the best Microsoft 365 cybersecurity scanning tools can only go so far.

1. 38% of CIS Controls Require Manual Review

According to the CIS Microsoft 365 Foundations Benchmark v5, 81 out of 130 controls are classified as automated—meaning they can be evaluated using PowerShell or Microsoft Graph commands.

That leaves 49 out of 130 controls (nearly 38%) that must be manually audited.

2. Scanners Can’t Audit Manual Controls

If a scanning tool claims you’re “100% compliant” with the CIS M365 Benchmark, that’s only true for the automated subset of controls.

In other words, you’re really only about 62% compliant overall.

This creates a dangerous false sense of security—especially for organizations that depend on scan reports for compliance reporting or risk assessments.

3. Complex Controls Defy Simple Automation

Some controls can’t be accurately measured by any tool. For example, break glass accounts can have a variety of usernames, but they all must meet a specific set of security requirements.

A scanning tool can only approximate which accounts are intended as break glass accounts—it can’t determine that with 100% certainty. As a result, automated tools can easily miss critical misconfigurations or assume compliance where there is none.

4. Redeemer Cyber Takes It Further

At Redeemer Cyber, we leverage all available automation built into the CIS Microsoft 365 Benchmark—but we don’t stop there.

Our Microsoft 365 Security Assessments include both:

• Automated checks for the 81 measurable controls,

• Custom Redeemer Cyber M365 controls, and

• A detailed manual review of the remaining 49 controls to ensure complete coverage

That means our assessments align with the full CIS Microsoft 365 Benchmark (v5), not just the automated portions most tools focus on.

5. Tools Have Their Place—but Not the Whole Story

Scanning tools are valuable for efficiency and repeatability, but they share one inherent limitation: they can only see what they’re programmed to look for.

Human expertise fills in the gaps—interpreting context, reviewing manual settings, and validating that security controls are both implemented and effective.

Final Thoughts

Automation can accelerate cybersecurity, but it can’t replace human oversight. Relying solely on scanning tools can leave significant blind spots in your Microsoft 365 security posture.

That’s why Redeemer Cyber combines automation and expert manual review to deliver the most complete, benchmark-aligned Microsoft 365 assessments available.

👉 Hire Redeemer Cyber for a thorough Microsoft 365 Security Assessment today—and make sure no control is left unchecked.

📩 Contact us at www.redeemercyber.com