シークレットスキャンの信頼性向上：大規模な誤検知削減への取り組みMaking secret scanning more trustworthy: Reducing false positives at scale

GitHub has added context-aware large language model reasoning to the verification stage of its secret scanning service, a change the company says reduces false positives at scale and makes security alerts more trustworthy and actionable. The update matters because alert fatigue is one of the most persistent problems in application security: when a tool flags too many strings that turn out to be harmless, developers and security teams begin to ignore its output, and genuine leaked credentials can slip through unnoticed.

Secret scanning works by inspecting code, commit history, issues, and other repository content for patterns that resemble credentials such as API keys, access tokens, private keys, and connection strings. Much of this detection has historically relied on regular expressions and known token formats supplied by GitHub and its partner issuers. That approach catches a wide range of real secrets, but pattern matching alone cannot always tell the difference between a live credential and a string that merely looks like one. Example values in documentation, placeholder tokens in test fixtures, randomly generated identifiers, and hashes can all trigger alerts that waste reviewer time.

The new step inserts an LLM into the verification pipeline so the system can reason about the surrounding context of a candidate secret rather than judging the string in isolation. According to GitHub, this context-aware analysis helps determine whether a detected pattern is plausibly a real, active credential or a benign lookalike. Reading nearby code, variable names, file paths, and usage patterns gives the model signals that a static regular expression cannot capture, which appears to be the main mechanism behind the reduction in noise.

This builds on existing capabilities rather than replacing them. GitHub already offers validity checks for many supported secret types, in which a detected token is checked against the issuing provider to see whether it is still active. That partner-backed validation is highly reliable when available, but it only covers credentials from participating services. The LLM-based reasoning is likely most useful for the long tail of secret types and ambiguous cases where direct validation is not possible, complementing the deterministic checks already in place.

Reducing false positives also has practical implications for push protection, a related feature that blocks commits containing detected secrets before they reach a remote branch. When detection is more precise, push protection is less likely to interrupt developers with spurious blocks, which tends to improve adoption and reduce the temptation to bypass the control. More accurate alerts similarly lower the triage burden on security teams who must review and dismiss findings, freeing attention for credentials that genuinely need rotation.

Secret scanning and push protection are part of GitHub Advanced Security, the paid add-on that also includes code scanning powered by CodeQL and dependency analysis through Dependabot. GitHub has been gradually expanding secret protection availability and repackaging some of these capabilities, and the broader direction is toward catching exposed credentials earlier in the development lifecycle. The company frames this LLM enhancement as part of that ongoing effort to make findings more dependable.

The move also reflects a wider industry trend of applying machine learning and language models to security tooling. Competing products from vendors such as GitGuardian and open-source scanners like TruffleHog have long grappled with the same precision-versus-recall trade-off, and several have introduced their own heuristics, entropy analysis, and validation steps to suppress noise. Using an LLM specifically to weigh contextual evidence is a notable application of the technology to a problem where accuracy directly affects whether a tool is trusted.

As with any LLM-based system, there are caveats worth keeping in mind. Model-driven classification is probabilistic, so it can introduce its own errors, and organizations handling sensitive code will want clarity on how content is processed during analysis. GitHub presents the feature as an improvement to verification rather than a guarantee, and the reported gains are described in terms of reducing noise at scale rather than eliminating false positives entirely. Teams evaluating the change should still treat any confirmed alert as a prompt to revoke and rotate the affected credential, since a real leaked secret remains a risk regardless of how it was detected.