Prbl's homepage has a section showing visitors real examples of what the scanner catches. One of those examples is a string of marketing copy:
code: "create_blog_posts.py:22: password='password123'",
That line is display text. It tells a visitor: here is what a hardcoded credential finding looks like. It is not a real assignment anywhere in the Prbl codebase. The scanner does not know that. It read the literal string password='password123' sitting inside a .tsx file and reported it as a HIGH severity hardcoded credential, on its own marketing page, in its own repo.
Why this happened
The rule that catches hardcoded credentials, PRBL-C001, is intentionally aggressive about exactly this pattern. A real password = 'password123' in application code is a serious finding and the rule has to fire on it without hesitation. It has no way to know, from the regex alone, that this particular instance is inside a string describing a finding rather than an actual assignment.
We had a second instance of the same root problem in a different file: prbl/rewriter/prompt.py, the module that builds prompts for the AI rewriter. That file is full of intentional “Before:” and “After:” code examples for every fixable rule, TLS verification disabled, JWT decoded without checking the signature, hardcoded session secrets, by design, so the rewriter knows what a fix looks like. Scanning it produced a wall of HIGH severity findings for content that exists specifically to teach the AI rewriter what bad code looks like.
Scan your own app for issues like these
Paste your live URL. We check what your app serves publicly for exposed keys and misconfigurations. No account, no install.
The fix
Two separate, narrow fixes, not a blanket exception for our own repo:
- A new suppression pattern that recognizes the
"path/file.ext:LINE: description"format common to security writing and marketing copy describing a finding. If a line looks like documentation referencing a finding elsewhere, it is not treated as a live assignment. Verified this does not suppress real credential assignments, only the documentation-prefixed format. prbl/rewriter/prompt.pyis now skipped entirely by the scanner. Its only purpose is holding intentional vulnerability examples for the rewriter, so there is no scenario where a real production bug would hide there.
Both fixes shipped with regression tests that check the fix works and that it does not accidentally suppress a real finding written in a similar style. Full test suite: 327 passing.
Why we are posting this instead of quietly fixing it
A security tool that never finds anything wrong with itself is not impressive, it is unverified. This one found two real false positives in its own codebase the same week it shipped, and the fix for each one is now a permanent regression test instead of a one-off patch. That is the actual argument for using a scanner: not that it is perfect, but that when it is wrong, the wrongness gets caught, explained, and locked down so it cannot happen again.
Common questions
How did the scanner flag its own marketing copy?
The homepage had example text describing what Prbl finds — something like 'sk_live_...' as an illustration of a Stripe key leak. The scanner, running on the Prbl repo itself, found that string, matched it against the Stripe key pattern, and returned a HIGH severity finding. It was correct about the pattern. It was wrong about the context — the string was documentation, not a live credential.
Why is this a false positive and not a real security issue?
A false positive is a finding that matches a pattern but is not a real security risk in context. Example text on a marketing page describing what a leaked key looks like is not itself a leaked key. The scanner was doing its job — pattern matching on a high-confidence signal — but lacked the context to distinguish 'this is an illustration' from 'this is a live credential'.
How was the false positive fixed?
Two changes: the homepage text was restructured so the example value is prefixed with a documentation marker, and a context-aware rule was added to suppress findings on strings that appear in documentation-prefixed positions. Each change shipped with a regression test verifying the fix does not suppress real credential assignments in similar positions.
Does a scanner that flags its own code prove it works?
It proves the pattern matching is working and aggressive enough to fire on realistic-looking values. Whether that is a feature or a bug depends on the context — in production code it would be the right call; in documentation it was a false positive. The interesting part is that the scanner found it in the same week it shipped, which is also how we know the regression test suite works.