A Registry Outage Looked Exactly Like a Real Vulnerability Alert
On 4 September one of our automated security checks failed. The build went red, the notification fired, and the message said what it always says when that check fails: vulnerable dependencies.
Nothing was vulnerable. The public advisory database that the check queries had timed out.
The check had no way to tell us that, because the tool it runs reports both outcomes identically. A vulnerability found and an API that did not answer both exit with the same code. From the outside — from the notification, the dashboard, the red tick — they are the same event.
This is a small incident with a large lesson, and the lesson is not about the specific tool. It is about what a security alert is actually telling you, and what happens to your team when it starts telling you the wrong thing.
Why a false red is expensive
The reflexive view is that a false alarm is harmless. The check was wrong, you looked, nothing was there, move on.
That is true once. The cost is cumulative.
A monitoring signal is only useful because of what people do when it fires. If a check goes red for a real vulnerability on Monday and a network timeout on Tuesday, you have not got one noisy check — you have degraded the meaning of every future red. The team learns, entirely rationally, that this particular alarm is often nothing. The next time it fires, it gets a slower look. Eventually it gets a glance.
A green-to-red flip that means “the registry was down” trains you to ignore the one that means “you shipped a known vulnerability”.
That is the actual damage. Not the twenty minutes spent investigating an outage, but the attention you will not spend six months from now on the one that matters.
For any small business running automated monitoring — endpoint alerts, backup verification, uptime checks, dependency scanning — this is the question worth asking of every alert you receive: can this signal distinguish between “I found a problem” and “I could not perform the check”? If it cannot, it will eventually be ignored, and the decision to ignore it will be made informally, by a tired person, on a busy day.
The obvious fix made it worse
The tool we use has a machine-readable output mode. The obvious move was to switch to it and parse the result properly rather than relying on the exit code.
We tried that, and it inverts the bug rather than fixing it.
In machine-readable mode, when the network call fails, the tool prints an error object and exits zero. Zero means success. So the check would pass, the build would go green, and the record would show that dependencies were audited and found clean — on a run where nothing was audited at all.
That is unambiguously the worse failure. A false red wastes time. A false green is a security control that reports itself as working while doing nothing, and nobody investigates a green tick.
This shape recurs constantly in monitoring. The two failure modes are not symmetrical, and the fix for one frequently creates the other. When you change how a check reports, work out what it now does when it cannot run at all — because that is the path nobody tests and the one that produces false confidence.
The race we lost while testing
The next design was to do both: run the machine-readable version to detect whether the service was reachable, then run the normal version to make the pass-or-fail decision.
That is a race, and it lost during testing. The first call came back with a result. The second call, moments later, timed out. Two calls to a flapping service can disagree, and when they do, the check behaves according to whichever answer arrived rather than according to reality.
We were fortunate that it failed while we were watching. A race that only manifests when a third-party API is degraded is otherwise the kind of defect that sits dormant for a year and then produces an inexplicable result on the worst possible day.
One question, one call. If two pieces of information have to be consistent with each other, they must come from the same observation.
What the check does now
Both facts — whether the service answered, and what it said — come from a single call.
The advisory API is genuinely unreliable at the moment. In local testing it was timing out roughly one attempt in two. So the check retries up to three times before concluding the service is unreachable, with a pause between attempts.
Three outcomes, and each one is distinct:
The service could not be reached after three attempts. The build emits a loud warning whose text says, explicitly, this run did NOT check for vulnerabilities; this is not a clean bill of health. It does not block the build, because a third-party outage should not stop us shipping. But it is on the record, in the log, in plain language, so nobody can later mistake that run for a passing audit.
The service answered and found nothing serious. It prints the counts — how many dependencies were audited, and how many advisories at each severity — and passes. The count matters: “audited 522 dependencies, critical 0, high 0” is evidence. “Passed” is not.
The service answered and found something at high or critical. It names each offending package, its severity, the advisory title and a link, and fails the build.
Each of those three branches was exercised against a real or synthetic payload before it shipped. A branch you have never watched execute is a branch you are guessing about.
The substantive result, from a run where the API did answer: 522 dependencies audited, zero critical, zero high. The advisories that started the work were genuinely resolved.
What to take from this
Ask whether your alerts can say “I don’t know”. Most cannot. A check with only two states — pass and fail — will eventually report “I could not run” as one of them, and whichever one it picks is wrong.
Treat a false green as a serious incident, not a lucky escape. If a control has ever reported success without performing its check, it is not a control yet. This is the single most common way security tooling fails quietly.
Log the evidence, not the verdict. “Audited 522 dependencies, 0 critical” tells you the check ran. “Passed” tells you a variable was set. The first can be audited later; the second cannot.
Exercise every branch before you trust it. Including, especially, the failure branches. They are the ones that will run at 3am while nobody is reading.
Distinguish outage from finding in the notification text itself. Not in a dashboard someone has to open. In the words the alert uses, because that is all most people will read.
None of this is exotic, and none of it is specific to dependency scanning. The same reasoning applies to your backup verification, your uptime monitoring, your antivirus reporting and your log alerting. If any of those can fail in a way that looks like success, that is the one to fix first.
If you would like a second opinion on whether your monitoring is telling you the truth, that is part of what we do in managed IT support and for businesses across Parramatta and Western Sydney.
Ash Ganda is the founder of Cloud Geeks, which provides managed IT support and cyber security for Australian small and medium businesses.
