You open a security report and see three red rows. One says a protective setting was not found. Another identifies a software product. The third says a check could not finish. Has the service got three security holes?

The colour alone cannot answer that. The entries describe different things. Before deciding how worried to be, it helps to follow how a report moves from something observed to a claim about what could go wrong.

A report entry begins the explanation

A tool is a program following instructions to perform checks. For example, it might ask a public website for a page and look for a particular instruction in the answer. A browser is the program people normally use to open websites; the computer system providing the reply is commonly called a server. The tool can examine an exchange without seeing everything the service does internally. 11

When the report calls attention to a result, that entry is a finding. The word does not mean every entry is a demonstrated way to break into the system. A finding might describe a possible weakness, a missing protection, information about the service or a limit of the check.

Think of a warning light in a car. The light deserves attention, but it is not the mechanic’s completed diagnosis. You need to know what triggered it. Ignoring it would be careless; announcing a ruined engine from the light alone would also be careless.

The same reasoning has three steps in a report. First, what did the check observe? Second, what problem does that observation support? Third, what should be done about the problem? If a report skips the middle step, a frightening conclusion can appear without the explanation needed to support it.

For example, “we did not receive the expected instruction” describes the tool’s observation. “This page lacks this particular blocking protection” needs an explanation of where the instruction could be supplied and how it would work. “Customer invoices have been stolen” requires very different evidence. These sentences are not stronger and weaker ways of saying the same thing.

A weakness and an event are different claims

A weakness that could undermine a security protection is called a vulnerability. A suspected vulnerability needs to be checked. A confirmed one has support for the problem described. Neither word means that somebody has already used it.

Here is a more concrete example. Several businesses use an invoice service. Each employee signs into their own business account. When an employee requests a document, the service needs to check whether that person may perform the requested action on that document. It must not assume that signing in gives access to all customers’ records. 3

In an agreed test environment, an assessor might find that an account belonging to one invented business receives an invented invoice belonging to another. That behaviour would support a specific finding about failed document permissions in the tested circumstances. It is different from a warning based on the appearance of the public website.

The assessor has demonstrated a weakness using agreed test data. To say that a real customer’s invoice was accessed without permission, the investigator would need evidence about that event. A harmful occurrence or security violation is an incident; finding the possibility of it is not the same as establishing that it happened.

This distinction is not a reason to delay repairs until harm is proved. A confirmed weakness can deserve urgent correction before anybody misuses it. Separately, the supplier may need to investigate whether real misuse occurred. Repairing the weakness and establishing the history answer two different questions.

What a missing page protection actually means

Return to our first red row. A web page can include small programs, for example to make a form respond to your choices. The browser may obtain those programs from the page or other sources. A website can give it rules restricting which sources or behaviour are allowed. These rules are called a Content Security Policy, often shortened to CSP. They can provide an additional protection against certain unwanted content, depending on what they allow and enforce. 5

A report might say it did not find that policy among the extra instructions sent alongside the page. Those instructions are called response headers. The visitor ordinarily reads the page rather than the headers, but the browser can use them in handling it. Some policy rules can also be included within the page, with limitations. A missing header therefore does not automatically establish that no policy exists in any form. 5

Even a policy that is present needs interpretation. It may be permissive, allowing much more than the report’s reader imagines. Or it may be in a report-only mode: it records or reports behaviour that would violate the rules, rather than blocking that behaviour. Monitoring can serve a useful purpose, but it is not the same protection as enforcing the rules. 5

So the reviewer should explain what the check actually received and what protection applied to this page. A missing or weak protection may be worth fixing. It does not automatically establish a separate flaw in the invoice service, and it does not establish that somebody stole documents.

If the initial report described a problem that checking does not confirm as stated, it is a false positive. Serious testing guidance explicitly recommends verifying findings and removing such mistaken reports. This protects the usefulness of valid findings; it is not an instruction to dismiss inconvenient warnings. 9

Recognising software and having no result

Our second row names a software product. Software is the collection of programs used to do work. A tool may recognise signs associated with a product, but recognition alone does not establish every weakness ever reported for that product. The evaluator needs to connect the clue to what is actually used and the conditions under which the particular weakness applies.

For the business reader, “we recognised this product” is therefore the start of a question. “We established this particular weakness here” is a more specific claim. If the supplier says the tool recognised the wrong thing, a qualified reviewer should resolve the difference. Neither the more frightening answer nor the more reassuring answer becomes correct merely because somebody prefers it.

The unfinished check in the third row is different again. Perhaps the tool lacked the account needed to reach the working service. Perhaps it started but could not obtain a usable response. It has not answered its intended question. Reporting guidance recommends explaining such limitations, including lack of access or account information. 8

“Not tested”, “could not finish” and “does not apply” should remain distinct from “passed” and “failed”. A supplier that does not offer document storage, for example, need not pass a test of a storage feature it does not provide. But a feature you rely on must not disappear from consideration simply because the report never examined it. These are recommendations for clear interpretation, not fixed labels every product is required to use.

Certainty, seriousness and the next step

Certainty asks how sure we are that the described problem exists. Seriousness asks what could happen because of it. Urgency asks what to do next in this situation. They are different questions: a serious suspicion may justify prompt investigation, while a well-established minor improvement may fit ordinary maintenance.

Counting red rows will not answer them. One demonstrated permissions failure can matter more to invoice privacy than a long list of unrelated website suggestions. That does not mean the smaller suggestions should be ignored; it means each needs its own explanation.

Take one important row and ask: “What did you observe, what could happen because of it, and what action do you recommend?” If somebody also claims that harm has already happened, ask what establishes that event. This keeps repair decisions grounded in the actual problem rather than in the colour of the report.

Sources

3 — OWASP API1:2023 Broken Object Level Authorization

5 — W3C Content Security Policy Level 3

8 — OWASP WSTG v4.2 Reporting

9 — OWASP WSTG v4.2 Introduction

11 — MDN: How the web works