“We checked the supplier.” You hear this before choosing an invoice service, and it sounds reassuring. But what did the person actually do? Read an existing report? Open the public website? Or deliberately test whether the service would allow an action it should refuse?

All three can provide useful information. They are not interchangeable, and more checks do not automatically mean more relevant checks. Understanding how the information was obtained helps you understand the answer you have received.

Reading something that somebody else collected

Suppose a colleague sends you a saved report about a supplier’s public website. You open that copy on your computer. Your reading does not require you to contact the supplier’s website. Somebody else has already gathered the information, and you are consulting their record.

This is a passive way to learn about the supplier: your action is reading an existing observation rather than creating a new interaction with the supplier’s service. The original collector may have contacted that service. “Passive” describes your own step; it does not magically make every earlier step passive as well.

Imagine the report says a particular support page was available at the time of the assessment. You have learnt that it was available then. You have not checked whether it remains available now or whether support staff answer requests promptly. Each of those would need different information.

A photograph helps explain the time issue. Seeing a photograph today does not turn it into a photograph taken today. Likewise, downloading an old security report gives you new access to old information. Good assessment reporting describes what was examined and when, and warns that the environment can change after the assessment. 8

The record may still be valuable. It can show an earlier concern and what the supplier was asked to address. A later explanation or check can then consider whether the condition changed. The error would be silently replacing “was observed at that time” with “is established now”.

Visiting a page creates a new exchange

Now you enter the supplier’s address and open the page. The program used to open websites is a browser. It asks a computer system serving the site for information, then uses the reply to display the page. The answering system is commonly called a server. This request and response are part of using the web. 11

The supplier’s system is involved in this second activity. It receives a request and returns information, or it may fail to return the information expected. Your observation can concern what happened in that exchange. An ordinary page visit is not, by itself, an attempt to break through a security protection.

For example, you select Help and receive a message saying that page could not be found. That is a fresh observation about this route to help. It does not tell you what would happen inside the service after sign-in, because you have not reached that part.

An automated tool can make a page request too. A program is simply instructions that a computer follows. Instead of a person selecting a link, the tool can be instructed to request a page and examine the answer. Automation can repeat the work consistently, but it does not change the question that a particular request can answer.

There is a further useful distinction. A tool may request a page and save the first reply without displaying the page or running its programs. A browser can go further: it can fetch additional material and run programs included by the page.

The saved reply lets a reviewer examine the information and accompanying instructions that arrived first. Watching the page load also lets the reviewer observe what happens as the browser fetches more material and runs the page’s programs. These are different observations. Neither means that every part of the service has been tested. 11 5

Some products use “passive” for checks that include ordinary public-page requests. Others reserve the word for existing third-party records. Rather than arguing about the label, ask whether the supplier’s system was contacted and what the tool did there. Security-assessment guidance recognises that methods have different benefits and limitations; the description of the work matters. 7

Deliberately checking whether protection holds

Consider a different question: can a user belonging to one business read another business’s invoice? A public-page request will not settle it. When a document is requested, the service needs to check the user’s permission for that particular record and action. Identifying a signed-in user is not enough. 3

A security assessor can examine that behaviour in a test environment agreed with the supplier. In our example, the environment contains two invented businesses and invented invoices. The assessor has approved test accounts for both. The expected result is that each account may use the documents it is allowed to use and is refused the other business’s documents.

This is more than looking at a page. The assessor deliberately investigates whether a protection can be made to fail. A check that intentionally varies an interaction to investigate a suspected condition is often called an active probe. The assessor records what the service actually does, not merely whether the page looks correct.

A planned, authorised assessment that tries to identify and examine exploitable weaknesses is called a penetration test. It may combine different techniques and a person’s reasoning with tools. It is not just a new name for a large automatic report, and it is not the whole of security. Application-testing guidance recommends a balanced combination of approaches, including reviews as well as technical testing. 9

The distinction also explains why a normal product demonstration is useful but limited. Watching an invoice successfully arrive shows the demonstrated workflow. It does not establish that the service refuses every inappropriate request. Testing for that is a different job from showing that the normal job works.

Permission matters before this work begins. The responsible parties should agree which systems and test accounts are included, which activities are allowed and when testing must stop. A buyer’s interest in a product is not permission to experiment on real customers’ records. The invented-data example is a recommended way to investigate the question without using somebody else’s invoices.

What an unfinished check means

Suppose the assessor was meant to examine the signed-in service but never received the agreed test accounts. The observation is that this part could not be tested. It has not passed, because the expected behaviour was never examined. Nor does the missing account automatically establish that the service has a security weakness. Reporting guidance recommends explaining missing access and other practical limitations. 8

The same care applies when a tool starts a check but receives no usable answer. Perhaps the service did not respond, the connection was interrupted or the tool could not continue. The attempt tells us that the question remains unanswered. A separate investigation is needed to explain why, rather than guessing that protection succeeded or failed.

No single method answers every business concern. An existing record may help you understand history; a page visit may establish current public behaviour; an agreed permissions test may investigate customer separation. None of those alone demonstrates that saved invoice copies can be recovered after a storage failure. Ask what the method was capable of examining before treating its result as a wider promise.

When someone next says “we checked it”, ask for a plain description: “We read this saved report”, “we requested these public pages”, or “we tested these permissions using approved accounts and invented documents.” Once the action is clear, the value and the remaining limits of the result become much easier to understand.

Sources

3 — OWASP API1:2023 Broken Object Level Authorization

5 — W3C Content Security Policy Level 3

7 — NIST SP 800-115 Technical Guide to Information Security Testing and Assessment

8 — OWASP WSTG v4.2 Reporting

9 — OWASP WSTG v4.2 Introduction

11 — MDN: How the web works