How is this different from a penetration test?

A penetration test attacks the running system and reports what it managed to do. An audit examines the design and the evidence and reports what would happen under attack. A test can prove a specific hole exists; an audit can tell you whether the arrangement makes holes cheap or expensive.

Can an audit certify that our system is secure?

No. We are not aware of any accredited scheme that certifies a language model system as secure, and nothing that examines a design can rule out an attack nobody has thought of yet. What an audit produces is a description of your exposure and an order to work in.

How often should one be done?

When the shape changes, not on a calendar. Adding a tool, connecting a new corpus, or letting the system act without confirmation each change what an attack is worth, and each is a better trigger than a date.

Someone who can see the code path that assembles the prompt and the permissions the tools hold. Most of the questions cannot be answered from the outside, which is also why a self-assessment is a reasonable starting point and a poor finishing one.

What does a result actually give us?

An ordered list. The value is not the score, which is a summary, but the identification of the one thing holding the verdict down, because that is the piece of work that changes the answer rather than improving the number.

What is an LLM security audit?

An LLM security audit examines a language model system for what it can be made to do by text it did not expect, and what the consequences are when that happens. It looks at how instructions and content are separated, what the system can reach, what leaves it, and whether anyone would be able to reconstruct an incident afterwards. It is a review of design and evidence, which is a different exercise from attacking a running system.

What it covers

The input path, meaning how text that you did not write reaches the model and whether anything distinguishes it from your own instructions. The action path, meaning what the system can call and what happens without a person confirming. The output path, meaning what is checked before something downstream consumes it and whether secrets can travel in it.

Then the parts nobody enjoys: whether anyone deliberately tries to break it, whether a failure blocks a release, and whether the logs would let you tell what happened.

What it is not

It is not a penetration test. A penetration test attacks a running system and reports what it achieved; an audit reads the design and the evidence and reports what would happen. Both are useful and they answer different questions, and the one you want depends on whether you are trying to find a specific hole or to know whether the shape is sound.

It is also not a certification. We are not aware of any accredited scheme that certifies a language model system as secure, and a report that reads like one is describing an opinion.

Why the deployment shape decides the questions

A model answering from its own weights, a model reading your documents, and a model that can act are three different exposures wearing the same interface. Grading a chatbot with no tools on how tightly its tool permissions are scoped invents a finding, and grading an agent without asking invents a pass.

So the honest first question is not how mature your controls are. It is what this system can actually reach.

Common questions

How is this different from a penetration test?: A penetration test attacks the running system and reports what it managed to do. An audit examines the design and the evidence and reports what would happen under attack. A test can prove a specific hole exists; an audit can tell you whether the arrangement makes holes cheap or expensive.
Can an audit certify that our system is secure?: No. We are not aware of any accredited scheme that certifies a language model system as secure, and nothing that examines a design can rule out an attack nobody has thought of yet. What an audit produces is a description of your exposure and an order to work in.
How often should one be done?: When the shape changes, not on a calendar. Adding a tool, connecting a new corpus, or letting the system act without confirmation each change what an attack is worth, and each is a better trigger than a date.
Who should run it?: Someone who can see the code path that assembles the prompt and the permissions the tools hold. Most of the questions cannot be answered from the outside, which is also why a self-assessment is a reasonable starting point and a poor finishing one.
What does a result actually give us?: An ordered list. The value is not the score, which is a summary, but the identification of the one thing holding the verdict down, because that is the piece of work that changes the answer rather than improving the number.

Find out where you stand

The assessment on this site asks up to 21 questions about what your system can reach, and returns a score, a verdict and, where a control is short, the one thing to fix first. It takes about ten minutes, and questions your deployment shape has no surface for are dropped rather than scored against you.

It is a self-assessment of design and evidence, not a penetration test, not an audit of a running system, and not a certification.

Run the assessment