CASA: Security Assessments for Web Apps and APIs

App Defense Alliance

What the Cloud Application Security Assessment covers, who genuinely has no choice about it, what the laboratory does to your application, and what the tiers mean.

What is this?

The App Defense Alliance, usually shortened to ADA, is an industry group that sets security standards for apps. It began at Google in 2019 and moved to the Linux Foundation in 2024, so it is no longer run by any single company. Google, Meta and Microsoft sit on its steering committee.

CASA stands for Cloud Application Security Assessment. It covers web applications and APIs reachable over the internet. An independent laboratory works through a published list of security requirements against your application and records a verdict on each one.

The requirements come from the OWASP Application Security Verification Standard, written as ASVS. Like MASVS it is free and public, so you can read the bar before you engage anyone.

There is one important difference from MASA, and it changes how you should think about the whole thing. CASA is used inside Google's own verification process for apps that request access to sensitive or restricted user data through Google APIs. If your app asks a user to connect their Gmail or Drive, Google wants assurance that your app can be trusted with that data. CASA is how that assurance gets produced. In that situation CASA is not a trust-building exercise you undertake voluntarily. It is a gate between you and a working product.

ADA does not test apps itself. It publishes the requirements, then authorises independent laboratories to do the testing. A lab has to earn that authorisation, and part of earning it is holding accreditation to ISO/IEC 17025, the international standard for testing laboratories. The point of the arrangement is that the people judging your app have no commercial interest in the answer.

Who is required to complete it

This question gets muddled because there are three different answers and people run them together. Nobody is required by law. Some organisations are required by a platform or a contract, which in practice is just as binding. Everyone else chooses.

The distinction that matters in practice: if a platform or a customer has made the assessment a condition of something you need, treat it as a hard requirement with a date attached. If nobody has asked, it is an investment decision, and you should judge it on whether the findings and the report are worth the cost to you.

The checklist is public, and free

This is the most useful thing to know before you spend any money, and most developers do not know it. The App Defense Alliance publishes its requirement sets openly on GitHub. You can read the exact bar you will be measured against, work through it yourself, and fix what fails before engaging anybody.

The profile folder holds the documents you need. The CASA Specification is the requirement list: an identifier and a one-line description, and deliberately nothing more. The CASA Test Guide is the one you actually want, because for each requirement it gives the description, the rationale, the evidence you must submit, the test procedure, the acceptance criteria and any scope limitations. The folder also carries a Burp audit scan configuration. The laboratory report templates and developer questionnaires sit separately under the repository's submission forms and templates area rather than in the profile folder.

Assurance levels: how much of the work is yours

How much you do, versus how much a laboratory does, depends on the assurance level. This is the single biggest driver of what an assessment costs you, so establish it early.

AL1 is the level most developers meet, and it is worth being precise about what it asks. You are not signing a statement that you comply. You are running the test procedure and producing evidence that demonstrates it, and a reviewer then judges whether that evidence is sufficient. Thin or ambiguous evidence gets sent back, and every round trip costs you time.

The principle is proportionality. An app reading a small amount of low-sensitivity data should not face the same process as an app with deep access to somebody's mailbox. Note that Google's own API verification process may also describe risk tiers when deciding what is required of your app; if you meet both terms, the assurance level is about who does the testing, while a risk tier is about how much scrutiny your access warrants. Confirm which applies to you rather than assuming they are the same thing.

What gets tested

The CASA requirement set covers input validation, authentication, cryptography, access control, error handling, security testing and configuration. In plain terms:

What the laboratory actually does

Most developers have never seen inside a laboratory engagement, and the phrase independent assessment hides a lot of specific process. Here is what actually happens, and why each part is there. Knowing this makes you a much better buyer. For the whole engagement phase by phase, including the acceptance checks and the independent review, see what the laboratory actually does.

Before the lab is allowed to accept the job

An authorised lab cannot simply take your money and start. Before it accepts, it has to satisfy itself of several things, and record that it did:

• Competence. It has people qualified for your assessment type. ADA specifies which certifications the engagement team must hold, and a lapsed certification does not count.
• Capacity and tooling. It has the qualified tools and the test environment your application needs, and the time to do the work properly.
• No conflict of interest. This one matters to you. A lab that has previously assessed or advised on your application, or that helped you build it, is barred from validating it. If a firm offers to fix your app and then certify it, that is not an arrangement an authorised lab can make.
• An independent reviewer. It has someone available who took no part in the testing and who will check the work. If no independent reviewer is available, the lab should decline the engagement, because it would not be able to release your report.
• Agreed scope. Which application, which build, which requirements apply, what is excluded and why, all written down before testing begins.

Who is on the engagement

How the lab handles your application

For a web application there is no package to hand over, so what matters instead is which environment is tested and how it is identified. The lab records the URLs and endpoints in scope, the environment, and your deployed version identifier if you have one. If you offer staging rather than production, that gets recorded, because a staging environment configured differently from production limits what the results actually mean. Be honest about the differences.

• A hash is recorded on receipt. The lab takes a cryptographic fingerprint of exactly what it received, so there is never an argument later about which version was tested.
• A clean environment is built for you alone. Your engagement gets its own test environment, isolated from other clients' work, and it is destroyed when the engagement closes.
• Tools are checked before use. The lab confirms each tool is at the version it has qualified and produces the expected result on a known input. A tool that fails that check is not used.
• Your data stays separated. One client's data must not be reachable from another client's engagement. This is a requirement, not a courtesy.

How evidence works, and why you should care

This is the real difference between a laboratory assessment and a consultant's report. For every requirement, the assessor has to record evidence good enough that a different competent assessor could repeat the work and reach the same conclusion. Not a summary, and not a conclusion. The actual input, the action taken and the observed result, captured at the time it happened.

Two consequences follow, and both are in your favour:

• A lab cannot fail you on a hunch. If the evidence does not demonstrate that a requirement is unmet, the assessor cannot record a Fail. Findings have to be shown, not asserted.
• A lab cannot pass you on your word. If you tell the assessor a control works and they do not verify it, that is recorded as your statement, not as a validated Pass, and the report has to disclose which requirements were self-attested rather than validated. This protects you too: it means nobody can later claim the lab checked something it did not.

How the verdict is reached

Each requirement in scope gets one of three verdicts. There is no partial credit and no overall score that hides the detail.

Watch the Inconclusive count on your report. Inconclusive is the easy place to park a difficult judgement, and a well-run lab monitors its own rate for exactly that reason. If a large share of your requirements came back Inconclusive, ask why: usually it means the scoping was wrong or something the lab needed was never provided. It is worth knowing when a laboratory is allowed to record Inconclusive, because it may not be used where the evidence would support a Fail.

The independent review

Before anything is released, the reviewer works through the engagement and checks, among other things:

• that the build tested is the build received, by reference to the recorded hash;
• that the approved environment and qualified tools were actually used;
• that every verdict is supported by contemporaneous evidence;
• that no Inconclusive verdict is standing in for a Fail the evidence would support;
• that requirements you self-attested, and requirements excluded from scope, are disclosed;
• that remediation guidance is specific to your application rather than generic advice;
• that nobody who was barred or recused took part.

If any of that fails, the reviewer sends it back or withholds the report. That review is itself a record, kept whether or not it changed anything, because a review that found nothing is evidence the control was operating.

What the report has to contain

A CASA report is not a free-form document. Certain elements are mandatory, and the reason each one is there is worth understanding:

What happens to the records afterwards

The lab keeps the whole record set, not just the report: the evidence, the verdicts, the environment configuration, the review record, every version of the report and every retest. Retention is typically at least two years after the certificate expires. That exists so that if a question arises later about how a verdict was reached, the answer can actually be produced.

It also means a lab cannot quietly revise history. Every attempt at your assessment, including one that failed, is a retained record with its own report. If a lab suggests it can make a failed attempt disappear, that is a serious problem with that lab.

What you are entitled to ask for

• the scope agreement in writing before testing starts;
• the name of your Quality Control Reviewer and the basis on which they are independent of the testing team;
• the dispute process, in writing, before you need it;
• written confirmation that no part of the fee depends on the verdict;
• a clear statement of what will be published, what will be sent to ADA, and what consent you need to give;
• the lab's retention period for your data, and what is destroyed at the end of the engagement.

A lab that is reluctant to answer any of those in writing is telling you something useful.

Specific to CASA

• Accounts at more than one privilege level. The lab needs at least two accounts, ideally at different privilege levels, because the only way to test whether user A can reach user B's data is to have both. One account makes the most important category of testing impossible.
• Written authorisation to test. If any part of your stack is hosted or operated by somebody else, the lab may need their permission before touching it. Sort this out early; it is a routine cause of delay.
• API documentation. An undocumented API is slow and expensive to test, and the lab may miss endpoints entirely. If you have a specification, provide it.
• A defined testing window for anything disruptive. Some testing can affect performance. Agree when it happens and have somebody available.

How it works from your side, step by step

How often you have to do it again

Annually. As with MASA, validation lasts twelve months. If you are relying on CASA for continued API access, treat the expiry date as a hard business deadline rather than a compliance date. Losing the access can break your product for your users, which is a different order of problem from a lapsed badge.

Start renewal at least two months out. Web applications change fast, and a year of feature work usually introduces something new to find.

The mistakes we see most often

• Assuming the lightest path applies. Teams plan for a self-assessment and discover they need hands-on laboratory testing. Confirm your assurance level first, always.
• Testing a staging environment unlike production. Different configuration, different authentication or mocked integrations mean the results say little about production.
• One test account. You cannot test whether user A can reach user B's data with only one user.
• Forgetting the APIs. Teams prepare the web interface carefully and forget the mobile app's back-end API, which is in scope and often less well protected.
• Ignoring configuration. The application code can be sound while the cloud storage bucket is world-readable.
• No permission to test third-party components. Work out early who owns each part of the stack and get written authorisation.

Frequently Asked Questions

Click any question to expand the answer.

Where to start

Confirm your assurance level. Get the CASA Test Guide from the repository and read the requirements that apply to you. Run a scan and fix what it finds. Then audit your access control by hand, because that is where the serious findings usually are. After that, talk to an authorised lab.

About Adayptus

Adayptus Consulting Private Limited is an application security testing firm based in Noida, India. We have been doing web, mobile and cloud security testing since 2018.

We are currently building our laboratory management system to ISO/IEC 17025:2017 and working towards authorisation as an App Defense Alliance Security Test Laboratory. We are not an ADA-authorised laboratory today. We will say so plainly on this page until that changes, because you should be able to trust what a security firm tells you about itself.

What we can do for you right now:

• Readiness testing. We test your app against the published CASA requirement set and tell you what would fail, before you pay for a formal validation. Finding out early is much cheaper than failing late. This is web application and API penetration testing aimed at a specific bar.
• Fixing what we find. We give you guidance specific to your application, pointing at the actual component and the actual change needed, not a link to a general guideline. Where the cause is in the code, secure code review is usually the faster route.
• Evidence preparation. We help you assemble the documentation, architecture information and test accounts an authorised lab will ask for, so your validation does not stall on paperwork.

Attribution and source