APIM Policy Patterns for AI Governance: Part 2 – Content Safety & Model Control

Part 1 covered the consumption side of an AI gateway: rate limits, token quotas, usage attribution and the telemetry needed to understand them. The next runtime decision is whether a request should be allowed through at all, followed by whether that consumer is entitled to use the model it asked for.

Those are separate controls, and I would keep them separate in the APIM policy as well.

I covered the wider architecture in From Azure Policy to APIM: Implementing Azure AI Guardrails. Azure Policy controls which models and resources are allowed into the Azure estate, while APIM makes decisions against the consumer and request at runtime.

Content safety: use the built-in policy first

Before writing regular expressions against jailbreak phrases or calling Azure AI Content Safety manually with send-request, I would start with APIM’s built-in llm-content-safety policy.

A baseline can be fairly small:

<inbound>
    <base />
    <llm-content-safety
        backend-id="content-safety-backend"
        shield-prompt="true"
        enforce-on-completions="true">
        <categories output-type="EightSeverityLevels">
            <category name="Hate" threshold="4" />
            <category name="SelfHarm" threshold="4" />
            <category name="Sexual" threshold="4" />
            <category name="Violence" threshold="4" />
        </categories>
        <blocklists>
            <id>company-blocklist</id>
        </blocklists>
    </llm-content-safety>
</inbound>

The details behind backend-id are important. It references an APIM backend configured for the Azure AI Content Safety resource rather than the Content Safety resource name or URL placed directly in the policy.

Microsoft currently requires APIM’s managed identity to have the Cognitive Services User role on the Content Safety resource. The backend URL must use the https://<service-name>.cognitiveservices.azure.com form, and the backend authorisation credentials need managed identity configured with the resource ID https://cognitiveservices.azure.com.

The full policy reference is available in Microsoft’s llm-content-safety documentation.

The category thresholds and Prompt Shields deal with different risks. Hate, SelfHarm, Sexual and Violence are the four harm categories available to the policy. With EightSeverityLevels, a threshold of 4 allows severities 0 to 3 and blocks 4 to 7.

shield-prompt="true" checks for user prompt attacks. A prompt does not need hateful, violent, sexual or self-harm content to be malicious; it may instead be trying to override the instructions or boundaries the application is supposed to follow.

enforce-on-completions="true" also applies the content-safety check to chat completions rather than checking only the incoming prompt.

Streaming needs different expectations. APIM buffers streamed output using a sliding window and stops forwarding further events when Content Safety finds a violation. Anything already sent to the client cannot be removed, and APIM does not return a 403 for that streaming case.

That client behaviour is something I would test rather than assume an application handles correctly.

Blocklists add another organisation-specific layer for terms the general classifiers are not intended to know about. The blocklist has to exist in the Azure AI Content Safety resource before APIM can reference its ID.

Content safety does not give you PII controls

llm-content-safety covers the four harm categories, while Prompt Shields covers prompt attacks. Neither tells you that a prompt contains a National Insurance number, address, payment card number or another piece of personal data.

I would treat PII as its own requirement.

Azure provides that capability through Text PII in Azure Language, which can detect and redact sensitive entities from unstructured text. For simple and strongly structured data, there are still cases where a deterministic check inside APIM can be useful as one layer.

This example looks through text-only user messages in a chat-completions-shaped request for a card-like sequence of 13 to 19 digits:

<set-variable name="promptText" value='@{
    var body = context.Request.Body.As<JObject>(preserveContent: true);
    var messages = body["messages"] as JArray;
    if (messages == null)
    {
        return string.Empty;
    }
    var parts = new List<string>();
    foreach (var message in messages)
    {
        if ((string)message["role"] == "user"
            && message["content"] != null
            && message["content"].Type == JTokenType.String)
        {
            parts.Add((string)message["content"]);
        }
    }
    return string.Join("\n", parts);
}' />
<set-variable name="containsCardCandidate" value='@(
    System.Text.RegularExpressions.Regex.IsMatch(
        (string)context.Variables["promptText"],
        @"\b(?:\d[ -]*?){13,19}\b"
    )
)' />
<choose>
    <when condition='@((bool)context.Variables["containsCardCandidate"])'>
        <return-response>
            <set-status code="400" reason="Bad Request" />
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-body>{ "error": "Prompt appears to contain payment card data. Remove it and resend." }</set-body>
        </return-response>
    </when>
</choose>

preserveContent: true is important because the request still needs to reach the model after APIM has inspected it. Reading the request body without preserving it consumes the original body stream.

I would keep the purpose of this regex narrow. It finds number-shaped candidates; it does not establish that the value is a payment card number and it certainly does not provide general PII detection. False positives, formatting differences and personal data that has no predictable pattern all make a growing collection of APIM regex checks difficult to justify.

Once the requirement includes names, addresses, phone numbers, government identifiers, health information or arbitrary free text, I would use a service designed for PII detection rather than trying to turn gateway XML into a data-loss-prevention engine.

Response redaction depends on the API contract

Rejecting a request is simpler than changing content that the model has returned. There are still cases where you may want to let the response through after removing a predictable value.

For a non-streaming OpenAI chat-completions-shaped response, APIM can rewrite the JSON body:

<outbound>
    <base />
    <choose>
        <when condition="@(context.Response.StatusCode >= 200 && context.Response.StatusCode < 300)">
            <set-body>@{
                var body = context.Response.Body.As<JObject>();
                var contentToken = body.SelectToken("choices[0].message.content");
                if (contentToken == null || contentToken.Type != JTokenType.String)
                {
                    return body.ToString();
                }
                var content = (string)contentToken;
                content = System.Text.RegularExpressions.Regex.Replace(
                    content,
                    @"\b(?:\d[ -]*?){13,19}\b",
                    "[REDACTED-CARD]"
                );
                ((JValue)contentToken).Value = content;
                return body.ToString();
            }</set-body>
        </when>
    </choose>
</outbound>

The qualification is more important than the snippet. This code assumes a JSON response with choices[0].message.content and that APIM has the complete body available to rewrite.

It does not automatically apply to the Responses API, multimodal output or a different provider’s response contract, and I would not use this pattern for streaming output. The built-in content-safety policy understands streaming; custom JSON rewriting means owning the response contract and buffering behaviour yourself.

If the gateway redacts something, I would record that the redaction occurred without copying the sensitive value into another system. Request ID, consumer identity and a flag such as piiRedacted=true are usually enough for operational telemetry.

Model entitlement should come from trusted identity

Content safety decides whether the request or response is acceptable. Model authorisation decides whether this consumer is entitled to use the model it requested.

A safe prompt can still be asking for a model the application should not have access to. I would base that decision on an identity APIM has already validated rather than a caller-controlled header such as X-Consumer-Tier.

Assume the Entra access token contains an ai_tier claim and the gateway validates it first:

<validate-azure-ad-token
    tenant-id="{{tenant-id}}"
    output-token-variable-name="jwt">
    <audiences>
        <audience>{{ai-gateway-audience}}</audience>
    </audiences>
    <required-claims>
        <claim name="ai_tier" match="any">
            <value>standard</value>
            <value>premium</value>
        </claim>
    </required-claims>
</validate-azure-ad-token>
<set-variable
    name="consumerTier"
    value='@(((Jwt)context.Variables["jwt"]).Claims["ai_tier"][0])' />
<set-variable
    name="requestedModel"
    value='@(
        (string)(
            context.Request.Body
                .As<JObject>(preserveContent: true)["model"]
            ?? ""
        )
    )' />
<set-variable name="modelAuthorised" value='@{
    var tier = (string)context.Variables["consumerTier"];
    var model = (string)context.Variables["requestedModel"];
    if (tier == "premium")
    {
        return model == "chat-standard"
            || model == "chat-premium";
    }
    return model == "chat-standard";
}' />
<choose>
    <when condition='@(!(bool)context.Variables["modelAuthorised"])'>
        <return-response>
            <set-status code="403" reason="Model Not Authorised" />
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-body>@{
                return new JObject(
                    new JProperty("error", "model_not_authorised"),
                    new JProperty("requested_model", context.Variables["requestedModel"]),
                    new JProperty("consumer_tier", context.Variables["consumerTier"])
                ).ToString();
            }</set-body>
        </return-response>
    </when>
</choose>

validate-azure-ad-token can return the validated token as a Jwt object through output-token-variable-name, which means the authorisation decision can use claims from the token APIM has already validated. The policy reference documents that behaviour and the required-claims support.

The aliases in the example are deliberate. I would rather expose stable client-facing names such as chat-standard and chat-premium than make every application depend on the exact deployment name or model version behind APIM.

Microsoft’s unified model API now uses the same model-alias pattern. A client can call a stable alias while the APIM configuration maps it to the backend model, allowing the platform to change the target without forcing every consuming application to change at the same time.

I would authorise the alias first and route it afterwards.

The Boolean modelAuthorised variable also keeps the APIM expression within supported set-variable types. set-variable accepts a defined set of basic values, including strings and Booleans, but arbitrary values such as string[] are not supported context-variable types.

Deployment approval and runtime entitlement solve different problems

The model controls in Azure Policy still have a role here.

Microsoft’s current built-in Foundry model deployment policies include Foundry model deployments should only use approved models, which restricts which models or publishers can be deployed.

That answers a platform question: which models are allowed to exist in this Azure scope?

The APIM authorisation above answers a consumer question: which of those deployed models can this application use?

An organisation may legitimately deploy several approved model families because different products need them. That does not mean every product should automatically be able to consume every deployment through the shared gateway.

I would therefore keep deployment governance in Azure Policy and consumer entitlement in APIM. Trying to express both through the same allow-list loses the identity context available at runtime.

The wider architecture is also reflected in Microsoft’s Azure AI Landing Zones repository, which now separates an AI Foundry Landing Zone from an AI Gateway Landing Zone using APIM for centrally managing and serving Foundry models.

Failure paths I would test

There are several behaviours I would prove in a non-production gateway before relying on this as a shared policy:

• llm-content-safety harm categories and shield-prompt are separate controls. Test an unsafe-content case and a prompt-attack case independently.
• enforce-on-completions can check chat completions, but a violation in a streaming response stops later events rather than returning the normal 403. Make sure the client handles a terminated stream.
• llm-content-safety does not provide PII detection. Test the PII path separately if it is part of the platform requirement.
• Reading a request with As<JObject>() without preserveContent: true can consume the body before the backend receives it.
• Custom blocklists need to exist and contain the expected terms in Azure AI Content Safety before APIM references them.
• Model entitlement should come from a validated claim or another gateway-controlled identity source rather than a value the caller can change.
• set-variable cannot hold an arbitrary string[]. Keep stored values to the supported APIM types or make the authorisation decision directly.
• Response rewriting is tied to the contract being parsed. A choices[0].message.content policy needs separate handling for a different response shape or streaming.
• Microsoft’s current documentation is inconsistent about llm-content-safety on the Consumption tier. The policy page’s APPLIES TO header excludes Consumption, while its gateway usage section and the central policy matrix currently include it. I would verify the behaviour on the tier being deployed rather than make architecture depend on either table alone.

Add the controls in layers

I would put the built-in content-safety policy in before creating custom filtering. Configure the harm categories that make sense for the workload, enable Prompt Shields where user input can reach the model, and decide deliberately whether completion checking belongs on the path.

The thresholds need representative testing. I would not choose a deliberately permissive threshold simply to reduce friction during rollout, and I would not make every workload inherit the same threshold without considering its users and purpose.

PII should then be treated as its own decision. A deterministic check can be useful for a small number of predictable identifiers, but workloads handling arbitrary user text need a PII capability designed for that input rather than an ever-growing set of APIM expressions.

Model authorisation should follow the identity model already used by the platform. If consumers arrive with validated Entra identities, use claims or another trusted entitlement you already own. Stable model aliases then give APIM a clean boundary between the name the consumer requests and the deployment the platform routes to.

Before I promoted the policy, I would test a normal request, each configured harm category at either side of its threshold, a benign-looking prompt attack, completion blocking, a streaming violation, the organisation blocklist, a standard consumer requesting a premium model, a missing or invalid model alias, and every path where the request body has already been inspected by another policy.

I would put those cases into the gateway’s regression tests rather than leave them as a manual checklist. Once several teams depend on the same APIM policy, a change to a shared fragment should have to prove that these decisions still behave the way the consumers expect.