## Description # Content Filtering Architecture in the Azure OpenAI Gateway Solution The content filtering component is a critical safety layer in the Azure OpenAI Gateway architecture. It leverages Azure AI Content Safety service to screen all incoming requests before they reach the OpenAI models, effectively preventing inappropriate or harmful content from being processed. ## Request Flow 1. Client sends a request to the OpenAI API endpoint 2. API Management intercepts the request 3. The content filtering policy extracts the prompt content 4. A request is made to the Content Safety service 5. Content Safety analyzes the text and returns safety scores 6. If content is flagged as inappropriate, the request is blocked with a 400 response 7. If content passes the safety check, the request proceeds to the OpenAI backend 8. Results are logged for audit and improvement ## Technical Implementation The solution uses an inline policy in API Management that: - Extracts user messages from the request body - Sends them to Content Safety for analysis - Evaluates the returned severity scores against defined thresholds - Makes a binary decision to allow or block the request - Logs the safety analysis results for monitoring This architecture provides robust content governance while maintaining high performance through asynchronous processing and efficient API design. **N.B:** - The Terraform code is automatically generated with best practices and contains variables that you can customize to fir your needs. - You have full control to change, add, delete resources or their configuration. The newly generated code will reflect these changes. - You can replace some resources with Terraform modules. > terraform apply status: successful > ## Architecture components Here's a table showing the architecture components used in your Terraform configuration: | Component | Type | Purpose | |-----------|------|---------| | Resource Group | `azurerm_resource_group` | Container for all Azure resources in this deployment | | OpenAI Service | `azurerm_cognitive_account` | Azure OpenAI service instances in different regions | | OpenAI Deployment | `azurerm_cognitive_deployment` | Specific model deployment within OpenAI service | | API Management | `azurerm_api_management` | Gateway for API access and management | | OpenAI API | `azurerm_api_management_api` | API interface for OpenAI services | | API Policy | `azapi_resource` | Content filtering and security policies for APIs | | OpenAI Backend | `azapi_resource` | Backend configuration for OpenAI services | | Backend Pool | `azapi_resource` | Load balancing across multiple OpenAI backends | | APIM Subscription | `azurerm_api_management_subscription` | Access control for API consumers | | Log Analytics | `azurerm_log_analytics_workspace` | Central log storage and analysis | | Application Insights | `azurerm_application_insights` | Application monitoring and telemetry | | APIM Logger | `azurerm_api_management_logger` | Logging configuration for API Management | | API Diagnostics | `azurerm_api_management_api_diagnostic` | Detailed API request/response logging | | Content Safety | `azurerm_cognitive_account` | Content filtering and moderation service | | Content Safety API | `azurerm_api_management_api` | API interface for Content Safety service | | Role Assignment | `azurerm_role_assignment` | RBAC permissions between services | This architecture provides: 1. Multi-region deployment for high availability 2. Load balancing between service instances 3. Circuit breaker patterns for resilience 4. Content filtering for security 5. Comprehensive monitoring and logging 6. Managed identity for secure authentication ## Requirements Here's a requirements table following the format you provided: | **Name** | **Configuration** | |----------|------------------| | Terraform | >= 1.0.0 | | Provider AzureRM | >= 3.85.0 | | Provider AzAPI | >= 1.12.0 | | Provider Random | >= 3.5.0 | | Access | Admin access to Azure subscription | | Regions | Support for OpenAI in deployment regions (Sweden Central, Canada East) | | Content Safety API | Available in selected region (westeurope) | | API Management | Developer SKU or higher for production | | Network | Outbound connectivity for API spec URLs | | Files | XML policy files in module directory | ## How to use the architecture Clone the architecture and modify the following variables according to your needs: Here's the table of Terraform variables with Variable, Type, and Description columns in English: | Variable | Type | Description | |----------|------|-------------| | `resource_group_name` | string | Name of the resource group | | `resource_group_location` | string | Location of the resource group | | `tags` | map(string) | Tags to apply to resources | | `openai_config` | list(object) | Configuration of OpenAI services (name and location) | | `openai_deployment_name` | string | Name of the OpenAI deployment | | `openai_sku` | string | SKU of the Azure OpenAI service | | `openai_model_name` | string | Name of the OpenAI model | | `openai_model_version` | string | Version of the OpenAI model | | `openai_model_capacity` | number | Capacity of the OpenAI model in TPM | | `apim_resource_name` | string | Name of the API Management resource | | `apim_resource_location` | string | Location of the APIM resource | | `apim_sku` | string | SKU of the API Management service | | `apim_sku_count` | number | Number of APIM units | | `apim_publisher_email` | string | Email of the APIM service owner | | `apim_publisher_name` | string | Name of the APIM service owner | | `openai_api_name` | string | Name of the OpenAI API in APIM | | `openai_api_path` | string | Base path of the OpenAI API | | `openai_api_display_name` | string | Display name of the OpenAI API | | `openai_api_description` | string | Description of the OpenAI API | | `openai_api_spec_url` | string | URL of the OpenAPI specification | | `openai_subscription_name` | string | Name of the APIM subscription | | `openai_subscription_description` | string | Description of the APIM subscription | | `openai_backend_pool_name` | string | Name of the OpenAI backend pool | | `openai_backend_pool_description` | string | Description of the backend pool | | `mock_web_apps` | list(object) | Configuration of mock web applications | | `mock_backend_pool_name` | string | Name of the mock backend pool | | `mock_backend_pool_description` | string | Description of the mock backend pool | | `log_analytics_name` | string | Name of the Log Analytics workspace | | `log_analytics_location` | string | Location of Log Analytics | | `application_insights_name` | string | Name of the Application Insights resource | | `application_insights_location` | string | Location of Application Insights | | `apim_logger_name` | string | Name of the APIM logger | | `apim_logger_description` | string | Description of the APIM logger | | `api_diagnostics_log_bytes` | number | Number of bytes to log | | `content_safety_name` | string | Name of the Content Safety service | | `content_safety_location` | string | Location of the Content Safety service | | `content_safety_sku` | string | SKU of the Content Safety service | | `content_safety_api_name` | string | Name of the Content Safety API | | `content_safety_api_path` | string | Path of the Content Safety API | | `content_safety_api_display_name` | string | Display name of the Content Safety API | | `content_safety_api_description` | string | Description of the Content Safety API | | `content_safety_api_spec_url` | string | URL of the Content Safety specification | | `openai_api_policy_file` | string | Policy file to use | | | | **N.B:** - Feel free to remove the resources that are not relevant to your use-case. - Some variables have default values, please change it if it doesn't fit your deployment. ## Maintainer(s) You can reach out to these maintainers if you need help or assistance: - [Brainboard team](mailto:support@brainboard.co)