## Description  This solution provides an enterprise-grade Azure OpenAI implementation with built-in redundancy, scalability, and monitoring. It features three separate OpenAI service pools distributed across multiple regions, all unified behind an API Management gateway that handles load balancing and access control. A comprehensive monitoring stack with Application Insights, Log Analytics, and custom dashboards offers detailed visibility into system performance and usage patterns. Key advantages include: High availability through multi-region deployment Enhanced token throughput via distributed model instances Centralized API management and security controls Complete observability with pre-configured analytics Flexible scaling options to accommodate growing demands This architecture is designed to support production workloads with the reliability, security, and governance features required for business-critical AI applications. **N.B:** - The Terraform code is automatically generated with best practices and contains variables that you can customize to fir your needs. - You have full control to change, add, delete resources or their configuration. The newly generated code will reflect these changes. - You can replace some resources with Terraform modules. > terraform apply status: successful > ## Architecture components | Component | Description | Purpose | |-----------|-------------|---------| | **Azure OpenAI Services** | | OpenAI Pool 1 | Multiple Azure OpenAI instances in different regions | Provides primary model deployment with geographic redundancy | | OpenAI Pool 2 | Multiple Azure OpenAI instances in different regions | Provides secondary model deployment with geographic redundancy | | OpenAI Pool 3 | Multiple Azure OpenAI instances in different regions | Provides tertiary model deployment with geographic redundancy | | **API Management** | | APIM Instance | Central API Management service | Acts as gateway, providing a single endpoint for all OpenAI interactions | | APIM Logger | Logging component for API Management | Captures detailed information about API requests and responses | | APIM Subscription | API key management | Controls access to the OpenAI APIs | | **Backend Pools** | | Backend Pool 1 | Configuration for OpenAI Pool 1 endpoints | Manages load balancing across Pool 1 instances | | Backend Pool 2 | Configuration for OpenAI Pool 2 endpoints | Manages load balancing across Pool 2 instances | | Backend Pool 3 | Configuration for OpenAI Pool 3 endpoints | Manages load balancing across Pool 3 instances | | **Monitoring** | | Log Analytics Workspace | Centralized logging repository | Stores and indexes all system logs for analysis | | Application Insights | Application monitoring service | Provides real-time telemetry and performance monitoring | | Usage Analysis Workbook | Custom dashboard | Visualizes OpenAI usage patterns and metrics | | Diagnostic Settings | Resource logging configuration | Captures detailed logs from all Azure resources | | Alerts | Automated notifications | Triggers notifications based on predefined thresholds | | Azure Monitor | Global monitoring platform | Provides a unified view of all infrastructure metrics | | Resource Health | Service health monitoring | Monitors the health status of Azure OpenAI services | | **Security** | | IP Restrictions | Network security rules | Limits API access to specified IP addresses | | Subscription Keys | API authentication | Secure access control for the OpenAI endpoints | ## Requirements | Name | Configuration | | --- | --- | | Terraform | all versions | | Provider: azurerm | >= 5.33.0 | | Provider: azapi | >= 1.11.0 | | Provider: random | >= 3.5.1 | | Provider: time | >= 0.10.0 | | Access | Admin access | | Azure CLI | Required for authentication | | Azure OpenAI | Resource provider registration required | | API Management | Resource provider registration required | | Log Analytics | Resource provider registration required | | Application Insights | Resource provider registration required | | Subscription | Access to create Cognitive Services | | Regions | Support for Azure OpenAI service | | Resource quotas | Sufficient OpenAI deployment quotas | ## How to use the architecture Clone the architecture and modify the following variables according to your needs: | Variable | Description | |---------------|-------------| | **Log Analytics & Application Insights** | | `log_analytics_name` | Name of the Log Analytics resource | | `log_analytics_location` | Location of the Log Analytics resource | | `application_insights_name` | Name of the Application Insights resource | | `application_insights_location` | Location of the Application Insights resource | | **API Management Logger** | | `apim_logger_name` | Name of the APIM Logger | | `apim_logger_description` | Description of the APIM Logger | | `api_diagnostics_log_bytes` | Number of bytes to log for API diagnostics | | **Workbook** | | `workbook_name` | Name of the Workbook | | `workbook_location` | Location of the Workbook | | `workbook_display_name` | Display name of the Workbook | | **OpenAI Configuration** | | `index` | Index used to generate unique names | | `openai_config_1` | List of OpenAI resources to create for Pool 1 | | `openai_config_2` | List of OpenAI resources to create for Pool 2 | | `openai_config_3` | List of OpenAI resources to create for Pool 3 | | `openai_deployment_name_1` | Name of deployment 1 | | `openai_deployment_name_2` | Name of deployment 2 | | `openai_deployment_name_3` | Name of deployment 3 | | `openai_sku` | Azure OpenAI SKU | | `openai_model_name_1` | Name of model 1 | | `openai_model_version_1` | Version of model 1 | | `openai_model_name_2` | Name of model 2 | | `openai_model_version_2` | Version of model 2 | | `openai_model_name_3` | Name of model 3 | | `openai_model_version_3` | Version of model 3 | | `openai_model_capacity` | Model capacity | | **API Management** | | `apim_resource_name` | Name of the API Management resource | | `apim_resource_location` | Location of the APIM resource | | `apim_sku` | The pricing tier of this API Management service | | `apim_sku_count` | The instance size of this API Management service | | `apim_publisher_email` | The email address of the service owner | | `apim_publisher_name` | The name of the service owner | | **OpenAI API Configuration** | | `openai_api_name` | The name of the APIM API for the OpenAI API | | `openai_api_path` | The relative path of the APIM API for the OpenAI API | | `openai_api_display_name` | The display name of the APIM API for the OpenAI API | | `openai_api_description` | The description of the APIM API for the OpenAI API | | `openai_api_spec_url` | Complete URL for the OpenAI API specification | | **OpenAI Subscription** | | `openai_subscription_name` | The name of the APIM subscription for the OpenAI API | | `openai_subscription_description` | The description of the APIM subscription for the OpenAI API | | **OpenAI Backend Pools** | | `openai_backend_pool_name_1` | The name of the OpenAI backend pool 1 | | `openai_backend_pool_name_2` | The name of the OpenAI backend pool 2 | | `openai_backend_pool_name_3` | The name of the OpenAI backend pool 3 | | `openai_backend_pool_description` | The description of the OpenAI backend pool | | **Additional Infrastructure** | | `tags` | Default tags to apply to all resources | | | | **N.B:** - Feel free to remove the resources that are not relevant to your use-case. - Some variables have default values, please change it if it doesn't fit your deployment. ## Maintainer(s) You can reach out to these maintainers if you need help or assistance: - [Brainboard team](mailto:support@brainboard.co)