// case study · platform engineering

Building a Production-Grade Multi-Cloud Infrastructure Platform for AI Workloads

How an engagement that started with individual Kubernetes deployment problems became a portable, secure, observable platform running one AI workload across AWS, Azure and Google Cloud.

Modern AI platforms are increasingly distributed across Kubernetes, multiple cloud providers, private registries, managed databases, secret-management systems and specialised worker workloads.

For one AI platform, our infrastructure consulting engagement evolved from solving individual Kubernetes deployment problems into designing and operationalising a portable, secure, observable, multi-cloud AI infrastructure platform.

The objective was not simply to deploy an application onto Kubernetes. The larger goal was an infrastructure foundation that could support:

  • Repeatable Kubernetes deployments
  • Multi-cloud execution across AWS, Azure and Google Cloud
  • Private and restricted environments
  • Automated Helm-based application delivery
  • Secure secret management
  • Centralised observability
  • Automated CI/CD
  • Database migration between clouds
  • Air-gapped and private-registry scenarios
  • Production-grade security and vulnerability management
  • Operational scalability and high availability

The result was a platform capable of taking a complex AI workload and deploying it consistently across different Kubernetes environments.

The resulting platform: one Helm deployment model, three clouds, shared security, observability and delivery layers. Click to enlarge.

1. The infrastructure challenge

The platform consisted of several classes of workloads: Kubernetes control-plane services, stateful infrastructure such as PostgreSQL and Redis, Vault for secret management, application services, database migration jobs, AI/worker workloads, Windows-based workers, observability components, and networking and gateway components.

Initially, infrastructure behaviour varied considerably between environments. A deployment that worked on one cloud could fail on another because of differences in:

  • Container registries
  • Kubernetes implementations
  • Storage classes
  • IAM
  • Networking
  • Load balancers
  • Node architectures
  • Secret managers
  • Certificate management
  • Database connectivity
  • Image availability

That made the central problem less about Kubernetes itself and more about abstraction and portability. The approach was to establish a common deployment model while injecting cloud-specific infrastructure through configuration and Terraform.

2. Designing a reusable Helm-based application platform

One of the largest parts of the engagement was the development and continuous evolution of a centralised Helm chart. Instead of maintaining independent manifests for every environment, the application was packaged into a reusable, Helm-based deployment model. The chart grew to support:

  • Application deployments
  • Stateful workloads
  • PostgreSQL
  • Redis
  • Vault
  • Migration jobs
  • Gateway API
  • Envoy Gateway
  • cert-manager
  • OpenTelemetry
  • RBAC
  • ServiceAccounts
  • Image pull secrets
  • Node affinity
  • Storage configuration
  • Environment overrides
  • External databases
  • External Redis
  • External Secrets Operator
  • Optional components
  • Cloud-specific config

This created a single infrastructure interface for deploying the platform.

Configuration-driven deployment

Rather than hard-coding infrastructure assumptions into manifests, the chart was progressively redesigned around values files:

global:
  storageClass: standard-rwo
  imagePullSecrets:
    - name: private-registry

gateway:
  enabled: true

nodeAffinity:
  enabled: false

Environment-specific files then override only the values that differ between development, QA, staging and production — which significantly reduced environment-specific branching.

3. Multi-cloud Helm architecture

The chart was designed to run on Google Kubernetes Engine, Azure Kubernetes Service and Amazon Elastic Kubernetes Service. The key architectural principle:

The application chart should remain largely cloud-agnostic; cloud-specific infrastructure should be provided by the underlying platform.

Storage

Different clusters expose different StorageClasses, so the chart made storage configuration explicit instead of relying on cluster defaults. That prevented PVCs stuck in Pending, database pods failing to start, Redis failing to initialise and unexpected storage performance.

Container registries

Images could come from different registries depending on the environment, so the chart supports configurable registries and image overrides — either registry/repository:tag or immutable registry/repository@sha256:<digest>. The same chart works with public, private or mirrored images.

Cloud-specific overrides

Image locations, storage classes, pull secrets, secret-manager authentication, networking, gateway configuration, service exposure and database endpoints were separated from the base deployment logic. This abstraction became essential as deployments expanded from one cloud to several.

4. Private registry and image supply chain

A major requirement was deploying without depending on public container registries. We built an image-management workflow that:

  1. Identifies all application and infrastructure images
  2. Scans images for vulnerabilities
  3. Mirrors images into private registries
  4. Supports different registries across clouds
  5. Overrides image locations through Helm values
  6. Tests the resulting deployment end-to-end

Images moved between Google Artifact Registry, Azure Container Registry, Amazon ECR and other private registries. Every infrastructure image — Envoy, cert-manager, PostgreSQL, Redis, Vault, CloudNativePG, OpenTelemetry and Kubernetes utilities — can be overridden, which is critical for restricted and air-gapped environments.

5. Vulnerability management and hardened images

Container security became an ongoing part of the lifecycle rather than a one-time activity. We repeatedly scanned application, infrastructure and base images, including OpenTelemetry, PostgreSQL, Envoy, cert-manager, Vault and Redis. For each finding we evaluated version upgrades, base-image upgrades, rebuilds, replacing vulnerable dependencies, moving to hardened images, or pinning known-good versions.

Infrastructure image versions were then tracked independently of application releases, establishing a clearer software supply chain for the platform.

6. OpenTelemetry and Datadog integration

The goal was to collect Kubernetes and application telemetry without every team building its own monitoring pipeline. The OpenTelemetry Collector runs as a DaemonSet and collects container logs, Kubernetes events, node, pod, container and volume metrics, and application telemetry. Logs can be filtered per namespace before being forwarded.

  1. Kubernetes workloads
  2. OpenTelemetry Collector
  3. Logs → relay layerMetrics → DatadogEvents → pipeline

7. A dedicated OTel forwarding layer

As requirements grew, a dedicated forwarding component gave telemetry a controlled path out of each cluster. It had to solve authentication, environment tagging, client identification, CI/CD, image management, connectivity, socket and HTTP forwarding, failure handling, persistent queueing and resource sizing — and was tested across development, QA and production. Static credential files were replaced with workload identity wherever supported.

8. AI infrastructure observability

For AI workloads, application metrics alone aren't enough. Teams also need visibility into worker utilisation, node capacity, pod scheduling, storage, job execution, database and Redis availability, gateway behaviour, image-pull failures and cluster scaling. We added node-utilisation dashboards, Kubernetes event collection and persistent-volume metrics across Elasticsearch, Kibana, Prometheus, Grafana and Datadog — moving toward one operational view instead of troubleshooting pods in isolation.

9. Gateway API, TLS and external access

Secure external access was implemented with Gateway API, Envoy Gateway and cert-manager, with TLS certificates automated through ACME.

  1. Internet
  2. DNS
  3. Cloud load balancer
  4. Envoy Gateway
  5. HTTPRoute
  6. Application service

We also documented the common failure modes — DNS not pointing at the load balancer, HTTP-01 challenges failing, port 80 blocked, gateway IP changes and certificate delays — turning a manual networking process into a repeatable capability.

10. Vault and secret management

Secrets needed a stronger model than Helm values or plain manifests. Vault was integrated with initialisation, Kubernetes and ServiceAccount authentication, policies, pre/post-install bootstrap jobs, unseal automation, and an HA Raft configuration. Workloads authenticate with their Kubernetes identity instead of long-lived credentials.

We also worked through Vault's lifecycle problems: webhook conflicts, bootstrap and initialisation failures, unseal issues, upgrade ordering and RBAC ownership conflicts.

11. External Secrets Operator and multi-cloud secrets

As the platform spread across clouds, secret management had to become cloud-aware. External Secrets Operator lets workloads consume secrets without storing secret material in Helm values.

  1. Cloud secret manager
  2. External Secrets Operator
  3. Kubernetes Secret
  4. Application pod
GCP Secret Manager→ ESO → GKEGCP Workload Identity
AWS Secrets Manager→ ESO → EKSIAM Roles for Service Accounts
Azure Key Vault→ ESO → AKSAzure workload identity

This removed a significant dependency on static cloud credentials.

12. External database architecture

The platform initially ran databases inside Kubernetes. As it matured, the chart gained support for both in-cluster PostgreSQL and managed PostgreSQL, and the same for Redis. That meant changes to values, connection configuration, secrets, ESO, migration jobs, startup ordering and readiness checks — and created a clean separation between application compute and persistent infrastructure.

13. Database migration across clouds

AWS Database Migration Service moved PostgreSQL from another cloud into AWS, with the supporting infrastructure in Terraform plus smoke tests and replication validation.

  1. Source PostgreSQL
  2. AWS DMS
  3. Target RDS
  4. Migration validation

Issues along the way — failed tables, replication problems, connectivity, configuration flags, SSL requirements and restart behaviour — were captured in an operational runbook rather than left in individual engineers' heads.

14. Multi-cloud networking

Moving an AI platform between clouds needs more than moving manifests: the networks have to talk securely. We implemented AWS ↔ GCP connectivity with site-to-site VPN, routing, VPC and subnet planning, all automated in Terraform.

GCPKubernetes + database
VPN / interconnect
AWSKubernetes + RDS

Connectivity was validated from Kubernetes workloads, not just from cloud consoles — for example, whether an EKS pod could actually reach a database endpoint in another cloud.

15. Terraform-based cloud infrastructure

As environments multiplied, hand-built infrastructure became hard to maintain, so everything was codified in reusable Terraform modules: VPCs, subnets, clusters, node groups, autoscaling, IAM, storage, RDS, VPN, interconnect, secret-manager integration, S3, organisational policies, Control Tower and service accounts. New environments came up faster and with far less drift.

16. AWS account governance

As the AWS footprint grew we put in organisational structure, environment accounts, Terraform state storage, Control Tower, Service Control Policies, permission sets, EKS access controls and tagging standards — plus preventive controls such as blocking unwanted default VPC and subnet creation. AWS management moved from per-account configuration to central governance.

17. CI/CD for Helm and infrastructure

A major objective was eliminating manual Helm deployments.

  1. Developer change
  2. Pull request
  3. Lint / validation
  4. Chart packaging
  5. Versioning
  6. Artifact registry
  7. Promotion: dev → QA → prod

Pipelines cover Helm linting, chart validation, image updates, packaging, version management, environment promotion, release creation and smoke tests. Pre-commit hooks catch Helm errors even earlier.

18. ArgoCD implementation

GitOps through ArgoCD made Kubernetes state declarative and auditable. Instead of engineers running helm upgrade --install per environment, desired state lives in Git and ArgoCD reconciles it.

  1. Git repository
  2. ArgoCD
  3. Kubernetes cluster
  4. Helm application

Making the chart ArgoCD-friendly took work on annotations, release-name handling, namespace flexibility, CRDs, install ordering, hooks, resource ownership, cluster-scoped resources and upgrade behaviour.

19. Octopus CD and environment promotion

Alongside GitOps, an environment-oriented CD model was built in Octopus Deploy with dev, QA and production channels, environment variables, Kubernetes targets and agents, release pipelines, smoke tests, manual validation stages, and install and upgrade workflows. The platform supports both GitOps-style and pipeline-driven deployment, depending on the environment.

20. Windows Kubernetes workers

AI workloads aren't always Linux-only. Dedicated Windows node pools came with right-sized nodes, taints, scheduling rules, larger disks, container compatibility testing, Windows-specific Dockerfile fixes and autoscaling — and the chart schedules Windows workers without affecting Linux workloads.

21. Scheduling, affinity and high availability

Default scheduling stops being enough once workloads are production-critical. Node affinity, taints and tolerations, topology spread constraints, disruption planning and node autoscaling now place each workload deliberately:

Linux application nodescontrol-plane services
Linux worker nodesAI / processing jobs
Windows nodesWindows-specific workers

22. Stateful workload reliability

Databases and queues add another layer: PostgreSQL HA, Vault HA, Redis topology and health checks, PV sizing and PVC expansion, database upgrade behaviour, migration ordering and StatefulSet configuration. Storage is configurable and safe PVC expansion is documented — important because AI processing generates a lot of temporary and persistent data.

23. Air-gapped and restricted environment support

One of the hardest requirements was removing reliance on external registries and Helm repositories. Dependencies are now packaged into the Helm artifact ahead of time instead of running helm dependency build during deployment. A pre-built package carries the application, dependencies, CRD installation logic, configuration and image overrides into a restricted cluster — and combined with private registries, that enables deployments with no outbound connectivity.

24. Automation around infrastructure images

  1. Upstream version
  2. Image build
  3. Vulnerability scan
  4. Private registry
  5. Helm values update
  6. CI validation
  7. Environment promotion

This removed manual synchronisation between container versions and Helm chart versions.

25. Cluster lifecycle management

Clusters were created, tested, resized, upgraded and removed across AWS, Azure and Google Cloud — covering node pools, autoscaling, storage, IAM, registry access, gateway, Helm and observability installation, smoke tests and cleanup. Temporary clusters became the standard way to test infrastructure changes:

  1. Terraform
  2. Temporary cluster
  3. Helm install
  4. Integration tests
  5. CI validation
  6. Production

26. Troubleshooting as infrastructure engineering

Many failures only appeared when the whole platform ran end-to-end:

  • ImagePullBackOff
  • Registry auth failures
  • Architecture mismatches
  • Helm ownership conflicts
  • CRD install issues
  • Certificate failures
  • Gateway connectivity
  • Vault init failures
  • Redis startup
  • PostgreSQL startup
  • PVC provisioning
  • Scheduling problems
  • VPC connector failures
  • Cloud networking
  • DMS replication
  • OTel forwarding
  • Node deallocation
  • Autoscaling problems

Rather than treating these as isolated bugs, we wrote troubleshooting guides and reusable diagnostic procedures.

A production platform must be operable by engineers who did not build it.

27. Documentation as an infrastructure deliverable

Documentation was a first-class output: cluster and network requirements, firewall rules, Helm installation, image configuration, private registries, gateway and TLS, Vault, External Secrets, OpenTelemetry, multi-cloud networking, database migration, storage expansion, CI/CD, AWS and Azure infrastructure, air-gapped deployment, troubleshooting and cluster access. It became an operational guide, not a list of install commands — and made the platform far easier to hand over.

28. The resulting AI infrastructure architecture

The platform is a stack of interconnected layers — shown in the architecture diagram at the top of this article:

ApplicationAI services and workers stay largely independent of the cloud underneath.
KubernetesHelm provides one consistent deployment model.
CloudTerraform provisions the cloud-specific infrastructure.
SecurityVault, ESO, IAM, workload identity, private registries, vulnerability scanning and org policies.
NetworkingGateway API, TLS, VPN, interconnects, VPCs and routing.
ObservabilityOpenTelemetry, Datadog, Elasticsearch, Kibana, Prometheus and Grafana.
DeliveryCI/CD, Helm packaging, ArgoCD and Octopus.

29. What this means from an AI infrastructure consulting perspective

The most important outcome wasn't any single Helm chart, Terraform module or cluster. It was an infrastructure abstraction layer for AI workloads — so the same application can move between clouds without being rewritten.

30. Key engineering principles

  1. Build the chart once, adapt the infrastructure. The deployment model shouldn't be rewritten for every cloud.
  2. Treat images as infrastructure dependencies. They need versioning, scanning, mirroring and promotion just like Terraform modules.
  3. Never assume cloud portability. Kubernetes is portable; storage, IAM, networking, load balancing, registries and identity are not.
  4. Secrets should follow workload identity. Replace static credentials with workload-native identity.
  5. Observability must be part of the platform. Logs and metrics are infrastructure capabilities, not per-app projects.
  6. Stateful services require explicit engineering. PostgreSQL, Redis, Vault, storage, backup and migration are designed separately from stateless workloads.
  7. Infrastructure must be testable. A chart that renders isn't necessarily deployable — install it end-to-end on real clusters.
  8. Documentation is part of the deliverable. Infrastructure only its authors understand isn't production-ready.

Conclusion

Building infrastructure for an AI platform is fundamentally different from deploying a conventional web application. AI workloads combine high compute variability, large worker fleets, stateful processing, mixed operating systems, complex networking, strict security, large container supply chains, rapid iteration and multi-cloud requirements.

This engagement addressed those challenges by building a reusable platform around Kubernetes, Helm, Terraform, CI/CD, OpenTelemetry, secure secret management and multi-cloud networking — one that grew from a single cluster into a multi-cloud platform while keeping a common deployment model.

AI infrastructure should be engineered as a platform, not assembled as a collection of one-off deployments.