Operating Linux infrastructure on-premise

Requirements

• exclusively open-source technologies with no licensing restrictions,
• high availability without a single point of failure,
• separate production and test environments,
• room for future growth without major architectural changes.

Stateless application components run in Kubernetes; stateful services run in virtual machines: PostgreSQL, Kafka, OpenSearch, Redis and S3-compatible storage.

Design and evolution of the solution

The design went through three variants — from an ideal architecture with new hardware, through the use of existing servers, to a final cost optimisation: 4× refurbished hypervisors for a merged K8s cluster with logical separation, and VMs on the existing Proxmox cluster. The chosen variant was the optimal compromise between performance, availability and cost.

Architecture and perimeter

• a multi-layer perimeter: anti-DDoS → F5 Load Balancer → application firewall → internal network,
• an HA Kubernetes API over VPN, access via certificates and RBAC,
• test/production separation via namespaces and network policies,
• PostgreSQL and Redis Master/Slave, Kafka and OpenSearch cluster, S3 gateway.

Integration and DevOps

Central logging with OpenSearch + Dashboards, a CI/CD pipeline in GitLab with build, test, deployment to Kubernetes and the option of a fast rollback.

Benefits

• modern HA infrastructure, efficient use of the client's existing hardware,
• fast deployment without delaying the project, transparent operating costs,
• a scalable solution and demonstrable cooperation between operations and development.