Essential-8 Security in Practice with Kubernetes

The Digital Earth Australia and Digital Earth Africa programs host and process very large volumes of open data on cloud infrastructure using primarily Kubernetes as an orchestration mechanism. The project is run out of an Australian government agency, Geoscience Australia, and as such, this infrastructure needs to meet the [Essential-8](https://www.cyber.gov.au/acsc/view-all-content/essential-eight/essential-eight-explained) security mitigation requirements. This talk presents the difficult balancing act of meeting the security needs of an openaccess, open data and open source platform. The principal workloads on this infrastructure include web applications, petabyte scale data processing and cross-continent data movement. Part of this platform enables scientific experimentation, which essentially allows arbitrary code execution too. Recent statistical operations on year-long continental Satellite imagery time-series for Africa consumed 4,000 CPUs and 50 TiB of RAM across 200 nodes. The infrastructure is mostly codified and open sourced [here](https://github.com/opendatacube/datacube-k8s-eks) as terraform modules. Applications are deployed via [Helm Charts](https://github.com/opendatacube/datacube-charts) and scanned using [Trivy](https://github.com/aquasecurity/trivy). The network within the cluster and to the outside is filtered using [Cilium](https://cilium.io/). Deployments are automated and audit-logged via GitOps and [Flux](https://github.com/fluxcd/flux). The ability for users to self-signup and run arbitrary code occasionally leads to some cycles being lost to Monero. This talk will cover general security principles as well as specific security incidents and mitigation actions taken in response. In conclusion, our stack meets Australian Government security practice, while enabling very large workloads to be executed in a flexible, on-demand computation environment that empowers scientific users. This talk will tell you how we do it.