Data Platform Engineer

Description:

At Kraken, we've developed a data platform that is used by Octopus Energy and our other clients' retail energy businesses around the world. The platform empowers users with self-service data analytics and automates data processing workflows, ranging from simple ETL jobs to ML training and prediction. The data platform team works across the whole customer domain, handling everything from natural language understanding of our customer communications to processing billions of smart meter readings every day to support clients in creating customised and market leading smart energy tariffs. As the volume, scope and geographical range of our data offerings rapidly expand, we're looking for an experienced Platform Engineer to join the team to help us build and maintain our platform, pipelines and data sources. We're passionate about building great technology to change the way customers use, and think about, energy for the good of the planet. This is a fantastic opportunity to work with us on data problems that genuinely move us closer to Net Zero, and support the energy transition. We employ software engineering best practices to design, test, and deploy our data platform and services.

What you'll have...

• First and foremost, we want our Platform Engineers to have a passion for writing high quality code and building lean processes. It would be helpful to have experience/expertise/knowledge in the following (in rough priority order):
• AWS
• Terraform/OpenTofu IAC
• Postgres
• CDC systems
• Kubernetes (EKS)
• Python
• Docker
• An interest or experience in data/network security would be a nice-to-have
• Datadog / Grafana / Prometheus
• Data related products (Airflow, Jupyter, Spark, dbt etc)
• The projects will be varied and we’re looking for someone who can work autonomously and proactively to scope problems and solve and deliver pragmatic solutions

Our tech stack

• Python as our main programming language
• Terraform for our infrastructure definition
• Kubernetes for data services and task orchestration
• Argo CD for application deployments
• Airflow purely for job scheduling and tracking
• CircleCI for continuous deployment
• Parquet and Delta file formats on S3 for data lake storage
• Postgres / Aurora for our relational databases
• Spark for data processing
• dbt for data modelling
• SparkSQL for analytics
• Streamlit for data applications