Summary

The team

Join our Data Infrastructure team and play a pivotal role in upholding the reliability, scalability, and efficiency of our robust Data platform. As a Senior Site Reliability Engineer (SRE) specialized in Data Infrastructure, you will collaborate closely with diverse cross-functional teams to conceive, execute, and oversee the foundational data infrastructure that empowers our array of applications and services.

As a key member of our Data Infrastructure team, you will:

1. Design the data governance mechanisms that ensure our lakehouse is easy to interact with, secure and in compliance with all applicable regulations.
2. Implement the infrastructure we use to ingest our data, store it, catalog it with the right metadata and capture its lineage.
3. Provide a state-of-the-art suite of BI tools for multiple teams within the company.
4. Guarantee the availability, high performance, scalability and cost efficiency of our data platform.

Your proficiency in cloud technologies, infrastructure as code, automation, monitoring, logging, user and machine AuthNZ, and certificate management will be instrumental in upholding the exceptional operational standards we set for our services.

The opportunity

1. Implement data infrastructure solutions (self service) that support the needs of 10+ business units and over 100 engineering and data analysts
2. Utilize Infrastructure as Code (IaC) principles to design, provision, and manage both on-premises and cloud (AWS) infrastructure components using tools such as Terraform
3. Develop and maintain automation scripts using bash/shell scripting and to automate operational tasks and deployments
4. Enhance and manage CI/CD pipelines to facilitate consistent software deployments across the data infrastructure
5. Implement robust data monitoring and alerting solutions to proactively detect anomalies and performance issues
6. Manage and implement role-based access control (RBAC) and permissions for a multitude of user groups and machine workflows across different environments
7. Manage and maintain real-time streaming data architecture using technologies like Kafka and Debezium Change Data Capture (CDC)
8. Ensure the timely and accurate processing of streaming data, enabling data analysts and engineers to gain insights from up-to-date information
9. Utilize Kubernetes to manage containerized applications within the data infrastructure, ensuring efficient deployment, scaling, and orchestration
10. Implement effective incident response procedures and participate in on-call rotations
11. Collaborate with data analysts, engineers, and cross-functional teams to understand requirements and implement appropriate solutions
12. Document architecture, processes, and best practices to enable knowledge sharing and support continuous improvement
13. Support AI/ML teams with their infra requests

Skills you should HODL

1. Proven experience (5+ years) working as a Site Reliability Engineer, Infrastructure Engineer, Data Infrastructure Engineer, or similar roles, with a focus on data infrastructure and security
2. Experience with maintaining real-time data processing technologies, such as Kafka and Flink clusters and Debezium instances
3. Working experience in managing hybrid multi-tenant cloud systems particularly on AWS
4. Infrastructure as Code tools such as Terraform, Terragrunt and Atlantis
5. Experience with containerization and orchestration tools, particularly Kubernetes, Nomad, and Docker
6. Solid understanding of bash/shell scripting and proficiency in at least one programming language (preferably Python or JVM languages)
7. Experience maintaining data-related technologies: Apache Airflow, Apache Spark, DBs, BI tooling
8. Experience solving data access management issues at large scale data-lake
9. Familiarity with CI/CD deployment pipelines and related tools
10. Strong problem-solving skills and the ability to troubleshoot complex systems
11. Experience with data-related technologies (databases, data lakes, airflow, spark) is a plus

Skills

• Communications Skills
• Development
• Problem Solving
• Software Engineering
• Team Collaboration