About Company
Job Description
Summary
We're looking for a Senior Data Engineer who can:
- Help set up the standards and foundation for the upcoming data engineering team.
- Establish data practicing pattern at organisational level.
- Collaborate across teams
- Promote data literacy across the engineering team & business
We're ideally looking for the candidate to be based within Australia or an APAC country that allows for a minimum of 5hr crossover collaboration time with the teams.
Key Responsibilities
The person coming into the role will be responsible for:
- Establishing data ingestion pipeline
- Establishing orchestration layer for common ETL task orchestration
- Establishing ETL project template for ETL tasks
- Fully automating data CI/CD pipeline
Having these attributes would be beneficial for the success in the role:
- Strong Engineering skills
- Strong Experience working with various tools and tech stacks
- Strong data modelling skills
- Strong understanding of data challenges e.g. 3Vs and corresponding technical solutions
- Problem-focused outlook
Skills, Knowledge & Expertise
Ideally you'd bring:
- Production experience with modern data pipeline setup and standards.
- Proficient experience and skills with data ingestion, wrangling and ETL tasks. SQL, Pyspark, numpy, pandas, jupyter notebooks. Ideal working experience with AWS stacks, such as EMR, Glue, Athena. Other tools such as Fivetrans, dbt.
- Production experience with task orchestrations, hands-on experience with Airflow or lambda step functions, or equivalent.
- Production experience with Data warehouse technologies such as Databricks, Snowflake, Redshifts or equivalent
- Sound understanding of infrastructure as code, with working knowledge of Docker, Terraform, cloud formation or equivalent.
- Sound knowledge/experience around data modelling and data governance, monitoring.
- Sound understanding of different paradigms of data store technologies. Such as Streams, Pub-Sub, SQL, No-SQL.
- Familiar with data analytics and BI tools.
- Experience with ML, statistics is definitely a plus
What's in it for you?
- Opportunity to set up a new function within Data Engineering
- Work on a consumer facing app used by thousands of merchants and consumers.
- As we grow, you grow - we will fast track your development with a career path that could lead to cross functional and international opportunities, if you're up for it!
- Learning, learning, learning
- Ability to work on innovative tech and bring your ideas to life.
- All the perks that come with being one of our early-stage employees!
If you are excited about working in a company at the forefront of innovation, taking ownership, and being part of an extremely high-performing team then wed love to hear from you!
Skills
- Software Engineering
© 2024
cryptojobs.com. All right reserved.
