Data Engineer at Jubilee Insurance

Job Ref. No. JLIL106

Role Purpose

The data engineer will oversee expanding and optimizing of our data architecture and data pipeline with the purpose of improving data analysis and ML workflows. The data engineer will be handling the design and construction of scalable data systems and research new use cases for data acquisition.

Main Responsibilities

Technical

• Help to design and develop a data estate that is performant, accessible, secure, scalable, maintainable, and extensible.
• Help to implement true CI/CD.
• Design and develop EDW using Snowflake, DBT etc.
• Design and develop AWS/Azure Data Lake
• Design and develop data ingestion pipelines
• Model EDW entities and ensure all data is complete, accurate, timely, and well documented
• Work towards the implementation of a true Self-Service analytics platform.

Governance & Data Protection

• Ensure that all work follows best security practices and fully adheres to DPA and other data regulations.
• Ensure that all work follows the correct approval and sign-off process before it is pushed into Production.
• Ensure that all work is documented and if needed, has a runbook in order to guarantee business continuity and support
• Work with others in the team to keep the Data Dictionary and Ubiquitous Language complete and up to date.

Process

• Follow existing processes and work to improve/identify gaps in these processes.
• Ensure the correct SDLC promotion processes are followed.
• Follow the correct sign-off processes to ensure that only approved releases are deployed into Production.
• Ensure that all AWS/Azure development follows CI/CD processes and is repeatable.

Team/People

• Evangelize about the data team across the business.
• Build relationships with members of the data team and the wider business team.
• Work closely and collaboratively with all members of the data team.
• Work closely with and learn from tech and team leads and challenge proposed solutions with your own ideas.

Academic/Professional Qualifications

• Bachelors’ degree in IT or related fields
• Some experience and knowledge of a coding language such as Python.
• Good experience and knowledge of the SQL query language.
• Some understanding of star schemas and data warehouse concepts.
• Some knowledge of AWS/Azure tools and technologies
• Beneficial - ETL and ELT experience - both batch and microservices-led.

Relevant Experience

• Minimum two years’ experience in a data engineer role or similar. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, et