OfficeAncoats
Data Engineer
О роли
Описание вакансии
About the company
- DF Capital is a specialist bank providing award-winning commercial finance, retail finance, and savings products to consumers and small businesses.
- Based in Manchester, serving thousands of customers across the UK and into Europe.
Responsibilities
- Design, develop and maintain scalable data pipelines and data integration solutions.
- Build and optimise SQL-based solutions within Snowflake.
- Create and maintain robust ELT processes supporting reporting and analytics requirements.
- Develop and manage data models that support business, operational and regulatory needs.
- Ensure data quality, consistency and integrity across multiple data sources.
- Document data flows, transformation logic and technical solutions.
- Design and maintain ingestion pipelines for both structured and semi-structured data.
- Source and integrate data from operational databases, applications and third-party platforms.
- Develop integrations with REST APIs and other external data services.
- Extract, process and transform semi-structured data formats such as JSON and XML.
- Monitor ingestion processes and implement controls to ensure data reliability and completeness.
- Work with business and technical stakeholders to understand source systems and data requirements.
- Build and maintain transformation layers using SQL and DBT.
- Design reusable and scalable data models following engineering best practices.
- Implement testing and validation to ensure high levels of data quality.
- Optimise transformation performance and maintainability.
- Develop and maintain automated workflows using Windmill.
- Schedule, orchestrate and monitor data processing activities.
- Identify opportunities to improve operational efficiency through automation.
- Support troubleshooting and resolution of production issues.
- Use GitHub for source control, collaboration and code reviews.
- Support and improve CI/CD processes for data platform deployments.
- Promote engineering best practices including version control, testing and release management.
- Maintain technical documentation and operational runbooks.
- Work closely with the Lead Data Engineer and wider stakeholders across the bank.
- Participate in architecture discussions and solution design.
- Contribute to the evolution of the data platform and engineering standards.
- Share knowledge and support continuous improvement initiatives within the team.
Requirements
- Strong SQL skills with experience developing complex queries, transformations and data models.
- Experience working with modern cloud-based data architectures and data platforms.
- Experience sourcing and integrating data from structured sources such as relational databases.
- Experience working with semi-structured data including JSON, XML and API-based integrations.
- Familiarity with Python for automation, scripting, API integration and data processing.
- Experience designing and maintaining data transformation pipelines.
- Good understanding of data warehousing concepts and dimensional modelling.
- Experience working with Git-based source control and collaborative development practices.
- Understanding of CI/CD principles and automated deployment processes.
- Knowledge of data quality, testing and monitoring methodologies.
- Experience working as a Data Engineer, ETL Developer or similar role.
- Experience delivering data solutions within a cloud environment.
- Ability to work independently and manage priorities effectively.
- Strong analytical and problem-solving skills.
- Excellent communication skills with both technical and non-technical stakeholders.
- Desirable: Experience with DBT for data transformation and modelling.
- Desirable: Experience with Windmill for workflow orchestration and automation.
- Desirable: Experience building internal data applications using Streamlit.
- Desirable: Experience with Snowflake administration and optimisation.
- Desirable: Knowledge of financial services, banking or regulated industries.
- Desirable: Exposure to regulatory reporting and data governance frameworks.
- Desirable: Experience working in Agile delivery teams.
Стек и навыки
С чем работаем