Lead Data Engineer

Roles

Made Tech

Lead Data Engineer

Explore roles

This role has expired

Made Tech

Lead Data Engineer

£80k

£100k

Multiple locations

Hybrid

Graduate

Junior

Mid

Senior

Leader

Description

Hybrid requirements: This role has flexible working patterns.

Application question:

Have you lived in the UK continuously for the last 4.5 years?

As a lead data engineer or architect at Made Tech, you’ll play a pivotal role in helping public sector organisations become truly data-led. You’ll join our data team in its mission to get data knowledge and skills out of silos and embedded into delivery teams. You’ll help implement efficient data pipelines and storage.You’ll help build data platforms to enable predictive government services that make a positive impact on people in the UK.

Your role at Made Tech

Define, shape and perfect data strategies in central and local government Help public sector teams understand the value of their data, and make the most of it Establish yourself as a trusted advisor in data driven approaches using public cloud services like AWS, Azure and GCP As employee growth is a huge focus here, we would expect you to contribute to our recruitment efforts and take on line management responsibilities

What skills and experience are we looking for?

We are looking for candidates with a range of skills and experience, please apply even if you don’t meet all the criteria as if unsuccessful we can provide you with feedback.

Enthusiasm for learning and self-development Proficiency in Git (inc. Github Actions) and able to explain the benefits of different branch strategies Gathering and meeting the requirements of both clients and users on a data project Strong experience in IaC and able to guide how one could deploy infrastructure into different environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes) and the different use cases for them Ability to create data pipelines on a cloud environment and integrate error handling within these pipelines. With an understanding how to create reusable libraries to encourage uniformity of approach across multiple data pipelines. Able to document and present an end-to-end diagram to explain a data processing system on a cloud environment, with some knowledge of how you would present diagrams (C4, UML etc.) To provide guidance how one would implement a robust DevOps approach in a data project. Also would be able to talk about tools needed for DataOps in areas such as orchestration, data integration and data analytics. Experience in improving resilience into a project by checking for software vulnerabilities and implement appropriate testing strategies (unit, integration, data quality etc.) Knowledge of SOLID, DRY and TDD principles and how to practically implement these into a project. Agile practices such as Scrum, XP, and/or Kanban Designing and implementing efficient data transformation processes at scale, both in batch and streaming use cases Owning the cloud infrastructure underpinning data systems through a DevOps approach Agile practices such as Scrum, XP, and/or Kanban People skills such as mentoring, supportive team player and performing line management duties To be able to demonstrate a commercial mindset when on projects to grow accounts organically with senior stakeholders

Desirable experience

Experience in the following things isn’t essential, but it’s highly desirable!

Working at a technology consultancy Working with Docker and virtual environments as part of the development and CI/CD process. Working with senior stakeholders to gather requirements and keep them engaged with Experience in working with a team of engineers using a variety of techniques such as pair programming or mob programming. Working with data scientists to productionise advanced data deliverables, such as machine learning models Working knowledge of statistics Working with multidisciplinary digital and technology teams Working within the public sector Working with data scientists to productionise advanced data deliverables, such as machine learning models

Role tech stack

Azure Databricks

AWS

PySpark

Microsoft Azure

Terraform

DBT

Python

Life at

Made Tech

Browse all roles

Culture overview

We are Transparent We're building an open and transparent company across the UK, full of people who are passionate about their work, who always strive to be better, who love to help others grow and who care about using digital and technology to create a better, fairer society. We Make a Difference We focus our efforts on building meaningful, open source services that can help improve citizens' lives. We find this type of work more challenging and more rewarding. Less short-lived, throwaway projects, more mission-critical systems and services. We Continuously Improve We see the importance and value in self-improvement. We have regular code dojos, show and tells, reviews and other reflective practices we aim to encourage growth. One of the biggest ways we have committed to developing our skills is Learn Tech Fridays. We Learn Learn Tech consists of a number of tracks, where every Friday afternoon the whole company comes together to focus on learning new or developing existing skills. This is a constantly evolving programme and we are working with our team's feedback to keep improving it going forward. We support individuals with 12 days' paid learning each year, plus an individual learning budget. We are here to stay We are building a lasting organisation in which our people can grow with the business. We try to think hard about how to best approach every aspect of what we do to create the most enjoyable and productive environment. We are Inclusive We want to foster a diverse, happy and well-supported team. D&I isn't just something we pay lip service to. The services we help build will be used by people around the country. So it’s important to us that the teams that build them reflect this diversity too. Wherever you are on your career path. Whatever your background and experience. If you’re passionate about using technology to make life better for everyone, we absolutely want to hear from you. Certifications We see workplace engagement as a vital part of our continued success in empowering our employees to have a positive impact on the public sector. We are extremely proud to have received certifications from Best Companies for our continued commitment to making Made Tech a great place to work. Technology’s 50 Best Companies to Work For in London for 2021 Technology’s 50 Best Companies to Work For in the South West for 2021 Outstanding to Work for 2-Star Accreditation for 2021

Employee benefits

Cycle To Work

Flexible Hours

Flexible Working

Laptop

Learning Allowance

Life Insurance

Pension

Season Ticket Loan

Unlimited Holidays

Office vibe

Beer Fridge

City Centre

Friday Drinks

Office Dog

Open Plan

Social Events

Location

Tech at

Made Tech

Go to profile

Leadership

Ben Collier

Head of Cloud and Engineering

Tech overview

We want to positively impact the future of the country by using technology to improve society, for everyone. We work with public sector organisations to modernise technology and working practices, and accelerate digital delivery so that citizens can benefit from better public services. We're transparent about what we do and how we do it, which is why we publish everything about us here: https://github.com/madetech/handbook We grow a team of polyglot programmers, comfortable using several programming languages. Our teams can work across a variety of technologies and you will need to be comfortable doing so too. Our teams have used Ruby with Rails and Sinatra, ES6 with React and Angular, C# with .NET Core, Java with Spring, Python with Django, and plenty more stacks. We don’t limit ourselves as a company to particular technologies and we expect all our engineers to be keen on learning new skills. You will need to be comfortable sharing your knowledge and skills with others. We’d love to hear some examples of mentoring, coaching and growing team members. Maybe you will have written some blog posts about your discipline, or perhaps even delivered a talk or two. High performing software delivery teams need to be empowered to deliver changes all the way through to production. To do this we combine our extensive cloud automation knowledge with DevOps culture. We've been using AWS from the start and as Advanced Partners are go to experts within the public sector. We use a range of IaaS, PaaS and FaaS depending on the needs of our users, in this case software teams, such as EC2, Lambda, ECS, Kubernetes, Heroku, CloudFoundry, Azure App Services, and more. We use VPC and PrivateLink for connecting to on-premise, legacy systems. We also use API Gateway, S3, CloudFront, SQS, SNS, SES, RDS, and many other services provided by AWS. You will have the opportunity to use and learn these technologies and we offer official AWS and other cloud training. We ensure we document our architecture and infrastructure as code, using technologies such as Terraform and OpenAPI. Containerisation is a big part of empowering our teams to develop, deploy and scale their applications, but so too is using AWS Lambda and avoiding the complexity of stateful services altogether. Right tool for the job. For us, DevOps is about culture rather than roles and titles. Even though this role is for someone with strong DevOps experience, the biggest impact you will have is coaching and helping teams use the platforms you build. You won't be building infrastructure in isolation or charged with deploying other peoples work into production. You'll empower teams with the mantra: you build it, you run it!