With a big investment into Databricks, and with a large amount of interesting data, this is the chance for you to come and be part of an exciting transformation in the way we store, analyse and use data in a fast paced organisation. You will join as a Senior Platform Data Engineer providing technical leadership to the Data Engineering team. You will work closely with our Data Scientists and business stakeholders to ensure value is delivered through our solutions.

Job Accountabilities

• Develop robust, scalable data pipelines to serve the easyJet analyst and data science community.
• Highly competent hands-on experience with relevant Data Engineering technologies, such as Databricks, Spark, Spark API, Python, SQL Server, Scala.
• Work with data scientists, machine learning engineers and DevOps engineers to develop, develop and deploy machine learning models and algorithms aimed at addressing specific business challenges and opportunities.
• Coach and mentor the team (including contractors) to improve development standards.
• Work with Business Analysts to deliver against requirements and realise business benefits.
• Build a documentation library and data catalogue for developed code/products.
• Oversight of project deliverables and code quality going into each release.

Requirements of the Role

Key Skills Required

• Technical Ability: has a high level of current, technical competence in relevant technologies, and be able to independently learn new technologies and techniques as our stack changes.
• Clear communication; can communicate effectively in both written and verbal forms with technical and nontechnical audiences alike.
• Complex problem-solving ability; structured, organised, process-driven and outcome-oriented. Able to use historical experiences to help with future innovations.
• Passionate about data; enjoy being hands-on and learning about new technologies, particularly in the data field.
• Self-directed and independent; able to take general guidance and the overarching data strategy and identify practical steps to take.

Technical Skills Required

• Significant experience designing and building data solutions on a cloud based, big data distributed system.
• Hands-on software development experience with Python and experience with modern software development and release engineering practices (e.g. TDD, CI/CD), and software deployment automation with GitHub actions or Azure DevOps.
• Experience in testing automation of data transformation pipelines, using frameworks like Pytest or dbt Unit Test.
• Comfortable writing efficient SQL and debugging.
• Data warehouse operations and tunning experience in schema evolution, indexing, partitioning.
• Hands-on IaC development experience with Terraform or CloudFormation.
• Understanding of ML development workflow and knowledge of when and how to use dedicated hardware.
• Significant experience with Apache Spark or any other distributed data programming frameworks (e.g. Flink, Hadoop, Beam)
• Familiarity with Databricks as a data and AI platform or the Lakehouse Architecture.
• Experience with data quality and/or and data lineage frameworks like Great Expectations, dbt data quality, OpenLineage or Marquez, and data drift detection and alerting.
• Understanding of Data Management principles (security and data privacy) and how they can be applied to Data Engineering processes/solutions (e.g. access management, data privacy, handling of sensitive data (e.g. GDPR)

Desirable Skills

• Experience in event-driven architecture, ingesting data in real time in a commercial production environment with Spark Streaming, Kafka, DLT or Beam.
• Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options for processing unbounded data (pubsub, message queues, event streaming etc)
• Understanding of the most commonly used Data Science and Machine Learning models, libraries and frameworks.
• Knowledge of the development lifecycle of analytical solutions using visualisation tools (e.g. Tableau, PowerBI, ThoughtSpot)
• Hands-on development experience in an airline, e-commerce or retail industry.
• Worked within the AWS cloud ecosystem.
• Experience of building a data transformation framework with dbt.