Make yourself visible and let companies apply to you.
Roles
Data Engineer Jobs in London
Overview
Looking for top Data Engineer jobs in London? Explore the latest opportunities on Haystack, your go-to IT job board connecting skilled data engineers with leading companies in the heart of the UK’s tech hub. Whether you're an experienced data engineer or just starting your career, find roles that match your skills and ambitions in London’s vibrant tech scene. Start your job search today and take the next step in your data engineering career!
Senior Technical Data Solution Designer
Queen Square Recruitment Ltd
London
Hybrid
Senior
£400/day - £600/day
RECENTLY POSTED

Location: London Paddington / Reading Green Park (Hybrid – 2 days onsite)
Contract: 6 Months
Rate: £465–£550 per day (Inside IR35)

It Means More

This opportunity offers more than just another contract role:

* More ownership – Act as the senior technical authority within your data product squad, shaping end-to-end solution design.

* More impact – Build trusted business layers and dashboards that directly influence strategic business decisions.

* More technical depth – Work hands-on with modern tooling including dbt, SQL, GCP, Tableau and Atlan within a hybrid data mesh environment.

The Role

We are seeking a Senior Technical Data Solution Designer to join a large-scale enterprise data transformation programme within a complex, fast-paced environment.

This is a pivotal, hands-on leadership role — the peak of the technical career pathway within the squad model. It is not a people management role, but a senior individual contributor position responsible for the end-to-end technical success of data products.

You will bridge the gap between architecture and execution, translating business requirements into scalable, secure, and high-quality data solutions. From trusted Business Layer design and Data Contracts through to intuitive dashboards, you will ensure solutions align with hybrid data mesh principles and engineering best practices.

Key Responsibilities

* Translate complex business requirements into scalable, performant and secure technical solutions

* Own hands-on solution design including Business Layer modelling and Data Contract definition

* Ensure alignment with hybrid data mesh architecture principles

* Lead best practice in dashboard design, UX and data storytelling using Tableau

* Deliver intuitive, insight-driven data products that answer key business questions

* Act as the senior technical authority within the squad, mentoring BI Specialists and Analytics Engineers

* Lead by example through hands-on development using SQL, dbt and GCP

* Champion coding, testing and documentation standards through peer review

* Lead technical aspects of Agile ceremonies and own technical backlog prioritisation in Jira

* Promote data governance, metadata management, lineage and data quality practices using Atlan

Essential Skills & Experience

* Expert-level hands-on SQL and dbt

* Strong experience with Tableau and Google Cloud Platform (GCP)

* Deep understanding of dimensional modelling and modern data warehousing

* Experience with data integration and data mesh principles

* Strong data visualisation and data storytelling capability

* Proven experience leading technical delivery within Agile squads

* Excellent communication skills across technical and non-technical stakeholders

Desirable

* Strong solution design mindset

* Focus on scalability and maintainability

* Ability to balance business and technical trade-offs

If you are interested in discussing this opportunity confidentially, please apply or get in touch directly

Head of Data – London – Messy Data / Web Crawlers / Data Aggregation
Avanti Recruitment
London
Hybrid
Leader
£90,000 - £100,000
RECENTLY POSTED

High Growth B2B SaaS | AI & Machine Learning Platform
£90,000 – £100,000 + Significant Equity

I am working with a scaling London based technology business where Machine Learning and AI sit at the core of the product.

The platform is built around a large-scale dataset created by crawling the web and collecting millions of listings from multiple external sources. That raw data is then structured, matched and analysed to power a suite of SaaS tools used across the market.

One of the core technical challenges is that the same entity can appear multiple times across different sources, often with slightly different attributes or descriptions. The platform uses a combination of neural networks and rule based systems to match records and build a reliable dataset over time. They are now hiring a Head of Data to take ownership of their data and AI platform.

The Opportunity

This is a hands-on leadership role where you will lead a team of 5–6 across Data Science and Data Engineering while remaining technically involved in modelling, machine learning deployment and architectural decisions.

The platform runs on AWS and includes large-scale crawlers ingesting significant volumes of external data. However, the architecture has evolved over time and data currently sits across multiple siloed systems.

They need someone who understands what good looks like in a modern data platform and can bring structure, coherence and technical direction.

You will take ownership of improving the data architecture, strengthening the matching systems and defining a clear data and AI roadmap for the business.

Key Areas of Focus

• Designing and improving a scalable AWS based data platform
• Rationalising fragmented data across multiple systems
• Leading the full machine learning lifecycle from ingestion and feature engineering through to deployment
• Improving neural network and rule based matching systems used to identify the same entities across different sources
• Embedding robust MLOps and model monitoring
• Building a coherent data architecture that supports future AI development
• Leading and developing a team of Data Scientists and Data Engineers

The Background That Works Well

Relevant experience in environments that deal with messy data from multiple external sources might include:

• Web crawling or large-scale web data ingestion
• Marketplace or data aggregation platforms
• Entity resolution, record linkage or deduplication systems
• Matching systems or recommendation engines
• Geospatial or address based data platforms
• ML systems deployed into production

You do not need sector experience, but you should be comfortable working with complex datasets where the same entity appears across multiple sources.

What They Are Looking For

• Strong Data Science background with hands on machine learning experience
• Experience deploying ML models into production environments
• Understanding of data engineering concepts and modern cloud data platforms
• Architectural thinking around data platforms and ML systems
• Experience working with messy, multi-source datasets
• Leadership experience managing small, high impact teams
• Someone proactive who can define standards and shape the data roadmap

Package

Salary: £90,000 – £100,000
Equity: Meaningful equity with real upside (3 year vesting)
Location: London – Hybrid working, typically 2–3 days in the office with a pragmatic approach to flexibility

This is a high ownership role where you will be shaping the foundations of a data and AI platform inside a scaling technology business.

If you enjoy solving complex data problems, building scalable machine learning systems and leading a small but impactful team, I would be keen to speak

Data Engineer (Marketing Analytics)
US TECH SOLUTIONS LIMITED
London
Fully remote
Mid - Senior
£44/hour - £46/hour
RECENTLY POSTED

Role: Data Engineer (Marketing Analytics)

Location: UK- Remote

Duration: Until end of Dec 2026

About the role:

Join a fast-paced Marketing Analytics & Operations team as a Data Engineer. You’ll work with survey and marketing datasets to design scalable pipelines, dashboards, and reporting solutions that drive actionable insights for high-impact marketing programs.

Key Responsibilities:

  • Build and maintain ETL pipelines, data models, and dashboards for marketing analytics
  • Work closely with data scientists, engineers, and marketers to translate data needs into actionable insights
  • Solve complex data integration challenges from structured & unstructured sources
  • Ensure data security, quality, and compliance while working with large-scale datasets
  • Conceptualize scalable data architectures and evaluate cost/benefit tradeoffs

Required Qualifications

  • 5+ years in data engineering, data analysis, or data science
  • Strong SQL, ETL, Python skills
  • Expertise in data modeling, visualization, and data warehousing
  • Collaborative mindset, able to work with multi-disciplinary teams

Nice-to-Have:

  • Experience in Marketing Technology / Analytics
  • Background in large tech environments
  • Comfortable participating in team/client calls
Business Data Analyst
Robert Half
London
In office
Mid - Senior
Private salary
RECENTLY POSTED

Robert Half Technology are assisting a market leading financial services organisation to recruit a Business Data Analyst on a 6 month contract to perm basis. Onsite required - London based

This role sits at the intersection of data, operations, and business strategy, with a strong focus on improving middle office processes, enhancing reporting capabilities, and driving automation initiatives.

You will work closely with Operations, Finance, Technology, and Front Office stakeholders to analyse business processes, identify efficiency opportunities, and deliver scalable data-driven solutions.

Role

  • The Business Data Analyst will analyse complex datasets to support operational, financial, and regulatory reporting
  • Develop and maintain SQL queries and data models to support business intelligence and reporting needs
  • Use Python to automate workflows, improve data quality checks, and streamline recurring processes
  • Build and maintain advanced Excel models and reporting tools (including Power Query, Power Pivot, VBA where applicable)
  • Review and optimise middle office processes, identifying control gaps and automation opportunities
  • Partner with stakeholders to gather business requirements and translate them into technical solutions
  • Drive process automation initiatives to improve accuracy, reduce manual intervention, and increase scalability
  • Support reconciliation, valuation, and trade lifecycle analysis where required
  • Contribute to data governance, documentation, and control frameworks
  • Assist with system enhancements and implementation projects

Profile

  • The Business Data Analyst will have strong SQL skills (query writing, joins, performance optimisation, data manipulation)
  • Proficiency in Python for data analysis and automation (e.g., pandas, scripting, APIs)
  • Advanced Excel skills (complex formulas, pivot tables, data modelling)
  • Proven experience working in a Middle Office role within a financial services environment
  • Strong understanding of trade lifecycle, reconciliations, valuations, and operational controls
  • Experience reviewing and improving operational processes
  • Demonstrated ability to drive automation and efficiency initiatives
  • Strong analytical thinking and problem-solving skills
  • Excellent communication skills with the ability to work across technical and non-technical teams
  • Experience with Advent Geneva
  • Exposure to data visualisation tools (e.g., Power BI, Tableau)
  • Experience with financial instruments (equities, fixed income, derivatives, alternatives)

Company

  • Market leading financial services organisation with offices in London
  • Onsite required - London based

Salary & Benefits

The salary range/rates of pay is dependent upon your experience, qualifications or training.

Robert Half Ltd acts as an employment business for temporary positions and an employment agency for permanent positions. Robert Half is committed to diversity, equity and inclusion. Suitable candidates with equivalent qualifications and more or less experience can apply. Rates of pay and salary ranges are dependent upon your experience, qualifications and training. If you wish to apply, please read our Privacy Notice describing how we may process, disclose and store your personal data: gb/en/privacy-notice.

Oracle FDI Reporting Lead - SC Cleared
Lorien
London
In office
Senior
Private salary
RECENTLY POSTED

A major UK organisation is seeking an experienced Fusion Data Intelligence (FDI) Specialist to support and enhance their enterprise-wide Oracle Fusion data, reporting, and analytics landscape. This is a hands-on technical role with significant ownership, working at the heart of complex HCM, ERP, and EPM data flows.
You will play a key role in maturing the organisation’s FDI capability - driving high-quality data modelling, optimising performance, strengthening governance, and enabling robust, scalable reporting solutions.

Key Responsibilities

  • Leverage FDI subject areas and enhance existing models to meet evolving business needs.
  • Optimise data performance and query execution across FDI environments.
  • Ensure data integrity, consistency, and alignment with underlying source systems.
  • Work closely with report writers and business teams to maintain and enhance Oracle Data Visualization reporting solutions across Fusion HCM, ERP, and Oracle EPM Cloud datasets.
  • Support and evolve data governance, security, and RBAC models for reporting solutions.
  • Document data models, business logic, transformations, and metadata for reuse and governance compliance.
  • Troubleshoot and resolve issues related to data modelling and reporting within FDI.
  • Lead best-practice approaches for FDI deployment, security, optimisation, and performance tuning.
  • Collaborate with Technology teams and third-party partners to plan, coordinate, and deliver aligned work packages.
  • Champion continuous delivery and DevOps practices, driving automation and modern engineering standards.

Essential Requirements

  • 3+ years’ experience working with Oracle Fusion Data Intelligence (FDI).
  • Strong understanding of Fusion data structures, particularly HCM and ERP, with Oracle EPM Cloud knowledge beneficial.
  • Experience loading EPM data sets into FDI.
  • Expertise in Oracle Cloud Infrastructure (OCI), Oracle Integration Cloud (OIC), Oracle PaaS, and Fusion Middleware.
  • Hands-on experience building data pipelines, including integration from custom schemas (eg, EPM connectors).
  • Demonstrated ability to develop semantic models in FDI.
  • Proven experience designing and extending security models, security contexts, and RBAC.
  • Strong SQL & PL/SQL skills.
  • Practical experience with performance tuning in reporting and data-intensive environments.
  • Ability to work independently and own delivery of end-to-end solutions.
  • Experience integrating with Master Data Management solutions (eg, EDM).
  • Ability to clearly articulate technical and functional changes to both technical and non-technical audiences.
  • Proven ability to lead technical and/or functional teams in an Agile environment.
  • Strong analytical, troubleshooting, and problem-solving capabilities.
  • Excellent communication, stakeholder management, organisation, and self-motivation.
  • Ability to manage and prioritise a demanding workload.

Desirable Skills

  • Oracle Cloud certifications (eg, Oracle Analytics Cloud Professional, OCI Foundations, Autonomous Data Warehouse Foundations).
  • Background in IT consulting, support, ITIL processes, SDLC, and understanding of server/database environments.
  • Hands-on experience with FDI implementation or support, including knowledge of FDI data structures.
  • Experience building advanced BIP and OTBI reports.
  • Experience with REST/SOAP Web Services.
  • Experience supporting FDI in public sector environments.

Guidant, Carbon60, Lorien & SRG - The Impellam Group Portfolio are acting as an Employment Business in relation to this vacancy.

Senior Data Engineer Big Data/ Hadoop/ Spark
Hays Specialist Recruitment Limited
London
Remote or hybrid
Senior
£700/day - £850/day
RECENTLY POSTED
+5

Your new companyWorking for a renowned financial services organisation Your new roleWe’re looking for a Senior Data Engineer to design and deliver scalable on prem, high-quality data solutions for low/ high-level data platforms that power analytical and business insights. This is a hands-on role suited to someone with strong data engineering and big data expertise, ideally gained within financial services. Joining leading commodities, metals, trading, and exchange group, you will support a strategic metals initiative focused on reducing on-prem platform costs and modernising legacy ETL processes. You’ll help design and build a new on-prem data platform aligned to the metals strategy while developing and maintaining scalable data pipelines and analytics infrastructure. Using Hadoop, Big Data, and Spark technologies, you will ensure data quality through automated validation, monitoring, and testing. You will also enable seamless integration across data warehouses and data lakes, contributing to a robust, scalable, and resilient enterprise data ecosystem.What you’ll need to succeed

  • Vast Data Engineering expertise with Big Data technologies.
  • Experience designing and building on-prem data platforms, from high-level architecture to detailed technical design.
  • Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning.
  • Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools.
  • Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such as MongoDB.
  • Proven delivery of streaming pipelines and real-time data processing solutions.
  • Improved job efficiency and reduced runtimes through Apache Spark optimisation and development.
  • Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines.
  • Delivered streaming pipelines and real-time data processing solutions.
  • Experience replacing legacy ETL tools (e.g., Informatica) with modern data engineering pipelines and platform builds.
  • Proven background working within financial services environments.

What you’ll get in returnFlexible working options available.What you need to do nowIf you’re interested in this role, click ‘apply now’ to forward an up-to-date copy of your CV, or call us now.

Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept the T&C’s, Privacy Policy and Disclaimers which can be found at hays.co.uk

Cloud Consultant
83zero Ltd
London
Hybrid
Junior - Mid
£50,000 - £70,000
RECENTLY POSTED

Google Cloud Consultant (Cloud/Data/AI)

Location: London (Hybrid)

Salary: £50,000 - £70,000 + benefits

Type: Permanent

83zero are partnered with a leading technology consultancy that helps organisations modernise their digital platforms and unlock value through cloud, data, and AI technologies.

Due to continued growth, they are looking for a Google Cloud Consultant to join their expanding cloud practice. This is an excellent opportunity for someone with hands-on GCP experience who is looking to develop their consulting career while working on large-scale cloud and data transformation projects.

The Role

As a Google Cloud Consultant, you will work closely with clients and internal teams to design and deliver solutions on Google Cloud Platform (GCP). You will contribute to cloud platform implementations, data solutions, and emerging AI capabilities while supporting senior consultants and architects on customer engagements.

This role offers the opportunity to gain exposure to a variety of cloud transformation projects while developing both your technical and consulting skill set.

Key Responsibilities

  • Support the design and implementation of cloud solutions on Google Cloud Platform (GCP)
  • Work with clients to understand technical requirements and contribute to solution design
  • Help deliver cloud migration, platform modernisation, and cloud-native solutions
  • Assist with the design and development of data platforms and analytics solutions
  • Contribute to the implementation of AI or machine learning solutions where appropriate
  • Develop and maintain infrastructure using Infrastructure-as-Code tools such as Terraform
  • Support the creation of technical documentation including solution designs and implementation plans
  • Collaborate with architects, engineers, and client stakeholders to ensure successful project delivery
  • Follow best practices around security, governance, and cost optimisation (FinOps)

Skills & Experience

Essential

  • Experience working with Google Cloud Platform (GCP) in a professional environment
  • Exposure to GCP services such as Compute Engine, Cloud Run, GKE, BigQuery, Cloud Storage, Pub/Sub, or Vertex AI
  • Experience supporting the implementation of cloud infrastructure or data platforms
  • Understanding of Infrastructure-as-Code, ideally using Terraform
  • Strong problem-solving skills and the ability to work collaboratively within project teams
  • Good communication skills and an interest in developing a client-facing consulting career

Desirable

  • Experience with data engineering, analytics platforms, or machine learning technologies
  • Exposure to cloud migration or modernisation programmes
  • Knowledge of security or compliance practices within cloud environments
  • Relevant certifications such as Google Associate Cloud Engineer or Professional Cloud Architect

What They’re Looking For

The ideal candidate is someone who enjoys working with modern cloud technologies and wants to build a long-term career in cloud consulting. You’ll be curious, technically capable, and comfortable working in collaborative project environments.

This is a great opportunity to join a fast-growing consultancy where you will work on cutting-edge cloud, data, and AI projects while continuing to build your technical expertise.

SC & DV Cleared Data Consultants and Analysts
VIQU IT Recruitment
London
Hybrid
Mid - Senior
£60,000
RECENTLY POSTED

SC & DV Cleared Data Consultants and Analysts – London – Up to £60,000 per annum

Are you a data-driven professional with a passion for delivering impactful solutions? Do you excel in translating complex data challenges into actionable insights? If so, Morela has an exciting opportunity for you.

Morela is proud to partner with one of the UK’s leading data consultancies, renowned for their innovative approach to data integration and analytics. As they continue to expand, they’re seeking a skilled SC & DV Cleared Data Consultant and Analysts to join their dynamic team. This role is perfect for someone who combines technical expertise with a consultative mindset, aiming to drive value across diverse projects.

You’ll be part of a forward-thinking consultancy that operates at the intersection of advanced technology and real-world impact. Serving both public and private sectors, their services encompass strategic advisory, digital transformation, and end-to-end data engineering. As a SC & DV Cleared Data Consultant or Analyst, you’ll play a crucial role in designing and implementing data solutions that empower clients to make informed decisions.

Core Responsibilities

  • Data Integration: Connect and harmonise data from multiple sources, ensuring clean, reliable, and compliant data pipelines.
  • Data Modelling: Design and manage data models to create logical data structures that enhance accessibility and usability.
  • Application Development: Build full-stack applications using tools such as Python, SQL, and relevant frameworks, delivering intuitive user interfaces and interactive workflows.
  • Data Governance: Implement data lineage tracking and access controls to uphold compliance with organisational and regulatory standards.
  • Client Collaboration: Engage with clients to understand their needs, provide training sessions, and promote best practices for effective adoption and utilisation of data platforms.
  • Continuous Improvement: Contribute to internal capability building by improving frameworks, delivery processes, and team development across the consultancy.

What We’re Looking For

  • SC, DV or eDV Cleared: In this position, you will already be security cleared to one of those levels.
  • Technical Expertise: Proven experience in data engineering, software development, or systems integration, preferably within AI, analytics, or enterprise platforms.
  • Problem-Solving Skills: Strong analytical abilities with a knack for troubleshooting and optimising complex data workflows.
  • Communication Skills: Excellent ability to engage with both technical and non-technical stakeholders, providing clear and confident guidance.
  • Adaptability: Comfortable working across various sectors and with diverse clients, adjusting approaches as needed.
  • Travel Readiness: Open to occasional travel (up to 25%) depending on project requirements.
  • Palantir Foundry experience is an added bonus!

As a SC or DV Cleared Data Consultant or Analyst, you’ll lead the charge on some of the most complex, high-impact data programmes in the UK, helping clients unlock the power of their data while mentoring the next generation of engineering talent.

Please do not hesitate to reply and reach out to Morela today to find out more!

Hedge Fund - Python Developer (Equities) - Trade life cycle - PnL - Kafka - Contract
Scope AT Limited
London
In office
Senior
Private salary
RECENTLY POSTED
+1

Our Hedge Fund client is looking for a Python Developer/Engineer Contract role

This team is responsible for the firms equity transaction data platform, including trade life cycle event processing, enrichment, and PnL calculations. The role is ideal for an engineer who enjoys building robust, high-throughput services and data pipelines in a fast-paced, delivery-focused environment.

Principal Responsibilities
Design and develop solutions for trade life cycle event processing, including corporate actions, expiries, and other post-trade events.
Build and operate Python-based services that perform large-scale data transformations and calculations.
Publish and distribute transaction and PnL data using Kafka, including AVRO-based schemas and streaming patterns.

Required Skills
Minimum of 6+ years of professional Python development experience, ideally in capital markets or a fintech firm.
Experience in finance: understanding of common financial asset classes; knowledge of equities corporate action processing, trade life cycle concepts, and/or P&L calculations is a strong plus.
Experience with Kafka (or equivalent streaming/messaging platforms) and schema-based event publishing (eg, AVRO).
Strong experience performing large-scale data calculations in Python using libraries like pandas, polars, and NumPy.
Experience building REST services using frameworks such as FastAPI and/or Flask.
Strong SQL skills and experience working with relational databases in production environments.
Hands-on experience with containerized deployments and modern infrastructure tooling (Docker, Kubernetes) and familiarity with cloud platforms.
Understanding of modern SDLC practices (testing strategy, CI/CD, release management, observability, and operational ownership).

Office based, 5 days per week.
Based in London.

Contract role inside IR35

By applying to this job you are sending us your CV, which may contain personal information. Please refer to our Privacy Notice to understand how we process this information. In short, in order to supply you with work finding services, we will hold and process your personal data, and only with your express permission we will share this personal data with a client (or a third party working on behalf of the client) by email or by upload to the Client/third parties vendor management system. By giving us permission to send your CV to a client, this constitutes permission to share the personal data that would be necessary to consider your application, interview you (Phone/video/face to face) and if successful hire you.
Scope AT acts as an employment agency for Permanent Recruitment and an employment business for the supply of temporary workers. By applying for this job you accept the Terms and Conditions, Data Protection Policy, Privacy Notice and Disclaimers which can be found at our website.

Group Head of Data - Enterprise Data Strategy - Microsoft Fabric - Permanent - London
Station
London
In office
Leader
£90,000 - £110,000
RECENTLY POSTED

Group Head of Data | Enterprise Data Strategy | Microsoft Fabric | Executive Leadership

Our London based client is seeking a Group Head of Data to define and deliver its enterprise-wide data strategy, modernise its platform capability, and lead a major transition to a Microsoft-first data architecture (Azure Synapse to Microsoft Fabric).

This is a high-impact leadership role, pivotal to shaping a advanced, data-driven organisation and unlocking measurable commercial value through trusted, modern data products.

As Group Head of Data the successful candidate will own the strategic direction of the enterprise data platform, build and scale a high performing data engineering function, and drive the evolution toward a modern, governed, self-service analytics ecosystem.

The Group Head of Data will operate as the senior authority across data strategy, architecture, governance, and platform engineering partnering closely with executive stakeholders to embed data at the heart of decision-making.

Key Responsibilities
Data Strategy & Leadership

  • Define and run a long-term enterprise data strategy.
  • Champion data culture, literacy, and ownership across the organisation.
  • Act as the executive lead for data architecture and engineering capability.

Platform Modernisation & Engineering

  • Own the enterprise data platform roadmap and architecture.
  • Lead a migration to Microsoft Fabric, driving value and minimising disruption.
  • Build and lead high-performing engineering teams across ingestion, transformation, modelling, integration, and operations.
  • Ensure the platform supports both near-Real Time and batch processing, governed self-service analytics, AI/ML workloads, and enterprise reporting.
  • Embed modern DataOps practices including CI/CD, automation, testing, orchestration, and monitoring.
  • Ensure performance, reliability, observability, and SLA adherence.

Governance, Quality & Trust

  • Implement enterprise data governance frameworks (ownership, stewardship, life cycle).
  • Embed Microsoft Purview for governance, cataloguing, and quality management.
  • Ensure compliance with regulatory and privacy standards.
  • Drive measurable improvements in data quality, transparency, and lineage.

Business Partnering & Enablement

  • Translate business strategy into scalable data initiatives.
  • Enable governed self-service analytics.
  • Support AI/ML and advanced analytics capability.
  • Guide business units on effective use of Microsoft analytics tooling.
  • Ensure data platform effectively supports analytics and AI initiatives across the organisation.

Operational & Commercial Excellence

  • Define operational standards across security, resilience, and optimisation.
  • Drive cost efficiency across the Microsoft data estate.
  • Manage third-party suppliers and delivery partners.

Required Skills

  • Deep experience across Microsoft’s modern data stack including Fabric, Synapse, Power BI, Azure Data Factory, Databricks, Azure SQL/Storage, and Purview.
  • Demonstrable track record of delivering enterprise-scale platform migrations - ideally Synapse to Fabric.
  • Strong expertise in modern data architecture, DataOps, automation, and governance.
  • Deep understanding of data security, privacy, and compliance frameworks.
  • Significant experience in senior data leadership roles.
  • Demonstrated success building and leading high-performing technical teams (including offshore models).
  • Experience delivering enterprise data transformation programmes.
  • Strong executive stakeholder engagement and influencing capability.
  • Experience enabling self-service analytics and data product thinking.

This is a great opportunity for a strategic data leader who combines architectural depth with commercial pragmatism. If you are excited by the opportunity to shape a modern, Microsoft-centric data ecosystem please share your CV and we will contact you at our earliest possible convenience.

Principal Data Engineer
VIQU IT Recruitment
London
Fully remote
Senior
£95,000
RECENTLY POSTED

Location: Remote
Salary: Up to £95,000
Sponsorship: Not available

VIQU have partnered with a nationally recognised organisation that has recently completed a full migration from on-premise infrastructure to Google Cloud Platform (GCP) and is now building a modern, domain-oriented Data Mesh. This is a critical leadership hire as they continue to scale their cloud data platform and embed engineering excellence across multiple product domains.

This is not a step-up role. They will only consider candidates with proven Principal or Lead-level experience, including clear ownership of engineers, standards, and cross-domain architectural direction.

The Role

As Principal Data Engineer, you will be the senior technical authority across data product squads, driving strategy, architecture, and engineering best practice.

Key responsibilities include:

  • Providing visible, hands-on technical leadership across multiple domain teams
  • Defining how projects are approached and ensuring delivery standards are upheld
  • Designing and governing scalable pipelines using BigQuery, Dataflow, Datastream, Data Fusion, Dataproc and Cloud Composer (Airflow)
  • Leading best practice across dbt, Terraform, Docker and CI/CD
  • Driving Data Mesh adoption, domain ownership, and interoperable data products
  • Setting modelling standards across raw, curated and semantic layers
  • Embedding governance covering lineage, MDM, data quality, PII and compliance
  • Mentoring and challenging senior engineers to maintain high standards

You will work closely with Architects and Platform Engineering to ensure performance, cost control, security and scalability across the GCP estate.

What They Need

  • 10+ years in Data Engineering
  • Clear, demonstrable experience leading engineers at Lead/Principal level
  • Deep, hands-on GCP experience (this is essential)
  • Strong SQL and Python expertise
  • Experience building cloud-native data platforms end-to-end
  • Strong background in Data Lake / Warehouse architecture
  • Proven experience implementing CI/CD and infrastructure as code
  • Confidence to challenge stakeholders and influence technical direction

Candidates without strong, proven leadership experience in enterprise-scale environments will not be considered.

Why Join?

You’ll be joining at a pivotal stage in the evolution of a 100% cloud-native platform. The foundations are in place, but the Data Mesh journey is still being shaped. This is a genuine opportunity to influence architecture, standards, and the long-term engineering strategy — fully remote, with executive backing and real scope for impact.

Apply now to speak with VIQU IT in confidence. Or contact Aaron Chiverton on the VIQU website. Know someone great? Refer them and receive up to £1,000 if successful (terms apply). For more exciting roles and opportunities, follow us on LinkedIn @VIQU IT Recruitment.

Senior Data Engineer, Vice President
State Street
London
Remote or hybrid
Senior
Private salary
RECENTLY POSTED

This job is with State Street, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly.

Job Description

We are looking for a Senior hands-on Data Engineer for Cloud Data Lake design and development. The candidate should have Financial Services experience in design, architecture and development of large scale enterprise applications using Cloud based Data engineering platforms. This hands-on role will partner with senior-level development managers, architects and business leadership to develop and execute the technology product roadmap.

Main Duties and Responsibilities:

• End to end Cloud Data Lake design & development including data ingestion, data modeling and data distribution.

• Build data integrations, hand-offs between on-prem/cloud-based systems.

• Design and develop high volume data ingestion components and solutions to collect data from various sources.

• Manage and use appropriate tools in the cloud data lake development and deployment.

• Responsible for developing/implementing cloud infrastructure to support current and future business needs.

• Participating in technical design, application build, configuration, unit testing, and production deployment.

• Ensuring all cloud solutions exhibit higher level of cost efficiency, performance, security, scalability, and reliability

Required Skills:

• Development experience in designing, developing and deploying complex applications.

• Strong tech skillset/experience in PL/SQL, Java/J2EE, Python, Scala, PERL/scripting .

• Hands on experience with Cloud platforms, preferably using Databricks.

• Experience in building systems using modern, scalable, resilient, cloud native architectures.

• Experience in Data Modeling and database design with strong skills in performance tuning.

• Hands on experience in setting up data pipelines for ELT jobs across multiple data sources.

• Hands on experience in big-data processing, troubleshooting and resolving database performance issues.

• Strong knowledge of Cloud Security in AWS/Azure, specifically IAM Roles & Polices, Security Groups and Encryption keys.

• Experience with optimizing database performance, scalability, ensuring data security and compliance.

• Experience with event based, micro-batch and batched high volume and high velocity transaction and data processing system.

• Proficiency with CICD and DevOps.

Education & Qualifications

Bachelor’s/Master’s Degree in a Computer Science, Engineering or Math or equivalent experience

10+ years’ experience in professional software design and development.

Strong analytical skills, with ability to collect, absorb, and analyze information quickly and independently

Proven ability to develop and work with cross functional processes and project timelines

Excellent follow-through, attention to detail and time management skills

About State Street Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement

]]>

Data Engineer - Data Vault
Randstad Digital
London
Hybrid
Senior - Leader
£60/hour - £80/hour
RECENTLY POSTED

Senior Data Engineer (Data Modernization) We are looking for a proactive, goal-oriented Senior Data Engineer to build and scale high-performance big data pipelines. This role is ideal for a collaborative problem-solver who excels in distributed environments and values technical excellence through CI/CD and Agile. The Essentials Pipeline Tech: 5+ years of experience with Kafka, Spark, Hadoop, and DBT . Modern Modeling: Expert skills in Data Vault Mandatory and Dimensional modeling. Cloud & DevOps: Strong AWS experience with a focus on CI/CD and writing secure code. Data Governance: Proven ability to embed quality, lineage, and monitoring into every pipeline. Leadership: A passion for guiding and coaching fellow team members both technically and procedurally. The Mission You will lead data modernization efforts, ensuring large-scale systems remain compliant and secure while maintaining a ‘right tool for the job’ mindset. You’ll bridge the gap between complex data engineering and actionable analytics/ML concepts. Apply If you have a passion for building well-governed, scalable systems and want to lead a high-impact team, please apply with your CV Randstad Technologies is acting as an Employment Business in relation to this vacancy. TPBN1_UKTJ

SC & DV Cleared Senior Data Consultants and Analysts
VIQU IT Recruitment
London
Hybrid
Senior
£70,000
RECENTLY POSTED

SC & DV Cleared Senior Data Consultants and Analysts – London – Up to £70,000 per annum

Are you a data-driven professional with a passion for delivering impactful solutions? Do you excel in translating complex data challenges into actionable insights? If so, Morela has an exciting opportunity for you.

Morela is proud to partner with one of the UK’s leading data consultancies, renowned for their innovative approach to data integration and analytics. As they continue to expand, they’re seeking a skilled SC & DV Cleared Senior Data Consultant and Analysts to join their dynamic team. This role is perfect for someone who combines technical expertise with a consultative mindset, aiming to drive value across diverse projects.

You’ll be part of a forward-thinking consultancy that operates at the intersection of advanced technology and real-world impact. Serving both public and private sectors, their services encompass strategic advisory, digital transformation, and end-to-end data engineering. As a SC & DV Senior Cleared Data Consultant or Analyst, you’ll play a crucial role in designing and implementing data solutions that empower clients to make informed decisions.

Core Responsibilities

  • Data Integration: Connect and harmonise data from multiple sources, ensuring clean, reliable, and compliant data pipelines.
  • Data Modelling: Design and manage data models to create logical data structures that enhance accessibility and usability.
  • Application Development: Build full-stack applications using tools such as Python, SQL, and relevant frameworks, delivering intuitive user interfaces and interactive workflows.
  • Data Governance: Implement data lineage tracking and access controls to uphold compliance with organisational and regulatory standards.
  • Client Collaboration: Engage with clients to understand their needs, provide training sessions, and promote best practices for effective adoption and utilisation of data platforms.
  • Continuous Improvement: Contribute to internal capability building by improving frameworks, delivery processes, and team development across the consultancy.

What We’re Looking For

  • SC, DV or eDV Cleared: In this position, you will already be security cleared to one of those levels.
  • Technical Expertise: Proven experience in data engineering, software development, or systems integration, preferably within AI, analytics, or enterprise platforms.
  • Problem-Solving Skills: Strong analytical abilities with a knack for troubleshooting and optimising complex data workflows.
  • Communication Skills: Excellent ability to engage with both technical and non-technical stakeholders, providing clear and confident guidance.
  • Adaptability: Comfortable working across various sectors and with diverse clients, adjusting approaches as needed.
  • Travel Readiness: Open to occasional travel (up to 25%) depending on project requirements.
  • Palantir Foundry experience is an added bonus!

As a SC or DV Cleared Senior Data Consultant or Analyst, you’ll lead the charge on some of the most complex, high-impact data programmes in the UK, helping clients unlock the power of their data while mentoring the next generation of engineering talent.

Please do not hesitate to reply and reach out to Morela today to find out more!

Technical Product Manager with AI
Harvey Nash
London
Hybrid
Mid - Senior
Private salary
RECENTLY POSTED

Description:

The client is looking to join Growth Marketing Technology team to help shape the future of marketing through data-driven innovation and AI-powered solutions. As Product Manager, you’ll play a pivotal role in developing scalable marketing technologies that enhance automation, measurement, and personalization across global marketing campaigns on platforms such as Google, Meta and Apple.

Job Title: Technical Product Manager (AI)

Location: London

Mode of working: 3 Days Hybrid Working

Duration: 12+ Months contract.

Responsibilities:

  • Drive product development across data, automation, and AI/ML capabilities to support marketing scalability and efficiency.
  • Collaborate with engineering, data science and analytics teams to deliver robust measurement and optimization tools.
  • Lead initiatives to integrate generative AI into marketing workflows, including content generation and ad creative optimization.
  • Conduct market and trend analysis to inform product strategy and identify opportunities for innovation.
  • Build deep understanding of data infrastructure and able to source data dictionaries, database schema for wider use

Skills:

  • 5+ years of product management experience, ideally in marketing platforms, data products and AI/ML applications.
  • Strong technical acumen with experience in data engineering, workflow orchestration, and machine learning operations.
  • Excellent communication skills with the ability to influence cross-functional teams and stakeholders.
  • Well-versed in design thinking principles
  • Great problem solving and analytical skills

Preferred:

  • Experience with Python, SQL, Java, HQL, Hive, Pyspark, or similar programming languages preferred
Quantitative developer
Anson McCade
London
In office
Mid - Senior
£170,000 - £260,000
RECENTLY POSTED
TECH-AGNOSTIC ROLE

€170,000 - 260,000 EUR
Onsite WORKING
Location: Central London, Greater London - United Kingdom Type: Permanent

About the Company

I am working with a start-up focused on innovation and excellence in the financial technology sector look to hire Quantitative developers for thier Milan office

About the Role

Build, maintain, and develop robust and optimized production-grade code to manage and execute orders across multiple MFT strategies and multiple exchanges in equities, crypto, and futures. Work with quant and machine learning researchers to continuously refine and improve data pipelines and execution algorithms. Provide an extra pair of hands for various other SE/QD workloads.

Responsibilities for a Quantitative developer

  • Build, maintain, and develop robust and optimized production-grade code.
  • Manage and execute orders across multiple MFT strategies and exchanges.
  • Collaborate with quant and machine learning researchers.
  • Refine and improve data pipelines and execution algorithms.
  • Support various SE/QD workloads.

Qualifications

  • Experience developing live execution systems across different products and platforms at top tier quant firms.

Required Skills

  • Strong programming skills in relevant languages.
  • Experience with MFT strategies and exchanges.
  • Knowledge of data pipelines and execution algorithms.

Preferred Skills

  • Familiarity with equities, crypto, and futures markets.

Pay range and compensation package

200-300k (depending on experience) + bonus + internal investment

If you are interested in the Quantitative developer role - then apply to this job ad

Reference: AMC/THO/QD002

Postcode: -

#thho

Junior AI Data Engineer
hireful
London
Hybrid
Junior
£35,000 - £45,000
RECENTLY POSTED

Take the next step in your data career & work with cutting-edge AI driven data migration projects!

Small, but mighty data transformation company, operating in the financial services & banking arena are looking for a Data Analyst / Data Engineer to work on a freshly built AI data migration & pipeline application platform.

A most excellent opportunity for an ambitious junior to move on in their career & work with agentic AI at the forefront of data manipulation.

Job title: Data Engineer / Data Analyst.

Location: We have a London office (Liverpool Street) but operate remotely across the UK. However, for this role, we d like someone commutable to London (London or home counties, maybe?) V flexible on office / home-based work though.

Salary: £40K - £45K + Bens

What you ll be doing:

  • Working with our newly built, in-house agentic AI data migration, transformation & schema building platform!

  • SQL / Python scripting: Prompt > test > validate > deliver.

  • Working closely with our Senior Data Engineer & Data Project delivery manager on client projects

  • Learning a great deal about complex financial data sets, migration, schemas & AI data platforms!

What you ll need to bring:

  • SQL & Python coding knowledge. Working with complex financial data (migration, ETL, Schemas, etc) would be great, but any sphere is totally fine.

  • Data related, STEM, Computer science, etc type degree, or equivalent experience.

  • Inquisitive, questioning nature with oodles of critical thinking!

That s it.

We d love to hear form you, so please do hit the apply now button, or get in touch to be considered.

Junior AI Data Engineer
hireful.
London
Hybrid
Junior
£35,000 - £45,000
RECENTLY POSTED

Take the next step in your data career & work with cutting-edge AI driven data migration projects!

Small, but mighty data transformation company, operating in the financial services & banking arena are looking for a Data Analyst / Data Engineer to work on a freshly built AI data migration & pipeline application platform.

A most excellent opportunity for an ambitious “junior” to move on in their career & work with agentic AI at the forefront of data manipulation.

Job title: Data Engineer / Data Analyst.

Location: We have a London office (Liverpool Street) but operate remotely across the UK. However, for this role, we’d like someone “commutable” to London (London or home counties, maybe?) V flexible on office / home-based work though.

Salary: £40K - £45K + Bens

What you’ll be doing:

  • Working with our newly built, in-house agentic AI data migration, transformation & schema building platform!

  • SQL / Python scripting: Prompt > test > validate > deliver.

  • Working closely with our Senior Data Engineer & Data Project delivery manager on client projects

  • Learning a great deal about complex financial data sets, migration, schemas & AI data platforms!

What you’ll need to bring:

  • SQL & Python coding knowledge. Working with complex financial data (migration, ETL, Schemas, etc) would be great, but any sphere is totally fine.

  • Data related, STEM, Computer science, etc type degree, or equivalent experience.

  • Inquisitive, questioning nature with oodles of critical thinking!

That’s it.

We’d love to hear form you, so please do hit the apply now button, or get in touch to be considered.

Data Architect- Insurance
McGregor Boyall
London
Remote or hybrid
Senior - Leader
£95,000 - £115,000
RECENTLY POSTED

Data Architect, Data Lake, AWS, Azure, Insurance, Cloud

We are seeking an experienced and forward-thinking Data Architect to lead and optimise our data architecture initiatives, with a strong focus on the insurance sector. You will be responsible for designing, implementing, and governing scalable, high-performance data solutions that support both internal objectives and client-facing initiatives.

  • Build modern data infrastructure using technologies such as Databricks, Snowflake, Data Lakes, Data Warehouses, and Lakehouse architectures.
  • Promote and implement data mesh principles to enhance data accessibility across the global organisation.
  • Utilise deep expertise in SQL and database management to optimise performance and scalability.
  • Apply industry knowledge to support insurance-specific data needs, including operational efficiency and regulatory compliance.
  • Possess expert-level knowledge in AWS or Azure cloud technologies
  • Extensive experience as a Data Architect within large, complex organisations-ideally with deep specialisation in the insurance industry.
  • A strong track record of designing and implementing successful enterprise-level data strategies.
  • Strong cloud experience in wither AWS or Azure.
  • Expertise across SQL, data solutions, and solution architecture for data lakes, data warehouses, and modern data frameworks.
  • Exceptional leadership, communication, and stakeholder engagement skills.
  • Solid understanding of the insurance domain, with experience in the wealth sector or exposure to technologies such as Aladdin or Guidewire considered advantageous.
  • A Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.

This opening can be based from multiple locations

Data Architect, Data Lake, AWS, Azure, Insurance, Cloud

McGregor Boyall is an equal opportunity employer and do not discriminate on any grounds.

Palantir Foundry SME/Lead Data Engineer
Synergetic
London
In office
Senior
Private salary
RECENTLY POSTED
+6

Location: London with occasional international travel

Contract - 6-12 months

Inside IR35

Rate: Negotiable DOE

Our client is looking for a Palantir Foundry SME, to take ownership of the platform architecture, data modelling standards, pipeline development and integration strategy. You will work closely with stakeholders across business, data, and engineering teams to translate operational problems into scalable Foundry solutions.

This is a hands-on engineering role combined with some architectural leadership.

Key Responsibilities

  • Design and implement scalable data pipelines within Palantir Foundry
  • Develop and maintain Ontology objects, operational workflows and applications within Foundry
  • Build and optimise data transformations using PySpark, SQL and Python
  • Integrate Foundry with cloud platforms such as Amazon Web Services, Microsoft Azure, or Google Cloud Platform
  • Implement CI/CD workflows using tools such as Jenkins, GitLab, or GitHub Actions
  • Work with containerised deployments using Docker and orchestration via Kubernetes
  • Implement governance, data lineage, and security best practices
  • Optimise performance, cost and reliability of Foundry pipelines
  • Build operational dashboards and data applications for business users
  • Mentor internal teams and establish platform standards and best practices
  • Act as the senior technical interface with Palantir

Essential Experience

  • Proven commercial experience implementing and scaling Palantir Foundry
  • Strong experience in Python, SQL and Spark (PySpark preferred)
  • Experience building data models and ontologies
  • Strong understanding of ETL/ELT design patterns
  • Cloud-native architecture experience (AWS/Azure/GCP)
  • Experience with REST APIs and system integrations
  • Familiarity with DevOps and Infrastructure-as-Code principles (eg Terraform)
  • Strong understanding of data governance and security controls

Desirable Skills

  • Experience with big data tooling (eg Apache Spark, Kafka)
  • Experience integrating Foundry with ERP/CRM platforms
  • Knowledge of data science and ML workflows
  • Experience in regulated or security-sensitive environments
  • Exposure to operational decision-support systems
Senior Data Platform Engineer
Haystack - Partnerships
London
Hybrid
Senior
Private salary
RECENTLY POSTED

Permanent | Hybrid Working | South East England

We are partnering with a fast-paced, customer-focused organisation undertaking a significant data transformation programme. With major investment in a modern lakehouse platform and access to rich, high-volume datasets, this is an opportunity to help reshape how data is stored, analysed and leveraged across the business.

As a Senior Platform Data Engineer, you will provide technical leadership within the Data Engineering team, working closely with Data Scientists, Machine Learning Engineers and business stakeholders to deliver scalable, high-value data solutions.

Key Responsibilities

  • Design and build robust, scalable data pipelines to serve analytics and data science communities
  • Provide hands-on expertise across modern data engineering technologies (e.g. Databricks, Spark, Python, SQL, Scala)
  • Collaborate with data scientists and ML engineers to develop and deploy machine learning models addressing key business challenges
  • Coach and mentor engineers (including contractors), raising development standards and engineering maturity
  • Partner with Business Analysts to translate requirements into measurable business outcomes
  • Oversee code quality and project deliverables across releases
  • Establish and maintain documentation libraries and data catalogues for developed products
  • Contribute to platform evolution and continuous improvement of data engineering practices

Core Skills & Experience

Technical Capability

  • Significant experience designing and delivering data solutions on cloud-based, distributed big data platforms
  • Strong hands-on software engineering experience in Python, with knowledge of modern development practices (TDD, CI/CD, automated deployment pipelines)
  • Experience with automated testing of data transformation pipelines (e.g. Pytest, dbt unit testing frameworks)
  • Strong SQL skills, including performance tuning and debugging
  • Experience in data warehouse optimisation (schema evolution, indexing, partitioning)
  • Infrastructure as Code experience (e.g. Terraform or CloudFormation)
  • Strong knowledge of distributed data processing frameworks such as Apache Spark (or similar technologies such as Flink, Hadoop or Beam)
  • Familiarity with lakehouse architecture principles and modern data platforms
  • Experience implementing data quality, lineage and observability frameworks
  • Understanding of data governance, security and privacy principles, including handling of sensitive data within regulated environments

Engineering & Collaboration

  • Comfortable working independently while aligning with overarching data strategy
  • Strong communication skills across technical and non-technical audiences
  • Structured, analytical problem-solver with a focus on outcomes and innovation
  • Passionate about modern data technologies and continuous learning

Desirable Experience

  • Experience building real-time or event-driven data pipelines in commercial production environments (e.g. streaming architectures, Kafka, Spark Streaming, Beam)
  • Understanding of unbounded data processing and streaming design patterns
  • Familiarity with machine learning workflows and model lifecycle considerations
  • Experience supporting analytical visualisation tools (e.g. Tableau, Power BI or similar platforms)
  • Experience working within airline, ecommerce, retail or similarly data-rich industries
  • Experience within AWS cloud environments
  • Experience building transformation frameworks using dbt
Page 1 of 7
Frequently asked questions
Haystack features a wide range of Data Engineer roles in London, including positions in startups, established tech companies, financial institutions, and more, covering junior to senior levels.
Most London-based Data Engineer jobs require valid work authorization or visa sponsorship. Job listings typically specify these requirements to help you apply accordingly.
Yes, many employers offer remote or hybrid working options for Data Engineers in London. You can filter job listings based on work location preferences on Haystack.
Key skills include proficiency in SQL, Python, ETL tools, cloud platforms like AWS or Azure, and experience with big data technologies such as Hadoop or Spark.
You can browse listings, create a profile, upload your CV, and apply directly through the platform. Many listings also provide contact details for recruiters if you prefer direct communication.