Make yourself visible and let companies apply to you.
Roles
Hadoop Jobs
Overview
Discover the best Hadoop jobs on Haystack — your go-to IT job board for top data engineering roles. Whether you're a Hadoop developer, architect, or analyst, explore thousands of updated Hadoop job listings from leading companies. Start your search now and take the next step in your big data career with Haystack!
Lead Data Engineering Consultant CGEMJP
Experis IT
Yorkshire
Hybrid
Senior
£423/day
RECENTLY POSTED

Role Title: Lead Data Engineering Consultant

Duration: contract to run until 21/11/2026

Location: Sheffield, Hybrid 3 days per week onsite

Rate: up to £423.78 p/d Umbrella inside IR35

Role purpose/summary

We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms.

The ideal candidate will possess hands-on expertise in the following areas:

  • Extensive enterprise experience with Hadoop, Spark, and Splunk.
  • Proficiency in object-oriented and functional Scripting, particularly in Python.
  • Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL).
  • Experience integrating large, disparate datasets using modern tools and frameworks.
  • Strong background in building and optimizing ETL/ELT data pipelines.
  • Familiarity with source control and implementing Continuous Integration, Delivery, and Deployment via CI/CD pipelines.
  • Experience supporting and collaborating with BI and Analytics teams in fast-paced environments.
  • Ability to pair program and work effectively with other engineers.
  • Excellent analytical and problem-solving abilities.
  • Knowledge of agile methodologies such as Scrum or Kanban is a plus.
  • Comfortable representing the team in standups and problem-solving sessions.
  • Capable of driving the creation of technical test plans and maintaining records, including unit and integration tests, within automated test environments to ensure high code quality.
  • Promote SRE (Site Reliability Engineering) culture by addressing challenges through data engineering.
  • Ensure service resilience, sustainability, and adherence to recovery time objectives for all delivered software solutions.

Soft Skills (Consultant):

  • Demonstrated ability and enthusiasm for enhancing team performance.
  • Strong active listening and effective communication skills.
  • Self-mastery, with a focus on positive mindsets and professional behaviours.
  • Maintains up-to-date expertise in current tools, technologies, and key areas such as cybersecurity, data privacy, consent, and data residency regulations.
  • Engages with industry groups and external vendors to represent and advance the client’s interests and influence.
  • Takes accountability for ensuring control and compliance throughout the engineering process.
  • Champions innovation and the adoption of advanced technologies and best practices within the domain.

All profiles will be reviewed against the required skills and experience. Due to the high number of applications we will only be able to respond to successful applicants in the first instance. We thank you for your interest and the time taken to apply!

Data Architect
Morson Edge
Newcastle upon Tyne
Hybrid
Mid - Senior
£69/hour
RECENTLY POSTED
+2

Data Architect (SC Cleared)

Location: Edinburgh or Newcastle preferred
Other locations considered: Luton, Bristol, Southampton
(Hybrid - predominantly onsite)

Contract: 6 months initially (potential to extend)
Rate: Up to £69.44 per hour umbrella (rate may be negotiable)
Clearance: Active SC clearance required
Nationality: UK Eyes Only - sole British nationality required

The Opportunity

We are seeking an experienced Data Architect to support a high-profile, security-critical programme within a UK government/defence environment.

This role focuses on the design, implementation and governance of enterprise-level data architectures, ensuring data is accurate, secure, accessible, and fit for purpose across complex systems. You will work closely with technical teams and stakeholders to take data solutions from initial requirements through to delivery, operating in a highly regulated setting.

What You’ll Be Doing

  • Design and implement enterprise data architecture solutions, including databases, data warehouses and data lakes
  • Develop and maintain conceptual, logical and physical data models
  • Define and enforce data management standards, policies and best practices
  • Work closely with data engineers to design and optimise ETL/ELT pipelines for structured and unstructured data
  • Ensure data quality, consistency and security across platforms
  • Collaborate with business and technical stakeholders to translate data requirements into technical solutions
  • Oversee data integration across cloud and on-prem environments
  • Support data governance initiatives including metadata management, MDM and compliance
  • Evaluate and recommend new data technologies, tools and platforms
  • Provide technical leadership and guidance to data engineering and analytics teams

What We’re Looking For

Essential:

  • Degree in STEM, Computer Science, Information Systems, Data Science, or similar (Master’s preferred)
  • Strong experience in data architecture, database design or data engineering
  • Proven experience delivering data-intensive systems end to end
  • Strong SQL skills and experience with relational databases (eg Oracle, SQL Server, PostgreSQL, MySQL)
  • Experience with data warehouse and lakehouse architectures
  • Experience designing and governing data models and data flows
  • Experience establishing and working within data governance frameworks
  • Strong understanding of data security and compliance requirements
  • Excellent communication, problem-solving and documentation skills

Desirable:

  • Experience with cloud data platforms (AWS, Azure or GCP)
  • Exposure to big data technologies (Hadoop, Spark, Kafka)
  • Familiarity with ETL/orchestration tools (eg Informatica, Talend, Airflow)
  • Experience with API integration and microservices data flows
  • Familiarity with AI/ML data pipelines or analytics platforms
  • Experience using UML/SysML

Security Requirements

Due to the nature of the work:

  • Candidates must hold active SC clearance
  • UK Eyes Only restrictions apply - pplicants must be sole British nationals (no dual nationality)

Why Apply

  • Work on a high-impact, security-critical programme
  • Opportunity to shape and influence enterprise data architecture
  • Strong likelihood of contract extension
  • Competitive rate with flexibility for the right candidate
  • Collaborative environment with experienced technical teams
Senior Data Scientist
Leidos
Bristol
Hybrid
Senior
£61,500 - £78,800
RECENTLY POSTED
+1

Description

Senior Data Scientist

Programme: LCST

Location: Bristol, occasional travel to customer and Leidos sites may be required

We’re ready for you to unleash your potential!

Role Overview:

We are seeking a Senior Data Scientist to join us on a permanent basis. You will apply advanced analytical, statistical, and machine‑learning techniques to support major UK Government programmes and the wider enterprise, working across multidisciplinary teams throughout the UK.

You will collaborate with delivery managers, engineers, architects, and customer stakeholders to design, develop, and operationalise data‑driven solutions that deliver meaningful impact. This role offers the opportunity to deepen your expertise, contribute to high‑value projects, and help shape the analytical capability within one of the strongest teams in the business.

Your Responsibilities Will Include:

  • Designing, developing, and validating statistical models and machine‑learning solutions
  • Conducting exploratory data analysis and feature engineering to uncover insights and support decision‑making
  • Working closely with data engineers to ensure high‑quality, scalable data pipelines for modelling
  • Translating analytical findings into clear, actionable recommendations for technical and non‑technical audiences
  • Supporting the development of analytical roadmaps, risk identification, and mitigation planning
  • Contributing to the operationalisation and monitoring of models in production environments
  • Evaluating emerging tools, frameworks, and methodologies to enhance analytical capability
  • Providing guidance and informal mentorship to junior data scientists and analysts
  • Ensuring responsible, ethical, and secure use of data and AI

Required Skills:

  • Experience in data science, machine learning, or applied analytics
  • Strong proficiency in Python and common ML libraries (pandas, NumPy, scikit‑learn, TensorFlow, PyTorch)
  • Experience designing, training, and evaluating machine‑learning models
  • Solid understanding of statistical methods, predictive modelling, and experimental design
  • Experience working with large or complex datasets, including data cleansing and feature engineering
  • Ability to communicate analytical concepts clearly to non‑technical stakeholders
  • Experience working in an agile delivery environment
  • Strong problem‑solving skills and attention to detail

Desired Skills:

  • Experience with cloud‑based ML services (AWS SageMaker, Azure ML, Databricks)
  • Familiarity with MLOps practices and model lifecycle management
  • Experience with NLP, deep learning, or computer vision
  • Knowledge of data visualisation tools (Power BI, Tableau)
  • Exposure to big‑data technologies (Spark, Hadoop)
  • AWS certifications
  • Experience supporting government or highly regulated environments

Clearance Requirements:

  • Clearance to Start BPSS
  • Clearance for Role SC
  • Applicants must have (or be able to obtain) SC clearance

Leidos UK! Join our team and discover a culture of collaboration, innovation, diversity, trust, caring management, communication transparency, work-life balance, and overall job satisfaction…

What we do for you:
At Leidos we are PASSIONATE about customer success, UNITED as a team and INSPIRED to make a difference. We offer meaningful and engaging careers, a collaborative culture, and support for your career goals, all while nurturing a healthy work-life balance.
We provide an employment package that attracts, develops and retains only the best in talent. Our reward scheme includes:
•    Contributory Pension Scheme
•    Private Medical Insurance
•    33 days Annual Leave (including public and privilege holidays)
•    Access to Flexible benefits (including life assurance, health schemes, gym memberships, annual buy and sell holidays and a cycle to work scheme)

•    Flexible Working Scheme

Commitment to Diversity:

We welcome applications from every part of the community and are committed to a truly diverse and inclusive culture.  We foster a sense of belonging, welcoming all perspectives and contributions, and providing equal access to opportunities and resources for everyone.  If you have a disability or need any reasonable adjustments during the application and selection stages please let us know, and we will respond in a way that best fits your needs.

Who We Are:

The Logistics Commodities & Services Transformation (LCST) Programme for the UK Ministry of Defence is a critical effort to enhance and improve the UK’s defence supply chain. The programme provides essential services such as storage and distribution for the MOD’s materiel, including a global freight service and the procurement and inventory management of 70,000 commodity NATO Stock Number (NSNs).

Working together as Team Leidos we are helping to transform the UK’s defence supply chain by providing an integration of a complex mixture of services, at low risk, using a modern suite of systems that will deliver one version of the truth. For the first time, the MOD will see the whole picture, as it is happening.

What Makes Us Different:

Purpose: you can use your passion and abilities at Leidos to keep the people you care about safe. We are at the forefront of machine learning, AI, cyber security and solutions. Using your skills in the technology frontline by helping to build a safer world.  You can inspire change.

Collaboration: having flexibility to do your job is one of our core benefits, enabling you to become part of our extraordinary team.  We have been empowering our people to work flexibly for years.  Whether you work from home, the office or on customer sites, we will give you the digital tools and the flexibility to work smarter and align your needs and ours.

People: Leidos empowers people from every background to be themselves and gives you the tools to learn new skills by enabling growth whilst developing. We believe that extraordinary people need opportunities to grow, to be inspired and to inspire others. At Leidos, we invest in technical academies, career rotations and a career development plans that enhance your future.

If you’re looking for comfort, keep scrolling. At Leidos, we outthink, outbuild, and outpace the status quo — because the mission demands it. We’re not hiring followers. We’re recruiting the ones who disrupt, provoke, and refuse to fail. Step 10 is ancient history. We’re already at step 30 — and moving faster than anyone else dares.

Original Posting:

For U.S. Positions: While subject to change based on business needs, Leidos reasonably anticipates that this job requisition will remain open for at least 3 days with an anticipated close date of no earlier than 3 days after the original posting date as listed above.

Pay Range:

£61,500.00-£78,800.00

The Leidos pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.

About Leidos

Leidos is an industry and technology leader serving government and commercial customers with smarter, more efficient digital and mission innovations. Headquartered in Reston, Virginia, with 47,000 global employees, Leidos reported annual revenues of approximately $16.7 billion for the fiscal year ended January 3, 2025. For more information, visit www.Leidos.com.

Pay and Benefits

Pay and benefits are fundamental to any career decision. That’s why we craft compensation packages that reflect the importance of the work we do for our customers. Employment benefits include competitive compensation, Health and Wellness programs, Income Protection, Paid Leave and Retirement. More details are available at www.leidos.com/careers/pay-benefits.

Securing Your Data

Beware of fake employment opportunities using Leidos’ name. Leidos will never ask you to provide payment-related information during any part of the employment application process (i.e., ask you for money), nor will Leidos ever advance money as part of the hiring process (i.e., send you a check or money order before doing any work). Further, Leidos will only communicate with you through emails that are generated by the Leidos.com automated system – never from free commercial services (e.g., Gmail, Yahoo, Hotmail) or via WhatsApp, Telegram, etc. If you received an email purporting to be from Leidos that asks for payment-related information or any other personal information (e.g., about you or your previous employer), and you are concerned about its legitimacy, please make us aware immediately by emailing us at LeidosCareersFraud@leidos.com.

If you believe you are the victim of a scam, contact your local law enforcement and report the incident to the U.S. Federal Trade Commission.

Commitment to Non-Discrimination

All qualified applicants will receive consideration for employment without regard to sex, race, ethnicity, age, national origin, citizenship, religion, physical or mental disability, medical condition, genetic information, pregnancy, family structure, marital status, ancestry, domestic partner status, sexual orientation, gender identity or expression, veteran or military status, or any other basis prohibited by law. Leidos will also consider for employment qualified applicants with criminal histories consistent with relevant laws.

Lead Data Engineer
Vallum Associates Limited
Sheffield
Hybrid
Senior
Private salary
RECENTLY POSTED

We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas:

Extensive enterprise experience with Hadoop, Spark , and Splunk.

Proficiency in object-oriented and functional scripting, particularly in Python .

Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL).

Experience integrating large, disparate datasets using modern tools and frameworks.

Strong background in building and optimizing ETL/ELT data pipelines .

Familiarity with source control and implementing Continuous Integration, Delivery, and Deployment via CI/CD pipelines.

Experience supporting and collaborating with BI and Analytics teams in fast-paced environments.

Ability to pair program and work effectively with other engineers.

Excellent analytical and problem-solving abilities.

Knowledge of agile methodologies such as Scrum or Kanban is a plus.

Comfortable representing the team in standups and problem-solving sessions.

Capable of driving the creation of technical test plans and maintaining records, including unit and integration tests, within automated test environments to ensure high code quality.

Promote SRE (Site Reliability Engineering) culture by addressing challenges through data engineering.

Ensure service resilience, sustainability, and adherence to recovery time objectives for all delivered software solutions.

TPBN1_UKTJ

Customer Support Engineer - GridOS
Stackstudio Digital Ltd.
Multiple locations
Remote or hybrid
Mid - Senior
Private salary
RECENTLY POSTED
+1

Role: Customer Support Engineer - GridOS (Kubernetes + Kafka)
Experience with GridOS or other grid data management platforms.
Knowledge of utility operations, grid modeling, and distributed energy resource management.
Familiarity with real-time data streaming and IoT integration.
Key Responsibilities:

  • Design and implement GridOS Data Fabric solutions to streamline data access and management for utilities.
  • Develop data integration pipelines that unify operational (OT), informational (IT), and external data sources (e.g., weather, traffic).
  • Configure and manage metadata catalogs to ensure effective data governance, security, and compliance.
  • Collaborate with business and IT teams to enhance data discovery, quality, and utilization across the enterprise.
  • Support real-time and batch data processing using big data technologies and cloud platforms.
  • Optimize data workflows to enable AI, machine learning, and advanced analytics applications.
  • Provide technical support and troubleshooting for GridOS Data Fabric deployments.
  • Stay up to date with advancements in grid modernization, data management, and energy analytics.

Required Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related field.
  • 3+ years of experience in data management, integration, or analytics for utilities or energy sectors.
  • Strong knowledge of data fabric architectures, metadata management, and data governance principles.
  • Experience with SQL, ETL tools, and big data technologies (e.g., Hadoop, Spark, Kafka).
  • Familiarity with cloud platforms (AWS, Azure, or Google Cloud) and data integration frameworks.
  • Strong problem-solving skills and ability to work in a cross-functional environment.
Lead Data Engineer - Hadoop - Spark - Python
Square One Resources
Sheffield
Hybrid
Senior
£600/day - £617/day
RECENTLY POSTED

Job Title: Lead Data Engineer - Hadoop, Spark, Pytthon
Location: Sheffield - 3 days per week in the office
Salary/Rate: Up to 617 per day inside IR35
Start Date: 02/03/2026
Job Type: Contract until November

We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms.

The ideal candidate will possess hands-on expertise in the following areas:

  • Extensive enterprise experience with Hadoop, Spark, and Splunk.
  • Proficiency in object-oriented and functional scripting, particularly in Python.
  • Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL).
  • Experience integrating large, disparate datasets using modern tools and frameworks.
  • Strong background in building and optimizing ETL/ELT data pipelines.
  • Familiarity with source control and implementing Continuous Integration, Delivery, and Deployment via CI/CD pipelines.
  • Experience supporting and collaborating with BI and Analytics teams in fast-paced environments.
  • Ability to pair program and work effectively with other engineers.
  • Excellent analytical and problem-solving abilities.
  • Knowledge of agile methodologies such as Scrum or Kanban is a plus.
  • Comfortable representing the team in standups and problem-solving sessions.
  • Capable of driving the creation of technical test plans and maintaining records, including unit and integration tests, within automated test environments to ensure high code quality.
  • Promote SRE (Site Reliability Engineering) culture by addressing challenges through data engineering.
  • Ensure service resilience, sustainability, and adherence to recovery time objectives for all delivered software solutions.

If you are interested in this opportunity, please apply now with your updated CV in Microsoft Word/PDF format.

Disclaimer
Notwithstanding any guidelines given to level of experience sought, we will consider candidates from outside this range if they can demonstrate the necessary competencies.
Square One is acting as both an employment agency and an employment business, and is an equal opportunities recruitment business. Square One embraces diversity and will treat everyone equally. Please see our website for our full diversity statement.

Data Engineer
MBDA UK
Manchester
Hybrid
Mid - Senior
£45,000 - £55,000
+2

Bolton

As a data engineer specialising in generative AI ; this role will see you working in a developing international and transversal structure. You will have the responsibility to evaluate, build and maintain data sets for internal customers whilst ensuring they can be maintained.
Salary: Circa £45,000 - £55,000 depending on experience
Dynamic (hybrid) working: 2-3 days per week on-site due to workload classification
Security Clearance: British Citizen
Restrictions and/or limitations relating to nationality and/or rights to work may apply. As a minimum and after offer stage, all successful candidates will need to undergo HMG Basic Personnel Security Standard checks (BPSS), which are managed by the MBDA Personnel Security Team.
What we can offer you:

  • Company bonus: Up to £2,500 (based on company performance and will vary year to year)
  • Pension: maximum total (employer and employee) contribution of up to 14%
  • Overtime: opportunity for paid overtime
  • Flexi Leave: Up to 15 additional days
  • Flexible working: We welcome applicants who are looking for flexible working arrangements
  • Enhanced parental leave: offers up to 26 weeks for maternity, adoption and shared parental leave -enhancements are available for paternity leave, neonatal leave and fertility testing and treatments
  • Facilities: Fantastic site facilities including subsidised meals, free car parking and much more

The opportunity:
The MBDA IM GenAI delivery Office department is looking for an experienced data engineer able to evaluate design, deploy, improve and support MBDA data sets.

You will ensure MBDA data pipelines are designed to be resilient, secure and responsive. You will use your data engineering expertise to collaborate with different internal customers regarding their data, ensuring they are optimised and secured for their needs.

You will provide your knowledge in data management and data quality to guarantee compliance to MBDA data governance. A key part of this role is keeping up to date with new technology, where you will provide insight on our technology roadmap and deliver cutting edge solutions to our internal customers.
What we’re looking for from you:

  • SQL technologies skills (e.g. MS SQL, Oracle )
  • noSQL technologies skills (e.g. MongoDB, InfluxDB, Neo4J )
  • Data exchange and processing skills (e.g. ETL, ESB, API )
  • Development (e.g. Python) skills
  • Big data technologies knowledge (e.g. Hadoop stack)
  • Knowledge in NLP (Natural Language Processing)
  • Knowledge in OCR (Object Character Recognition)
  • Knowledge in Generative AI (Artificial Intelligence) would be advantageous
  • Experience in containerisation technologies (e.g. Docker) would be advantageous
  • Knowledge in the industrial and / or defence sector would be advantageous

Our company: Peace is not a given, Freedom is not a given, Sovereignty is not a given

MBDA is a leading defence organisation. We are proud of the role we play in supporting the Armed Forces who protect our nations. We partner with governments to work together towards a common goal, defending our freedom.

We are proud of our employee-led networks, examples include: Gender Equality, Pride, Menopause Matters, Parents and Carers, Armed Forces, Ethnic Diversity, Neurodiversity and more

We recognise that everyone is unique, and we encourage you to speak to us should you require any advice, support or adjustments throughout our recruitment process.

Follow us on LinkedIn (MBDA), X Instagram (MBDA_UK) and Glassdoor or visit our MBDA Careers website for more information.

Data Architect
Stable Resources Ltd
Oxfordshire
Hybrid
Senior
£700/day - £750/day

24 months until December 2027

Hybrid occasional travel to Reading

Rate TBD

Role Requirements:

  • Subject to clearance requirements
  • Experience: Proven track record as a Data Architect in complex, multinational environments.
  • Technical Expertise: Strong knowledge of Azure services (Data Lake, Data Factory, Synapse) and familiarity with emerging cloud technologies.
  • Security Awareness: Ability to design architectures with strict security boundaries, protective markings, export-control constraints, and multi-tenant segregation.
  • Collaboration: Comfortable working in greenfield projects and multinational defence contexts with varying data access classifications.

Key Responsibilities:

  • Define and implement end-to-end data architecture across multiple business areas.
  • Ensure architectural consistency across nations and business domains in collaboration with the Data Architect Manager.
  • Evaluate and adopt new technologies to support secure multinational collaboration.
  • Design and maintain distributed data environments and modern data warehousing patterns.
  • Implement solutions that comply with security, regulatory, and export-control requirements.
  • Extensive experience in Data standards
  • ISO 10303 experience
  • Experience in working with engineering data integration

Desirable Skills:

  • Hands-on experience with tools such as Databricks and Hadoop, and distributed data platforms.
  • Knowledge of modern data warehousing and big data processing frameworks.
  • Ability to work with Azure-based ecosystems and integrate emerging services.
  • Strong understanding of data governance in highly regulated environments.
  • Excellent communication and stakeholder management skills.
  • ERP
  • PLM
  • Manufacturing
Data Engineer
Lynx Recruitment Limited
Ruislip
Fully remote
Senior - Leader
£65,000

DataEngineer-RemoteWorking

LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions.

Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions.

Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures

DevelopingandmaintainingETL/ELTprocessesusingmoderntools

Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms

Ensuringdataquality,reliability,andoperationalexcellence

Leadingthedesignanddeliveryofnewdataproductsandpipelines

Collaboratingcloselywithstakeholdersandtechnicalteams

Providingtechnicalleadershipandmentoringwithindeliveryteams

Whattherelookingfor: Strongexperienceindataengineeringroles

Provenleadershiporlinemanagementexperience

Hands-onexperiencewithcloudplatforms(AWSorAzure;GCPbeneficial)

StrongSQLskillsandexperiencewithrelationaldatabases

Pythondevelopmentexperienceandfamiliaritywithframeworks(e.g.Flask,Django)

ExperiencebuildingandorchestratingETLpipelines(e.g.Airflow,Luigi,Argo)

Exposuretobigdatatechnologies(e.g.Spark,Kafka,Hadoop)

ExperienceworkinginAgileenvironments

Solidunderstandingofdataarchitectureanddataqualitybestpractices

TPBN1_UKTJ

Customer Support Engineer - GridOS
Stackstudio Digital Ltd.
UK
Hybrid
Mid - Senior
Private salary
+1

Role:Customer Support Engineer - GridOS (Kubernetes + Kafka) Experience with GridOS or other grid data management platforms.
Knowledge of utility operations, grid modeling, and distributed energy resource management.
Familiarity with real-time data streaming and IoT integration. Key Responsibilities:
Design and implement GridOS Data Fabric solutions to streamline data access and management for utilities.
Develop data integration pipelines that unify operational (OT), informational (IT), and external data sources (e.g., weather, traffic).
Configure and manage metadata catalogs to ensure effective data governance, security, and compliance.
Collaborate with business and IT teams to enhance data discovery, quality, and utilization across the enterprise.
Support real-time and batch data processing using big data technologies and cloud platforms.
Optimize data workflows to enable AI, machine learning, and advanced analytics applications.
Provide technical support and troubleshooting for GridOS Data Fabric deployments.
Stay up to date with advancements in grid modernization, data management, and energy analytics. Required Qualifications:
Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related field.
3+ years of experience in data management, integration, or analytics for utilities or energy sectors.
Strong knowledge of data fabric architectures, metadata management, and data governance principles.
Experience with SQL, ETL tools, and big data technologies (e.g., Hadoop, Spark, Kafka).
Familiarity with cloud platforms (AWS, Azure, or Google Cloud) and data integration frameworks.
Strong problem-solving skills and ability to work in a cross-functional environment.

Data Architect - Active SC required
Matchtech
Newcastle upon Tyne
Hybrid
Mid - Senior
Private salary
+2

Location: Edinburgh, Newcastle, Luton, Bristol OR Southampton (fully onsite at one of these UK based sites)

Duration: 6 month initial contract

Inside IR35

Role details:

Our client, a prominent player in the Defence & Security sector, is looking for Data Architects to join their team on a contract basis. Whilst the site preference is Edinburgh or Newcastle, the team can also support contractors working from their Luton, Bristol or Southampton site. Due to the secure environments, you will be required onsite at least 4 days per week.

Note, active SC clearance required!

Key Responsibilities:

Design and implement enterprise-level data architecture solutions, including databases, data warehouses, and data lakes.
Develop and maintain logical, physical, and conceptual data models.
Define data management standards, policies, and best practices.
Work with data engineers to design and optimise ETL/ELT pipelines for structured and unstructured data.
Ensure data quality, consistency, and security across platforms.
Collaborate with business stakeholders to understand data requirements and translate them into technical solutions.
Evaluate and recommend new data technologies, tools, and platforms to enhance data capabilities.
Oversee data integration across cloud and on-premises environments.
Support data governance initiatives, including metadata management, master data management (MDM), and compliance.
Provide technical leadership and mentorship to data engineering and analytics teams.

Job Requirements:

Bachelor’s degree in STEM, Computer Science, Information Systems, Data Science, or related field (Master’s preferred).
Strong experience in data architecture, database design, or data engineering.
Proficiency in SQL and database technologies (e.g., Oracle, SQL Server, PostgreSQL, MySQL).
Knowledge of data warehouse and lakehouse architectures.
Familiarity with ETL/ELT and orchestration tools (e.g., Informatica, Talend, Apache Airflow).
Experience establishing and maintaining data governance frameworks.
Experience complying with data security requirements.
Experience designing data models.
Excellent problem-solving, communication, and documentation skills.
Familiarity with AI/ML data pipelines and analytics platforms.

Preferred Qualifications:

Experience with cloud data platforms (e.g., AWS, Azure, Google Cloud).
Experience with big data technologies (e.g., Hadoop, Spark, Kafka).
Experience with UML/SYSML.
Strong understanding of API integration and microservices data flow.
If you have a strong background in data architecture and are eager to collaborate with IT teams, data engineers, system engineers, and stakeholders to define data models, data flow processes, and governance frameworks, apply now

Data Architect - SC Cleared
CBS Butler
Edinburgh
Hybrid
Senior - Leader
£550/day - £650/day
+2

Role: Data Architect

Location: Edinburgh or Luton (hybrid/onsite as required)
Engagement: Contract - Inside IR35
Rate: £550 - £650 per day
Security Clearance: Active SC Clearance required

Project Description

We are seeking an experienced Data Architect to lead the design, implementation, and governance of enterprise-scale data architectures within a secure environment. You will be responsible for ensuring data across systems is accurate, accessible, secure, and aligned to business and technical objectives.

The role requires extensive experience delivering data-intensive systems end-to-end, from initial requirements capture through architecture design and implementation to production delivery. You will work closely with customers and stakeholders to translate business needs into robust, scalable data solutions.

You will act as a senior technical authority, collaborating proactively with IT teams, data engineers, system engineers, and business stakeholders to define data models, data flows, and governance frameworks.

Key Responsibilities

  • Design and implement enterprise-level data architecture solutions, including databases, data warehouses, data lakes, and lakehouse platforms
  • Develop and maintain conceptual, logical, and physical data models
  • Define and enforce data management standards, policies, and best practices
  • Work closely with data engineers to design and optimise ETL/ELT pipelines for structured and unstructured data
  • Ensure data quality, consistency, security, and compliance across platforms
  • Collaborate with business stakeholders to gather requirements and translate them into technical solutions
  • Oversee data integration across cloud and on-premises environments
  • Support and lead data governance initiatives, including metadata management, MDM, and regulatory compliance
  • Evaluate and recommend new data technologies, tools, and platforms
  • Provide technical leadership and mentorship to data engineering and analytics teams

Skills and Experience Required

  • Active SC Clearance (mandatory)
  • Bachelor’s degree in STEM, Computer Science, Information Systems, Data Science, or related field (Master’s preferred)
  • Strong experience in data architecture, database design, or senior data engineering roles
  • High proficiency in SQL and database technologies (eg Oracle, SQL Server, PostgreSQL, MySQL)
  • Strong knowledge of data warehouse, data lake, and lakehouse architectures
  • Experience with ETL/ELT frameworks and orchestration tools (eg Informatica, Talend, Apache Airflow)
  • Proven experience establishing and maintaining data governance frameworks
  • Strong understanding of data security, access controls, and compliance requirements
  • Excellent problem-solving, communication, and documentation skills
  • Familiarity with AI/ML data pipelines and analytics platforms

Preferred Qualifications

  • Experience with cloud data platforms (AWS, Azure, GCP)
  • Experience with big data technologies (Hadoop, Spark, Kafka)
  • Familiarity with UML/SysML
  • Strong understanding of API integration and microservices data flows
Page 1 of 1
Frequently asked questions
Our platform features a wide range of Hadoop-related jobs including Hadoop Developer, Hadoop Administrator, Data Engineer, Big Data Analyst, and Hadoop Architect positions across various industries.Most Hadoop job listings require some prior experience or knowledge of Hadoop ecosystem components like HDFS, MapReduce, Hive, or Spark. However, there are also entry-level positions or internships for candidates willing to learn.Yes, we list both remote and onsite Hadoop job opportunities. You can filter your job search by location or remote work preferences to find roles that suit your needs.Typical skills include proficiency with Hadoop Distributed File System (HDFS), MapReduce programming, Hive, Pig, Spark, Java or Scala programming, data modeling, and experience with big data tools and platforms.New Hadoop job postings are updated regularly, often daily, ensuring you have access to the latest opportunities in the Big Data and Hadoop fields.
Feedback
Contact