Data Engineer

Salary: £40 - 72K + Benefits
Location: London or Manchester (aligned to office for client on-site requirements)
Working Pattern: Hybrid / On-site depending on client needs
Security Clearance: SC Clearance required

You will join a people-focused digital consultancy supporting data-driven services across the UK public sector. The organisation values collaboration, inclusion, and work-life balance, and actively supports continuous learning and professional development through access to training, modern engineering tools, and supportive multidisciplinary teams.

The consultancy works closely with government departments and public sector organisations to design, build, and operate secure, scalable data platforms that enable advanced analytics, data science, and machine learning. Diversity and inclusion are core values, and hiring decisions are based on skills, experience, and potential. Empowering individuals and building strong teams are central to delivering meaningful outcomes for clients and citizens.

This role is suited to Data Software Engineers with a strong technical foundation and an interest in data engineering, data science, and machine learning. You will work within agile, multidisciplinary teams alongside data scientists, platform engineers, and stakeholders to build robust data processing systems in secure environments.

Role Responsibilities

Design, build, and maintain scalable data processing and integration systems to support data science and analytics workloads.

Develop high-quality, well-tested software using a Test-Driven Development (TDD) approach.

Collaborate closely with data scientists to enable effective use of data for analytics and machine learning.

Build and operate cloud-based solutions, with a strong focus on AWS services.

Work with messaging, streaming, or data flow technologies to support real-time and batch data processing.

Contribute to infrastructure and platform automation using Infrastructure as Code.

Participate in agile ceremonies, technical design discussions, and code reviews.

Ensure solutions meet security, performance, and reliability requirements within public sector environments.

What You Will Bring to the Team

A strong interest in data, particularly data engineering, data science, or machine learning.

A solid technical background, with experience in Java, Python, TypeScript, or similar languages.

Experience developing software using TDD or a strong willingness to adopt TDD practices.

Strong problem-solving skills and the ability to work collaboratively within multidisciplinary teams.

Good communication skills, with the ability to explain technical concepts to both technical and non-technical stakeholders.

A proactive mindset with attention to detail and a commitment to quality.

Desirable Skills and Experience

Experience working with cloud platforms, ideally AWS.

A strong Linux background.

Experience with data integration and messaging technologies such as Apache NiFi, Apache Kafka, RabbitMQ, or similar tools.

Experience using Infrastructure as Code tools such as Terraform or CloudFormation.

Previous experience working in a consultancy or public sector delivery environment.

Familiarity with secure or regulated environments.

Reasonable Adjustments:

Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people of all backgrounds and perspectives. Our success is driven by our people, united by the spirit of partnership to deliver the best resourcing solutions for our clients.

If you need any help or adjustments during the recruitment process for any reason, please let us know when you apply or talk to the recruiters directly so we can support you.

Data Platform Engineer – Warton – SC Cleared – 12 Month Contract

About Your Role:

As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on premises platforms.
You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams.
A strong command of Apache NiFi is essential for this role.
You will design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure data ingestion, transformation, and delivery.
You should be adept at identifying and resolving issues within NiFi flows, managing performance bottlenecks, and implementing robust error handling strategies.
You’ll work closely with cross-functional teams including data architects, compliance officers, and cybersecurity specialists to integrate data from various systems such as databases, APIs, and cloud platforms.

About You:

As an experienced Data Platform Engineer, your skills and experience may include;

• Strong experience of Apache NiFi
• Experience designing and optimizing data flows for batch, real-time streaming, and event-driven architectures.
• Ability to identify and resolve flow issues, optimize performance, and implement error-handling strategies.
• Optional scripting skills for creating custom NiFi processors.
• Knowledge of data modelling, replication, and query optimization.
• Hands-on experience with SQL and NoSQL databases is desirable.
• Familiarity with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery) would be beneficial.
• Data Platform Management
• Comfortable operating in hybrid environments (cloud and on-prem).
• Experience integrating diverse data sources and systems

Key Responsibilities:

• Design our scalable, secure and cost-effective cloud hybrid data solution aligned with Omnia enterprise architecture and Army training objectives.
• Design innovative data models and metadata systems to interpret and enhance business needs, promoting data as a strategic asset.
• Develop transitional data architectures and road maps in alignment with Omnia digital transformation objectives, enabling the evolution of integrated training solutions.
• Collaborate with cross-functional teams and partners to ensure data architectural integrity and alignment.
• Translate complex technical concepts for a non-technical audience, fostering understanding and buy-in from stakeholders.
• Support horizon scanning to identify and assess emerging technologies and determine their potential impact on leveraging data analytics to improve training delivery.
• Membership and contribution to the Omnia Architecture Board(s) to review, define, refine and uphold data architectural principles, policies, and standards across Omnia Training.
• Work with engineering teams to support and guide the implementation of a variety of solutions across multiple domains, in an agile environment.
• Manage and lead prototyping and research activity as required to realise the best solutions.
• Provide oversight and advice to engineers undertaking the design of data models and support the management of data dictionaries.
• Ensure the security and compliance of data environments through the implementation of appropriate architecture principles, security controls and governance frameworks.
• Author, review and contribute to technical documentation.
• Produce High- and low-level design artifacts for data storage, processing, and retrieval systems.
• Stay current with data technologies, making recommendations for use based on business value. Drive the adoption of these technologies.
• Serve as the Subject Matter Expert (SME) for the Omnia Data Lakehouse, providing strategic guidance, technical oversight, and deep expertise in its architecture, implementation, and optimisation.

Who we are looking for:

The Data Architect will provide strategic and practical leadership across the data architecture landscape of the Army Collective Training Service (ACTS). This role is pivotal in shaping and governing the enterprise data architecture to ensure it aligns with the ACTS vision and supports data as a mission-critical asset. Working closely with the Data Lead, Enterprise Architect and Chief Engineer, you will design, evolve, and maintain a secure, scalable, and interoperable data architecture, leveraging cloud services from AWS, Azure, and OCP where appropriate.

This role requires a systems-thinking mindset, strong stakeholder engagement skills, and the ability to work across engineering teams in a complex and evolving environment.

Essential Skills and Experience:

• Proven experience designing and implementing scalable, secure, and interoperable data architectures in complex environments with cloud platforms (AWS, Azure, GCP, OCI) and cloud-native data services including use of services like S3, Lambda, Glue, Data Factory, etc.
• Experience designing and supporting implementation of large-scale data pipelines, data warehousing and Lakehouse technologies.
• Strong skills in conceptual, logical, and physical data modelling using tools like ER/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases.
• Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop).
• Experience with API management and gateway tools and services.
• Experience of RESTful APIs for ingesting and exposing data.
• Proven track record with data governance, quality, lineage, and security practices.
• Experience with real-time/streaming data technologies, data Ingestion / ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub).
• Experience in, or knowledge of, DevSecOps Tooling and Processes.
• Self-starter with the ability to appropriately prioritise and plan complex work in a rapidly changing environment.
• Strong critical thinker with problem solving aptitude.
• A working knowledge of MoD or Government IT Security environments and requirements at various classifications.

Desirable Skills and Experience:

• Hands-on experience with containerisation and orchestration (Docker, Kubernetes, Red Hat OpenShift).
• Exposure to infrastructure-as-code tools (Terraform, CloudFormation).
• Experience with BI/visualisation tools (e.g., Tableau, Power BI, Looker, Elastic Stack).
• Knowledge of compliance frameworks (GDPR, HIPAA, CCPA) and their impact on data systems.
• Relevant Data and Architecture certification such as TOGAF, MODAF, AWS/Azure Certified Data Architect or Solutions Architect, DAMA Certified Data Management Professional (CDMP)

The role:

This is more than a job - it’s a mission. You will be part of a high-impact, collaborative environment, where we expect everyone to live the values and standards of the British Army. Every person in our team plays a critical role in delivering OMNIA’s vision; designing, delivering, and transforming collective training so the British Army is ready to fight and win.

You’ll work in a matrix organisation and report operationally through OMNIA Training and functionally through the Cloud & Platform Lead. Ultimately, you’ll work for the British Army, championing innovation, and helping shape the future of military collective training.

Key Responsibilities:

• Design, deploy, and support integrated system and application solutions across physical, virtual, and cloud environments through CI/CD pipelines
• Work with engineering teams to translate operational and technical requirements into implementable system architectures and solution designs
• Support the deployment, configuration, and integration of systems and applications within development and operational environments
• Ensure system solutions meet performance, reliability, security, and interoperability requirements across platforms and environments
• Act as the technical liaison between engineering teams and supply chain partners, ensuring technical specifications, standards, and integration requirements are clearly communicated and understood
• Support supplier engagement by reviewing technical proposals, validating solution compatibility, and assisting with integration planning
• Coordinate with engineering and supply chain teams to ensure delivered components and systems meet required technical and operational specifications
• Support and conduct system testing, validation, and troubleshooting to ensure successful deployment and operational readiness
• Collaborate with cross-functional teams including software engineers, network engineers, architects, and programme teams to support system delivery
• Support the integration of third-party technologies and vendor solutions into the broader systems architecture
• Contribute to the development and maintenance of technical documentation, including design documentation, integration guides, and deployment procedures
• Maintain awareness of emerging technologies and tools that could enhance system performance, integration, or operational capability
• Any other duties required to meet the needs of the programme

Who we are looking for:

We’re after individuals who want to serve. You’ll have a mission focus, and the enthusiasm and drive to ‘get things done’. You’ll want to work in collaboration with other defence training organisations, and the British Army. You won’t let bureaucracy get in the way of what needs to be done, you’ll learn lessons and share these lessons across the team. You won’t necessarily have a military background, but you’ll understand what it means to serve and to put the mission first.

The OMNIA Systems Solutions Engineers will be responsible for responsible for the discovery, deployment and integration of a wide range of applications, bespoke technology and solutions in relations to cloud platforms and modelling & simulation for the Army Collective Training Service (ACTS). Responsible to the Cloud & Platform Lead, this position will be critical to the successful technology and interoperability of the entire solution. The position is suited to an all-round engineer who thrives on the challenge of tackling integration and deployment wide range of solutions across a broad spectrum of technologies, including simulation systems.

This position requires a customer first mindset, strong stakeholder engagement skills, and the ability work in a multidisciplinary engineering team in a complex and evolving environment.

Essential Skills and Experience:

• Engineering background with a degree in a relevant discipline or equivalent professional experience
• Proven experience designing, deploying, and supporting integrated systems and application solutions across hybrid environments (cloud and on-premise)
• Strong understanding of systems integration principles across software, infrastructure, and networking layers.
• Ability to translate operational and technical requirements into practical system architectures and deployment plans.
• Experience supporting technical deployments within structured delivery environments (e.g. Agile or SAFe).
• Strong analytical and problem-solving capability with experience diagnosing complex system issues
• Understanding of system security principles and governance requirements
• Experience designing and deploying containerised solutions using orchestration technologies (e.g. Docker, Kubernetes)
• Experience implementing CI/CD pipelines and DevOps tooling (e.g. GitHub, ArgoCD, Helm, Maven, SonarQube, Quay or equivalent)
• Experience using monitoring and observability platforms (e.g. Prometheus, Fluentd, ELK stack, Grafana)
• Experience integrating and supporting database solutions (e.g. MongoDB, PostgreSQL, MSSQL)
• Experience deploying and operating systems within cloud platforms such as Azure or AWS
• Experience supporting system testing, validation, and troubleshooting during deployment and operational phases
• Holds, or is eligible to obtain, UK Security Clearance (SC)

Desirable Skills and Experience:

• Experience working with Red Hat OpenShift and Ceph storage platforms
• Programming experience (e.g. Python, JavaScript)
• Experience working with event streaming and data pipeline technologies (e.g. Kafka, Apache NiFi, Apache Spark)
• Experience delivering multi-system integration projects within defence, government, or highly regulated environments.
• Experience implementing and configuring training simulation technologies across Live, Virtual, and Constructive (LVC) environments
• Experience working with external suppliers and vendors to deliver integrated technical solutions, including reviewing technical documentation to ensure alignment with system/architecture requirements and standards
• Experience supporting procurement or supply chain teams by translating technical specifications and integration requirements
• Understanding of interoperability standards or integration frameworks used within enterprise or defence systems

Benefits and Work Culture

Benefits

• Competitive salaries.
• 25 days holiday + statutory public holidays, plus opportunity to buy and sell up to 5 days (37hr)
• Contributory Pension Scheme (up to 10.5% company contribution)
• Company bonus scheme (discretionary).
• 6 times salary ‘Life Assurance’ with pension.
• Flexible Benefits scheme with extensive salary sacrifice schemes, including Health Cashplan, Dental, and Cycle to Work amongst others.
• Enhanced sick pay.
• Enhanced family friendly policies including enhanced maternity, paternity & shared parental leave.
• Car / Car allowance (dependant on grade/ role)
• Private Medical Insurance (dependant on grade)

Work Culture

• 37hr working week, although hours may vary depending on role, job requirement or site-specific arrangements.
• Early 1.30pm finish Friday, start your weekend early!
• Remote, hybrid and site based working opportunities, dependant on your needs and the requirements of the role.
• A grownup flexible working culture that is output, not time spent at desk, focussed. More formal flexible working arrangements can also be requested and assessed subject to the role. Please enquire or highlight any request to our Talent Acquisition team to explore the flexible working possibilities.
• Up to 5 paid days volunteering each year.

Our client, a prominent entity within the Defence & Security sector, is currently searching for a dedicated and skilled DevOps Systems Engineer (Systems Solutions Engineer) to join their team in Wiltshire.

This is more than a job - it’s a mission. You will be part of a high-impact, collaborative environment, contributing to the vision of designing, delivering, and transforming collective training for the UK forces.

Key Responsibilities:

• Designing, deploying, and supporting integrated system and application solutions across physical, virtual, and cloud environments through CI/CD pipelines
• Translating operational and technical requirements into implementable system architectures and solution designs
• Supporting the deployment, configuration, and integration of systems and applications within development and operational environments
• Ensuring system solutions meet performance, reliability, security, and interoperability requirements across platforms and environments
• Acting as the technical liaison between engineering teams and supply chain partners
• Reviewing technical proposals and assisting with integration planning
• Coordinating with engineering and supply chain teams to ensure delivered components and systems meet required technical and operational specifications
• Supporting system testing, validation, and troubleshooting to ensure successful deployment and operational readiness
• Collaborating with cross-functional teams including software engineers, network engineers, architects, and programme teams
• Integrating third-party technologies and vendor solutions into the broader systems architecture
• Developing and maintaining technical documentation
• Maintaining awareness of emerging technologies and tools

Job Requirements:

We are looking for individuals who have a mission focus and the enthusiasm and drive to ‘get things done’. You should be able to collaborate effectively with defence organisations. The ideal candidate will possess the following skills and experience:

• Engineering background with a degree in a relevant discipline or equivalent professional experience
• Proven experience designing, deploying, and supporting integrated systems and application solutions across hybrid environments (cloud and on-premise)
• Strong understanding of systems integration principles across software, infrastructure, and networking layers
• Ability to translate operational and technical requirements into practical system architectures and deployment plans
• Experience supporting technical deployments within structured delivery environments (e.g. Agile or SAFe)
• Strong analytical and problem-solving capability with experience diagnosing complex system issues
• Understanding of system security principles and governance requirements
• Experience designing and deploying containerised solutions using orchestration technologies (e.g. Docker, Kubernetes)
• Experience implementing CI/CD pipelines and DevOps tooling (e.g. GitHub, ArgoCD, Helm, Maven, SonarQube, Quay or equivalent)
• Experience using monitoring and observability platforms (e.g. Prometheus, Fluentd, ELK stack, Grafana)
• Experience integrating and supporting database solutions (e.g. MongoDB, PostgreSQL, MSSQL)
• Experience deploying and operating systems within cloud platforms such as Azure or AWS
• Experience supporting system testing, validation, and troubleshooting during deployment and operational phases

Desirable Skills and Experience:

• Experience working with Red Hat OpenShift and Ceph storage platforms
• Programming experience (e.g. Python, JavaScript)
• Experience working with event streaming and data pipeline technologies (e.g. Kafka, Apache NiFi, Apache Spark)
• Experience delivering multi-system integration projects within defence, government, or highly regulated environments
• Experience implementing and configuring training simulation technologies across Live, Virtual, and Constructive (LVC) environments
• Experience working with external suppliers and vendors to deliver integrated technical solutions
• Experience supporting procurement or supply chain teams by translating technical specifications and integration requirements
• Understanding of interoperability standards or integration frameworks used within enterprise or defence systems

Benefits:

• Competitive salaries
• 25 days holiday + statutory public holidays, with an option to buy and sell up to 5 days
• Contributory Pension Scheme (up to 10.5% company contribution)
• Company bonus scheme (discretionary)
• 6 times salary ‘Life Assurance’ with pension
• Flexible Benefits scheme including Health Cashplan, Dental, and Cycle to Work amongst others
• Enhanced sick pay
• Enhanced family friendly policies including enhanced maternity, paternity & shared parental leave
• Car / Car allowance (dependant on grade/ role)
• Private Medical Insurance (dependant on grade)

Work Culture:

• 37hr working week, although hours may vary depending on role or job requirement
• Early finish on Fridays, starting the weekend at 1.30pm
• Remote, hybrid, and site-based working opportunities
• A flexible working culture that is output focussed
• Up to 5 paid days volunteering each year

If you are a capable and committed DevOps Systems Engineer (Systems Solutions Engineer) looking for a new opportunity to develop your career within the Defence & Security sector, we would love to hear from you. Apply now to join our client’s dynamic and talented team in Wiltshire.

DV Cleared DevOps / Software Engineer - Contract
550- 600 per day Outside IR35
Condensed week available - work 3 long days, paid for 5
Active MOD DV Clearance required

We are looking for an experienced DV Cleared DevOps / Software Engineer to join a high-impact technical environment. This role suits someone who thrives in modern DevOps ecosystems, enjoys building secure, scalable platforms, and wants the flexibility of a condensed working week.

You will work across the full software delivery lifecycle, helping to design, build, automate, and maintain robust cloud-native and containerised platforms.

Key Responsibilities

• Design, build and maintain CI/CD pipelines for reliable software delivery
• Deploy and manage Kubernetes clusters and application platforms
• Collaborate with engineers to develop, test, and release software efficiently
• Implement automation and infrastructure as code
• Ensure systems remain secure, resilient, and scalable
• Deliver software fixes, upgrades, and performance improvements
• Contribute to workflow improvements and DevOps best practices

Technical Skills (Must Have)

• CI/CD & Automation: GitLab CI, Terraform, Ansible, Helm Charts
• Scripting & APIs: Python, PowerShell, REST APIs
• Containerisation & Platforms: Kubernetes cluster build and application delivery
• Application Technologies: Apache NiFi, Elastic ECK, Artifactory
• Secrets Management: Hashicorp Vault
• Operating Systems: RedHat Linux and Windows

Experience & Background

• Experience across the full software development lifecycle
• Strong understanding of automation and DevOps tooling
• Knowledge of secure software delivery practices
• Comfortable working with multiple programming / scripting languages
• Ability to collaborate across teams and support continuous improvement

Key Details

• Contract Rate: 550- 600 per day
• Status: Outside IR35
• Clearance: Active MOD DV clearance required
• Working Pattern: Option for condensed week (3 long days, paid for 5)

Job Title: Software Engineer

Location: London

Salary: TBC - We are booking interviews in next week! Please call or email for a slot

As a Software Engineer you’ll play a pivotal role in developing cutting edge DevSecOps software systems and applications. You’ll be at the forefront of driving innovation and safeguarding digital infrastructures in the UK.

The Role:

So, what will you be doing as a Software Engineer?

Typically, you will be involved in one or more of the following:

• Planning, conducting and coordinating software development activities.
• Working with customers and key stakeholders to identify requirements and ready to advise on and influence the adoption of suitable architectures and solutions.
• Design and development of tests to debug software.
• Working with customer requirements that are to be programmed into business/mission applications, whilst utilising the appropriates standards, processes and tools throughout the entire development lifecycle.
• Mentoring of junior engineers.
• Analysis of system capabilities to resolve questions of program intent, output requirements, input data acquisition, programming techniques, and controls.
• Championing automation in the form of IaC, CI/CD and test automation.

Extra info:

• Expert knowledge of current Java.
• Proficient in another language such as Python, C++, JavaScript, TypeScript, React.
• Advanced knowledge of AWS Services such as EC2, EKS, Fargate, IAM, S3, Lambda.
• Advanced knowledge of CI/CD software pipelines for Cloud deployments.
• Expert knowledge of developing software for cloud deployment - specifically AWS.
• Expert in developing software for microservice and or serverless architectures.
• Advanced understanding the development challenges moving on-prem workload to cloud.
• Experience of developing on Linux.
• Advanced understanding of mocking and unit testing frameworks such as JUnit 5.x (Jupiter).
• Expert with Git version control.
• Proficient in Agile Development using SCRUM.
• Advanced knowledge of Oracle/ Relational Databases and/or Mongo.
• Experience of GitLab CI/CD Pipelines.
• Experience of Apache NiFi.
• Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence.
• Experience of Elasticsearch and Kibana.
• Advanced knowledge of Hibernate.

We would love to hear from you if you have any of the following Qualifications or Certifications:

• Oracle Java Certification.
• AWS Certification.
• ISTQB Certification.
• OSCP Certification.
• CCNA Certification.

My client is keen to get this position filled ASAP, so if you feel you have the skills and desire to provide passion and drive as the next Software Engineer, this is the opportunity for you!

You will be required to undergo DV clearance for this role.

So, if you are interested in joining a well-established, renowned, globally recognised organisation working with experts in their field as the next Software Engineer, hit that apply button now!

Services advertised by Gold Group are those of an Agency and/or an Employment Business.
We will contact you within the next 14 days if you are selected for interview. For a copy of our privacy policy please visit our website.

DV Cleared DevOps / Software Engineer - Contract £550-£600 per day | Outside IR35 Condensed week available - work 3 long days, paid for 5 Active MOD DV Clearance required We are looking for an experienced DV Cleared DevOps / Software Engineer to join a high-impact technical environment. This role suits someone who thrives in modern DevOps ecosystems, enjoys building secure, scalable platforms, and wants the flexibility of a condensed working week. You will work across the full software delivery lifecycle, helping to design, build, automate, and maintain robust cloud-native and containerised platforms. Key Responsibilities Design, build and maintain CI/CD pipelines for reliable software delivery Deploy and manage Kubernetes clusters and application platforms Collaborate with engineers to develop, test, and release software efficiently Implement automation and infrastructure as code Ensure systems remain secure, resilient, and scalable Deliver software fixes, upgrades, and performance improvements Contribute to workflow improvements and DevOps best practices Technical Skills (Must Have) CI/CD & Automation: GitLab CI, Terraform, Ansible, Helm Charts Scripting & APIs: Python, PowerShell, REST APIs Containerisation & Platforms: Kubernetes cluster build and application delivery Application Technologies: Apache NiFi, Elastic ECK, Artifactory Secrets Management: Hashicorp Vault Operating Systems: RedHat Linux and Windows Experience & Background Experience across the full software development lifecycle Strong understanding of automation and DevOps tooling Knowledge of secure software delivery practices Comfortable working with multiple programming / scripting languages Ability to collaborate across teams and support continuous improvement Key Details Contract Rate: £550-£600 per day Status: Outside IR35 Clearance: Active MOD DV clearance required Working Pattern: Option for condensed week (3 long days, paid for 5)