Ready to keep things running smoothly? Join our tombola team!
At tombola, we pride ourselves on building our own exceptional games and platforms in-house. That means keeping everything running flawlessly is paramount! We’re seeking a Lead Site Reliability Engineer (SRE) to join us and help ensure our critical systems and services are always reliable, available, and performing at their best.
What will you be doing?
As an SRE, you’ll be instrumental in implementing automation, monitoring, and incident response strategies to minimize downtime and optimize our operations. You’ll collaborate closely with our development, infrastructure, and security teams, balancing exciting new feature delivery with rock-solid system stability.
Key Accountabilities and Responsibilities:
Team Leadership and Development
Providing leadership, management, and development for direct reports through effective 1-to-1s, objective setting (OKRs), and performance management.
Making team goals clear and ensuring they align with our broader business objectives.
Collaborating with other teams and departments to achieve shared success.
Partnering with our People Partner for tech to build robust team management practices.
System Reliability and Availability
Ensure system uptime: Monitor and maintain the availability and reliability of critical systems and services, meeting all uptime SLAs (Service Level Agreements).
Incident management: Quickly respond to incidents, investigate root causes, and ensure effective postmortems and continuous improvement processes are in place.
Failure detection and response: Proactively identify potential failures or performance bottlenecks before they impact users, and respond to failures and outages effectively.
Monitoring and Alerting
Implement monitoring systems: Set up and maintain robust monitoring systems (e.g., Dynatrace) for application performance, infrastructure health, and system metrics.
Alerting: Create and manage alerting systems to notify us about issues or potential risks in a timely manner, minimizing impact on our players.
Metrics collection: Define and track key metrics (e.g., uptime, latency, request rates) to measure system health and performance.
Incident Response
Incident resolution: Work quickly to resolve incidents, minimize downtime, and restore service as fast as possible.
Post-incident analysis: After resolving incidents, perform root cause analysis (RCS), including a post-incident review, and document findings to prevent similar issues in the future.
Automation and Efficiency
Automate manual tasks: Automate repetitive operational tasks to boost efficiency, reduce human errors, and accelerate delivery.
Infrastructure automation: Utilise Terraform, Git, and TeamCity to automate infrastructure provisioning and configuration management.
Deployment pipelines: Help develop and maintain automated deployment pipelines (e.g., CI/CD) to streamline releases and reduce manual intervention.
Capacity Planning and Scaling
Plan for scalability: Ensure our systems can scale efficiently to meet demand, both horizontally (adding more servers) and vertically (increasing server resources).
Optimize resource usage: Monitor and optimize resource usage (CPU, memory, storage) to ensure the infrastructure handles increasing loads without waste.
Forecasting and capacity planning: Analyze usage trends and plan for future capacity needs to ensure systems remain reliable and perform under varying workloads.
Performance Optimization
Performance tuning: Continuously analyze system performance and make optimizations (e.g., tuning databases, improving response times, optimizing caching).
Load testing and stress testing: Conduct load and stress tests to ensure systems can handle high traffic and performance demands without failure.
Infrastructure Management
Cloud infrastructure management: Manage AWS cloud resources and ensure proper scaling, cost optimization, resilience and environment consistency.
Disaster recovery planning: Develop and maintain disaster recovery strategies to ensure quick recovery in case of catastrophic failures.
Collaboration with Development Teams
Work with development teams: Collaborate with development teams to ensure that new features and applications are designed with reliability, scalability, and operational needs in mind.
Service ownership: Promote a culture of service ownership where developers are involved in operational aspects, ensuring they are accountable for the operational success of their services.
Performance and reliability feedback: Provide feedback to developers about performance and reliability issues encountered in production and recommend improvements.