Lead Software Engineer - DevOps

Experience: 10+ Years

Location: Bangalore

Role: Full-time / Technical Lead

Role Summary

We are seeking a highly capable Technical Lead with deep expertise in Big Data (Spark, Hadoop, Hive), Kubernetes platforms, Azure cloud, PostgreSQL, CI/CD (GitHub Actions/Jenkins), and DevOps automation. The ideal candidate will be a strong hands-on engineer who also brings mature leadership capabilities, sound architectural judgment, and exceptional software craftsmanship practices.

Key Responsibilities

1. Software Engineering & Craftsmanship

• Develop high quality, maintainable code using Software Craftsmanship principles such as:

  • Test-Driven Development (TDD)
  • Continuous Integration / Continuous Delivery / Continuous Deployment
  • Clean code, refactoring, SOLID principles

• Participate actively in design reviews and code reviews to ensure engineering excellence.

• Implement quick POCs using the latest tech stack to validate ideas and technical feasibility.

• Apply design patterns and engineering best practices for scalable and maintainable solutions.

• Collaborate in all phases of the SDLC, with strong understanding of Agile/Scrum or continuous delivery environments.

2. Big Data Engineering & Distributed Systems

• Lead development of large-scale ETL/ELT pipelines using Apache Spark (Java/Scala).

• Oversee data processing frameworks in Hadoop and Hive, especially within Azure-based ecosystems.

• Optimize Spark/Hive workloads for cost, performance, and reliability.

3. Cloud, Kubernetes & Infrastructure Engineering

• Lead architecture and operations of distributed applications running on:

  • Kubernetes (On-Prem + AKS)
  • Docker

• Drive adoption of Helm, container standards, and cluster best practices.

• Build and maintain Azure infrastructure using Terraform (IaC) with reusable, scalable modules.

• Ensure end-to-end observability of clusters using Prometheus, ELK, APM tools, and custom Python scripts.

4. DevOps Engineering & CI/CD Leadership

• Define and own the platform CI/CD strategy using:

  • GitHub Actions
  • Jenkins
  • Terraform automation

• Build standardized CI/CD frameworks for 50+ microservices, leveraging Helm and automated build workflows.

• Automate DNS creation, certificate management, APM integration, user management, Vault policy creation, and repository migrations (200+ repos).

• Champion an automation-first mindset across the engineering and SRE teams.

• Coordinate with developers and testers for smooth deployment and production readiness.

5. Database Engineering

• Provide leadership on PostgreSQL cluster management:

  • HA setup
  • Performance tuning
  • Failover mechanisms
  • Monitoring dashboards

• Automate PostgreSQL operations using Python and infrastructure scripts.

6. Observability, Stability & Support

• Ensure reliability and availability of services across stacks (Java, Angular, .NET).

• Implement monitoring and alerting using Prometheus, Grafana, ELK, Elastic APM.

• Oversee production stability through proactive dashboards and automated health checks.

• Provide guidance for L2/L3 support and participate in incident review and resolution.

7. Leadership & Collaboration

• Lead a cross-functional team of developers, DevOps engineers, and data engineers.

• Translate business requirements into technical designs and actionable backlogs.

• Mentor team members on coding practices, DevOps processes, and cloud-native patterns.

• Collaborate with distributed teams and communicate effectively with technical and non-technical stakeholders.

Required Skills & Qualifications

• 10+ years of experience in software engineering / data engineering / DevOps roles.

• Strong expertise in:

  • Apache Spark (Java/Scala)
  • Hadoop, Hive on Azure
  • Kubernetes, Docker
  • Azure Kubernetes Service (AKS)
  • Ansible
  • Jenkins & GitHub Actions
  • PostgreSQL (HA + automation)

• Strong understanding of CI/CD, SDLC, Agile, and continuous delivery.

• Good understanding of networking fundamentals, DNS, LBs, ingress controllers.

• Experience working on production support and high-availability systems.

Soft Skills

• Strong leadership and mentoring capabilities.

• Excellent communication skills with distributed teams.

• Ability to analyze complex problems and deliver quick, practical solutions.

• High sense of ownership and accountability.

Nice-to-Have

• Experience with Azure Data Lake, Databricks, or Synapse.

• Exposure to microservices architecture and domain-driven design.