Module Lead – Data Platform (Databricks, AWS/Azure)

About the role

We are seeking a hands‑on Module Lead to own a module of our data platform built on Databricks, running on AWS or Azure. The role involves daily coding, guiding a small engineering team, and ensuring timely, high‑quality delivery.

Key responsibilities

• Design, build and deliver the assigned module on Databricks, ensuring reliability, performance and cost efficiency.
• Break work into clear tasks, provide estimates, track progress and meet delivery deadlines.
• Review code, enforce coding and testing standards, and implement data quality checks, monitoring and alerts.
• Guide and support engineers, collaborate with Lead Engineer and architects, and communicate status, risks and dependencies.

Required profile

• 8‑10 years of software or data engineering experience, with at least 2 years leading a module or small team.
• Strong hands‑on experience with Databricks (Spark/PySpark, Delta Lake, Workflows, Unity Catalog).
• Experience with AWS (S3, Glue, Lambda, IAM, Redshift) or Azure (Data Factory, ADLS Gen2, Synapse, Key Vault).
• Proficient in SQL and Python for building batch and real‑time data pipelines.

Required skills

• Databricks, Spark, PySpark, Delta Lake, Unity Catalog
• AWS services (S3, Glue, Lambda, IAM, Redshift) or Azure services (Data Factory, ADLS Gen2, Synapse, Key Vault)
• SQL, Python
• CI/CD, Git‑based releases