Lead Data Engineer

About the Role

HackerRank's data platform is entering an exciting new phase after a successful multi-year modernization, migrating from Redshift to StarRocks + Apache Hudi and dramatically reducing export latencies. As the Lead Data Engineer, you will be a senior individual contributor responsible for owning complex platform decisions, collaborating cross-functionally with AI, Product, and Go-to-Market teams, and delivering data-driven features that directly drive revenue. This is a unique opportunity to shape the future of data at HackerRank.

What You Will Do

• Own and enhance the data platform, ensuring its performance, reliability, and security at scale, utilizing technologies like StarRocks (OLAP), Apache Hudi (Data Lake), Trino, Spark, and Apache Ranger.
• Construct the next-generation AI-optimized data layer, creating clean, structured datasets to power natural language querying and AI add-on features for HackerRank for Work customers.
• Take ownership of in-product data features, including exports, insights dashboards, interview analytics, and the self-serve Custom Reports interface.
• Enable self-service data pipelines for internal teams (AI Platform, Analytics, Go-to-Market), reducing ad-hoc requests and scaling data access organization-wide.
• Implement robust data security measures, including access controls, Apache Ranger policies, and confidence-scoring guardrails for AI-generated outputs.
• Lead technical design reviews and establish engineering standards for the data team.
• Partner with Product Managers and business stakeholders to proactively identify and scope AI-enabled data use cases.

Who You Are

• 6+ years of data engineering experience, with at least 2 years in a senior or lead capacity.
• Deep hands-on expertise with OLAP databases such as StarRocks, ClickHouse, Druid, or similar.
• Strong experience with data lake technologies like Apache Hudi, Iceberg, or Delta Lake.
• Proficiency with distributed query engines (Trino / Presto) and batch/streaming compute using Apache Spark.
• Solid understanding of data security, RBAC, and access control tools like Apache Ranger.
• Comfortable operating in a hybrid AWS and open-source, self-managed environment.
• Excellent communication skills, capable of translating technical decisions for non-technical stakeholders and independently driving cross-functional projects.

Even Better If You Have

• Hands-on experience with AI/LLM-adjacent data work, including confidence scoring, agentic pipelines, RAG architectures, or vector stores.
• Prior exposure to agentic workflows and experience operationalizing emerging AI concepts at production scale.
• Experience scaling data infrastructure within a SaaS or B2B product company.
• Familiarity with natural language querying interfaces or building data products for end-customer consumption.

You Will Thrive in This Role If

• You are energized by working on a platform that is both technically mature and has significant greenfield potential.
• You proactively connect data capabilities to business outcomes without waiting for a roadmap.
• You care deeply about how other teams utilize data, not just about the pipelines that produce it.
• You possess genuine curiosity about AI and a desire to be at the intersection of data and intelligence.
• You excel in lean, cross-functional environments where your decisions have a visible, company-wide impact.

About HackerRank

HackerRank empowers companies like NVIDIA, Amazon, and Microsoft to hire and upskill the next generation of developers based on skills, not pedigree. Their platform is trusted by over 2,500 innovative companies to build strong engineering teams. Software is evolving into an era of human-AI collaboration, and HackerRank is defining the future of technical talent assessment. People at HackerRank are passionate about their work's impact and are dedicated to customer success.