Senior Data Engineer

About the Job

Onapsis is dedicated to eliminating blind spots in cybersecurity for the world's most critical business applications. We provide cybersecurity solutions for on-premises, cloud, and hybrid environments, protecting the SAP and Oracle landscapes of nearly 30% of the Forbes Global 100. We are seeking a Senior Data Engineer to join our team and contribute to high-impact cybersecurity solutions.

Your Legacy: What You Will Be Doing

You will work directly with Principal Engineers to evaluate, scope, propose, and build features that fulfill business solution requirements for customer protection. You will play a direct role in establishing the technical foundation for a new product offering. Additionally, you will collaborate with Engineering and DevOps to deliver high-quality products and services, ensuring adherence to security best practices with security and IT professionals.

Responsibilities

• Architect and Design Scalable Data Solutions: Design, develop, and maintain Data Lakehouse solutions (Iceberg/Delta Lake/Hudi), applying industry best practices and optimizing data structure for various access patterns.
• Data Pipeline Development: Implement ETL/ELT pipelines using cloud technologies (Spark/pySpark/Glue, Kinesis Streams/Iceberg) to load data into a Lakehouse for efficient ML processing and UI reporting.
• Data Modeling and Processing: Implement data models and processing frameworks (Spark, Kafka, Snowflake) to ingest, transform, and load large datasets into Data Lakehouse technologies (Apache Iceberg, Apache Delta Lake, or Apache Hudi), ensuring high availability and reliability.
• Advanced Data Integration: Develop solutions to integrate multiple data sources into Snowflake or similar data warehouses for real-time analytics and reporting.
• AI/ML Integration: Collaborate with cross-functional teams to co-develop AI-driven features, identifying patterns and anomalies in client data using AI/ML technologies (Python).
• Compliance and Security: Ensure compliance with industry standards and secure best practices (SOX, SOC 1/2) by implementing data governance frameworks, monitoring pipelines, and optimizing cloud database architectures.
• Stakeholder Collaboration: Work closely with stakeholders (analysts, engineers, product managers) to understand data needs, propose solutions, and drive data-driven decision-making.
• Data Infrastructure Monitoring: Continuously monitor, troubleshoot, and enhance data pipelines using CI/CD tools (Docker, Jenkins, GitHub Actions) and orchestrate workflows with Apache Airflow.
• Leadership and Mentorship: Provide hands-on mentorship and technical guidance to junior engineers, including code reviews and architecture discussions.
• Documentation and Governance: Establish comprehensive documentation for data architecture, governance, and processes.

Qualifications

• 5+ years of proven experience as a Data Engineer or in a similar role, with a deep understanding of data architecture and cloud-based ETL/ELT frameworks.
• Strong experience with AWS (preferred) or Azure, specifically with Glue, EMR, S3, Lambda. Databricks and Snowflake experience is a bonus.
• Proficiency in big data technologies such as Apache Spark, Kafka, Hadoop, and Databricks.
• Proficiency with Python libraries for data processing and ML (e.g., Pandas, NumPy, Polars, Scikit-learn, PyTorch, TensorFlow).
• Hands-on experience building real-time data processing and AI/ML-driven analytics solutions (SageMaker, Bedrock, NLP, Power BI).
• Ability to architect and manage data lakehouse solutions (Iceberg/Delta Lake/Hudi) or classic warehouse solutions (Redshift, Snowflake).
• Familiarity with compliance and audit requirements (SOX, SOC 1/2, GDPR) and implementing data governance and security frameworks.
• Strong problem-solving skills focused on data integrity, scalability, and performance optimization.
• Experience with CI/CD tools (Jenkins, GitHub Actions, Docker) and data orchestration platforms (Apache Airflow).

Preferred Qualifications

• Experience with advanced data architecture principles (medallion architecture, materialized views, task scheduling).
• Experience using BI tools (e.g., Power BI, Tableau) for real-time analytics and operational reporting.

What We Offer

• A role in shaping the future of protecting critical business applications and a career that grows with the company.
• A unique culture of high achievement and teamwork.
• Supportive and humble colleagues who are top problem solvers and innovators.
• Financial security through competitive compensation and incentives.

Location

Heidelberg. This is a hybrid role, so candidates must be commutable to Heidelberg.