✨ AI Insights & Summary
Prolific is at the forefront of the AI revolution, recognizing that the true differentiator in AI development lies in the quality and diversity of human-generated data. This Senior Data Engineer role is a pivotal opportunity to shape the company's data infrastructure, directly impacting how AI models are built and refined. You'll work in a modern, cloud-native environment, contributing to scalable data solutions that empower data scientists, AI engineers, and product teams, making this a prime position for engineers passionate about building the foundation for ethical and advanced AI.
About Prolific
Prolific is pioneering the human data infrastructure essential for advancing AI development. In an era where AI technologies are rapidly commoditizing, the distinctiveness of products and models hinges on the quality and diversity of human-generated data. We are building the platform that connects researchers and companies with a global participant pool, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is fundamental to developing more accurate, nuanced, and aligned AI systems. We believe the next significant leap in AI capabilities will stem from integrating diverse human perspectives and behaviors into its development. By providing this critical human data infrastructure, Prolific is positioned at the vanguard of the next wave of AI innovation—one that truly reflects the breadth and best of humanity.
The Role: Senior Data Engineer (6 Month Contract)
We are seeking a Senior Data Engineer to construct scalable data solutions supporting various teams across the business, including data scientists, analysts, AI engineers, and product teams. You will contribute to both the underlying platform and the application of data within our products, operating within a modern, cloud-native environment where privacy, security, and sound data practices are integral to our development process.
What You'll Bring to the Role
- Technical Expertise: 3+ years building and deploying production-grade data systems, with strong SQL skills and proficiency in Python or another object-oriented language (Java, Scala, etc.).
- Cloud-Native Familiarity: Exposure to Terraform, Kubernetes, or similar Infrastructure as Code (IaC) and container orchestration tools. While deep expertise isn't required, comfort in this environment is key.
- Data Services Experience: Experience designing data APIs or services that expose data to applications, or a strong interest in bridging the analytical/operational boundary.
- Data Governance: A thoughtful approach to data quality, privacy, and security, understanding the importance of how data is used alongside how it's moved.
- Collaboration: Strong collaboration skills and the ability to work effectively across teams with diverse priorities (engineers, data scientists, AI engineers, product).
- Mindset: A pragmatic, curious mindset, staying current with the field and knowing when to adopt new tools versus leveraging existing solutions.
- Pipeline Management: Hands-on experience with data pipeline tools (Airflow, dbt) and a strong ability to optimize for performance and reliability.
- Quality Focus: Commitment to continuously improving product quality, security, and performance through rigorous testing and code reviews.
- Documentation: Meticulous approach to creating and maintaining architecture and systems documentation.
- Collaborative Mindset: Ability to work across teams to understand and address diverse data needs while maintaining data integrity.
- Growth Orientation: Desire to continually stay updated with advancements in data engineering practices and technologies.
- Problem-Solving: Exceptional analytical skills to troubleshoot complex data issues and implement effective solutions.
- Independence: Capability to independently deliver medium-sized features while contributing to team objectives.
What You'll Be Doing in the Role
- Build and maintain data pipelines that support analytics, ML workloads, and product-facing applications, covering sources from internal databases and SaaS platforms to streaming systems.
- Evolve our data platform in conjunction with cloud platform engineers, utilizing infrastructure-as-code (Terraform) and Kubernetes-based deployments (Argo) to ensure scalability, reliability, and self-service capabilities.
- System Architecture: Design and implement scalable data infrastructure capable of handling our growing data volume and complexity.
- Develop data services and APIs to provide trusted data to product applications, effectively bridging analytical and operational systems.
- Own data quality and observability by implementing monitoring, testing, and alerting to ensure early issue detection and maintain high data trust.
- Partner with AI engineers, data scientists, analysts, and product teams to understand their data requirements and design optimal solutions.
- Uphold stringent data privacy, security, and compliance practices in all development efforts, particularly concerning data flow into product contexts.
- Technical Documentation: Create and maintain comprehensive documentation for data flows, models, and systems to facilitate knowledge sharing.
- Security and Compliance: Ensure all data systems comply with security best practices and regulatory requirements.
Why Prolific is a Great Place to Work
Working with us places you at the cutting edge of AI innovation, offering access to our unique human data platform and opportunities for groundbreaking research. Join us to enjoy a competitive salary, comprehensive benefits, and the flexibility of remote working within an impactful, mission-driven culture.
Links to More Information
- Benefits
- External Handbook
- Website
- Youtube
Privacy Statement
By submitting your application, you consent to Prolific collecting your personal data for recruitment and global organizational planning purposes. Prolific’s Candidate Privacy Notice details the personal information Prolific may process, the purposes for processing, and your rights regarding your personal information.