⨠AI Insights & Summary
Trase Systems is at the cutting edge of AI Agent innovation, even outperforming industry giants on leaderboards. This Staff Software Engineer role is a company-critical position, offering a unique opportunity to define the core execution model and platform architecture for Trase OS, the "agentic operating system" powering enterprise AI deployments. If you thrive on designing robust, scalable, and secure distributed systems where failure is a normal condition and correctness under pressure is paramount, this role offers the chance to build foundational primitives for mission-critical applications.
About Trase Systems
Co-founded in 2023 by Joe Laws and Grant Verstandig, Trase Systems is dedicated to making AI uncomplicated for enterprises. Our end-to-end platform empowers leaders to deploy, manage, and optimize AI, focusing on bridging the "last mile" of adoption to unlock AI's full potential while driving efficiency and cost savings. We are recognized leaders in AI Agent innovation, topping the Hugging Face GAIA Leaderboard.
About The Role
As a Staff Software Engineer, you will own the core execution model and platform architecture of Trase OS, the foundational "agentic operating system" for Trase deployments in regulated environments. Your responsibilities will include defining abstractions and APIs that connect workflows, agents, tools, and product surfaces, ensuring the system's correctness, scalability, and extensibility. This is a company-critical role where your work will set the technical direction for the platform, acting as a force multiplier across all engineering teams. The focus on clean abstractions and correctness-under-failure is crucial due to operating long-lived agents in sensitive sectors like healthcare and defense, where auditability and reliability are non-negotiable.
Why This Role Is Needed
Trase OS is an orchestration-heavy system coordinating complex, long-lived workflows and agents. As the platform evolves, the primary risks shift to system design quality. This role is essential to:
- Define durable abstractions for the platform execution model.
- Ensure correctness and determinism in workflow execution, especially under failure.
- Translate evolving product requirements into a coherent platform architecture.
- Enable teams to build on Trase OS without introducing systemic complexity.
What Makes This Role Hard
- Designing systems where failure is anticipated, and correctness must be maintained across retries, restarts, and partial executions.
- Balancing elegant abstractions with real-world constraints like performance, security, and multi-tenancy.
- Making foundational decisions that impact all products and teams.
- Ensuring the system remains understandable and auditable despite increasing complexity and scale.
Responsibilities
- Develop the core execution model, including state machine, lifecycle, resource model, and failure semantics.
- Design platform APIs/SDKs for connecting workflows, agents, tools, and product surfaces, managing versioning and compatibility.
- Guarantee correctness through idempotency, deterministic replays, compensating actions, and data integrity.
- Engineer reliability at scale with controls for concurrency, rate limiting, backpressure, sharding, and workload isolation.
- Integrate security and governance features like RBAC/ABAC, policy enforcement, and fine-grained audit trails.
- Deliver comprehensive observability, including distributed tracing, structured logging, metrics, and evaluation hooks.
- Own quality through design reviews, test strategies (unit, property, chaos), performance baselines, SLOs, incident response, and postmortems.
- Mentor senior engineers and collaborate with Product, Security, and Customer teams.
- Make pragmatic choices for storage, queueing, and compute, creating efficient "paved roads" for other teams.
- Define system boundaries and reduce cross-service coupling.
- Drive platform-wide standards for correctness, reliability, and API design.
- Balance short-term delivery with long-term architectural integrity.
Requirements
- 10+ years of experience building distributed/platform systems, with significant experience defining architecture across teams.
- Experience building mission-critical runtimes or workflow/orchestration systems.
- Deep expertise in durable execution (state machines, event sourcing, saga/compensation, idempotency, at-least-once semantics).
- Proven track record with security & governance in production systems (auth, RBAC, audit, policy).
- Hands-on experience with observability tools (Grafana or equivalent), including trace correlation.
- Strong systems design skills across storage, queues, schedulers, and evented architectures; performance tuning experience.
- Proficiency in a modern language (Go, Rust, Java, TypeScript) and cloud-native stacks (containers, CI/CD, IaC).
- Comfort operating in regulated or high-assurance environments; bias toward correctness, clarity, and documentation.
- Proven ability to influence technical direction and drive adoption of architectural standards.
- Ability to incorporate advanced LLM capabilities into system design and architecture.
Nice to Have
- Prior work on workflow engines (Temporal/Cadence, Argo, Airflow) or serverless runtimes.
- Experience with policy engines (OPA), secrets/KMS, or data handling controls (PII/PHI).
- ML/LLM evaluation frameworks, tool/plugin architectures, or embedding model governance.
- Government or healthcare experience (HIPAA, audit readiness) and multi-tenant isolation.
Salary & Benefits
- Salary Range: $180,000 - $245,000
- Benefits:
- Career track opportunity with potential for rapid advancement.
- 100% employer-paid comprehensive health care (medical, dental, vision) for you and your family.
- 14 weeks paid maternity and paternity leave at full pay.
- Unlimited PTO (with management approval).
- Professional development and continued learning opportunities.
- Optional 401K, FSA, and equity incentives.
- Mental health benefits through Tara Mind.
Equal Opportunity Employer
Trase Systems is an Equal Opportunity Employer. Consideration for employment is given without regard to race, sex, color, religion, sexual orientation, gender identity, national origin, protected veteran status, or disability.