Post a Job
Staff Software Engineer, Batch Compute
- Tecton
-
Verified
Job Description
We are building Rift (talk, blog post) - a new fully managed compute environment that allows data scientists to construct powerful batch and streaming pipelines in Python. Our new environment leverages popular open-source technologies such as Ray, Arrow, and DuckDB. We also have deep integrations with Spark platforms (Databricks, EMR, Dataproc) and data warehouses (e.g. Snowflake, BigQuery, RedShift), along with performant training data pipelines and a workload orchestration platform.
As a staff-level engineer on the Batch Compute team, you’ll play a critical role in architecting, designing, and scaling the core compute engines and storage architecture used by every Tecton customer. You'll contribute to the performance of our query optimizer, from parsing & optimization to plan selection. Think of this team as the “beating heart” of Tecton.
This role is a unique opportunity that combines customer-obsessed product focus with platform and data engineering innovation and helps companies accelerate their path to real-time AI. You will be working in one or more of the following areas to build the next generation of Tecton infrastructure:
- Distributed compute and resource management
- Query optimization and distributed execution
- Cross-platform integrations with state-of-the-art data platforms
Responsibilities
Qualifications
