Sword Health

Senior Data Engineer - Data Platform

Job Description

Architect a scalable lakehouse by migrating workloads to Apache Iceberg. You’ll develop a Jobs API, build Spark/Flink pipelines, and establish data contracts. Requires proficiency in Python and Kafka with a platform-first, collaborative mindset.

At Sword Health, data is at the core of our mission to build a pain-free world. Our Data Team plays a central role in enabling a truly data-driven organization, ensuring that every decision is guided by reliable, actionable insights that directly impact millions of lives worldwide.


What you’ll be doing:
  • Spearhead the migration of existing workloads to the Iceberg format, establishing and maturing the foundational lakehouse architecture.
  • Architect and construct robust batch and streaming data pipelines utilizing Spark and Flink technologies.
  • Collaborate closely with the Backend Engineering team on API integrations and the establishment of formal data contracts.
  • Contribute to the development of a unified lineage and governance framework utilizing DataHub.
  • Provide comprehensive support to the Core Team in the successful adoption of new data platform capabilities.

  • What you need to have:
  • Demonstrated proficiency with Python and PySpark.
  • Hands-on experience with data lake formats (e.g., Iceberg, Delta Lake, or Hudi).
  • Solid understanding of Kafka and event-driven architectures.
  • Experience in building and orchestrating data pipelines at scale.
  • Strong SQL proficiency and comprehensive data modeling knowledge.
  • Familiarity with workflow orchestration tools (e.g., Airflow, Dagster, or similar).
  • Platform-oriented mindset: developing solutions for broad organizational use, not solely individual purposes.
  • Ownership mentality: committed to seeing problems through to resolution.
  • Clear communication skills: ability to articulate complex technical concepts to non-technical stakeholders.
  • Highly collaborative: excels in working alongside backend engineers, data engineers, and analysts.
  • Pragmatic approach: adept at balancing ideal solutions with practical delivery timelines.
  • Bonus: Demonstrated expertise with Flink or comparable streaming frameworks.
  • Bonus: Proficiency in DBT and familiarity with the modern data stack.
  • Bonus: Experience with modern data platforms such as BigQuery, Trino, Snowflake, or Databricks.
  • Bonus: Proven background in developing self-service data platforms.

  • To ensure you feel good solving a big Human problem, we offer:
  • A stimulating, fast-paced environment with lots of room for creativity.
  • A bright future at a promising high-tech startup company.
  • Career development and growth, with a competitive salary.
  • The opportunity to work with a talented team and to add real value to an innovative solution with the potential to change the future of healthcare.
  • A flexible environment where you can control your hours (remotely) with unlimited vacation.
  • Access to our health and well-being program (digital therapist sessions).
  • Remote or Hybrid work policy.
  • To get to know more about our Tech Stack, check here.