Staff Software Engineer - Big Data (Gen AI)-9260

Job Description

Position: Staff Software Engineer - Big Data (Gen AI)
Experience: 8 to 13 Years
Location: Remote/Bangalore
 
Job Responsibilities:
 
·        Be a thought leader and forward thinker, help drive an innovative vision for our various products and platforms, design and launch strategic machine learning (ML) solutions and drive business-wide innovation.
·        Take the lead in the end-to-end software development lifecycle, encompassing design, testing, deployment, and operations, lead technical discussions and strategy, and participate hands-on in design reviews, code reviews, and implementation.
·        Craft high-performance, Big Data Lakehouse architectures such as Hudi, Delta or Iceberg.
·        Develop Big Data Platform to solve big data pipeline and processing.
·        Mentor and develop other engineers on the team, establish technical direction and foster team culture.
·        Uphold the highest standards of technical rigor in engineering and operational excellence, build highly resilient and scalable systems, and champion operational and process improvements.
 
Basic Qualifications:
 
·        Degree in mathematics/computer science or related discipline.
·        5+ years of experience in the complete software development lifecycle including design, coding, code reviews, testing, build processes, deployments, and operations.
·        4+ years of experience in Spark/PySpark with an in-depth knowledge of its advanced features and libraries.
·        3+ years of experience in Data Lakehouse solution such as Hudi/Iceberg/Delta with an in-depth knowledge of its advanced features and libraries.
·        2+ years of experience in leading the design and architecture of large distributed systems preferably on cloud platforms (e.g., AWS, Azure, Google Cloud).
·        Proficient in Docker, Kubernetes, and modern CI/CD practices.
 
Preferred Qualifications:
 
·        MS or PhD in Computer Science or equivalent experience in ML.
·        Experience working with Big Data Infra such as AWS EMR.
·        Experience with NoSQL and document databases.
·        Proven ability to handle big data, optimize workflows, and improve system performance.
 
Come work with a team of highly talented engineers, and advance with us to achieve new heights every day!