Job Description
Elevate your expertise to new heights at 3Pillar, the premier destination for trailblazing senior software engineers like yourself. Picture collaborating with industry leaders, spearheading transformative projects that redefine urban living, establishing new media channels for enterprise companies, or driving innovation in healthcare. Our mission is to revolutionize the tech landscape, and your skills are integral to achieving this ambitious goal.
Your role extends beyond the realm of code composition; you will architect transformative solutions that reshape entire industries. Join us and spearhead the charge toward a tech future that transcends the ordinary.
Key Responsibilities:
Understanding the business requirements and implementing the technical solution
Design, development, and maintenance of scalable data pipelines using Java and big data technologies
Creating data pipelines, versioning, and change management
Managing the complexity inherent in versioned data pipelines
Working with relational (SQL) and NoSQL databases, ensuring data integrity and performance
Development, maintenance, and troubleshooting of ETL/ELT processes for efficient data ingestion, transformation, and storage.
Automating and optimizing data workflows such as data ingestion, aggregation, and ETL processing
Designing, building, and maintaining batch or real-time data pipelines in production
Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures
Writing secure and scalable data exposing APIs for data consumers Training and mentoring of junior engineers
Executing complex activities within the current methodology and quality standards, showcasing success across diverse engagements
Promoting client success across the team by collaborating with engineers, designers, and managers to understand user pain points, anticipate potential problems, and iterate on solutions that drive client success
Minimum Qualifications:
Demonstrated expertise with a minimum of 5+ years of experience as a software engineerProficiency in Java 8+ and frameworks like Spring Boot with exposure to Kafka, SparkProficient in data pipeline and workflow management tools like AirflowAdvanced SQL skills and experience with relational and NoSQL databases like MySQL, and MongoDBExposure to working on Data Lakes & Data Warehouses solutionsExcellent problem-solving, communication, and organizational skillsProven ability to work independently and with a team.
Nice to have:Prior experience working with IOT devices would be an advantageExperience with Big Data technologies like Map Reduce, Hadoop, Hive, etc.Experience with data visualization tools like PowerBI, Tableau, AWS QuickSight, etc.Experience with any public cloud (AWS/Azure/GCP)