Data Engineer (data warehouse, Azure Data Factory, SQL)

Job Description

Job Title: Data Engineer (ETL Specialist)
Location: Remote
Employment Type: Full-Time

About Us:
DMV IT Service LLC is a trusted IT consulting firm, delivering tailored solutions to businesses and individuals since 2020. We specialize in optimizing IT infrastructure, providing expert guidance, and supporting workforce needs with top-tier staffing services. Our consulting expertise spans system administration, cybersecurity, networking, and IT operations, empowering clients to achieve their technology goals. With a client-focused approach, we also offer online training and job placements to build long-term IT success.

Job Overview:
We are seeking a skilled Data Engineer with extensive experience in ETL processes and data warehousing, particularly within the healthcare sector. The successful candidate will be responsible for developing and managing ETL pipelines, primarily using Azure Data Factory and Snowflake, to ensure efficient data transformation and storage that meets the needs of our end users.

Requirements

• Develop and manage effective working relationships with other departments and groups to coordinate ETL processes.
• Communicate efficiently with ETL architects to understand requirements and transform data accordingly.
• Assist in the overall architecture of the ETL design, providing proactive inputs and implementing automated ETL flows.
• Investigate data issues within ETL pipelines, notify end-users, and propose adequate solutions.
• Develop idempotent ETL processes using Azure Data Factory dataflows and pipelines.
• Work within Snowflake Virtual Warehouses and automate data pipelines using Snowpipe.
• Capture changes in data dimensions and maintain versions using Stream sets in Snowflake.
• Optimize data movement, including data at rest in the database, to enhance performance.
• Build and manage a highly efficient orchestrator for scheduling jobs, executing workflows, and performing data quality checks.
• Test ETL systems, perform root cause analysis, resolve production issues, and conduct routine database and data flow testing.
• Document implementations, test cases, and build deployment documents for CI/CD.

Required Skills:

• 5+ years of Data engineering experience with a focus on Data Warehousing.
• Extensive experience with ETL tools like Informatica PowerCenter, SSIS, and Azure Data Factory.
• Strong proficiency in SQL with various databases like Oracle, Snowflake, and SQL Server.
• Familiarity with version control systems like GitHub or SVN.
• Experience with healthcare data standards such as HL7 and FHIR.

Preferred Skills:

• Batch or PowerShell scripting experience.
• Python scripting capabilities.
• Data modeling experience in a data warehouse environment.
• Familiarity with Informatica Intelligent Cloud Services and API design in Snowflake and ADF.
• Azure certifications in data engineering or analytics.