Data Engineer II

Job Description

About Mediavine

Mediavine is a fast-growing advertising management company representing over 10,000 websites in the food, lifestyle, DIY, and entertainment space. Founded by content creators, for content creators, Mediavine is a Top 20 Comscore property, exclusively reaching over 125 million monthly unique visitors. With best-in-class technology and a commitment to traffic quality and brand safety, we ensure optimal performance for our creators.

About This Role

The Data & Analytics team consists of data analysts, data engineers and analytics engineers working to build the most effective platform and tools to help uncover opportunities and make decisions with data here at Mediavine. We partner with Product, Support, Ad Operations and other teams within the Engineering department to understand behavior, develop accurate predictors and build solutions that provide the best internal and external experience possible.

A Data Engineer at Mediavine will help build and maintain our data infrastructure. Building scalable data pipelines, managing transformation processes, and ensuring data quality and security at all steps along the way. This will include writing and maintaining code in Python and SQL, developing on AWS, and selecting and using third-party tools like Rundeck, Metabase, and others to round out the environment. You will be involved in decisions around tool selection and coding standards.

Our current data engineering toolkit consists of custom Python data pipelines, AWS infrastructure including Kinesis pipelines, Rundeck scheduling converting to Prefect orchestration, dbt for transformation and Snowflake as our data warehouse platform. We are open to new tools and expect this position to be a part of deciding the direction we take. 

Essential Responsibilities

• Create data pipelines that make data available for analytic and application use cases
• Develop self-healing, resilient processes that do not require constant care and feeding to run smoothly
• Create meaningful data quality notifications with clear actions for interested parties including other internal teams and other members of the data and analytics team
• Leading projects from a technical standpoint, creating project Technical Design Documents
•  Support data analysts and analytics engineers ability to meet the needs of the organization
• Participate in code reviews, understanding coding standards, ensuring test coverage and being aware of best practices
• Build or implement tooling around data quality, governance and lineage, in the dbt framework and Snowflake but external to that as needed
• Provide next level support when data issues are discovered and communicated by the data analysts
• Work with data analysts and analytics engineers to standardize transformation logic in the dbt layer for consistency and ease of exploration by end users
• Enable analytics engineers and data analysts by providing data modeling guidance, query optimization and aggregation advice

Requirements

• 3+ years of experience in a data engineering role
• Strong Python skills (Understands tradeoffs, optimization, etc)
• Strong SQL skills (CTEs, window functions, optimization)
• Experience working in cloud environments (AWS preferred, GCS, Azure)
• Experience managing complex dbt environments with hundreds or more flows
• An understanding of how to best structure data to enable internal and external facing analytics
• Familiarity with calling APIs to retrieve data (Authentication flows, filters, limits, pagination)
• Experience working with DevOps to deploy, scale and monitor data infrastructure
• Scheduler experience either traditional or DAG based
• Experience using LM-powered tools for code generation, documentation, and architectural diagramming
• Comfortable working with multi-TB cloud data warehouses (Snowflake preferred, Redshift, Big Query)
• Experience with other DBMS systems (Postgres in particular)
• Ability to travel up to approx 15%

Nice to haves:

• Experience with web analysis such as creating data structures that support product funnels, user behavior, and decision path analysis 
• Understanding of Snowflake external stages, file formats and snowpipe
• Experience managing the semantic layer in either dbt or Snowflake
• Experience with orchestration tools particularly across different technologies and stacks
• Knowledge of Ad Tech, Google Ad Manager and all of its fun quirks (so fun)
• The ability to make your teammates laugh (it wouldn’t hurt if you were fun to work with is what I’m saying)
• Familiarity with event tracking systems (Snowplow, etc)
• Experience with one or more major BI tools (Omni, Sigma, Metabase, etc.)

Benefits

• 100% remote
• Comprehensive benefits including Medical, Dental, Vision, Disability, and Life Insurance
• 401(k) with company matching
• Generous PTO
• Wellness initiatives and employer-sponsored mental health resources
• Professional development opportunities
• Inclusive, collaborative, and entrepreneurial company culture