Design and implement scalable data platforms using Snowflake, Databricks, Delta Lake, and cloud technologies.
Build batch and real-time data pipelines using PySpark, Kafka, and Spark Structured Streaming.
Develop AI-ready data architectures supporting analytics, ML, LLMs, and RAG applications.
Design semantic models, data governance, metadata, and data lineage solutions.
Implement vector databases, embedding pipelines, and retrieval solutions for AI applications.
Build and manage ML/LLMOps pipelines, model deployment, monitoring, and CI/CD.
Ensure data security, RBAC, compliance, and governance across the platform.
Mentor engineering teams and define architecture best practices.
Required Skills
12+ years of experience in Data Engineering/Data Architecture.
Strong experience with **Snowflake, Databricks, PySpark, Kafka, Delta Lake, SQL, and Python**.
Hands-on experience with **AWS (S3, Glue, Redshift, Bedrock, Kinesis)** or Azure.
Experience with **LangChain, LlamaIndex, OpenAI/Bedrock, RAG, Vector Databases (Pinecone, ChromaDB, FAISS, OpenSearch)**.
Good understanding of ML/LLMOps, Data Governance, Data Lineage, and CI/CD.
Excellent communication and stakeholder management skills.