Best Egg

Lead Software Engineer II, AI Operations

Job Description

Best Egg is hiring a Lead Software Engineer II for AI Operations to design, ship, and operate production-grade LLM applications, agents, and automations across the business. You’ll own the end‑to‑end path from prototype to stable deployment—building RAG pipelines, instituting evals and guardrails, and driving cost/performance optimization. Our stack includes Python, Metaflow on Outerbounds, AWS (including Bedrock), OpenAI/ChatGPT, and Cursor; Databricks is being evaluated and available where it makes sense. Your work will accelerate delivery, reduce LLM unit costs, and improve output quality for use cases like agent assist, compliance automation, process automation, and QA—treating AI Ops as a force multiplier for the enterprise.  


Key Responsibilities
  • Build and ship LLM apps & agents: Deliver internal copilots and customer/agent-facing automations with clear SLAs, rollbacks, and observability from day one. 
  • Own RAG pipelines: Design ingestion, chunking, embeddings, indexing, hybrid search/rerank, and retrieval evaluation; track retriever quality via offline golden sets and online metrics. 
  • AWS Infrastructure & Orchestration: Design and implement scalable AWS architectures, including AWS AI features such as Bedrock, IAM, knowledge bases, secure secrets and policy enforcement, automated provisioning, and resource-usage governance as core platform capabilities.
  • Observability & SRE for AI: Add tracing, prompt/agent version lineage, eval dashboards, and regression alerts; establish golden datasets and canary tests. 
  • Guardrails & governance: Enforce PII redaction, safety filters, role-based access, audit logs, and human‑in‑the‑loop review paths to control quality and risk. 
  • CI/CD for AI artifacts: Version and deploy prompts, tools, agents, and retrieval pipelines; support blue/green and shadow deploys with automatic rollback triggers. 
  • Cost & performance: Cut run‑rate spend through caching, truncation, batching, autoscaling, and model routing; establish clear unit economics per workflow. 
  • Developer enablement: Provide templates, SDKs, and high‑quality abstractions that let product teams ship safely without bespoke plumbing; improve developer experience. 
  • Platform integration: Build primarily in Python and Metaflow (Outerbounds); deploy on AWS (Bedrock + core services) and OpenAI; use Cursor in daily workflows; help evaluate and, when appropriate, run on Databricks.
  • Production posture: Participate in on‑call, author runbooks, and remove single‑thread risk for AI services; drive reliability and resilience akin to ML Ops.  

    • What You’ll Need to Succeed:
  • Experience: 5–10 years of professional software engineering (or equivalent) with 2+ years building AI/LLM applications; portfolio of shipped AI projects (links to code, demos, or case studies).
  • Exploration: Demonstrated passion for relentless exploration of the latest AI models, frameworks, and tooling, ensuring constant adoption of state-of-the-art innovations in the workflow.
  • LLM product engineering: Hands‑on with some/all of OpenAI, Bedrock, Huggingface/Ollama/vLLM; MCP servers and function/tool calling, multi‑turn orchestration, streaming, and prompt/version management.
  • RAG expertise: Practical experience designing and tuning retrieval systems (chunking, embeddings, hybrid search, reranking), integration with vector database, and measuring retrieval quality.
  • Full‑stack or equivalent backend depth: Comfortable building APIs/services and simple UIs where needed; strong fundamentals in Python and modern packaging/testing.
  • DevOps & deployment: CI/CD, containers, cloud fundamentals (AWS), and runtime performance tuning; experience operating services in production.
  • Platform & orchestration: Metaflow (Outerbounds) preferred; Databricks familiarity is a plus; ability to integrate data/feature pipelines and schedule/operate flows.
  • Observability & testing for AI: Tracing and logging, expertise in tools like Datadog, Dynatrace or Grafana where relevant for AI monitoring is essential.
  • Cost, quality, and risk mindset: Comfortable optimizing latency/throughput/cost, and implementing guardrails for PII/safety/compliance. 
  • Collaboration & mentorship: Partner effectively with data scientists, analysts, and engineers; promote best practices and high‑leverage abstractions. 
  • Bonus points: Fine‑tuning or distillation experience; Kubernetes or FastAPI exposure; familiarity with Snowflake or similar warehousing for retrieval sources.  

    • Apply Now
    Job Summary

    Solutions Architect, Blue Yonder WMS Required

    Remote role, anywhere in the US Open Sky Group is a dynamic and innovative global supply chain solution and consulting firm, strategically located near Raleigh, North Car

    • USA Only
    • yesterday

    Solution Architect

    We are seeking a Solution Architect to design and drive scalable, cloud-native solutions that support fleet optimization and operational efficiency across a large-scale v

    • Portugal Only
    • 7 hours ago

    Tech Lead (Remote First)

    We’re hiring three Tech Leads to support our Engineering teams across Business Acquisition & Tooling (BAT), Accounts Tooling & Automation (ATAT), and Renewals & Pricing (

    • Lead
    • Canada Only
    • yesterday

    Staff Software Engineer (Libraries- Platform)

    Staff Software Engineer, (Libraries Platform) The role: At Chainguard, we think the best platform work is invisible: the libraries just appear, the builds just work, an

    • United Kingdom Only
    • 2 days ago