Lead Assistant Manager

Key Responsibilities

• Monitor and maintain health of production AI models (GenAI and traditional ML).
• Troubleshoot data/model/infra issues across model pipelines, APIs, embeddings, and prompt systems.
• Collaborate with Engineering and Data Science teams to deploy new versions and manage rollback if needed.
• Implement automated logging, alerting, and retraining pipelines.
• Handle prompt performance drift, input/output anomalies, latency issues, and quality regressions.
• Analyze feedback and real-world performance to propose model or prompt enhancements.
• Conduct A/B testing, manage baseline versioning and monitor model outputs over time.
• Document runbooks, RCA reports, model lineage and operational dashboards.
• Support GenAI adoption by assisting in evaluations, hallucination detection, and prompt optimization.
  

Must-have Skills

• 1+ year of experience in Data Science, ML, or MLOps.
• Good grasp of ML lifecycle, model versioning, and basic monitoring principles.
• Strong Python skills with exposure to ML frameworks (scikit-learn, pandas, etc.).
• Basic familiarity with LLMs and interest in GenAI (OpenAI, Claude, etc.).
• Exposure to AWS/GCP/Azure or any MLOps tooling.
• Comfortable reading logs, parsing metrics, and triaging issues across the stack.
• Eagerness to work in a production support environment with proactive ownership.
  

Nice-to-Have Skills

• Prompt engineering knowledge (system prompts, temperature, tokens, etc.).
• Hands-on with vector stores, embedding models, or LangChain/LlamaIndex.
• Experience with tools like MLflow, Prometheus, Grafana, Datadog, or equivalent.
• Basic understanding of retrieval pipelines or RAG architectures.
• Familiarity with CI/CD and containerization (Docker, GitHub Actions).
  

Ideal Candidate Profile

• A strong starter who wants to go beyond notebooks and see AI in action.
• Obsessed with observability, explainability, and zero-downtime AI.
• Wants to build a foundation in GenAI while leveraging their traditional ML skills.
• A great communicator who enjoys cross-functional collaboration.