Senior Data and AI Analyst

The ideal candidate for this role will be responsible for Business Process Analysis, which involves designing and maintaining analytics frameworks for business processes and financial workflows. You will develop dashboards and reports to provide real-time insights into key business metrics. Analyzing user interaction patterns with AI features to enhance adoption and effectiveness will be a crucial part of your role. Additionally, you will create and manage ETL pipelines to ensure data quality and accessibility, as well as generate actionable insights from complex financial datasets to inform product strategy. In terms of AI Platform Analysis, you will be tasked with developing and implementing frameworks to evaluate the performance of LLM-based systems and AI agents. This includes analyzing AI model outputs, response quality, and automation effectiveness. You will also track and report on key AI performance metrics such as accuracy, latency, and automation rates. Collaboration with ML engineers to identify areas for model improvements based on data insights and creating dashboards for monitoring AI system performance and reliability will be part of your responsibilities. A/B testing of different AI configurations and prompt strategies will also be conducted. Regarding the Strategic Impact aspect of the role, you will provide data-driven recommendations for enhancing both AI capabilities and business processes. Partnering with product teams to define and track success metrics for AI features and collaborating with customer success teams to analyze user feedback and AI system effectiveness will be essential. Building and maintaining documentation for AI performance metrics and analytics processes is another key responsibility. **Required Qualifications:** - 5+ years of experience in data analysis, with recent exposure to AI/ML systems - Proficiency in SQL and Python for data manipulation and analysis - Experience analyzing AI/ML system performance and metrics - Expertise in business intelligence tools (e.g., Tableau, Power BI, Looker) - Familiarity with large-scale financial datasets and ETL processes - Strong statistical analysis skills and experience with A/B testing - Experience with cloud-based data warehouses, preferably Snowflake - Bachelor's degree in Statistics, Mathematics, Computer Science, or a related field **Preferred Qualifications:** - Experience analyzing LLM-based systems and generative AI applications - Knowledge of NLP metrics and evaluation frameworks - Experience with financial systems and processes (AP, AR, Vendor Management) - Familiarity with prompt engineering and LLM performance optimization - Experience with real-time analytics and streaming data - Understanding of data privacy and compliance requirements in finance - Master's degree in a related field **Technical Skills Required:** **Data Analysis & Visualization:** Python (Pandas, NumPy, Matplotlib, Seaborn, Plotly, Dash), SQL, Tableau/Power BI/QuickSight, Excel **AI/ML Analytics:** LLM evaluation metrics, AI monitoring frameworks, Prompt effectiveness analysis **Data Infrastructure:** Snowflake or similar data warehouses, AWS/Azure data services, ETL tools and frameworks, Data modeling **Statistics & Testing:** Statistical analysis, A/B testing, Hypothesis testing, Experimental design This role requires a candidate with expertise in data analysis, AI/ML systems, and a strong technical skill set in various tools and frameworks related to data analysis and AI analytics.,