139 Web Scraping Jobs - Page 5

Who We Are Applied Materials is the global leader in materials engineering solutions used to produce virtually every new chip and advanced display in the world. We design, build and service cutting-edge equipment that helps our customers manufacture display and semiconductor chips- the brains of devices we use every day. As the foundation of the global electronics industry, Applied enables the exciting technologies that literally connect our world- like AI and IoT. If you want to work beyond the cutting-edge, continuously pushing the boundaries of"science and engineering to make possible"the next generations of technology, join us to Make Possible® a Better Future. What We Offer Location: Bangalore,IND At Applied, we prioritize the well-being of you and your family and encourage you to bring your best self to work. Your happiness, health, and resiliency are at the core of our benefits and wellness programs. Our robust total rewards package makes it easier to take care of your whole self and your whole family. Were committed to providing programs and support that encourage personal and professional growth and care for you at work, at home, or wherever you may go. Learn more about our benefits . Youll also benefit from a supportive work culture that encourages you to learn, develop and grow your career as you take on challenges and drive innovative solutions for our customers."We empower our team to push the boundaries of what is possible"”while learning every day in a supportive leading global company. Visit our Careers website to learn more about careers at Applied. Job Expectations The candidate will be responsible for leading a team of data scientists who provide analytics services for Applied Materials installed base. The deliverables include developing new service capabilities, piloting them, and commercializing them in partnership with Engineering and Service Product Managers. The team will work with field engineers and product engineering teams to understand the requirements, bring forward creative ideas, develop proofs-of-concept, architect, design, develop, and modify algorithms into production code, provide production support, and train end-users. The skill sets in the team include descriptive statistical analysis, predictive statistical analysis using AI/ML, data visualization and analytics process automation, data cleansing, complex image processing, and text processing. Candidate should be willing to learn and adopt semiconductor industry as their career domain. Key Responsibilities Responsible for managing completion of assignments, projects and programs to support Applieds service business. Scope of algorithm development includes research, design, code development, implementation, and proliferation. Execute projects as needed to support the business. Review and monitor progress to milestones on development programs. Develop roadmaps for algorithmic development programs. Oversee algorithmic concept and feasibility for algorithmic modules, including problem statement definition, data gathering, literature review, concept selection, risks, and implementation constraints. Oversee documentation of algorithmic development and deployment, including integration into required systems, user testing, and user training. Oversee software and hardware implementation. Interact with internal and external customers to define gaps, identify opportunities, define program scope and deliverables, and proliferate solutions to the user base. Present to management for project reviews, interact with project stakeholders, run regular cadence meetings and work in alignment with team and organization goals. Responsible for technical development of team, objective setting, and performance management. Develop growth plan for the team, including identification of new areas of impact. Preferred programming and data science skills includePython, C++, Unix, Image Processing, Deep Learning, AI/ML, NLP, GenAI, Text Mining, Database Design and Management, Web Scraping, GPU Optimization. Proficient in business processes and software such as Microsoft Word/ Excel/ Powerpoint/ Teams, Atlassian JIRA and Confluence. Highly organized and detail-oriented. Ability to build and maintain positive and productive inter-departmental working relationships. Ability to work in a cross-functional organization and multitask on multiple projects. Drive team members to deliver programs on time and on budget. Excellent oral and written communication, organizational, analytical, and interpersonal skills. Interest in building a career in the semiconductor industry. Functional Knowledge Demonstrates comprehensive understanding of concepts and principles within own job family and knowledge of other related job families. Business Expertise Applies in-depth understanding of how own discipline integrates within the segment/function. Leadership Manages multiple related teams, sets organizational priorities and allocates resources. Problem Solving Identifies and resolves complex technical, operational and organizational problems. Impact Impacts the business results of a team or area by supporting and funding of projects, products, services and/or technologies and developing policies and plans. Guided by business unit, department or sub-functional business plans. Interpersonal Skills Influences others internally and externally, including senior management. Position requires understanding of Applied Materials global Standards of Business Conduct and compliance with these standards at all times. This includes demonstrating the highest level of ethical conduct reflecting Applied Materials core values. Education Bachelors, Masters, or Ph.D. Degree in Computer Science, Mathematics, or Engineering with a concentration in data science or AI/ML. Experience 15 years of experience Computer Science/ Mathematics/ Engineering background with 15 years of experience in performing statistical analysis, designing and developing Image Processing/ Computer Vision Algorithms, handling and analyzing large volumes of data. Semiconductor background is an added advantage. Prior team leadership experience is required. Additional Information Time Type: Full time Employee Type: Assignee / Regular Travel: Yes, 20% of the Time Relocation Eligible: Yes Applied Materials is an Equal Opportunity Employer. Qualified applicants will receive consideration for employment without regard to race, color, national origin, citizenship, ancestry, religion, creed, sex, sexual orientation, gender identity, age, disability, veteran or military status, or any other basis prohibited by law.

Summary To enhance user profiling and risk assessment, we are building web crawlers to collect relevant user data from third-party sources, forums, and the dark web. We are seeking a Senior Web Crawler & Data Extraction Engineer to design and implement these data collection solutions. Job Responsibilities Design, develop, and maintain web crawlers and scrapers to extract data from open web sources, forums, marketplaces, and the dark web. Implement data extraction pipelines that aggregate, clean, and structure data for fraud detection and risk profiling. Use Tor, VPNs, and other anonymization techniques to safely crawl the dark web while avoiding detection. Develop real-time monitoring solutions for tracking fraudulent activities, data breaches, and cybercrime discussions. Optimize crawling speed and ensure compliance with website terms of service, ethical standards, and legal frameworks. Integrate extracted data with fraud detection models, risk scoring algorithms, and cybersecurity intelligence tools. Work with data scientists and security analysts to develop threat intelligence dashboards from collected data. Implement anti-bot detection evasion techniques and handle CAPTCHAs using AI-driven solvers where necessary. Stay updated on OSINT (Open-Source Intelligence) techniques, web scraping best practices, and cybersecurity trends. Requirements 5+ years of experience in web crawling, data scraping, or cybersecurity data extraction. Strong proficiency in Python, Scrapy, Selenium, BeautifulSoup, Puppeteer, or similar frameworks. Experience working with Tor, proxies, and VPNs for anonymous web scraping. Deep understanding of HTTP protocols, web security, and bot detection mechanisms. Experience parsing structured and unstructured data from JSON, XML, and web pages. Strong knowledge of database management (SQL, NoSQL) for storing large-scale crawled data. Familiarity with AI/ML-based fraud detection techniques and data classification methods. Experience working with cybersecurity intelligence sources, dark web monitoring, and OSINT tools. Ability to implement scalable, distributed web crawling architectures. Knowledge of data privacy regulations (GDPR, CCPA) and ethical data collection practices. Nice to Have Experience in fintech, fraud detection, or threat intelligence. Knowledge of natural language processing (NLP) for analyzing cybercrime discussions. Familiarity with machine learning-driven anomaly detection for fraud prevention. Hands-on experience with cloud-based big data solutions (AWS, GCP, Azure, Elasticsearch, Kafka).

Position: Data Management Intern Location: Noida - Sector 126 Working days- 6 days, (10:30 am to 7:30 pm) Skills: Data entry, Basic excel, Data Correction, Web scraping. Company's Website- https://leverage.biz/ Experience: pursuing or freshers Role & responsibilities Web Scraping: You'll be responsible for extracting course information from university websites using the Octoparse Tool . Manual Data Correction/Management: You'll ensure the accuracy and quality of the extracted data through manual review and correction. Data Gathering: Collect structured and unstructured data through web scraping and other methods. Data Cleaning: Ensure data accuracy by identifying inconsistencies, duplication and errors in collected data-sets. Preferred candidate profile Education: Any field of study. Technical Skills: Basic understanding in Excel and data entry tools, Basic knowledge of web scraping tools (e.g., Octoparse) is a plus. Analytical Skills: Strong attention to detail and data accuracy. Communication Skills: Able to communicate professionally in office with cross functional teams Time Management: Ability to handle multiple tasks and meet deadlines in a fast-paced environment. Problem-Solving Skills: A proactive approach to addressing data collection challenges.

Skill Expectations Must-Have Skills: Strong hands-on experience in Python development Experience working with Fast API Data migration and data engineering experience (ETL, pipelines, transformations) Experience in web scraping and data extraction techniques Experience working with GCP Good-to-Have / Optional Skills: Exposure to FastAPI Experience or familiarity with GenAI / LLMs Knowledge of testing frameworks (e.g., PyTest) and CI/CD pipelines

Good knowledge and in Python, sql, Perl with 6+ years experience. Good problem solving skill. Ability to understand the data and its relations Capability to learn new technologies in short span of time. Should be able to work in Sprint and meet deadlines. Flexible Work time. Mandatory Skills: Python - Basics, Pandas, Web scrapping, File and XML Handling, Extracting/Manipulating -Excel/CSV/Any File Formats.Perl - Basics, CPAN modules, File and Web scrapping/Handling.** Work from option is available

We are looking for a highly skilled and self-motivated Python Developer with strong expertise in Flask, API integrations (like Amazon and stock market platforms), Selenium automation, and AI/LLM-based applications . The ideal candidate will support and enhance our data-driven platforms, build intelligent automation systems, and contribute to innovation in Business Intelligence and digital workflows. Key Requirements: Proficiency in Python with 2+ years of hands-on experience. Strong experience with Flask or similar Python web frameworks. Solid understanding of RESTful API development and consumption. Hands-on expertise in Selenium for browser automation tasks. Practical experience or projects using Large Language Models (LLMs), GPT, or AI toolkits . Familiarity with version control (Git), Docker, and deployment environments. Strong analytical and problem-solving skills. Excellent communication and collaboration abilities. Knowledge of Accounts will be added advantage. Send your resume to mandakranta.mahapatra@jaytea.com with the subject: "Application for Python Developer Position", including: - Current CTC - Expected CTC - Notice Period

Web scraping ( Imm joiner) Job Responsibilities: Develop and maintain web scraping scripts using Python and Selenium to extract data from websites and APIs Understanding requirement of customer needs, identify source and scrape/collect data and integrate the data crawled and scraped into our databases. Writing efficient, reusable, testable, and testing scalable code. Build code that is easily readable, properly documented, and follows key coding standards. Use Beautiful Soup and other scraping tools to clean and process data for analysis Research and learn new technologies to design code reviews and application enhancements/ upgrades Performance tuning and automation of application. Maintain communication with corporate clients, Coordination with team and management. Coordinating with development teams to determine application requirements. Prioritized daily workflows, including all projects and task Experience: Proficiency with Selenium and Beautiful Soup Depth knowledge of Data Structures, Operators, Datatypes, Casting, Methods, Conditions, Loops, Functions, Arrays, OOPS concept, iterators, Modules and MySQL integration with Python Strong understanding of web scraping techniques and best practices HTML, CSS, XPATH, Regex and HTML DOM. Knowledge of Libraries like Selenium, Requests, Scrappy, BeautifulSoup, NumPy, Pandas, Matplotlib, SciPy, TensorFlow, Keras, PyTorch, Plotly and Tableau. Good hands-on experience in Python languages and debugging with GDB Database & File Format: SQL, MySQL, Mongo dB, JSON, XML, Excel, CSV Development Tools: Microsoft Visual Studio, Visual Code, PyCharm, Power bi Knowledge of Rest API will be added as plus point

Company Overview BOLD is an established and fast-growing product company that transforms work lives. Since 2005,weve helped more than 10,000,000 folks from all over America(and beyond!) reach higher and do better. A career at BOLD promises great challenges, opportunities, culture and the environment. With our headquarters in Puerto Rico and offices in San Francisco and India, were a global organization on a path to change the career industry. Position Overview BOLD is seeking a highly skilled professional to spearhead the development of cutting-edge browser automation technology for our Expert Apply product. You will play a key role in designing scalable automation frameworks, tackling challenges in bot detection, and optimizing system performance. You'll also be responsible for building and monitoring metrics to ensure system reliability and robustness. If you are passionate about large-scale automation and system reliability, we want to hear from you. Role & responsibilities Design and architect scalable and robust enterprise-level automation applications using Python. Develop applications that run on PODs (Kubernetes), ensuring high availability and reliability. Debug complex issues in applications and devise solutions that enhance stability and performance. Identify performance bottlenecks within applications through profiling and metrics analysis. Optimize existing code to improve performance and efficiency, ensuring the system can handle high traffic loads. Utilize automation frameworks and tools such as Playwright, Chromium, and stealth browser for web automation tasks. Implement message handling to facilitate communication between different services. Develop web scraping solutions to gather and process data from various online sources. Analyze and troubleshoot software issues, providing timely resolutions to ensure system reliability Collaborate with cross-functional teams to understand user requirements and translate them into technical specifications. Review and enhance code quality through thorough testing and code reviews. Stay updated with industry trends and emerging technologies, integrating best practices into the development process Document architecture, design choices, and implementation details for future reference and knowledge sharing. Ensure compliance with security and data privacy standards throughout the application lifecycle. Preferred candidate profile Strong programming skills in Python like expertise in string manipulation and regular expression to effectively handle and process the text data during web scrapping and automation tasks. Deep understanding of OOP principles, including encapsulation, inheritance, and polymorphism, to design robust and maintainable software systems. Knowledge of common design patterns (e.g., Singleton, Factory, Observer) to enhance system design, improve code reusability, and implement best practices in software architecture. Solid foundation in algorithms (sorting, searching, parsing) and data structures (lists, dictionaries, trees) to solve complex problems efficiently and effectively during software development. Good understanding of how modern browsers function, including rendering engines, Java Script engines, HTTP protocols, and browser APIs. Experience optimizing scraping strategies based on browser behavior and performance. Experience with caching technologies (e.g. Redis, in-memory caching) Experience with messaging protocols (e.g. Azure service bus, Kafka, RabbitMQ) working knowledge and proven experience in containerization using Docker. Understanding of DevOps practices and CI/CD pipelines. Excellent communication skills and the ability to collaborate across time zones Excellent analytical and problem-solving skills. Knowledge of cloud computing, Amazon Web Services or Microsoft Azure

We are looking for a Python Developer who has expertise in web scraping and backend development. The ideal candidate should be proficient in Python frameworks, data extraction techniques, and API integration.

Job Information Job Opening ID ZR_2129_JOB Date Opened 05/03/2024 Industry Technology Job Type Work Experience 3-5 years Job Title DB developer -Python City Chennai Province Tamil Nadu Country India Postal Code 600001 Number of Positions 5 Good knowledge and in Python, sql, Perl with 6+ years experience. Good problem solving skill. Ability to understand the data and its relations Capability to learn new technologies in short span of time. Should be able to work in Sprint and meet deadlines. Flexible Work time. Mandatory Skills: Python - Basics, Pandas, Web scrapping, File and XML Handling, Extracting/Manipulating -Excel/CSV/Any File Formats.Perl - Basics, CPAN modules, File and Web scrapping/Handling.** Work from option is available check(event) ; career-website-detail-template-2 => apply(record.id,meta)" mousedown="lyte-button => check(event)" final-style="background-color:#2B39C2;border-color:#2B39C2;color:white;" final-class="lyte-button lyteBackgroundColorBtn lyteSuccess" lyte-rendered=""> I'm interested

Role & responsibilities Develop and implement Python scripts for web scraping using Selenium WebDriver to extract relevant data from client websites. Clean, transform, and manipulate extracted data using Python libraries (e.g., Pandas, BeautifulSoup) for schema (Structured data) markup implementation Write well-documented, maintainable, and efficient Python code adhering to best practices. Collaborate with SEOs and the Director of SEO to understand client requirements and translate them into technical solutions. Stay up-to-date on the latest trends and developments in web scraping, schema (Structured data) markup, and SEO best practices. Assist with testing and debugging developed scripts to ensure accuracy of schema (Structured data) implementation without any error. Experience working in Automation through AI agents Experience working with machine learning and AI (Artificial Intelligence) integration using Python. Preferred candidate profile Having 4-5 years of working experience in Python programming. Strong understanding of Python syntax, data structures, Iterator, Generators, Exception Handling, File handling, OOPs, Data Structures, ORM and object-oriented programming concepts. Proficiency in using web scraping libraries like Selenium WebDriver and Beautiful Soup. Must be familiar with Web Frameworks like HTML, CSS, JavaScript, Django or Flasks. Good knowledge of machine learning & ML frameworks like NumPy, Pandas, Kera's, scikit-learn, PyTorch, TensorFlow or Microsoft Azure Machine Learning will be added advantage. Must be familiar with development tools like Jupyter Notebook, IDLE, PyCharm or VS Code. Must be familiar with Scrum methodology, CI/CD, Git, Branching/Merging and test-driven software development. Candidates worked in product-based companies will be preferred. Excellent analytical and problem-solving skills. Ability to work independently and as part of a team. Strong communication and collaboration skills. A passion for SEO and a desire to learn about schema (Structured data) markup. Familiarity with cloud platforms (AWS, GCP, Azure DevOps, Azure Blob Storage Explorer) Experience with API integration. Experience working with AI (Artificial Intelligence) integration with Python to automate SEO tasks with Google Gemini, GenAI (Generative AI) & ChatGPT 4. Experience working in Automation through AI agents Good verbal and written communication skills.

Contract Duration – 3 Months Minimum 5yrs in Python Must Have: Python Web Scrapping (Selenium/Beautifulsoup) Pycharm Html, CSS, Oops AI Automation Good to have: Schema/ Structure Data Cloud Storage - Azure(blob storage)/AWS

About Us: LdotR is an online brand protection service company, offering businesses the right solution and services to protect, manage and benefit from their digital assets in the online space. We work across all digital platforms - Domains, Website, Social Media, Online Marketplaces, and App Stores to identify, assess and nullify brand infringements. About the Role: We are looking for an experienced Data Scraping Specialist to help us extract and structure data from leading social media platforms at scale. The ideal candidate will have hands-on expertise with scraping tools, APIs, and large-scale data processing. Key Responsibilities: Design and develop custom scraping solutions to extract public data from platforms like Instagram, Facebook, X (Twitter), LinkedIn, YouTube, etc. Handle large-scale scraping tasks with efficiency and resilience against rate-limiting and platform-specific restrictions. Clean, normalize, and structure the scraped data for analysis or downstream applications. Maintain scraping scripts to adapt to frequent platform changes. Ensure compliance with data protection policies and terms of service. Required Skills: Proficiency in Python and scraping libraries (e.g., Scrapy, BeautifulSoup, Selenium, Playwright). Experience with API integration (official or unofficial social media APIs). Familiarity with rotating proxies, headless browsers, and CAPTCHA-solving techniques. Strong understanding of data structuring formats like JSON, CSV, and databases (MongoDB, PostgreSQL, etc.). Experience with cloud-based scraping and storage solutions (AWS/GCP preferred). Good to Have: Knowledge of NLP or data analytics for social media sentiment or trend analysis. Understanding of GDPR and CCPA compliance. Prior work with third-party scraping platforms or browser automation tools. What We Offer: Opportunity to work on impactful, large-scale data projects. Flexible work arrangements. Competitive compensation based on experience and delivery.

Responsibilities:As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. Annual bonus Health insurance Provident fund

Key Responsibilities: Develop and maintain automated web scraping scripts using Python libraries such as Beautiful Soup, Scrapy, and Selenium. Optimize scraping pipelines for performance, scalability, and resource efficiency. Handle dynamic websites, CAPTCHA-solving, and implement IP rotation techniques for uninterrupted scraping. Process and clean raw data, ensuring accuracy and integrity in extracted datasets. Collaborate with cross-functional teams to understand data requirements and deliver actionable insights. Leverage APIs when web scraping is not feasible, managing authentication and request optimization. Document processes, pipelines, and troubleshooting steps for maintainable and reusable scraping solutions. Ensure compliance with legal and ethical web scraping practices, implementing security safeguards. Requirements: Education : Bachelors degree in Computer Science, Engineering, or a related field. Experience : 2+ years of Python development experience, with at least 1 year focused on web scraping. Technical Skills : Proficiency in Python and libraries like Beautiful Soup, Scrapy, and Selenium. Experience with regular expressions (Regex) for data parsing. Strong knowledge of HTTP protocols, cookies, headers, and user-agent rotation. Familiarity with databases (SQL and NoSQL) for storing scraped data. Hands-on experience with data manipulation libraries such as pandas and NumPy. Experience working with APIs and managing third-party integrations. Familiarity with version control systems like Git. Bonus Skills : Knowledge of containerization tools like Docker. Experience with distributed scraping solutions and task queues (e.g., Celery, RabbitMQ). Basic understanding of data visualization tools. Non-Technical Skills : Strong analytical and problem-solving skills. Excellent communication and documentation skills. Ability to work independently and collaboratively in a team environmen

About Us : We are a dynamic and innovative team specializing in building scalable and efficient web scraping solutions for complex e-commerce platforms and diverse web environments. Our team is proficient in a wide range of technologies and advanced web scraping techniques, ensuring high-quality data extraction and delivery Job Summary: We are seeking a talented Web Scraping Engineer / Data Extraction Specialist to join our growing team. The ideal candidate will have a strong background in web scraping, data extraction, and backend technologies. You will be responsible for designing, developing, and maintaining robust web scraping solutions, handling dynamic content, and overcoming anti-crawling measures. Responsibilities: Develop and maintain scalable web scraping scripts using Python, JavaScript, and related frameworks (e.g., Scrapy, Selenium, Puppeteer, Beautiful Soup, Cheerio.js). Implement advanced web scraping techniques, including API interception, sitemap parsing, and handling dynamic content. Design and build data pipelines for efficient data extraction, processing, and storage. Manage and optimize data extraction workflows, ensuring high speed and accuracy. Implement anti-crawling solutions, including IP rotation, proxy management, and CAPTCHA bypassing. Collaborate with cross-functional teams to gather requirements and deliver complex data solutions. Utilize backend technologies such as Flask, FastAPI, Django, Node.js, Spring Boot, and relational databases (PostgreSQL, MySQL) for data storage and API development. Work with cloud platforms like Azure and leverage services such as AzureML and ADLS GEN2. Employ data processing techniques using libraries like NumPy and Pandas. Use tools like Postman, MITM, and DevTools for API testing and network traffic analysis. Apply machine learning and NLP techniques for data analysis and processing (e.g., sentiment analysis, content classification). Set up and manage server-side scraping environments. Monitor and troubleshoot scraping scripts to ensure optimal performance. Required Skills: Strong proficiency in Python and/or JavaScript. Experience with web scraping libraries and frameworks (Scrapy, Selenium, Puppeteer, Beautiful Soup, Cheerio.js). Knowledge of backend technologies (Flask, FastAPI, Django, Node.js, Spring Boot). Experience with relational databases (PostgreSQL, MySQL). Understanding of HTTP/HTTPS protocols and API communication. Familiarity with cloud platforms (Azure). Ability to handle dynamic content and JavaScript-heavy websites. Experience with anti-crawling techniques (IP rotation, proxies, CAPTCHA bypassing). Data processing and analysis skills (NumPy, Pandas). Experience with API testing tools (Postman). Knowledge of machine learning and NLP concepts is a plus. Strong problem-solving and debugging skills. Excellent communication and collaboration skills. Experience: 5+ years of professional experience in web scraping or related fields. Education: Bachelor's degree in Computer Science, Engineering, or a related field

Develop and maintain scalable Python-based applications, microservices, and automated tools using Django. Design, implement, and maintain RESTful APIs and backend logic for various modules (college search, applications, data processing, etc.).

Overview The Data Technology team at MSCI is responsible for meeting the data requirements across various business areas, including Index, Analytics, and Sustainability. Our team collates data from multiple sources such as vendors (e.g., Bloomberg, Reuters), website acquisitions, and web scraping (e.g., financial news sites, company websites, exchange websites, filings). This data can be in structured or semi-structured formats. We normalize the data, perform quality checks, assign internal identifiers, and release it to downstream applications. Responsibilities As data engineers, we build scalable systems to process data in various formats and volumes, ranging from megabytes to terabytes. Our systems perform quality checks, match data across various sources, and release it in multiple formats. We leverage the latest technologies, sources, and tools to process the data. Some of the exciting technologies we work with include Snowflake, Databricks, and Apache Spark. Qualifications Core Java, Spring Boot, Apache Spark, Spring Batch, Python. Exposure to sql databases like Oracle, Mysql, Microsoft Sql is a must. Any experience/knowledge/certification on Cloud technology preferrably Microsoft Azure or Google cloud platform is good to have. Exposures to non sql databases like Neo4j or Document database is again good to have. What we offer you Transparent compensation schemes and comprehensive employee benefits, tailored to your location, ensuring your financial security, health, and overall wellbeing. Flexible working arrangements, advanced technology, and collaborative workspaces. A culture of high performance and innovation where we experiment with new ideas and take responsibility for achieving results. A global network of talented colleagues, who inspire, support, and share their expertise to innovate and deliver for our clients. Global Orientation program to kickstart your journey, followed by access to our Learning@MSCI platform, LinkedIn Learning Pro and tailored learning opportunities for ongoing skills development. Multi-directional career paths that offer professional growth and development through new challenges, internal mobility and expanded roles. We actively nurture an environment that builds a sense of inclusion belonging and connection, including eight Employee Resource Groups. All Abilities, Asian Support Network, Black Leadership Network, Climate Action Network, Hola! MSCI, Pride & Allies, Women in Tech, and Women’s Leadership Forum. At MSCI we are passionate about what we do, and we are inspired by our purpose – to power better investment decisions. You’ll be part of an industry-leading network of creative, curious, and entrepreneurial pioneers. This is a space where you can challenge yourself, set new standards and perform beyond expectations for yourself, our clients, and our industry. MSCI is a leading provider of critical decision support tools and services for the global investment community. With over 50 years of expertise in research, data, and technology, we power better investment decisions by enabling clients to understand and analyze key drivers of risk and return and confidently build more effective portfolios. We create industry-leading research-enhanced solutions that clients use to gain insight into and improve transparency across the investment process. MSCI Inc. is an equal opportunity employer. It is the policy of the firm to ensure equal employment opportunity without discrimination or harassment on the basis of race, color, religion, creed, age, sex, gender, gender identity, sexual orientation, national origin, citizenship, disability, marital and civil partnership/union status, pregnancy (including unlawful discrimination on the basis of a legally protected parental leave), veteran status, or any other characteristic protected by law. MSCI is also committed to working with and providing reasonable accommodations to individuals with disabilities. If you are an individual with a disability and would like to request a reasonable accommodation for any part of the application process, please email Disability.Assistance@msci.com and indicate the specifics of the assistance needed. Please note, this e-mail is intended only for individuals who are requesting a reasonable workplace accommodation; it is not intended for other inquiries. To all recruitment agencies MSCI does not accept unsolicited CVs/Resumes. Please do not forward CVs/Resumes to any MSCI employee, location, or website. MSCI is not responsible for any fees related to unsolicited CVs/Resumes. Note on recruitment scams We are aware of recruitment scams where fraudsters impersonating MSCI personnel may try and elicit personal information from job seekers. Read our full note on careers.msci.com

Job Responsibilities. Collaborate with data scientists, software engineers, and business stakeholders to understand data requirements and design efficient data models.. Develop, implement, and maintain robust and scalable data pipelines, ETL processes, and data integration solutions.. Extract, transform, and load data from various sources, ensuring data quality, integrity, and consistency.. Optimize data processing and storage systems to handle large volumes of structured and unstructured data efficiently.. Perform data cleaning, normalization, and enrichment tasks to prepare datasets for analysis and modelling.. Monitor data flows and processes, identify and resolve data-related issues and bottlenecks.. Contribute to the continuous improvement of data engineering practices and standards within the organization.. Stay up-to-date with industry trends and emerging technologies in data engineering, artificial intelligence, and dynamic pricing. Candidate Profile. Strong passion for data engineering, artificial intelligence, and problem-solving.. Solid understanding of data engineering concepts, data modeling, and data integration techniques.. Proficiency in programming languages such as Python, SQL and Web Scrapping.. Understanding of databases like No Sql , relational database, In Memory database and technologies like MongoDB, Redis, Apache Spark would be add on... Knowledge of distributed computing frameworks and big data technologies (e.g., Hadoop, Spark) is a plus.. Excellent analytical and problem-solving skills, with a keen eye for detail.. Strong communication and collaboration skills, with the ability to work effectively in a teamoriented environment.. Self-motivated, quick learner, and adaptable to changing priorities and technologies.. (ref:hirist.tech).

Data Acquisition & Web Application Developer Experience: 3 - 6 Years Exp Salary : USD 1,851-2,962 / month Preferred Notice Period : Within 30 Days Shift : 10:00AM to 7:00PM IST Opportunity Type: Remote Placement Type: Permanent (*Note: This is a requirement for one of Uplers' Clients) Must have skills required : APIS, data acquisition, Web scraping, Agile, Python Good to have skills : Analytics, Monitoring, stream processing, Web application deployment, Node Js GPRO Ltd (One of Uplers' Clients) is Looking for: Data Acquisition & Web Application Developer who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you. Role Overview Description Job Title: Data Acquisition & Web Application Developer About the Project: We are seeking a skilled full-stack developer to build a specialised web application designed to aggregate and present public information on individuals, such as company executives and leaders. This tool will serve as a comprehensive profile generator, pulling data from diverse online sources including news outlets, social media and other platforms. The primary goal is to provide users with a centralised, easily navigable view of a person's online presence, latest news and public information. Project Overview: The core of this project involves developing a robust data acquisition layer capable of scraping and integrating information from various online sources. This data will then be presented through a user-friendly web interface. The application should allow users to input a person's name and receive an aggregated view of relevant public data. Key Responsibilities: Develop and Implement Data Acquisition Layer: Design and build systems to scrape and collect data from specified sources, including news websites (e.g., Bloomberg.com, Reuters, BBC.com, Financial Times), social media (e.g., X, LinkedIn), and media platforms (e.g., YouTube, podcasts). Integrate with APIs: Utilize official APIs (e.g., Bloomberg data, Reuters, FinancialTimes, Google Finance) where available and prioritized. Evaluate and integrate with third-party scraping APIs (e.g., Apify, Oxylabs, SerpApi) as necessary, considering associated risks and subscription models. Handle Hybrid Approach: Implement a strategy that leverages licensed APIs for premium sources while potentially using third-party scrapers for others, being mindful of terms of service and legal/ethical considerations. Direct scraping of highly protected sites like Bloomberg, Reuters, and FT should be avoided or approached with extreme caution using third-party services. Design Data Storage and Indexing: Determine appropriate data storage solutions, considering the volume of data and its relevance over time. Implement indexing and caching mechanisms to ensure efficient search and retrieval of information, supporting near real-time data presentation. Develop Web Application Front-End: Build a basic, functional front-end interface similar to the provided examples ("Opening Screen," "Person profile"). This includes displaying the aggregated information clearly. Implement User Functionality: Enable users to: Input a person's name for searching. Sort displayed outputs by date. Click through links to access the original source of information. Navigate to a new search easily (e.g., via a tab). Consider Stream Processing: Evaluate and potentially implement stream processing techniques for handling near real-time data acquisition and updates. ¢ Ensure Scalability: Design the application to support a specified level of concurrent searches (estimated at 200 for the initial phase). ¢ Build Business Informational Layer: Develop a component that tracks the usage of different data services (APIs, scrapers) for monitoring costs and informing future scaling decisions. ¢ Technical Documentation: Provide clear documentation for the developed system, including data flows, API integrations, and deployment notes. Required Skills and Experience: ¢ Proven experience in web scraping and data acquisition from diverse online sources. ¢ Strong proficiency in developing with APIs, including handling different authentication methods and data formats. ¢ Experience with relevant programming languages and frameworks for web development and data processing (e.g., Python, Node.js, etc.). ¢ Knowledge of database design and data storage solutions. ¢ Familiarity with indexing and caching strategies for search applications. ¢ Understanding of potential challenges in web scraping (e.g., anti-scraping measures, terms of service). ¢ Experience in building basic web application front-ends. ¢ Ability to consider scalability and performance in system design. ¢ Strong problem-solving skills and ability to work independently or as part of a small team. ¢ Experience working with foreign (western based) startups and clients. Ability to work in agile environments and ability to pivot fast. Desirable Skills: ¢ Experience with stream processing technologies. ¢ Familiarity with deploying and managing web applications (though infrastructure design is flexible). ¢ Experience with monitoring and analytics for application usage. How to apply for this opportunity: Easy 3-Step Process: 1. Click On Apply! And Register or log in on our portal 2. Upload updated Resume & Complete the Screening Form 3. Increase your chances to get shortlisted & meet the client for the Interview! About Our Client: A web app aggregating real-time info on individuals for financial services professionals About Uplers: Our goal is to make hiring and getting hired reliable, simple, and fast. Our role will be to help all our talents find and apply for relevant product and engineering job opportunities and progress in their career. (Note: There are many more opportunities apart from this on the portal.) So, if you are ready for a new challenge, a great work environment, and an opportunity to take your career to the next level, don't hesitate to apply today. We are waiting for you!

Python Developer Location: Sector-1, Noida (Work from Office) Experience: Minimum 3 years Education: B.E./B.Tech Primary Role: Responsible for performing web scraping and crawling to extract and structure data from various websites. Handle data cleaning, transformation, and storage in structured formats. Write efficient and scalable Python scripts to manage high-volume data extraction tasks. Monitor and manage log files using automation scripts. Key Skills: Proficiency in Python with hands-on experience in web scraping and crawling . Strong working knowledge of BeautifulSoup , Selenium , NumPy , Pandas , and Pytest . Good understanding of JavaScript , HTML , and SQL (preferably MS SQL ). Experience with MongoDB is an added advantage. Ability to integrate multiple data sources and databases into a single pipeline. Solid understanding of: Python threading and multiprocessing Event-driven programming Scalable and modular application design Preferred Skills: Practical experience in writing and maintaining web crawlers and scrapers . Familiarity with anti-bot mechanisms and techniques to bypass them responsibly. Exposure to handling large datasets and ensuring data accuracy and completeness. Experience with automated testing using Pytest .

Role & responsibilities Data Pipeline Development: Design, develop, and maintain data pipelines utilizing Google Cloud Platform (GCP) services like Dataflow, Dataproc, and Pub/Sub. Data Ingestion & Transformation: Build and implement data ingestion and transformation processes using tools such as Apache Beam and Apache Spark. Data Storage Management: Optimize and manage data storage solutions on GCP, including BigQuery, Cloud Storage, and Cloud SQL. Security Implementation: Implement data security protocols and access controls with GCP's Identity and Access Management (IAM) and Cloud Security Command Center. System Monitoring & Troubleshooting: Monitor and troubleshoot data pipelines and storage solutions using GCP's Stackdriver and Cloud Monitoring tools. Generative AI Systems: Develop and maintain scalable systems for deploying and operating generative AI models, ensuring efficient use of computational resources. Gen AI Capability Building: Build generative AI capabilities among engineers, covering areas such as knowledge engineering, prompt engineering, and platform engineering. Knowledge Engineering: Gather and structure domain-specific knowledge to be utilized by large language models (LLMs) effectively. Prompt Engineering: Design effective prompts to guide generative AI models, ensuring relevant, accurate, and creative text output. Collaboration: Work with data experts, analysts, and product teams to understand data requirements and deliver tailored solutions. Automation: Automate data processing tasks using scripting languages such as Python. Best Practices: Participate in code reviews and contribute to establishing best practices for data engineering within GCP. Continuous Learning: Stay current with GCP service innovations and advancements. Core data services (GCS, BigQuery, Cloud Storage, Dataflow, etc.). Skills and Experience: Experience: 5+ years of experience in Data Engineering or similar roles. Proficiency in GCP: Expertise in designing, developing, and deploying data pipelines, with strong knowledge of GCP core data services (GCS, BigQuery, Cloud Storage, Dataflow, etc.). Generative AI & LLMs: Hands-on experience with Generative AI models and large language models (LLMs) such as GPT-4, LLAMA3, and Gemini 1.5, with the ability to integrate these models into data pipelines and processes. Experience in Webscraping Technical Skills: Strong proficiency in Python and SQL for data manipulation and querying. Experience with distributed data processing frameworks like Apache Beam or Apache Spark is a plus. Security Knowledge: Familiarity with data security and access control best practices. • Collaboration: Excellent communication and problem-solving skills, with a demonstrated ability to collaborate across teams. Project Management: Ability to work independently, manage multiple projects, and meet deadlines. Preferred Knowledge: Familiarity with Sustainable Finance, ESG Risk, CSRD, Regulatory Reporting, cloud infrastructure, and data governance best practices. Bonus Skills: Knowledge of Terraform is a plus. Education: Degree: Bachelors or masters degree in computer science, Information Technology, or a related field. Experience: 3-5 years of hands-on experience in data engineering. Certification: Google Professional Data Engineer

What we are looking for The ideal candidate will possess hands-on expertise in designing and deploying advanced web scraping solutions, leveraging Node.js and other technologies. A significant focus will be on overcoming bot detection challenges, building scalable and resilient scraping systems, and ensuring the efficiency and scalability of data acquisition pipelines. This is a highly technical, hands-on role ideal for someone passionate about solving complex scraping and infrastructure challenges. Things you will be doing Advanced Web Scraping: Develop and maintain high-performance scraping systems using Node.js, Python, or other relevant technologies. Handle JavaScript-heavy and asynchronous content using tools like Puppeteer, Playwright, or custom solutions in Node.js. Implement advanced bot detection bypass techniques, including: CAPTCHA solving using automation, AI/ML, or third-party services. Advanced proxy management and IP rotation strategies. User-agent, cookie, and header spoofing. Build robust error-handling mechanisms to adapt to changes in website structures or anti-scraping measures. Bot Detection and Anti-Scraping Expertise: Analyze and reverse-engineer advanced bot detection systems and anti-scraping mechanisms, including rate-limiting, behavioral analysis, and fingerprinting. Design and implement techniques to bypass WAFs (Web Application Firewalls) and server-side protections using Node.js libraries and tools. Monitor, log, and analyze bot detection patterns to ensure system adaptability. Create innovative solutions to blend scraping traffic with legitimate user behavior. Infrastructure and Networking: Architect and maintain scalable infrastructure using containerization tools like Docker and orchestration platforms such as Kubernetes. Leverage cloud platforms (AWS, GCP, Azure) for distributed scraping and data acquisition. Utilize Node.js and related tools to optimize network configurations for high-throughput scraping, including proxy and load balancer configurations. Automate deployment and scaling of scraping systems using CI/CD pipelines. Performance and Optimization: Ensure optimal performance of scraping systems by reducing latency and optimizing resource utilization. Develop robust monitoring and logging systems to track and troubleshoot issues in real time. Optimize pipelines for scalability, fault tolerance, and high availability. Compliance and Security: Ensure adherence to legal, ethical, and regulatory standards (e.g., GDPR, CCPA) for all scraping activities. Safeguard data acquisition systems from detection, blocking, and external threats. Respect website terms of service while implementing efficient scraping solutions. Skills you need in order to succeed in this role Technical Skills: 3+ years of hands-on experience in web scraping or data engineering. Expertise in Node.js for building and optimizing scraping systems. Deep expertise in handling advanced bot detection systems and anti-scraping mechanisms. Strong knowledge of programming languages such as Python and JavaScript. Advanced understanding of networking concepts, including HTTP/HTTPS protocols, WebSockets, DNS, and API integrations. Experience with containerization tools (Docker) and orchestration platforms (Kubernetes). Proficiency in cloud platforms (AWS, GCP, Azure) for scalable data acquisition pipelines. Familiarity with tools like Puppeteer, Playwright, Scrapy, or Selenium. Problem-Solving Expertise: Proven ability to reverse-engineer anti-bot measures such as CAPTCHA, IP blocks, and fingerprinting. Strong debugging and optimization skills for network and scraping pipelines.

Job Overview: The ideal candidate will have strong Python programming skills and experience with web scraping frameworks and libraries like Requests, BeautifulSoup, Selenium, Playwright or URLlib. You will be responsible for building efficient and scalable web scrapers, extracting valuable data, and ensuring data integrity. This role requires a keen eye for problem-solving, the ability to work with complex data structures, and a strong understanding of web technologies like HTML, CSS, DOM, XPATH, and Regular Expressions. Knowledge of JavaScript would be an added advantage. Responsibilities: • As a Web Scraper, your role is to apply your knowledge set to fetch data from multiple • online sources • Developing highly reliable web Scraper and parsers across various websites • Extract structured/unstructured data and store them into SQL/No SQL data store • Work closely with Project/Business/Research teams to provide scrapped data for analysis • Maintain the scraping projects delivered to production • Develop frameworks for automating and maintaining constant flow of data from multiple • sources • Work independently with minimum supervision • Develop a deep understanding of the data sources on the web and know exactly how, when, and which data to scrap, parse and store this data Required Skills and Experience: • Experience as Web Scraper of 1 to 2 years. • Proficient knowledge in Python language and working knowledge of Web Crawling/Web scraping in Python Requests, Beautifulsoup or URLlib and Selenium, Playwright. • Must possess strong knowledge of basic Linux commands for system navigation, management, and troubleshooting. • Must have expertise in proxy usage to ensure secure and efficient network operations. • Must have experience with captcha-solving techniques for seamless automation and data extraction. • Experience with data parsing - Strong knowledge of Regular expression, HTML, CSS, DOM, XPATH. Knowledge of Javascript would be a plus SLO Technologies Private Limited Registered Office Address: IQS Tower, 5th Floor, Baner Road, Baner, Pune Maharashtra 411045 CIN: U74120MH2015PTC267292 Phone: 7900151368/ 8652865168 Email: info@advarisk.com Website: www.advarisk.com • Must be able to access, manipulate, and transform data from a variety of database and flat file sources. MongoDB & MYSQL skills are essential. • Must possess strong knowledge of basic Linux commands for system navigation, management, and troubleshooting. • Must be able to develop reusable code-based scraping products which can be used by others. • GIT knowledge is mandatory for version control and collaborative development workflows. • Must have experience handling cloud servers on platforms like AWS, GCP, and LEAPSWITCH for scalable and reliable infrastructure management. • Ability to ask the right questions and deliver the right results in a way that is understandable and usable to your clients. • A track record of digging in to the tough problems, attacking them from different angles, and bringing innovative approaches to bear is highly desirable. Must be capable of selfteaching new techniques. Behavioural expectations: • Be excited by and have positive outlook to navigate ambiguity • Passion for results and excellence • Team player • Must be able to get the job done by working collaboratively with others • Be inquisitive and an analytical mind; out-of-the-box thinking • Prioritize among competing opportunities, balance consumer needs with business and product priorities, and clearly articulate the rationale behind product decisions • Straightforward and professional • Good communicator • Maintain high energy and motivate • A do-it-yourself orientation, consistent with the companys roll-up the- sleeves” culture • Proactive

Role & responsibilities Job Overview: We are looking for a highly motivated Junior Data Engineer with a passion for web scraping and web crawling to join our team. The ideal candidate will have strong Python programming skills and experience with web scraping frameworks and libraries like Requests, BeautifulSoup, Selenium, Playwright or URLlib. You will be responsible for building efficient and scalable web scrapers, extracting valuable data, and ensuring data integrity. This role requires a keen eye for problem-solving, the ability to work with complex data structures, and a strong understanding of web technologies like HTML, CSS, DOM, XPATH, and Regular Expressions. Knowledge of JavaScript would be an added advantage. Responsibilities: As a Web Scraper, your role is to apply your knowledge set to fetch data from multiple online sources Developing highly reliable web Scraper and parsers across various websites Extract structured/unstructured data and store them into SQL/No SQL data store Work closely with Project/Business/Research teams to provide scrapped data for analysis Maintain the scraping projects delivered to production Develop frameworks for automating and maintaining constant flow of data from multiple sources Work independently with minimum supervision Develop a deep understanding of the data sources on the web and know exactly how, when, and which data to scrap, parse and store this data Required Skills and Experience: Experience as Web Scraper of 1 to 2 years. Proficient knowledge in Python language and working knowledge of Web Crawling/Web scraping in Python Requests, Beautifulsoup or URLlib and Selenium, Playwright. Must possess strong knowledge of basic Linux commands for system navigation, management, and troubleshooting. Must have expertise in proxy usage to ensure secure and efficient network operations. Must have experience with captcha-solving techniques for seamless automation and data extraction. Experience with data parsing - Strong knowledge of Regular expression, HTML, CSS, DOM, XPATH. Knowledge of Javascript would be a plus Preferred candidate profile Must be able to access, manipulate, and transform data from a variety of database and flat file sources. MongoDB & MYSQL skills are essential. • Must possess strong knowledge of basic Linux commands for system navigation, management, and troubleshooting. • Must be able to develop reusable code-based scraping products which can be used by others. • GIT knowledge is mandatory for version control and collaborative development workflows. • Must have experience handling cloud servers on platforms like AWS, GCP, and LEAPSWITCH for scalable and reliable infrastructure management. • Ability to ask the right questions and deliver the right results in a way that is understandable and usable to your clients. • A track record of digging in to the tough problems, attacking them from different angles, and bringing innovative approaches to bear is highly desirable. Must be capable of selfteaching new techniques. Behavioural expectations: • Be excited by and have positive outlook to navigate ambiguity • Passion for results and excellence • Team player • Must be able to get the job done by working collaboratively with others • Be inquisitive and an analytical mind; out-of-the-box thinking • Prioritize among competing opportunities, balance consumer needs with business and product priorities, and clearly articulate the rationale behind product decisions • Straightforward and professional • Good communicator • Maintain high energy and motivate • A do-it-yourself orientation, consistent with the companys roll-up the- sleeves culture • Proactive