237 Web Scraping Jobs

As a Junior Data Engineer, your role involves developing, maintaining, and optimizing data pipelines to ensure seamless extraction, transformation, and loading (ETL) of data. You will also be responsible for building and maintaining web crawlers and scraping solutions to gather structured and unstructured data from various online sources. Collaboration with the Data Science and Analytics teams to ensure data collected meets the required standards for analysis is crucial. Working with relational and non-relational databases to store and retrieve large datasets will be part of your daily tasks. Monitoring data quality processes, ensuring data integrity, and assisting in performance tuning and ...

As a Tech Analyst at Benori, your role will involve working with the sales and delivery teams on various activities related to the integration of AI/Gen AI, data extraction, management, and automation. Key Responsibilities: - Design, test, and refine prompt structures to elicit optimal responses from AI models, ensuring high accuracy and relevance for specific use cases. - Develop and implement RAG pipelines that combine prompt engineering with retrieval systems, enabling models to draw on relevant contextual information from external databases or knowledge bases. - Leverage existing APIs provided by different tools and services to connect applications, exchange data, and automate processes ...

As an ideal candidate for the role, you will be responsible for the following: Role Overview: You will be developing Python and C# scripts to extract and process data from PDF documents, perform data transformation and analysis using pandas, and automate web processes using Selenium. Additionally, you will handle complex PDFs with OCR tools and LLM models, design RPA solutions with UiPath Studio, and conduct quality assurance testing on bots and scripts. Troubleshooting data extraction issues, writing unit and integration tests, and performing API integration and web scraping are also key parts of your responsibilities. Furthermore, you will work with front-end technologies like HTML, CSS, a...

Role Overview: As a Web Scraper, your main responsibility is to apply your expertise to extract data from various online sources. You will be tasked with developing reliable web scrapers and parsers for different websites. Your role involves extracting structured/unstructured data and storing it in SQL/NoSQL data stores. Additionally, you will collaborate closely with Project/Business/Research teams to provide the scraped data for analysis. It will be your duty to maintain the scraping projects that are delivered to production and to create frameworks for automating and ensuring a continuous flow of data from multiple sources. You should be able to work independently with minimal supervision...

As a BI Architect in Gen AI, your role involves architecting a scalable and performant BI platform that serves as a single source of truth for key business metrics across functions such as Sales, Marketing, Supply Chain, and Finance. You will be advising the BI Lead by aligning architecture decisions with long-term strategy and business priorities. Additionally, you will design a GenAI-powered natural language interface on top of BI dashboards to enable business users to query KPIs, trends, and anomalies conversationally. Your responsibilities also include selecting and implementing appropriate architectures like RAG using vector stores and Knowledge Graphs to support intelligent, context-aw...

Build and maintain scalable data pipelines, optimize LLM-based AI workflows, perform web/social media scraping, ensure data quality, integrate APIs, mentor junior engineers, conduct code reviews, contribute to architecture and technical decisions.

Role Overview: As a Lead software/Python Web Developer at Hypersonix.ai, you will be responsible for managing the interchange of data between the server and the users. Your primary focus will be on developing all server-side logic to ensure high performance and responsiveness to requests from the front-end. Additionally, you will integrate front-end elements into the application and require a basic understanding of front-end technologies. Key Responsibilities: - Writing reusable, testable, and efficient code. - Designing and implementing low-latency, high-availability, and performant applications. - Integrating user-facing elements developed by front-end developers with server-side logic. - ...

You are an experienced Senior Network Security Automation Tester with 5+ years of Industry experience and hands-on experience in End-to-End Solution testing in Security SDWAN area. Your key responsibilities include: - Strong understanding of network security concepts, protocols, and technologies. - QA Experience with VPN technologies (e.g., IKev2, IKev1, IPSec, SSL/TLS etc). - QA Experience with SD-WAN technologies and solutions. - Hands-on experience with network devices, L2/L3 protocols, and traffic generation tools (Ixia, Spirent). - Knowledge of next-generation network security standards (e.g., Post-Quantum Cryptography) and best practices. - Proficient in Python and standard libraries. ...

Key Responsibilities1 Design, develop, and maintain robust Python scripts for data processing, cleaning, and transformation Work extensively with Pandas for complex data manipulation and reporting Automate Excel workflows using OpenPyXL and win32com libraries Build and optimize web scraping solutions using Selenium (and other scraping tools if required) Maintain and scale existing Python-based applications and automation pipeline Troubleshoot, debug, and optimize performance of data workflows and scraping processes Ensure data accuracy, reliability, and process scalability Collaborate with stakeholders to understand business requirements and deliver automated solutions Take ownership of exis...

Role Overview: As the Data Analytics and Insights Manager at Piramal Pharma Solutions, you will play a crucial role in driving data-driven decision-making using advanced analytics, artificial intelligence (AI), and machine learning techniques. Your primary focus will be on utilizing technology to automate data extraction, generate actionable insights, and provide predictive analytics to support business growth. You will be responsible for automating data integration from diverse sources, identifying key trends through data mining, and delivering strategic insights to optimize marketing and sales strategies. Key Responsibilities: - Utilize advanced data extraction techniques, automation tools...

As a strategy leader in Tracxn's Data Operations & Strategy team, your role involves owning the strategy, KPIs, and automation roadmap for a large-scale data organization. You will drive operational excellence, AI-led innovation, and team leadership. **Key Responsibilities** - Define and execute a long-term strategy for Tracxns Data department. - Translate business goals into operational priorities, KPIs, and success metrics. - Identify and implement new data acquisition models, automation tools, and AI/ML solutions. - Lead large-scale SaaS-style data pipelines involving APIs, web scraping, AI/LLMs, and human curation. - Improve accuracy, turnaround time, and scalability of data operations. ...

As a developer at our company, your role involves the complete development, testing, and deployment of applications and systems. You will be responsible for the following key tasks: - Experience in Python with Flask framework - Proficiency in Python for web scraping and MongoDB for backend operations - Ability to utilize MongoDB-based REST API effectively - Hands-on experience in developing large-scale applications - Proficient in using JIRA for product requirements and Bitbucket for source control - Capable of translating Figma UI design screens into UI specifications - Knowledge of Azure Cosmos DB - Strong understanding and respect for Intellectual Property rights To be successful in this ...

You are a highly skilled and experienced Senior Node.js Developer with expertise in web scraping and web automation using tools like Chrome DevTools Protocol (CDP). Your solid understanding of web automation techniques, along with strong proficiency in Node.js and TypeScript, will be essential for designing and implementing robust web automation solutions to ensure efficiency and scalability. - Develop and maintain web scraping and automation scripts using Puppeteer, CDP, or other web automation tools. - Design and build scalable, high-performance systems for web data extraction and automation. - Collaborate with cross-functional teams to gather requirements and implement solutions. - Ensure...

We are hiring Backend Python Developer, with a strong focus on web development and automation along with web crawling/scraping, Selenium, RESTful APIs/FastAPI, Docker, Kubernetes, MS SQL Server and MongoDB exp. Years of experience - 5 to 8 Location - Hyderabad Interested candidates pleas drop your resumes to praveenkumard@ritefit.ai

Role Overview: As an automation engineer, your focus will be on designing and implementing intelligent automation solutions using Python, GenAI models, and RPA techniques. You will be responsible for developing scalable automation frameworks, performing data extraction/parsing from multiple sources, and integrating systems to support business workflows. Key Responsibilities: - Automation Development: Build and maintain RPA solutions for tasks such as web scraping (traditional and AI-driven), PDF parsing, and OCR-based data extraction. - GenAI Integration: Implement intelligent data parsing pipelines using LLMs (OpenAI, Google Gemini, Grok, etc.) and prompt engineering. - Application Developm...

Job Summary: We are seeking an accomplished and highly experienced Senior Data Analyst / Tech Lead with over 5 years of proven expertise in analytics and technology leadership, also preferably within the E-commerce domain. This role requires a strategic thinker who can manage and lead the entire technology and analytics function, while driving innovation, efficiency, and business impact. The ideal candidate will possess deep knowledge of Power BI, Python, SQL, Power Apps, and advanced Excel, along with strong leadership, client-handling, and project management skills. Key Responsibilities: creation of dynamics data scrappers, webcrawleds, auto bots, automation of process and data analytics. ...

Russell Investments is actively hiring for Artificial Intelligence, Analyst for Mumbai, Goregaon (E) location. Interested applicants please apply through below mentioned link: https://russell.wd5.myworkdayjobs.com/russellinvestments/job/Mumbai/Analyst--AI_R-03416 Job Description This role is responsible for supporting and growing the AI strategy, platform and deliverables at Russell Investments. We are looking for a curious and analytical individual who will research, develop, implement, and maintain processes to meet the needs of our AI strategy and deliver on business objectives. This is an excellent opportunity to take advantage of emerging trends and technologies and make a real-world di...

Job Title: Python Developer Data Extraction & Automation Role Type: Hybrid Location: Bengaluru (Bangalore) Experience: 3+ years Shift Timings: 6am to 3pm and 2pm to 11pm About the Role We are looking for a Python Developer with strong programming fundamentals to build and maintain automated tools for large-scale data extraction. You will design scrapers, work with APIs, transform raw data into usable formats, and ensure high reliability of our data pipelines. Your work will directly enable data-driven decisions by delivering accurate, clean, and consistent data at scale. Key Responsibilities Design, develop, and maintain automated data extraction and web scraping tools using Python. Extract ...

As an AI Engineer at GenAI & Applied ML, you will have the opportunity to work with cutting-edge AI technologies, particularly in Generative AI, Large Language Models (LLMs), and applied machine learning. Your role will involve designing, building, and deploying AI solutions using platforms like OpenAI, Hugging Face, and Cohere, specifically for applications in document intelligence, automation, and Q&A systems. You will collaborate with product and engineering teams to integrate AI features into scalable platforms, utilizing tools like Lang Chain, Flask, and AWS/GCP to deploy models effectively. - Integrate RAG (Retrieval-Augmented Generation), prompt engineering, and zero/few-shot learning...

As an experienced automation engineer, your role involves designing, building, and maintaining sophisticated web scraping systems and autonomous application agents. You will combine technical expertise in web automation with strategic thinking to develop tools that efficiently collect data and interact with web applications. - Design and develop robust web scraping architectures capable of collecting structured data from complex websites, including those with anti-bot measures. - Build intelligent agents that can autonomously navigate job application workflows, including form completion and document submission. - Implement sophisticated rate limiting, proxying, and browser fingerprinting tec...

An exciting opportunity awaits you in a small yet strategically vital Modernization team within the Global Research business. This team is part of Corporate and Investment Banking Global Research and is dedicated to delivering innovative digital products, experiences, and solutions to both internal and external clients. These offerings enhance the curation and consumption of our Global Research product across diverse client channels. The Research Modernization team is committed to identifying new opportunities and spearheading execution and change throughout the organization. **Role Overview:** As a Python Developer within the Research Modernization team, you will bring dynamic energy, stron...

Design, develop, test, and deploy scalable Python applications. Write reusable, efficient, and maintainable code. Integrate applications with third-party APIs, databases, and services. Collaborate with front-end developers, product managers, and QA teams. Troubleshoot, debug, and upgrade existing systems. Ensure code quality, security, and performance. Participate in code reviews and provide constructive feedback. Document technical designs, processes, and implementation details.

Role & responsibilities We are seeking a highly skilled Backend Python Developer with a strong background in web development, automation, and web scraping . The ideal candidate will have hands-on experience building scalable backend systems, working with RESTful APIs (preferably FastAPI), and implementing web crawling solutions using Selenium. This role requires a proactive problem-solver with excellent technical and communication skills. Key Responsibilities: Design, develop, and maintain backend services using Python. Build and optimize RESTful APIs, preferably with FastAPI. Develop and maintain web scraping/crawling solutions using Selenium. Work with databases including MS SQL Server and...

Job Responsibilities. Collaborate with data scientists, software engineers, and business stakeholders to understand data requirements and design efficient data models.. Develop, implement, and maintain robust and scalable data pipelines, ETL processes, and data integration solutions.. Extract, transform, and load data from various sources, ensuring data quality, integrity, and consistency.. Optimize data processing and storage systems to handle large volumes of structured and unstructured data efficiently.. Perform data cleaning, normalization, and enrichment tasks to prepare datasets for analysis and modelling.. Monitor data flows and processes, identify and resolve data-related issues and ...

Role Overview: As a developer in this role, you will be responsible for the complete development, testing, and deployment of applications and systems. Your primary focus will be on Python with Flask, Python-Web Scraping, MongoDB in the backend, and utilizing MongoDB-based REST API. You should have experience working on large-scale applications and be familiar with using JIRA for product requirements and Bitbucket for source control. Additionally, you will be expected to interpret Figma UI design screens as part of the UI specification and have knowledge of Azure Cosmos DB. Key Responsibilities: - Develop, test, and deploy applications and systems using Python with Flask - Implement Python-We...