Web Scraping Data Analyst Sagacious Infosystems Pvt ltd

2.0 - 6.0 years

0 Lacs

karnataka

On-site

As a skilled Web Scraping Data Analyst, you will be responsible for collecting, cleaning, and analyzing data from various online sources. Your expertise in Python-based scraping frameworks, data transformation, and experience with proxy/VPN rotation and IP management will be crucial in building data pipelines that support our analytics and business intelligence initiatives. Your key responsibilities will include designing, developing, and maintaining robust web scraping scripts using tools like Python, BeautifulSoup, Scrapy, Selenium, etc. You will also implement IP rotation, proxy management, and anti-bot evasion techniques, deploy scraping tools on cloud-based or edge servers, and monitor scraping jobs for uptime and efficiency. Additionally, you will parse and structure unstructured or semi-structured web data into clean, usable datasets, collaborate with data analysts and data engineers to integrate web-sourced data into internal databases and reporting systems, conduct exploratory data analysis (EDA), and ensure compliance with website scraping policies, robots.txt, and relevant data privacy regulations. To excel in this role, you should have proficiency in Python and experience with libraries like Requests, BeautifulSoup, Scrapy, Pandas. Knowledge of proxy/VPN usage, IP rotation, and web traffic routing tools (e.g., Smartproxy, BrightData, Tor, etc.), familiarity with cloud platforms (AWS, Azure, or GCP) and Linux-based environments, experience deploying scraping scripts on edge servers or containerized environments (e.g., Docker), solid understanding of HTML, CSS, JSON, and browser dev tools for DOM inspection, strong analytical mindset with experience in data cleansing, transformation, and visualization, good knowledge of SQL and basic data querying, and ability to handle large volumes of data and build efficient data pipelines. Preferred qualifications for this role include experience with headless browsers like Puppeteer or Playwright, familiarity with scheduling tools like Airflow or Cron, background in data analytics or reporting using tools like Tableau, Power BI, or Jupyter Notebooks, and knowledge of anti-captcha solutions and browser automation challenges. This is a full-time position with the work location being in person.,

Posted 4 days ago

Apply

Web Scraping Engineer CloudHire

4.0 - 8.0 years

0 Lacs

haryana

On-site

As an experienced automation engineer, you will be responsible for designing, building, and maintaining sophisticated web scraping systems and autonomous application agents. Your role will involve combining technical expertise in web automation with strategic thinking to develop tools that efficiently collect data and interact with web applications. Your key responsibilities will include designing and developing robust web scraping architectures capable of collecting structured data from complex websites, including those with anti-bot measures. You will also build intelligent agents that can autonomously navigate job application workflows, including form completion and document submission. Implementing sophisticated rate limiting, proxying, and browser fingerprinting techniques to ensure reliable operation will be crucial. Additionally, you will create data processing pipelines to clean, normalize, and enrich collected information, develop monitoring systems to detect and adapt to website changes, and build user-friendly interfaces for controlling automation systems. It is essential to stay current with evolving anti-automation technologies and develop countermeasures. To excel in this role, you should have 4+ years of experience with web scraping and automation technologies, strong programming skills in Python and JavaScript, and advanced experience with browser automation libraries such as Puppeteer, Playwright, or Selenium. Expertise in parsing HTML/DOM structures and handling dynamic content (AJAX, SPAs), experience implementing IP rotation, user-agent cycling, and browser fingerprinting solutions, and knowledge of data storage solutions for organizing and querying large datasets are required. Familiarity with headless browser environments, containerization, and deploying automation scripts in Lambda AWS is also necessary. Preferred qualifications include experience with proxy management services and residential IP networks, knowledge of machine learning techniques for solving CAPTCHAs and pattern recognition, experience with natural language processing for enhancing form completion logic, familiarity with ethical and legal considerations surrounding web automation, experience developing against enterprise authentication systems, and understanding of job application workflows and ATS (Applicant Tracking Systems). In return, we offer a competitive salary and benefits package, a flexible work environment, cutting-edge technical challenges, and the opportunity to shape the development of next-generation automation systems.,

Posted 5 days ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.