3 - 7 years

0 Lacs

Posted:19 hours ago| Platform: Shine logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

As a Backend Engineer specializing in Data Collection & Processing, your role will involve designing and implementing scalable data ingestion pipelines to collect structured and unstructured data from various sources such as browsers, APIs, and external sources. You will be responsible for developing robust ETL/ELT pipelines to clean, transform, and normalize large datasets. Additionally, maintaining secure, high-performance RESTful and event-driven APIs will be a crucial part of your responsibilities. Key Responsibilities: - Design and implement scalable data ingestion pipelines for collecting structured and unstructured data from browsers, APIs, and external sources. - Develop robust ETL/ELT pipelines to clean, transform, and normalize large datasets. - Build and maintain secure, high-performance RESTful and event-driven APIs. - Integrate with third-party data sources including public APIs, internal systems, and dark web monitoring services. - Ensure high availability, fault tolerance, and real-time performance across services. - Collaborate closely with data scientists, threat analysts, and frontend engineers to support end-to-end data workflows. - Implement security, authentication, and authorization best practices across backend systems. - Write clean, maintainable, and well-tested code. Qualifications: - Bachelors or Masters degree in Computer Science, Engineering, or related field. - 3+ years of backend development experience with languages such as Python, Node.js, Go, or Java. - Strong understanding of REST APIs, microservices architecture, and event-driven systems. - Experience with data pipeline technologies: Apache Kafka, Redis Streams, RabbitMQ, Airflow, or similar. - Experience working with databases like PostgreSQL, MongoDB, Elasticsearch, or ClickHouse. - Familiarity with containerized environments (Docker, Kubernetes). - Exposure to data privacy and security best practices. - Experience with cloud platforms (AWS, GCP, or Azure). Additional Details: As a Backend Engineer for Data Collection & Processing, you will have the opportunity to work on critical aspects of the data pipeline architecture that directly contribute to fraud detection and secure transactions. Your role will involve collaborating with various teams to ensure the efficiency and effectiveness of data workflows.,

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You

bengaluru, karnataka, india

pune, maharashtra, india

bengaluru, karnataka, india