Web Scraping Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Web Scraping Engineer - Global Team based in India.

This is a fully remote engineering opportunity focused on building reliable, scalable web data extraction systems. You will design, refactor, and maintain robust web scrapers that support critical business insights and customer-facing products. The role involves solving complex challenges around dynamic websites, browser rendering, session management, and anti-bot protections. You will collaborate with analysts and technical stakeholders to ensure high-quality, resilient data pipelines. As part of a growing engineering function, you will have meaningful influence over tooling, processes, and technical best practices. The role offers flexibility while working primarily within an 11 AM–8 PM IST schedule.

• Design, build, refactor, and maintain robust web scraping solutions, improving reliability, maintainability, performance, and scalability.
• Apply clean coding practices, modular architecture, code reviews, and sustainable engineering approaches across scraping projects.
• Implement advanced scraping techniques involving cookies, headers, user-agent rotation, proxies, fingerprinting, session management, and dynamic content handling.
• Navigate complex DOM structures and browser-rendered experiences while developing resilient data extraction workflows.
• Partner with analysts and other stakeholders to understand requirements, align scraping targets, and maintain high standards of data quality.
• Create monitoring, logging, metrics, and alerting solutions to identify scraper failures and performance issues quickly.
• Troubleshoot bottlenecks, availability issues, and scaling challenges while proactively improving scraper efficiency and resilience.
• Recommend new tools, methodologies, and technologies that strengthen web data extraction capabilities.
• Document solutions and provide guidance to internal stakeholders on effective use of scraped data.
• Stay current with evolving web technologies, bot-detection techniques, and innovative approaches to data extraction.

• 3+ years of professional experience working with web scraping frameworks such as Selenium, Playwright, or Puppeteer.
• Strong understanding of HTTP, RESTful APIs, HTML parsing, browser rendering, and TLS/SSL mechanics.
• Hands-on expertise with advanced fingerprinting and evasion techniques, including browser fingerprint spoofing and request-signature manipulation.
• Deep experience managing cookies, headers, session states, and proxy rotation, including residential and data-center proxies.
• Experience developing or maintaining logging, monitoring, metrics, and alerting systems for high-availability applications.
• Strong troubleshooting and analytical abilities, with a focus on optimizing scraping solutions for reliability, efficiency, and scalability.
• Ability to communicate clearly and effectively in English with both technical and non-technical stakeholders.
• Strong ownership mindset, attention to detail, and enthusiasm for solving complex technical problems.
• Ability to work collaboratively in a distributed, cross-functional environment and proactively identify opportunities for improvement.

• Fully remote position based in India, with standard working hours of 11 AM–8 PM IST and flexibility around working hours.
• Competitive salary and comprehensive benefits package.
• Paid vacation time and parental leave.
• Team events and opportunities to connect with colleagues in a distributed environment.
• Learning reimbursement to support continuous professional development.
• Career growth based on impact, skill development, ownership, and contribution rather than tenure or office presence.
• A collaborative culture built around learning, self-improvement, respect, trust, and meaningful technical ownership.
• Inclusive equal-opportunity workplace committed to supporting diverse perspectives and backgrounds.