Senior Data Product Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Product Engineer based in Brazil.

This is a senior-level opportunity to help transform a research-driven data collection platform into a scalable, production-grade data product for cyber risk analytics. You will design and operate distributed web-scraping systems capable of collecting large volumes of data from thousands of internet sources.

• Architect, build, and scale distributed web-scraping systems capable of reliably collecting data from thousands of online sources.
• Develop robust approaches for handling anti-scraping and bot-protection mechanisms, including proxy rotation, CAPTCHAs, rate limiting, IP blocking, browser fingerprinting, and headless browsing.
• Build and maintain data extraction workflows using technologies such as Scrapy, Playwright, Selenium, Puppeteer, BeautifulSoup, lxml, XPath, and CSS selectors.
• Refactor and productionize existing Python data-collection pipelines, strengthening reliability, observability, error handling, retry mechanisms, monitoring, and alerting.
• Build schedulable, containerized ingestion workflows using Airflow or comparable orchestration technologies.
• Design PostgreSQL schemas, views, partitioning strategies, and efficient data-access patterns for processed web data.
• Develop and maintain cloud infrastructure and data services using AWS technologies including S3, EKS, IAM/IRSA, Parameter Store, and ECR.
• Build and maintain the data-access layer using GraphQL and Hasura, while collaborating with Data Science and SaaS teams on reliable API contracts.
• Support the integration of existing ML components and ensure dependable movement of data from collection and ingestion through consumption.
• Take end-to-end ownership of the data product, contribute to code reviews, and continuously improve engineering practices, architecture, and system reliability.

• Strong hands-on experience designing and operating advanced web-scraping systems at scale.
• Demonstrated experience working with anti-scraping and bot-protection mechanisms, including proxy pools, headless browsers, rate limiting, IP blocking, fingerprinting, or similar techniques.
• Strong professional experience with Python and data processing, including production-grade pipeline development.
• Extensive experience with web-scraping and DOM-parsing technologies such as Scrapy, Playwright, Selenium, Puppeteer, BeautifulSoup, lxml, XPath, or CSS selectors.
• Hands-on AWS experience, particularly with S3 and boto3; familiarity with EKS, IAM/IRSA, Parameter Store, and ECR is highly valuable.
• Experience building reliable, schedulable data pipelines with Airflow or an equivalent orchestration platform.
• Strong knowledge of PostgreSQL, SQL, relational database fundamentals, schema design, and data-access patterns.
• Ability to take end-to-end ownership of technical solutions, make sound architectural decisions, and work effectively with a high degree of autonomy.
• Experience participating in code reviews and maintaining strong software engineering standards.
• Familiarity with GraphQL, Hasura, Redis, Elasticsearch, Docker, Kubernetes, Helm, or GitHub Actions is an advantage.
• Experience with cybersecurity data, NLP/ML pipelines, or data-as-a-product environments is a plus.
• English proficiency at B2 level or higher, with the ability to communicate effectively with technical and cross-functional stakeholders.

• 100% remote work across LATAM.
• Flexible working hours.
• Contractor engagement with compensation paid in USD.
• Flexible, self-managed vacation time.
• Sick leave, personal days, and public holidays.
• Paternity and maternity leave.
• Study leave and moving days.
• Company-provided equipment and work materials.
• Training in technology and engineering best practices.
• Access to books, technical talks, and continuous learning opportunities.
• In-house English classes.
• Continuous feedback and 1:1 career development sessions.
• Internal events and team activities.
• Birthday day off.