Software Engineer

Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure.

Built by engineers, for engineers. From large-scale GPU orchestration to inference optimization, we own the hard problems across compute, storage, networking and applied AI.

Listed on Nasdaq (NBIS) and headquartered in Amsterdam, we have a global footprint with R&D hubs across Europe, the UK, North America and Israel. Our team of 1,500+ includes hundreds of engineers with deep expertise across hardware, software and AI R&D.

We are building an agent-native search platform designed specifically for AI systems rather than human users. Our product provides programmatic, low-latency, and observable search APIs that AI agents use to retrieve, filter, and reason over real-world information at scale.

• Design, implement, and operate web-scale crawling systems for acquiring content from the internet.
• Build ingestion workflows for internal and external data sources, including crawlers, structured feeds, and partner integrations.
• Develop crawl scheduling, prioritisation, recrawl policies, and freshness strategies.
• Build systems for URL discovery, deduplication, content extraction, and crawl orchestration.
• Ensure reliable operation of crawling infrastructure under high-throughput conditions.
• Define observability and quality metrics for crawl coverage, freshness, throughput, and content quality.
• Monitor resource usage, bandwidth consumption, and infrastructure cost.
• Collaborate with indexing and ML teams to ensure acquired content meets retrieval and ranking requirements.
• Enable safe experimentation with crawling strategies and content acquisition policies.

• 8+ years of experience building backend or distributed systems.
• Strong Go or C++ expertise.
• Experience with large-scale distributed systems (10k+ RPS, billions of URLs, high-throughput pipelines).
• Understanding of web protocols (HTTP, DNS, TLS), crawling, scraping, and content extraction.
• Experience operating production systems and debugging failures in distributed environments.
• Strong understanding of scalability, fault tolerance, and resource management.
• Strong candidates may also have experience with web crawling; building streaming data pipelines and event-driven systems; Kafka, Pulsar, NATS, RabbitMQ, or similar messaging platforms; designing distributed schedulers, queues, and asynchronous processing systems; Spark, Flink, Beam, or MapReduce; ad tech, social networks, search engines, or other large-scale content platforms.
• Coding interviews are part of the process.

• Competitive compensation.
• Career growth and learning opportunities.
• Flexibility and ownership.
• Collaborative and innovative culture.
• Opportunity to work on impactful AI projects.
• International environment and talented teams.
• Applicants must be authorized to work in the country in which they apply and will be required to provide proof of employment eligibility as a condition of hire.