Senior Data Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. The partner is looking for a Senior Data Engineer - Full Stack based in India.

• Partner with business and technical stakeholders to understand workflows, clarify objectives, translate ambiguous requirements into technical solutions, and establish delivery plans.
• Design, develop, and maintain end-to-end data products using Databricks, Delta Lake, SQL, Python, PySpark, and related technologies.
• Build reliable batch, incremental, streaming, and near-real-time data pipelines using Kafka and comparable event-streaming technologies.
• Design event-driven architectures and integrate operational systems with downstream data consumers.
• Develop backend services, REST APIs, and integrations that expose governed data to applications and operational workflows.
• Build lightweight applications, dashboards, and user interfaces in collaboration with product, analytics, BI, and UX teams.
• Rapidly prototype solutions, validate concepts with stakeholders, and transition successful prototypes into scalable production capabilities.
• Create scalable data models and curated datasets supporting analytics, reporting, AI/ML initiatives, and operational decision-making.
• Implement data-quality, security, lineage, and governance controls using Databricks, Unity Catalog, and comparable technologies.
• Establish automated testing, CI/CD, monitoring, alerting, documentation, and deployment practices across the complete data-product lifecycle.
• Optimize pipelines, queries, streaming workloads, APIs, and applications for performance, reliability, scalability, and cost efficiency.
• Troubleshoot and resolve issues across source systems, streaming platforms, pipelines, data models, APIs, applications, and downstream consumers.
• Collaborate with platform and product engineering teams to turn recurring stakeholder requirements into reusable data capabilities.
• Lead technical design and code reviews, mentor engineers, and contribute to stronger full-stack data-engineering practices and standards.

• 5+ years of experience in data engineering, software engineering, or a related discipline, including ownership of production data solutions.
• Strong proficiency in SQL, Python, and PySpark, with experience developing reliable, production-grade data pipelines and products.
• Hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog, or comparable data-platform and governance technologies.
• Experience building and supporting streaming or near-real-time pipelines using Kafka, Kinesis, Event Hubs, or similar technologies.
• Strong understanding of event-driven architecture, message processing, schema evolution, data consistency, and streaming reliability.
• Experience delivering full-stack solutions spanning data pipelines, backend services or APIs, and lightweight user-facing applications.
• Experience developing REST APIs, services, and integrations, ideally using Python frameworks such as FastAPI, Flask, or comparable tools.
• Experience with AWS, Azure, or GCP and cloud-native architecture patterns.
• Strong knowledge of data modeling, data warehousing, distributed processing, and analytics-oriented data design.
• Experience with Git, automated testing, CI/CD, monitoring, and production deployment practices.
• Demonstrated ability to work directly with stakeholders, navigate ambiguity, and translate business challenges into practical technical solutions.
• Strong communication, analytical, problem-solving, technical leadership, and end-to-end ownership skills.
• Ability to balance rapid delivery with maintainability, security, governance, scalability, and operational reliability.
• Experience with React or another modern frontend framework is a plus.
• Familiarity with infrastructure as code, containerization, and automated cloud deployment is advantageous.
• Experience with AI/ML pipelines, feature engineering, retrieval systems, or generative AI use cases is beneficial.
• Knowledge of data observability, platform engineering, data-product management, or reusable data-platform capabilities is valued.
• Background in forward-deployed engineering, solutions engineering, technical consulting, or stakeholder-embedded delivery is helpful.
• Prior experience mentoring engineers and working in Agile or Scrum environments is a plus.

• Remote work within India.
• Generous time-off policies.
• Comprehensive benefits designed to support employees' wellbeing and professional needs.
• Education and learning support.
• Wellness and lifestyle resources.
• Opportunity to work with modern data and cloud technologies, including Databricks, Spark, Kafka, and cloud-native platforms.
• Exposure to full-stack data-product development spanning ingestion, modeling, APIs, applications, governance, and production operations.
• Opportunities for technical leadership, mentoring, innovation, and professional growth.