About the company
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Staff Data Engineer based in the United States.
Responsibilities
- Lead enterprise-wide data architecture strategy and define long-term data platform design principles
- Evaluate and approve major technical designs, establish engineering best practices, and advise leadership on data technology decisions
- Champion data governance, data democratization, and scalable standards
- Design, develop, and optimize ETL pipelines using Apache Airflow, dbt, Python, and Snowflake
- Build and manage large-scale data ingestion and transformation workflows
- Develop streaming pipelines using Confluent Kafka
- Use Spark and other big data technologies for batch processing and streaming analytics
- Build and maintain CI/CD pipelines and automate deployments
- Establish enterprise-wide data quality frameworks, testing procedures, and validation processes
- Monitor pipeline health, freshness, availability, performance, and SLA compliance
- Optimize cloud infrastructure and containerized workloads
Requirements
- Master's degree or foreign equivalent in Data Science, Applied Computer Science, Computer Science, or related field
- At least 4 years of experience in each of the following:
- Developing and optimizing scalable data pipelines using Apache Airflow and Python
- Building streaming data pipelines using Kafka
- Tuning big data systems using Kafka and Hive
- Using Apache Spark for batch processing and streaming analytics
- Writing complex SQL queries
- Using AWS for data processing and large-volume workloads
- Automating code and process deployments using Python
- Working with cloud-based infrastructure
- Building Datadog dashboards
- Using Docker and Kubernetes
- Implementing CI/CD pipelines using Jenkins
- Using GitHub and working with Windows and Linux
- Strong understanding of enterprise data architecture, data modeling, data quality, and pipeline reliability
- Experience collaborating with cross-functional teams
Conditions
- Annual salary ranging from $175,000 to $200,000
- Fully remote work arrangement, with telecommuting available anywhere in the United States
- Paid parental leave
- 401(k) retirement plan with employer matching contributions
- Flexible vacation time
- Medical, dental, and vision coverage
- Commuter benefits