About the company
Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI.
Responsibilities
- Perform advanced Troubleshooting and Root Cause Analysis to resolve performance and reliability issues in Spark, SQL, Delta, Streaming, and Databricks runtime features using tools like Spark UI metrics, Mosaic AI Model Service, DAGs, and event logs.
- Discover requirements for continuous monitoring to detect early performance issues working with R&D and NOC teams to optimize the DNB customer environments.
- Build Rapid POCs, Test/Deploy/Monitor the solutions built by Databricks Engineering to address customer challenges and showcase advanced Spark/ML/AI runtime capabilities aligned with their business goals.
- Develop comprehensive playbooks and maintain a knowledge base of common issues and solutions for Spark, ML, and AI workflows.
- Train customer engineering and business teams on best practices in performance tuning, debugging, and effectively leveraging Databricks Features.
- Pilot new best practices processes/ programs, champion process improvements, and collaborate with cross-functional teams to enhance the customer experience.
- Advocate for customers in business review meetings and maintain close relationships as a trusted advisor and primary technical point of contact.
- Collaborate onsite with Field Engineering, Sales, and Product teams during customer engagements and technical presentations to provide rapid solutions to production-impacting issues.
Requirements
- 8–12 years of experience designing, building, and troubleshooting distributed computing applications, with 4+ years delivering production-scale Spark/ML/AI solutions using Python, Java, or Scala.
- Hands-on expertise with Data Lakes, SQL-based databases, and Cloud-based Data Warehousing/ETL tools like Snowflake, Redshift, Bigquery, etc.
- Deep knowledge of Spark core internals, Delta/Iceberg, JVM optimization, and memory management, with additional proficiency in AI ecosystems like Machine Learning, Deep Learning, and Generative AI.
- Practical experience with AWS, Azure, or GCP, coupled with expertise in building and managing CI/CD pipelines, monitoring, and alerting systems.
- 3–5 years in customer-facing roles such as Technical Account Manager or Solutions Architect.
- Proven ability to anticipate, identify, and mitigate risks while planning solutions for production challenges.
- Proven ability to work with cross-functional teams and senior leadership to address roadblocks, mitigate risks, and drive customer success while creating impactful documentation for self-service solutions.