Job Description

Job Details

• Implement Databricks Auto Loader for efficient data ingestion and ETL pipelines.

• Optimize data workflows for performance and scalability.

• Ensure data governance and security compliance using Unity Catalog.

• Collaborate with Data Scientists and Analysts to operationalize models.

Required Qualifications:

• Hands-on experience with Databricks (Executive-level exception requirement).

• Expertise in AI/BI Genie, Dashboards, Auto Loader, MLOps workflows, and Unity Catalog.

• Strong programming skills in Python, SQL, and/or Scala.

• Experience in cloud-based data platforms (Azure, AWS, or GCP).

• Excellent problem-solving and collaboration skills.

Preferred:

• Experience with Lilly’s data ecosystem (if applicable).

• Knowledge of data governance, security, and compliance best practices.

Required Knowledge, Skills, and Abilities

• Python
• Databricks
• Etl Pipelines
• SQL