Job Description
Key Responsibilities:
- Design, develop, and maintain scalable and efficient data pipelines using Azure Databricks and its native services, such as Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, and Azure Blob Storage.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and implement solutions that meet business needs.
- Perform data profiling, cleansing, and transformation to ensure data quality, consistency, and accuracy throughout the data lifecycle.
- Optimize data pipelines for performance, reliability, and cost-effectiveness, leveraging best practices and automation techniques.
- Implement data security and compliance measures to protect sensitive information and ensure regulatory compliance.
- Monitor and troubleshoot data pipeline issues, proactively identifying and resolving issues to minimize downtime and ensure data availability.
- Document technical designs, data lineage, and system configurations to support knowledge sharing and facilitate collaboration.
- Stay current with industry trends, emerging technologies, and best practices in data engineering and cloud computing, and recommend opportunities for innovation and optimization.
Qualifications:
- Bachelor’s or master’s degree in computer science, Engineering, or a related field.
- 3 to 6 years of experience working as a Data Engineer, with a focus on cloud-based data platforms and services, particularly Azure Databricks and its native services.
- Strong proficiency in SQL, Python, and Spark, with hands-on experience developing ETL processes and data pipelines.
- Experience working with Azure cloud services such as Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage, and Azure Data Lake Storage.
- Knowledge of data modelling techniques, dimensional modelling, and data warehousing concepts.
- Excellent analytical and problem-solving skills, with the ability to troubleshoot complex data issues and implement effective solutions.
- Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
Copyright © 2024 SAP SuccessFactors, Inc. All rights reserved. These online services are SuccessFactors confident
