Senior Data Engineer

Job Title: Senior Data Engineer (Azure Data Engineer)

Job Location: Egypt (Remote/Hybrid)

Experience: at least 5 years in Data Engineering/Data Pipelines using Azure

We’re looking for a Senior Data Engineer with 5+ years of experience building and managing data pipelines, primarily using Azure technologies. You’ll be responsible for ingesting, transforming, and integrating data from diverse sources to support analytics, master data management, and business decision-making.

Roles & Responsibilties:

Data Pipeline Development:

• Build scalable ETL/ELT pipelines using Azure Data Factory.
• Ensure smooth data movement across staging, processing, and analytical layers.
• Optional: Use Azure Synapse Analytics for advanced processing.

Data Source Analysis:

• Analyze multiple structured and unstructured data sources to understand schemas, data quality, and relationships.
• Assess and document the metadata, formats, and business rules of incoming data sets.

Data Profiling & Matching:

• Perform comprehensive data profiling to assess completeness, accuracy, uniqueness, and consistency of data.
• Develop logic to match and reconcile records across different systems using keys, rules, or fuzzy matching techniques.

Master Data Management (MDM):

• Design and implement processes for building and maintaining golden master records.
• Support creation and maintenance of master data across domains such as customer, product, asset, etc.
• Handle data deduplication and standardization.

ETL & Data Pipelines:

• Build scalable data pipelines using Azure Data Factory for ingesting, transforming, and cleansing data from multiple sources.
• Ensure efficient data movement across staging, processing, and analytical layers.
• Leverage Azure Synapse Analytics (optional) for advanced data processing and analytics.

Collaboration & Governance:

• Work closely with business analysts, data stewards, and application owners to understand data definitions and ownership.
• Enforce data governance standards and best practices.

Performance & Optimization:

• Monitor data workflows and optimize performance for large-scale processing.
• Troubleshoot data issues and implement robust logging and monitoring.

Required Qualifications:

• 5+ years of experience in data engineering and pipeline development.
• Strong hands-on expertise with Azure Data Factory and SQL.
• Proficiency in Python or PySpark for data processing.
• Experience in data profiling, matching, and master data management (MDM).
• Fluent in Arabic and English.

Preferred Skills:

• Experience with Azure Synapse, Azure Purview, or similar tools.
• Familiarity with data modeling, cataloging, and privacy standards (e.g., GDPR).
• Knowledge of ETL tools like Talend, SSIS, or Informatica is a plus.