Role Summary:

A Senior Data Engineer is responsible for building and maintaining data systems, constructing datasets that are easy to analyze and support company requirements. Senior data engineers provide hands-on coding with the ability to outline solution design with coding best practices and perform code reviews.

The Senior Data Engineer is responsible for overseeing junior data engineering activities and aiding building the business’ data collection systems and processing pipelines. The role of the Senior Data Engineer is responsible for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department.
 

Primary Responsibilities:

• Data Pipeline Development: Design, develop, and maintain scalable ETL/ELT data pipelines for structured and unstructured data.
• Database Optimization: Collaborate with engineers to optimize database schemas and ensure efficient query performance.
• Data Governance & Security: Implement data governance best practices, ensuring compliance with HIPAA, GDPR, and industry regulations.
• Data Architecture: Develop data warehouse and data lake solutions to support analytics and machine learning workloads.
• Automation & Integration: Automate data processes using Python, SQL, or Spark, integrating with cloud platforms like Azure, AWS, or GCP.
• BI & Analytics Support: Work closely with BI & analytics teams to support dashboarding and reporting needs.
• Monitoring & Troubleshooting: Ensure high data pipeline performance, troubleshoot issues, and implement proactive monitoring.
• Metadata Management: Establish data cataloging, metadata management, and lineage tracking for better data governance.