Job Details
Experience Needed:
Career Level:
Education Level:
Salary:
Job Categories:
Skills And Tools:
Job Description
( Our Client Multinational digital & technology company operate in over 26 countries from 7 office-hubs across the globe )
- Our Data Engineer is responsible for implementing data ingestion, continuous integration, monitoring & orchestration on cloud for company business entity. Needs to assist the team in the successful execution, performance optimization of the cloud data warehouse & cost estimation of serverless cloud components.
Responsibilities include: - Design, construct, install and maintain data management systems using Spark/PySpark, AWS Glue, Dataflow, or similar cloud ETL Tools
- Handle vast amounts structured, semi-structured data on cloud
- Executing Data orchestration, Workflows & ETL Scheduling Tools like Apache Airflow, luigi & step functions.
- Well versed in one of the scripting languages: Python (preferably), Scala or Java, Bash/Shell Scripting and SQL.
- Recommend different ways to constantly improve data reliability and quality.
- Employ an array of technological languages and tools to connect systems together
- Recommend different ways to constantly improve data reliability and quality
- Should be able to clearly communicate results & ideas within the team
- Communicate effectively to all levels of the organization.
- Help in translating the ETL processes from SQL Warehouse/SAP ETL to Cloud Standard and other ETL Tools
Job Requirements
- University degree in Computer Science Engineering or Statistics
- 3+ years of experience in building back-end applications and 2+ years in large-scale data/software systems with high performance, scalability and availability parameters
- Thorough understanding of the security features, access layers and data monitoring components on AWS or GCP
- Must have a clear understanding of data unification, centralization, and data lakes on cloud
- Good understanding and know-how of using microservices
- Decent exposure to Cloud data warehouses like BigQuery, Redshift or Snowflake & proficient with Cloud Standard SQL
- Good working knowledge in building Data Lakes, Data warehouses, Architecture, & cloud infrastructure components