Job Details
Skills And Tools:
Job Description
The successful candidate will be responsible for expanding and optimizing the company's data and data pipeline architecture. This includes optimizing data flow and collection for cross-functional teams and different source systems. The ideal candidate should have experience in building and optimizing data pipelines and systems from the ground up and has the ability to work and enhance in current data pipelines. The Data Engineer will also collaborate with data analysts and data scientists to support different business teams on data initiatives. They will ensure that the optimal data delivery architecture is consistent throughout ongoing projects. In addition, the Data Engineer should be self-directed and comfortable with supporting the data needs of multiple teams, systems, and products
Job Requirements
- Bachelor’s degree in computer science, Information Systems, software/computer engineering, or another quantitative field.
- 2+ year of experience with the suite of open source big data technologies and platforms (Cloudera/Hortonworks, Hadoop Ecosystems, Spark, Kafka, Hive).
- Familiarity with Hadoop ecosystem and its components: obviously, a must.
- Hands-on experience in HDFS, Hive, Sqoop, Spark, Impala, Kudo and Airflow is a must.
- Advanced work experience in SQL and database manipulation, ETL development, and data warehousing, is a must.
- Good knowledge of Linux/UNIX commands and shell scripting.
- Building complete infrastructure to ingest, transform & store data for further analysis & business requirement and maintaining the data security and privacy.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- Experience with NoSQL databases, including Postgres and Cassandra is a plus.
- Excellent communication skills and dynamic team player.
- Demonstrated ability to adapt to new technologies and learn quickly.
Featured Jobs
Similar Jobs
- Senior Data Warehousing & Busi...The Micro, Small & Medium Enterprise Development Agency - Dokki, Giza4 days ago
