Senior Data Engineer
- 10Viewed
- 0In Consideration
- 7Not Selected
Job Details
Skills And Tools:
Job Description
Re-Vision:
Re-Vision is the leading provider of mobile self-scanning solutions. Benefiting from over 10 years of experience and built around retail best practices, our My-Scan Omni-Channel platform is implemented in different supermarkets and hypermarkets in over 30 countries all over the world. Re-Vision’s short-term roadmap will be mainly focused on expanding the product portfolio with Artificial Intelligence/Machine Learning to detect fraud and upsell to the customers.
Role:
Data Engineers are responsible for implementing, expanding, and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams.
They combine raw information from different sources to create consistent and machine-readable formats. They also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling.
Data Engineers support our software developers, architects, data analysts, and data scientists on data initiatives and ensure optimal data delivery architecture that is consistent throughout ongoing projects.
The right candidate will be excited by the prospect of optimizing or even re-designing our company's data architecture to support our next generation of products and data initiatives.
Responsibilities:
- Create and maintain optimal data pipeline architecture.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Work with data and analytics experts to achieve greater functionality in our data systems.
- Explore ways to enhance data quality and reliability.
- Identify opportunities for data acquisition.
Location: Gouna, Red Sea, Egypt.
Accommodation allowance is provided.
Job Requirements
- Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets.
- Advanced SQL knowledge and experience working with relational databases, query authoring (SQL), as well as familiarity with a variety of databases.
- Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
- A successful history of manipulating, processing, and extracting value from large disconnected datasets.
- Solid knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Strong project management and organizational skills.
- Experience supporting and working with cross-functional teams in a dynamic environment
- Experience with relational SQL and NoSQL databases, including Postgres and MongoDB.
- Experience with AZURE cloud services: ACR, AKS, Data factory, Event Hub,.etc.
- Experience with microservices like Kubernetes and managing the deployment for other services.
- Data modeling: knowledge of data modeling for both data warehousing and Big Data.
- Foundation software engineering: Agile, DevOps, architecture design, service-oriented architecture.
- Cloud Data Integration tool: AWS Glue or Azure Data Factory.
- Hadoop Ecosystems tools like Hive, HBase, Spark, Kafka is a plus.
- CI: Git.
- Bash Scripting.
- Programming: Experience with at least one scripting language (e.g. Python, Scala).
- Machine Learning, Deep Learning: A certain level of understanding in these areas is an asset in order to be able to work in collaboration with data scientists.
- Ability to make sound decisions, even in the absence of complete information.
- Strong attention to detail and ability to think critically and conceptually.
- Certified Azure Data Engineer Associate or similar certification is a plus.
