Yassin Dinana
Big Data Engineer at Coformatique
New Cairo, Cairo, EgyptWork Experience
Big Data EngineerFull Time
- Job Details:o Led the research and development of a versatile data pipeline architecture, handling structured and unstructured data in diverse formats (videos, text, images, audio) using various tools. o Led the data engineering team, assigning tasks, communicating with the client, and leading standup and progress meetings. o Designed and implemented a robust Apache NiFi flow for a batch pipeline to ingest data from multiple sources and efficiently transmit it to various HDFS (Data Lake) directories, optimizing data handling and storage format such as Parquet, Avro, and JSON. o Developed a stream processing pipeline utilizing Apache Kafka, Spark Structured Streaming, HDFS, InfluxDB, Grafana, and Hive for real-time processing. o Automated both batch and stream processing pipelines using Apache Airflow, enabling smooth and automated execution of all services and scripts. o Implemented testing techniques using a large dataset to validate the pipeline's functionality.
Big Data EngineerFull Time
- Job Details:o Assisted in designing and building a data lake, data pipelines, and ETL on the Big Data platform using multiple tools. o Processed data using a combination of technologies, including Cloudera Data Platform (CDP), Apache Flink, Confluent Kafka, Apache Spark, HDFS, Impala, Hive, Cassandra, and Python. The data was processed in batch mode, near real-time, and real-time streaming. o Providing an intelligent solution to business problems using Machine Learning, and Deep Learning algorithms. o Developed multiple reports using Tableau for the client to visualize the appropriate data. o Designed and implemented data quality checks to ensure the accuracy, completeness, and consistency of the data. o Developed and maintained data integration workflows and automation scripts to improve the efficiency and scalability of the data processing pipelines. o Conducted performance tuning and optimization of the data processing pipelines to minimize the processing time and improve the overall throughput.
Machine Learning Engineer - InternInternship
Jul 2020 - Nov 2020 -4 months
Egypt , Cairo
- Job Details:o Developed and evaluated ML algorithms aimed at increasing performance, quality, and accuracy of a hardware emulation project. o Conducted data exploration, feature engineering, and data preprocessing on various datasets to ensure data quality for use in machine learning models. o Provide insights into the data (e.g., cluster identification and spatial relationship of different phenotypes) using machine learning and statistical learning for the clients/researchers to assist their researches. o Worked with cross-functional teams to design and develop scalable, robust, and high-performance models used for predictive analysis. o Conducted experiments and provided analysis to evaluate the effectiveness and efficiency of machine learning models and applied multiple optimization techniques. o Improved the performance of existing ETL processes and SQL queries to generate ground truth datasets by introducing new functionalities. o Stayed up-to-date with the latest machine learning research, techniques, and tools to continually improve skills and expertise.
Software Enginer - InternInternship
Jun 2020 - Jul 2020 -1 month
Egypt , Cairo
- Job Details:o Collaborated with cross-functional teams to design, develop, and deploy software solutions that met customer requirements. o Integrated third-party APIs and services into software applications to extend functionality and enhance user experience. o Participated in Agile/Scrum methodology to manage project development, including sprint planning, daily standups, and retrospectives. o Utilized Git for version control, branching, and merging of software code. o Analyzed and debugged software issues, providing timely solutions to ensure minimal downtime.
Education
MSc in Artificial Intelligence
University of AberdeenJan 2020 - Jan 2021 - 1 Year
BEng in Electronics and Electrical
University of WestminsterJan 2017 - Jan 2020 - 3 yr
Skills
View More
Languages
English
FluentArabic
FluentFrench
Advanced
Training & Certifications
Introduction to Generative AI
Google·2023