Usman Ahmed

Senior Data Consultant

• Job Details:Utilizing Azure Fabric to ingest data from data lake, executing complex transformations with Spark SQL and PySpark. Creating refined data views and presented insightful dashboards using Power BI for business decision-making. Designing and executing Azure pipelines (Azure Data Factory, Azure Data Lake, Databricks, SSIS), optimizing extraction, transformation, and loading. Proficient in Azure DevOps for CI/CD pipelines, automating deployments, and version control to enhance development efficiency. Applied medallion architecture, performing unit testing in development and migrating changes through testing to production environments.

Senior Data Engineer

• Job Details:Automated manual processes, enhanced data delivery efficiency post-Talend migration to Python/PySpark with Airflow. Led data engineering initiatives, extracting diverse datasets using Python/Spark, Hadoop, Talend, and Informatica. Formatted data into Parquet and CSV, orchestrated seamless ingestion into PostgreSQL with Airflow, empowering AI team. Performed data analysis and engineering for clients like AT&T, DirecTV, Bouygues, and Virgin Media, utilizing data warehousing. Built efficient ETL/ELT pipelines to transfer millions of records from Snowflake/Databricks/Palantir using Spark/Talend. Conducted feature engineering on CRM data, building predictive models with ML algorithms (GBM, R-part, Random Forest). Set up containerized environment with Docker, enhancing scalability for Airflow using Docker Swarm on multiple Linux servers.

Data Engineer II

• Job Details:Developed and maintained robust data warehouse solutions, optimized ETL processes to enhance data accessibility and reliability. Led the implementation of ETL processes using Informatica Big Data Management and Talend to handle high-volume data. Migrated MySQL to GP/PostgreSQL, established new data sources with Talend and PySpark, and developed testing scenarios. Led CI/CD implementation for ETL, maintaining version-controlled scripts on GitHub/GitLab for efficiency and collaboration.