Big Data Developer / Specialist

Big Data Developer Job Description:

We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.

Big Data Developer Responsibilities:

Expertise of Python, Flask, Hadoop, Spark, Spark Streaming, Spark-Sql, Kafka, Impala, Memsql, MySql, MapReduce, HDFS, SHARK, HIVE, Linux, Tableau and Mongo-DB.

Working knowledge of Apache Flume, Storm, Nifi, tensorflow, Cassandra, Solr and Elasticsearch.

Developed data processing algorithm with a very high throughput time using kafka and spark

Designed and implemented the architecture for real time full text searching, historical jobs and advance analytics using big data tools and technologies.

Integrated multiple components to avoid data loss, increase reliability and performance of the system.

Implemented, configured and create web service for Data Modelling.

Interest extraction algorithms (designed algorithms & coded scripts for classifying users according to their interests (based on user generated content)

Expertise in using Linux and Python for data analysis, text mining and system administration.

Automation of manual tasks so that they run without human intervention

Specialties: Data Ingestion, ETL, Data Cleansing, Data Processing, Data Warehousing, Business Intelligence, Text Mining, Data Mining, Automation, linux Admin.

Dealing with Twitter API's and tweet data structure

Big Data Developer Qualifications:

Degree in Computer Science.

Strong organizational and project management skills..

Good problem solving skills.

Excellent in Algorithms and Object Oriented programming