Job Details
Experience Needed:
Career Level:
Education Level:
Salary:
Job Categories:
Skills And Tools:
Job Description
The Big data engineer will be part of the big data division that applies scientific, mathematical and social principles to design, build, and maintain technology products, systems and solutions. These technology products and solutions provide amazing customer experiences while meeting the needs of the business.
Job Duties
- This position will be responsible for implementation and support of the Enterprise Hadoop environment, including design, capacity planning, cluster set up, performance tuning, monitoring, infrastructure planning, scale up/out and administration.
- Should work towards highly available and performing within agreed on service levels.
- The candidate will apply a strong understanding of Hadoop eco system such as HDFS, YARN, Map Reduce, HBase, Zookeeper, Pig, Hadoop streaming, and Hive.
- The candidate should have experience in installing, administering, and supporting Linux operating systems/hardware in an enterprise environment.
- System administration and programming skills.
- Storage, performance tuning and capacity management of Hadoop clusters and Hadoop MapReduce routines.
- Setup, configuration and management of security for Hadoop clusters using SSL, Kerberos and integration with LDAP/AD at an Enterprise level.
- Monitor Hadoop cluster connectivity and security.
- Manage and review Hadoop log files.
- AWS or Cloud Technology experience preferred.
- Develop and document best practices.
Job Requirements
Skills and Qualifications
- Bachelor's Degree in Computer Science or equivalent experience.
- Two years experience as Big data engineer in a large production environment.
- Strong experience in HDFS support, maintenance, backup, restore, disaster recovery, failovers management and troubleshooting.
- A deep understanding of Big Data, Hadoop Architecture and YARN/Map Reduce concepts
- Good verbal and written communication skill.
- Working knowledge of Linux operating system.
- Network and system administration experience, preferably in large-scale data center environments.
- Highly proficient in one of the following programming languages: Java, Go, C/C++.
- Good scripting skills and ability to pick up new ones.
- Systematic problem solving approach and knowledge of algorithms, data structures and complexity analysis.
