Hadoop Developer Job Description Template

Company Introduction

{{ Write a short and catchy paragraph about your company. Make sure to provide information about the company’s culture, perks, and benefits. Mention office hours, remote working possibilities, and everything else that you think makes your company interesting. }}

Job Description

We are looking for a Hadoop developer to help us build large-scale data storage and processing software and infrastructure. Knowledge of existing tools is essential, as is the capacity to write software using the Hadoop API.

Responsibilities

• Write software to interact with HDFS and MapReduce.
• Assess requirements and evaluate existing solutions.
• Build, operate, monitor, and troubleshoot Hadoop infrastructure.
• Develop tools and libraries, and maintain processes for other engineers to access data and write MapReduce programs.
• Develop documentation and playbooks to operate Hadoop infrastructure.
• Evaluate and use hosted solutions on AWS / Google Cloud / Azure. {{If you’d like to use hosted solutions}}
• Write scalable and maintainable ETLs. {{If you need to run ETLs}}
• Understand Hadoop’s security mechanisms and implement Hadoop security. {{If you need fine-grained security within your organization}}
• Write software to ingest data into Hadoop.

Skills

• You know the JVM runtime, the Java language, and ideally another JVM-based programming language. {{Mention Java version if it matters to your existing code}}
• You know computer science fundamentals, particularly algorithmic complexity.
• You know trade-offs in distributed systems.
• You’re proficient at software engineering principles that produce maintainable software and you can use them in practice.
• You have worked with a Hadoop distribution.
• You have worked with one or more computation frameworks, such as Spark.
• You’re familiar with HBase, Kafka, ZooKeeper, or other Apache software. {{Add or remove Apache software based on need}}
• You know Linux and its operation, networking, and security.
• You know how to efficiently move large data around.