Hadoop Engineer, Remote

Our client and partner, the world's largest travel site, operating at scale for almost half a million unique visitors each month, needs to expand their data platform team.

As a data-driven company, this user-generated platform needs a Hadoop Engineer who will share with his teammates the responsibility of maintaining the company's petabyte-scale data lake and the ecosystem of tools that surround it. This position open at Pentalog requires hands-on operational experience with Big Data technology and securing distributed platforms.

The tech stack is a hybrid open-source Hadoop cloud, house-built distributed ETL tools, and public cloud technologies that help to drive the analytics of the company. A successful engineer will be able to solve complex security problems in the hybrid-cloud environment while working on a team spanning multiple time-zones.

Responsibilities

• Write code, as you create today's solutions for tomorrow's problems;
• Use your expertise to build the kinds of tools that expand the modern Big Data ecosystem;
• Work with a team of talented engineers to identify the shortcomings in Big Data technologies;
• Apply cloud engineering best practices and develop cloud-first solutions to Big Data challenges.

Job requirements

• 3+ years of in-depth technical experience with modern Big Data technologies such as Hadoop, Hive, Presto and Spark;
• Familiarity with the Linux operating system and the Bash shell;
• General software engineering expertise (Java - preferred, Python and Kotlin);
• The ability to communicate effectively in an operations environment;
• Willingness to "get your hands dirty" in a modern Big Data environment
• Very good level of English.