Elastic Hadoop on scale-out POWER Servers

You must have heard about the new scale-out POWER8™servers from IBM as well as OpenPOWER members. Some of you might also want to evaluate it for your big data workloads, but have the following concerns:

  • Will Apache Hadoop work on IBM Power® servers?
  • Can my IT team use these Power servers without requiring any reskilling?
  • Can I manage the system similar to any other Linux® and Hadoop system?
  • Can I use my existing configuration automation mechanism based on Puppet with Power servers?
  • Can I use OpenStack to set up a virtual Hadoop cluster using the Power servers? ,
  • and more …

You are not alone to ponder over these questions. As a matter of fact, I have heard these questions numerous times! These are some of the very common questions from people who are either new to Power servers or have only heard about traditional Power servers running IBM PowerVM.

These questions are blockers, which prevent users like you from exploiting the advantages of IBM Power Architecture for specific workloads, such as big data.

There is a new range of servers from IBM and OpenPOWER members that are very similar to the servers based on Intel architecture from usability and management point of view. These are usually referred to as scale-out servers. You don’t have to do any re-learning to leverage the benefits of the Power platform for your big data workloads running on these scale-out Power servers.

You don’t need a different management stack to work with the scale-out Power servers. You can use these servers as bare metal servers running standard Linux distributions or you can use them as virtualized servers leveraging kernel-based virtual machine (KVM) and manage them by leveraging OpenStack, oVirt, Chef, Puppet, and so on.

As a proof point, few of my colleagues and I have written a white paper describing Apache Hadoop cluster setup on scale-out Power servers running KVM and managed by OpenStack. This is our humble attempt to address the questions that is there in the mind of almost any new Power server user. I welcome your feedback, suggestions, and ideas.

Pradipta Kumar Banerjee

I'm a Cloud and Linux/ OpenSource enthusiast, with 16 years of industry experience at IBM. You can find more details about me here - Linkedin

You may also like...