Looks like the Big data world is now aiming for some standardization. Recently, a new alliance of Apache Hadoop distribution vendors, solution providers and customers got formed. The alliance named Open Data Platform (ODP) aims to create a reference core of select Big data related projects like Hadoop, Ambari etc.
According to the ODP website:
“The Open Data Platform Initiative (ODP) is a shared industry effort focused on promoting and advancing the state of Apache Hadoop® and Big Data technologies for the enterprise.”
A Hadoop based Big data solution has many components, and there are number of offerings (or distributions) in the market, which packages the required components for easy deployment and usage. However, these offerings (or distributions) are not compatible with one another. It’s usually very difficult to migrate from one platform to another without affecting the applications. ODP is trying to fix this problem.
ODP will provide a tested and certified reference base from, which vendors and solution providers can build their products and solutions. There are many similar analogies available in other fields – for example in the area of Software Defined Networks (SDN) there is OpenDaylight. Then there is Linux Standard Base that was created to reduce the differences between individual Linux distributions, thereby making application porting and certification across distributions easier. There are many such examples.
A brief word on the proposed ODP reference core. Many of you working with Big data solutions would know about Apache Bigtop project, which provides for a tested and integrated Hadoop distribution. How ODP core is related to Bigtop is not very clear to me.
The founding members of ODP are GE, Hortonworks, IBM, Infosys, Pivotal, International Telco, SAS, AltiScale, Capgemini, CenturyLink, EMC, Splunk, Verizon and VMware.
Surprisingly, the top two Hadoop distribution providers – Cloudera and MapR, are not part of this alliance. Any standardization effort missing the important players is ripe ground for competing standards. It needs to be seen how ODP will help with standardization and Big data application interoperability between multiple Hadoop distributions. Only time will tell !