This guide supposed to work as a helper guide beside the official docs. Apache Helix is a cluster management framework which registers all the cluster details including the cluster itself, nodes and resources. Helix automates the management of partitioned, replicated and distributed resources hosted on a cluster. In this guide, we are avoiding various theory questions which commonly arise while talking around Apache Helix such as comparison with Apache Mesos, even comparison with Apache Zookeeper. Helix has wider use cases. The official website and documentation on Helix should be enough to get started.
Dependencies of Apache Helix varies according to the goal (dependency is a complex matter for the tools related to DevOps and data sciences). Apache Zookeeper and some database such as PostgreSQL are kind of obvious dependencies.
Download Apache Zookeeper from the official website. Download Apache Helix. Unzip Apache Zookeeper into the desired location which peoples refer as the home of zookeeper – zookeeper_home. Navigate to zookeeper_home/conf location and make a copy of zoo_sample.conf to zoo.conf. Open and edit zoo.conf if needed. Like an example, here are two important lines :
---
1 2 3 4 | # snapshot storage dataDir=/tmp/zookeeper # port clientPort=2181 |
Unzip Apache helix to into desired location – helix_home. Go to zookeeper_home/bin and issue the below command to start zookeeper server:
1 | sudo ./zkServer.sh start |
The Zookeeper activities can be viewed from zookeeper_home/bin/zookeeper.out. Apache Helix provides utility scripts inside helix_home/bin. Navigate to helix_home/bin.
Building Helix admin
1 2 3 | git clone git://git.apache.org/helix.git cd helix mvn clean install package -DskipTests |
Helix supports bootstrapping a cluster configuration based on a YAML file :
1 | https://helix.apache.org/0.7.1-docs/tutorial_yaml.html |
Creating supercluster on Zookeeper
1 2 3 4 5 6 | cd helix-core/target/helix-core-pkg/bin chmod +x ./helix-admin.sh ./helix-admin.sh --addCluster mySuperCluster --zkSvr <ZKSERVER:PORT> Creating helix controller instances to super cluster ./helix-admin.sh --addNode mySuperCluster myController-1_12345 --zkSvr <ZKSERVER:PORT> ./helix-admin.sh --addNode mySuperCluster myController-n_12345 --zkSvr <ZKSERVER:PORT> |
You can start all of the controller instances :
1 2 | ./run-helix-controller.sh --cluster mySuperCluster --mode DISTRIBUTED --controllerName myController-1_12345 --zkSvr <ZKSERVER:PORT> ./run-helix-controller.sh --cluster mySuperCluster --mode DISTRIBUTED --controllerName myController-n_12345 --zkSvr <ZKSERVER:PORT> |
This ends this guide.
Tagged With apache helix how to get help