mHUB - 360Science for Spark- Installation

 

To install 360Science for Spark to your home directory simply download the tar and extract:

cd ~
wget http://get.360science.com/downloads/360ScienceSpark-1.7-linux.tar.gz
tar -zxvf 360ScienceSpark-1.7-linux.tar.gz

Installation Contents

The tar will extract to a folder called mHubSpark. This folder contains the following:

lib Folder containing mHUB native shared object library.
libstdc++ Folder containing C++ standard libraries required by mHUB.
samples Folder containing sample applications.
HubSpark.jar Jar containing the 360Science for Spark package.
matchITHub.jar Jar containing the mHUB Java interface.
readme Text file containing overview of product.
release_notes Text file containing release notes.

Native Libraries

The native library files should be placed in folder, e.g. /usr/local/lib64, that can be synced with all the processing nodes. Execute the following steps:

cd mHubSpark/lib
chmod 777 ./libmatchITHub.so
sudo cp libmatchITHub.so /usr/local/lib64
cd ../libstdc++
chmod 777 ./lib*.so
sudo cp lib*.so /usr/local/lib64

Ensure that the files in /usr/local/lib64 are synced with all processing nodes in the cluster and that spark-env.sh contains these lines:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64
SPARK_YARN_USER_ENV="LD_LIBRARY_PATH=$LD_LIBRARY_PATH"