Toggle navigation
Home
安装部署
Archives
Tags
Spark Installation
环境搭建
hadoop
2019-05-06 06:51:53
45
0
0
louyj
环境搭建
hadoop
## Install Scala wget http://downloads.lightbend.com/scala/2.10.6/scala-2.10.6.tgz tar zxvf scala-2.10.6.tgz vi /etc/profile export SCALA_HOME=/home/hadoop/scala-2.10.6 export PATH=$PATH:$SCALA_HOME/bin source /etc/profile scala -version ## Install Spark wget http://d3kbcqa49mib13.cloudfront.net/spark-1.6.2-bin-hadoop2.6.tgz tar zxvf spark-1.6.2-bin-hadoop2.6.tgz cd spark-1.6.2-bin-hadoop2.6/conf cp spark-env.sh.template spark-env.sh vi spark-env.sh export JAVA_HOME=/opt/jdk1.8.0_91 export SCALA_HOME=/home/hadoop/scala-2.10.6 export HADOOP_HOME=/home/hadoop/hadoop-2.6.4 export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop export YARN_HOME=/home/hadoop/hadoop-2.6.4 export YARN_CONF_DIR=${YARN_HOME}/etc/hadoop export SPARK_HOME=/home/hadoop/spark-1.6.2-bin-hadoop2.6 export SPARK_LOCAL_DIRS=/home/hadoop/spark-1.6.2-bin-hadoop2.6 export SPARK_LIBARY_PATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$HADOOP_HOME/lib/native export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$SCALA_HOME/bin export SPARK_MASTER_IP=x.x.x.x export SPARK_DRIVER_MEMORY=1G vi slaves cp slaves.template slaves linode01 linode02 linode03 vi /etc/profile export SPARK_HOME=/home/hadoop/spark-1.6.2-bin-hadoop2.6 export PATH=$PATH:/opt/jdk1.8.0_91/bin:$SCALA_HOME/bin:$SPARK_HOME/bin source /etc/profile ## Start Spark sbin/start-all.sh http://x.x.x.x:8080
Pre:
Mount HDFS by NFS
Next:
Impala Installation
0
likes
45
Weibo
Wechat
Tencent Weibo
QQ Zone
RenRen
Submit
Sign in
to leave a comment.
No Leanote account?
Sign up now.
0
comments
More...
Table of content
louyj
No Leanote account? Sign up now.