生产环境实战spark （6）分布式集群 5台设备 Scala安装
生产环境实战spark （6）分布式集群 Scala安装
1，scala官网下载scala安装到本地。
这里使用spark 2.1.x版本，而spark 2.1 版本需要scala2.1.1版本以上的兼容版本，因此下载安装 scala-2.11.8
下载地址：http://www.scala-lang.org/download/all.html
Spark runs on Java 7+, Python 2.6+/3.4+ and R 3.1+. For the Scala API, Spark 2.1.0 uses Scala 2.11. You will need to use a compatible Scala version (2.11.x).
2，从本地使用winscp工具将scala 2.11.8上传到master节点。 上传后检查 Last login: Wed Apr 19 09:27:17 2017 from 132.150.75.19
[root@master ~]# cd /usr/local
[root@master local]# ls
bin etc games include jdk1.8.0_121 lib lib64 libexec rhzf_setup_scripts rhzf_spark_setupTools sbin share src
[root@master local]# cd rhzf_spark_setupTools
[root@master rhzf_spark_setupTools]# ls
jdk-8u121-linux-x64.tar.gz scala-2.11.8.zip
[root@master rhzf_spark_setupTools]#
3，解压scala进行安装。 [root@master rhzf_spark_setupTools]# unzip scala-2.11.8.zip
Archive: scala-2.11.8.zip
creating: scala-2.11.8/
creating: scala-2.11.8/man/
creating: scala-2.11.8/man/man1/
inflating: scala-2.11.8/man/man1/scala.1
inflating: scala-2.11.8/man/man1/scalap.1
inflating: scala-2.11.8/man/man1/fsc.1
inflating: scala-2.11.8/man/man1/scaladoc.1
inflating: scala-2.11.8/man/man1/scalac.1
creating: scala-2.11.8/bin/
inflating: scala-2.11.8/bin/scalac
inflating: scala-2.11.8/bin/fsc
inflating: scala-2.11.8/bin/fsc.bat
inflating: scala-2.11.8/bin/scala
inflating: scala-2.11.8/bin/scalap
inflating: scala-2.11.8/bin/scaladoc.bat
inflating: scala-2.11.8/bin/scaladoc
inflating: scala-2.11.8/bin/scalac.bat
inflating: scala-2.11.8/bin/scala.bat
inflating: scala-2.11.8/bin/scalap.bat
creating: scala-2.11.8/doc/
creating: scala-2.11.8/doc/tools/
inflating: scala-2.11.8/doc/tools/index.html
inflating: scala-2.11.8/doc/tools/scalap.html
creating: scala-2.11.8/doc/tools/images/
inflating: scala-2.11.8/doc/tools/images/scala_logo.png
extracting: scala-2.11.8/doc/tools/images/external.gif
inflating: scala-2.11.8/doc/tools/scala.html
creating: scala-2.11.8/doc/tools/css/
inflating: scala-2.11.8/doc/tools/css/style.css
inflating: scala-2.11.8/doc/tools/fsc.html
inflating: scala-2.11.8/doc/tools/scalac.html
inflating: scala-2.11.8/doc/tools/scaladoc.html
inflating: scala-2.11.8/doc/README
inflating: scala-2.11.8/doc/LICENSE.md
creating: scala-2.11.8/doc/licenses/
inflating: scala-2.11.8/doc/licenses/mit_jquery-ui.txt
inflating: scala-2.11.8/doc/licenses/mit_sizzle.txt
inflating: scala-2.11.8/doc/licenses/apache_jansi.txt
inflating: scala-2.11.8/doc/licenses/bsd_asm.txt
inflating: scala-2.11.8/doc/licenses/mit_tools.tooltip.txt
inflating: scala-2.11.8/doc/licenses/mit_jquery-layout.txt
inflating: scala-2.11.8/doc/licenses/mit_jquery.txt
inflating: scala-2.11.8/doc/licenses/bsd_jline.txt
inflating: scala-2.11.8/doc/License.rtf
creating: scala-2.11.8/lib/
inflating: scala-2.11.8/lib/scala-parser-combinators_2.11-1.0.4.jar
inflating: scala-2.11.8/lib/scala-reflect.jar
inflating: scala-2.11.8/lib/scalap-2.11.8.jar
inflating: scala-2.11.8/lib/scala-continuations-library_2.11-1.0.2.jar
inflating: scala-2.11.8/lib/config-1.2.1.jar
inflating: scala-2.11.8/lib/akka-actor_2.11-2.3.10.jar
inflating: scala-2.11.8/lib/scala-continuations-plugin_2.11.8-1.0.2.jar
inflating: scala-2.11.8/lib/scala-actors-migration_2.11-1.1.0.jar
inflating: scala-2.11.8/lib/scala-xml_2.11-1.0.4.jar
inflating: scala-2.11.8/lib/jline-2.12.1.jar
inflating: scala-2.11.8/lib/scala-library.jar
inflating: scala-2.11.8/lib/scala-compiler.jar
inflating: scala-2.11.8/lib/scala-swing_2.11-1.0.2.jar
inflating: scala-2.11.8/lib/scala-actors-2.11.0.jar
[root@master rhzf_spark_setupTools]# ls
jdk-8u121-linux-x64.tar.gz scala-2.11.8 scala-2.11.8.zip
复制到/usr/local目录：
[root@master rhzf_spark_setupTools]# mv scala-2.11.8 /usr/local
[root@master rhzf_spark_setupTools]# cd /usr/local
[root@master local]# ls
bin etc games include jdk1.8.0_121 lib lib64 libexec rhzf_setup_scripts rhzf_spark_setupTools sbin scala-2.11.8 share src
4，配置linux环境变量。vi /etc/profile
export JAVA_HOME=/usr/local/jdk1.8.0_121
export SCALA_HOME=/usr/local/scala-2.11.8
export PATH=.:$PATH:$JAVA_HOME/bin:$SCALA_HOME/bin
在命令行中输入source /etc/profile，使刚才修改的SCALA_HOME及PATH配置文件生效 。[root@master local]# source /etc/profile
5，单机master节点上scala安装完成，版本检查。
[root@master local]# scala -version
Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
[root@master local]#
6，编写脚本，在woker的节点也安装scala。
[root@master rhzf_setup_scripts]# ls
rhzf_hosts_scp.sh rhzf_ssh.sh
[root@master rhzf_setup_scripts]# vi rhzf_scala.sh
#!/bin/sh
for i in 238 239 240 241
do
scp -rq /usr/local/scala-2.11.8 root@10.*.*.$i:/usr/local/scala-2.11.8
scp -rq /etc/profile [email protected].$i:/etc/profile
ssh root@10.*.*.$i source /etc/profile done
[root@master rhzf_setup_scripts]# ls
rhzf_hosts_scp.sh rhzf_scala.sh rhzf_ssh.sh
[root@master rhzf_setup_scripts]# chmod u+x rhzf_scala.sh
[root@master rhzf_setup_scripts]# ./rhzf_scala.sh
[root@master rhzf_setup_scripts]#
7， 在woker的节点检查scala安装完成。 [root@worker01 ~]# scala -version
Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
[root@worker01 ~]#
Last login: Tue Apr 18 12:52:16 2017 from master
[root@worker02 ~]# scala -version
Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
[root@worker02 ~]#
Last login: Tue Apr 18 12:52:01 2017 from worker02
[root@worker03 ~]# scala -version
Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
[root@worker03 ~]#
Last login: Tue Apr 18 12:08:22 2017 from 132.150.75.19
[root@worker04 ~]# scala -version
Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
[root@worker04 ~]#
