Hadoop/spark安装实战（系列篇3） Hadoop 伪分布模式安装
Hadoop/spark安装实战（系列篇3） Hadoop 伪分布模式安装
Hadoop 的安装与配置 伪分布模式安装
1/解压缩hadoop
tar -zxvf hadoop-1.2.1-bin.tar.gz
2/ 将文件移到/usr/local
mv hadoop-1.2.1 /usr/local
3/设置环境变量HADOOP_HOME，修改文件“/etc/profile”
[root@localhost local]# vi /etc/profile
export HADOOP_HOME=/usr/local/hadoop-1.2.1
export PATH=.:$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin
生效
[root@localhost local]# source /etc/profile
检验
[root@localhost local]# echo $PATH
.:.:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/l
ocal/jdk1.7.0_79/bin:/root/bin:/usr/local/jdk1.7.0_79/bin:/usr/local/ha
doop-1.2.1/bin
[root@localhost local]#
4 修改hadoop 环境变量脚本文件hadoop-env.sh
增加
export JAVA_HOME=/usr/local/jdk1.7.0_79
5 修改hadoop 核心配置文件core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
<description> hadoop.tmp.dir</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://spark0:9000</value>
<description>HDFS fs.default.name</description>
</property>
</configuration>
6 hdfs 配置文件hdfs-site.xml，存储副本数1份
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
<description>1</description>
</property>
</configuration>
7 MapReduce 配置文件mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>spark0:9001</value>
<description>JobTracker </description>
</property>
</configuration>
8格式化文件系统
在第一次使用之前需要进行格式化，只在第一次启动的时候格式化，不要每次启
动都格式化
[root@localhost bin]# hadoop namenode -format
15/09/12 06:31:11 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = localhost.localdomain/127.0.0.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 1.2.1
STARTUP_MSG: build =
-r
1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
STARTUP_MSG: java = 1.7.0_79
************************************************************/
15/09/12 06:31:13 INFO util.GSet: Computing capacity for map BlocksMap
15/09/12 06:31:13 INFO util.GSet: VM type = 32-bit
15/09/12 06:31:13 INFO util.GSet: 2.0% max memory = 1013645312
15/09/12 06:31:13 INFO util.GSet: capacity = 2^22 = 4194304
entries
9 启动hadoop 程序
[root@localhost bin]# start-all.sh
[root@localhost bin]# start-all.sh
Warning: $HADOOP_HOME is deprecated.
starting namenode, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-namenode-localhost.localdomain.out
localhost: starting datanode, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-datanode-localhost.localdomain.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-secondarynamenode-
localhost.localdomain.out
starting jobtracker, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-jobtracker-localhost.localdomain.out
localhost: starting tasktracker, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-tasktracker-localhost.localdomain.out
检验
[root@localhost bin]# jps
[root@localhost bin]# jps
27375 Jps
26949 DataNode
27046 SecondaryNameNode
26851 NameNode
27240 TaskTracker
27124 JobTracker
[root@localhost bin]#
10 检验 在本地物理机浏览器输入
NameNode 'spark0:9000'
Started: Sat Sep 12 06:34:38 PDT 2015
Version: 1.2.1, r1503152
Compiled: Mon Jul 22 15:23:09 PDT 2013 by mattf
Upgrades: There are no upgrades in progress.
Browse the filesystem
Namenode Logs
-----------------------------------------------------------------------
---------
Cluster Summary
7 files and directories, 1 blocks = 8 total. Heap Size is 31.32 MB /
966.69 MB (3%)
Configured Capacity : 5.65 GB DFS Used : 28.01 KB Non DFS
Used : 3.17 GB DFS Remaining : 2.47 GB DFS Used% : 0 % DFS
Remaining% : 43.82 % Live Nodes : 1 Dead Nodes : 0
Decommissioning Nodes : 0 Number of Under-Replicated Blocks : 0
-----------------------------------------------------------------------
---------
NameNode Storage:
Storage Directory Type State /home/hadoop/tmp/dfs/name IMAGE_AND_EDITS
Active
-----------------------------------------------------------------------
---------
This is Apache Hadoop release 1.2.1
至此 hadoop安装完成
