WARN mapred.JobClient: Error reading task outputNo route to host

主從節點的/etc/hosts檔案配置如下

[hadoop@master ~]$ cat /etc/hosts

127.0.0.1 localhost localdomain.localhost

::1 localhost6.localdomain6 localhost6

192.168.100.11 master.oracle.com master

192.168.100.12 slave1.oracle.com slave1

[hadoop@master ~]$

在主從節點執行jps都正常：

[hadoop@master ~]$ jps // 在主節點執行

4690 Jps

2413 JobTracker

2350 SecondaryNameNode

2229 NameNode

[hadoop@master ~]$

[hadoop@slave1 ~]$ jps // 在從節點執行

2248 TaskTracker

27778 Jps

2175 DataNode

[hadoop@slave1 ~]$

[hadoop@master ~]$ hadoop dfs -ls // 在主從節點執行此命令也正常

Found 5 items

drwxr-xr-x - hadoop supergroup 0 2012-10-16 13:24 /user/hadoop/dfsdir

drwxr-xr-x - hadoop supergroup 0 2012-10-16 14:08 /user/hadoop/output

drwxr-xr-x - hadoop supergroup 0 2012-10-16 14:14 /user/hadoop/output1

drwxr-xr-x - hadoop supergroup 0 2012-10-16 14:40 /user/hadoop/output2

drwxr-xr-x - hadoop supergroup 0 2012-10-16 14:43 /user/hadoop/output3

[hadoop@master ~]$

可在測試map/reduce的時候報錯：

[hadoop@master ~]$ hadoop jar hadoop-0.20.2-examples.jar wordcount dfsdir/test.txt output3

12/10/16 14:42:40 INFO input.FileInputFormat: Total input paths to process : 1

12/10/16 14:42:40 INFO mapred.JobClient: Running job: job_201210161256_0009

12/10/16 14:42:41 INFO mapred.JobClient: map 0% reduce 0%

12/10/16 14:42:50 INFO mapred.JobClient: map 100% reduce 0%

12/10/16 14:43:04 INFO mapred.JobClient: Task Id : attempt_201210161256_0009_r_000000_0, Status : FAILED

Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.

12/10/16 14:43:04 WARN mapred.JobClient: Error reading task outputNo route to host

12/10/16 14:43:04 WARN mapred.JobClient: Error reading task outputNo route to host

12/10/16 14:43:19 INFO mapred.JobClient: Task Id : attempt_201210161256_0009_r_000000_1, Status : FAILED

Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.

12/10/16 14:43:19 WARN mapred.JobClient: Error reading task outputNo route to host

12/10/16 14:43:19 WARN mapred.JobClient: Error reading task outputNo route to host

12/10/16 14:43:36 INFO mapred.JobClient: Task Id : attempt_201210161256_0009_r_000000_2, Status : FAILED

Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.

12/10/16 14:43:36 WARN mapred.JobClient: Error reading task outputNo route to host

12/10/16 14:43:36 WARN mapred.JobClient: Error reading task outputNo route to host

12/10/16 14:43:54 INFO mapred.JobClient: Job complete: job_201210161256_0009

12/10/16 14:43:54 INFO mapred.JobClient: Counters: 12

12/10/16 14:43:54 INFO mapred.JobClient: Job Counters

12/10/16 14:43:54 INFO mapred.JobClient: Launched reduce tasks=4

12/10/16 14:43:54 INFO mapred.JobClient: Launched map tasks=1

12/10/16 14:43:54 INFO mapred.JobClient: Data-local map tasks=1

12/10/16 14:43:54 INFO mapred.JobClient: Failed reduce tasks=1

12/10/16 14:43:54 INFO mapred.JobClient: FileSystemCounters

12/10/16 14:43:54 INFO mapred.JobClient: HDFS_BYTES_READ=12

12/10/16 14:43:54 INFO mapred.JobClient: FILE_BYTES_WRITTEN=62

12/10/16 14:43:54 INFO mapred.JobClient: Map-Reduce Framework

12/10/16 14:43:54 INFO mapred.JobClient: Combine output records=2

12/10/16 14:43:54 INFO mapred.JobClient: Map input records=1

12/10/16 14:43:54 INFO mapred.JobClient: Spilled Records=2

12/10/16 14:43:54 INFO mapred.JobClient: Map output bytes=20

12/10/16 14:43:54 INFO mapred.JobClient: Combine input records=2

12/10/16 14:43:54 INFO mapred.JobClient: Map output records=2

哪位大神遇到過此問題，給支個招吧！！！

問題終於解決：

[hadoop@master hadoop]$ hadoop jar hadoop-0.20.2-examples.jar wordcount dfsdir/test.txt output

12/10/16 15:30:10 INFO input.FileInputFormat: Total input paths to process : 1

12/10/16 15:30:11 INFO mapred.JobClient: Running job: job_201210161528_0001

12/10/16 15:30:12 INFO mapred.JobClient: map 0% reduce 0%

12/10/16 15:30:19 INFO mapred.JobClient: map 100% reduce 0%

12/10/16 15:30:31 INFO mapred.JobClient: map 100% reduce 100% // 執行成功了

12/10/16 15:30:33 INFO mapred.JobClient: Job complete: job_201210161528_0001

12/10/16 15:30:33 INFO mapred.JobClient: Counters: 17

12/10/16 15:30:33 INFO mapred.JobClient: Job Counters

12/10/16 15:30:33 INFO mapred.JobClient: Launched reduce tasks=1

12/10/16 15:30:33 INFO mapred.JobClient: Launched map tasks=1

12/10/16 15:30:33 INFO mapred.JobClient: Data-local map tasks=1

12/10/16 15:30:33 INFO mapred.JobClient: FileSystemCounters

12/10/16 15:30:33 INFO mapred.JobClient: FILE_BYTES_READ=30

12/10/16 15:30:33 INFO mapred.JobClient: HDFS_BYTES_READ=12

12/10/16 15:30:33 INFO mapred.JobClient: FILE_BYTES_WRITTEN=92

12/10/16 15:30:33 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=16

12/10/16 15:30:33 INFO mapred.JobClient: Map-Reduce Framework

12/10/16 15:30:33 INFO mapred.JobClient: Reduce input groups=2

12/10/16 15:30:33 INFO mapred.JobClient: Combine output records=2

12/10/16 15:30:33 INFO mapred.JobClient: Map input records=1

12/10/16 15:30:33 INFO mapred.JobClient: Reduce shuffle bytes=30

12/10/16 15:30:33 INFO mapred.JobClient: Reduce output records=2

12/10/16 15:30:33 INFO mapred.JobClient: Spilled Records=4

12/10/16 15:30:33 INFO mapred.JobClient: Map output bytes=20

12/10/16 15:30:33 INFO mapred.JobClient: Combine input records=2

12/10/16 15:30:33 INFO mapred.JobClient: Map output records=2

12/10/16 15:30:33 INFO mapred.JobClient: Reduce input records=2

[hadoop@master hadoop]$

錯誤原因：

[root@master ~]# cat /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=master.oracle.com // 這是在報錯時的主機名配置

GATEWAY=192.168.100.1

[root@master ~]#

[hadoop@master hadoop]$ hostname

master.oracle.com

[hadoop@master hadoop]$

我在/etc/sysconfig/network 中設定主機名的時候設定的是master.oracle.com，而在/etc/hosts 中配置的主機名為：

192.168.100.11 master

後來把主機名更改為如下：

[hadoop@master hadoop]$ cat /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=master

GATEWAY=192.168.100.1

[hadoop@master hadoop]$

[hadoop@master hadoop]$ hostname

master

[hadoop@master hadoop]$

[hadoop@master hadoop]$ cat /etc/hosts

127.0.0.1 localhost localdomain.localhost

::1 localhost6.localdomain6 localhost6

192.168.100.11 master

192.168.100.12 slave1

[hadoop@master hadoop]$

其他節點的修改一樣，注意這三地方的主機名都要一樣：

/etc/sysconfig/network

/etc/hosts

$hostname

[@more@]