RHEL5 hadoop2.4.1 rsync配置

thamsyangsw發表於2014-07-21
首先下載對應linux的rsync軟體:
本例中,下載   22-Jun-2014 21:58  420K  RHEL5 and CentOS-5 x86 32bit

服務端配置

[root@master software]#pwd

/root/software

[root@master software]#ll
… …

-rwxr--r--1 root root    429855 Jul 18 17:52 rsync-3.1.1-1.el5.rfx.i386.rpm

[root@master software]# rpm -ivh rsync-3.1.1-1.el5.rfx.i386.rpm
warning: rsync-3.1.1-1.el5.rfx.i386.rpm:Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...               
########################################### [100%]

file /etc/xinetd.d/rsync from install of rsync-3.1.1-1.el5.rfx.i386 conflicts with file from package rsync-2.6.8-3.1.i386       
file /usr/bin/rsync from install of rsync-3.1.1-1.el5.rfx.i386 conflicts with file from package rsync-2.6.8-3.1.i386
       
file /usr/share/man/man1/rsync.1.gz from install of rsync-3.1.1-1.el5.rfx.i386 conflicts with file from package rsync-2.6.8-3.1.i386
       
file /usr/share/man/man5/rsyncd.conf.5.gz from install of rsync-3.1.1-1.el5.rfx.i386 conflicts with file from package rsync-2.6.8-3.1.i386

從日誌資訊可以看出和本機已經安裝的rsync-2.6.8-3.1.i386衝突了。需要先解除安裝該包。

[root@master software]# rpm -e rsync-2.6.8-3.1

warning: /etc/xinetd.d/rsync saved as /etc/xinetd.d/rsync.rpmsave

[root@master software]# rpm -qa|grep rsync

[root@master software]# rpm -ivh rsync-3.1.1-1.el5.rfx.i386.rpm

warning: rsync-3.1.1-1.el5.rfx.i386.rpm:Header V3 DSA signature: NOKEY, key ID 6b8d79e6

Preparing...               
########################################### [100%]
  
1:rsync                 
########################################### [100%]

[root@master software]#rpm -qa|grep rsync

rsync-3.1.1-1.el5.rfx

[root@master software]#vi /etc/rsyncd.conf

#global options
strict modes=yes
port =873

log file=/var/log/rsyncd.log

pid file=/usr/local/rsync/rsyncd.pid

#mode options

[hadoop]

max connections=10

uid=root

gid =root

path = /home/hadoop/hadoop-2.4.1

ignore errors

read only = no

write only = no

hosts allow = *

#hosts deny =

auth users = root

screts file = /home/hadoop/rsync.pass

#(注意:這裡的secrets單詞寫錯了,導致後面許可權驗證通不過)

[root@master software]#vi /home/hadoop/rsync.pass

root:beijing

[root@master software]#chmod 777 /home/hadoop/rsync.pass

[root@master ~]# vi /etc/xinetd.d/rsync

# default: off

# description: The rsync server is a good addition to an ftp server, as it \

# allows crc checksumming etc.

service rsync

{       
#disable        = yes
       
disable = no
(把這個引數改成no       
socket_type     = stream
       
wait            = no
       
user            = root

server          = /usr/bin/rsync       
server_args     = --daemon
       
log_on_failure  += USERID

}

[root@master ~]#chkconfig --list|grep sync       
rsync:          on

[root@master software]#rsync --daemon

failed to create pid file /usr/local/rsync/rsyncd.pid:No such file or directory

[root@master software]#mkdir /usr/local/rsync/

[root@master software]#rsync –daemon

[root@master software]#netstat -an |grep 873

tcp        0     
0 0.0.0.0:873                
0.0.0.0:*                   LISTEN     

tcp        0     
0 :::873                     
:::*                        LISTEN   

[root@master software]#ps -ef|grep rsync

root     29351    1  0 17:36 ?        00:00:00 rsync --daemon

root     31429 2714  0 17:41 pts/1    00:00:00 grep rsync

在客戶端slave-5測試同步

[root@slave-5 ~]# rsync -vzrtopg --progress --delete root@master::hadoop /home/hadoop/testrsync

Password:

@ERROR: auth failed on module hadoop

rsync error: error starting client-server protocol (code 5) at main.c(1296) [receiver=2.6.8]

可見由於使用者驗證未透過,該操作失敗了,下面用了幾種方式進行調整並測試

方法一(not works):

雖然該方法對本例未奏效,但是建議chmod 600  /home/hadoop/rsync.pass

[root@master software]# ll /home/hadoop/rsync.pass

-rwxrwxrwx 1 root root 13 Jul 21 17:23 /home/hadoop/rsync.pass

[root@master software]# chmod 600 /home/hadoop/rsync.pass

[root@master software]# ll /home/hadoop/rsync.pass

-rw-------1 root root 13 Jul 21 17:23 /home/hadoop/rsync.pass

[root@slave-5 ~]# rsync -vzrtopg --progress --delete root@master::hadoop /home/hadoop/testrsync

Password:

@ERROR: auth failed on module hadoop

rsync error: error starting client-server protocol (code 5) at main.c(1296) [receiver=2.6.8]

方法二(not works):

在客戶端更新跟伺服器相同版本的rsync(後經測試,低版本的客戶端也是能夠正常從服務端同步檔案的)

[root@slave-5 ~]#  rpm -e rsync-2.6.8-3.1

[root@slave-5 software]# rpm -ivh rsync-3.1.1-1.el5.rfx.i386.rpm

warning:rsync-3.1.1-1.el5.rfx.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6

Preparing...                ###########################################[100%]

   1:rsync  ########################################### [100%]

[root@slave-5 software]# rsync --daemon

Failed to parse config file: /etc/rsyncd.conf

[root@slave-5 software]# touch /etc/rsyncd.conf

[root@slave-5 software]# rsync –daemon

[root@slave-5 software]# rsync -vzrtopg --progress --delete /home/hadoop/testrsync

Password:

@ERROR: auth failed on module hadoop

rsync error: error starting client-server protocol (code 5) at main.c(1635) [Receiver=3.1.1]

方法三(not works):

[root@slave-5 ~]# mkdir /home/hadoop/testrsync

[root@slave-5 ~]# rsync -vzrtopg --progress --delete /home/hadoop/testrsync

Password:

@ERROR: auth failed on module hadoop

rsync error: error starting client-server protocol (code 5) at main.c(1635) [Receiver=3.1.1]

方法四(not works):重啟下服務端試試

[root@master software]# ps -ef|grep rsync

root     29351    1  0 17:36 ?        00:00:00 rsync --daemon

root     31614 2714  0 18:17 pts/1    00:00:00 grep rsync

[root@master software]# kill -9 29351

[root@master software]# ps -ef|grep rsync

root     31616 2714  0 18:18 pts/1    00:00:00 grep rsync

[root@master software]# rsync --daemon

failed to create pid file /usr/local/rsync/rsyncd.pid: File exists

[root@master software]# cd /usr/local/rsync/

[root@master rsync]# rsync --daemon

failed to create pid file /usr/local/rsync/rsyncd.pid: File exists

[root@master rsync]# ll

total 4

-rw-r--r-- 1 root root 6 Jul 21 17:36 rsyncd.pid

[root@master rsync]# cat rsyncd.pid
29351

[root@master rsync]# rm -rf rsyncd.pid

[root@master rsync]# rsync –daemon

[root@master rsync]# ps -ef|grep rsync

root     31629    1  0 18:19 ?        00:00:00 rsync --daemon

root     31634 2714  0 18:20 pts/1    00:00:00 grep rsync

方法五(works):檢視下伺服器端的日誌,終於發現問題在哪裡了

[root@master rsync]# tail -f /var/log/rsyncd.log

2014/07/21 18:20:35 [31636] params.c:Parameter() - Ignoring badly formed line in config file: ignore errors

2014/07/21 18:20:35 [31636] Unknown Parameter encountered:"screts file"

2014/07/21 18:20:35 [31636] IGNORING unknown parameter "screts file"

2014/07/21 18:20:35 [31636] connect from slave-5 (100.12.56.227)

2014/07/21 18:20:39 [31636] auth failed on module hadoop from slave-5 (100.12.56.227) for root: no matching rule

2014/07/21 18:23:11 [31651] params.c:Parameter() - Ignoring badly formed line in config file: ignore errors

2014/07/21 18:23:11 [31651] Unknown Parameter encountered: "screts file"

2014/07/21 18:23:11 [31651] IGNORING unknown parameter "screts file"

2014/07/21 18:23:11 [31651] connect from slave-5 (100.12.56.227)

2014/07/21 18:23:13 [31651] auth failed on module hadoop from slave-5 (100.12.56.227) for root: no secrets file

[root@master rsync]# vi  /etc/rsyncd.conf

#global options

strict modes=yes

port=873

log file =/var/log/rsyncd.log

pid file =/usr/local/rsync/rsyncd.pid

#mode options

[hadoop]

max connections=10

uid=root

gid=root

path=/home/hadoop/hadoop-2.4.1

ignore errors

read only=no

write only=no

hosts allow=*

#hosts deny=

auth users=root

secrets file=/home/hadoop/rsync.pass

在客戶端再進行測試

[root@slave-5 selinux]#rsync -vzrtopg --progress --delete root@master::hadoop /home/hadoop/testrsync

在服務端觀察下日誌如下:

[root@master rsync]# tail -f /var/log/rsyncd.log

2014/07/21 18:20:35 [31636] params.c:Parameter()- Ignoring badly formed line in config file: ignore errors

2014/07/21 18:20:35 [31636] Unknown Parameter encountered: "screts file"

2014/07/21 18:20:35 [31636] IGNORING unknown parameter "screts file"

2014/07/21 18:20:35 [31636] connect from slave-5 (100.12.56.227)

2014/07/21 18:20:39 [31636] auth failed on module hadoop from slave-5 (100.12.56.227) for root: no matching rule

2014/07/21 18:23:11 [31651] params.c:Parameter() - Ignoring badly formed line in config file: ignore errors

2014/07/21 18:23:11 [31651] Unknown Parameter encountered: "screts file"

2014/07/21 18:23:11 [31651] IGNORING unknown parameter "screts file"

2014/07/21 18:23:11 [31651] connect from slave-5 (100.12.56.227)

2014/07/21 18:23:13 [31651] auth failed on module hadoop from slave-5 (100.12.56.227) for root: no secrets file

2014/07/21 18:27:31 [31675] params.c:Parameter() - Ignoring badly formed line in config file: ignore errors

2014/07/21 18:27:31 [31675] connect from slave-5 (100.12.56.227)

2014/07/21 18:27:33 [31675] rsync on hadoop/ from root@slave-5 (100.12.56.227)

2014/07/21 10:27:33 [31675] building file list

2014/07/21 10:27:54 [31675] sent 143747859 bytes  received 110760 bytes  total size 238829369

在客戶端檢視同步的資料夾

[root@slave-5 selinux]#cd /home/hadoop/testrsync

[root@slave-5 testrsync]#ll

total 56

drwxr-xr-x 2 hadoop hadoop  4096 Jun 21 14:05 bin

drwxr-xr-x 3 hadoop hadoop  4096 Jun 21 14:05 etc

drwxr-xr-x 2 hadoop hadoop  4096 Jun 21 14:05 include

drwxr-xr-x 3 hadoop hadoop  4096 Jun 21 14:05 lib

drwxr-xr-x 2 hadoop hadoop  4096 Jun 21 14:05 libexec

-rw-r--r-- 1 hadoop hadoop 15458 Jun 21 14:38 LICENSE.txt

drwxrwxr-x 2 hadoop hadoop  4096 Jul 21 16:22 logs

-rw-r--r-- 1 hadoop hadoop   101 Jun 21 14:38 NOTICE.txt

-rw-r--r-- 1 hadoop hadoop  1366 Jun 21 14:38 README.txt

drwxr-xr-x 2 hadoop hadoop  4096 Jun 21 14:05 sbin

drwxr-xr-x 4 hadoop hadoop  4096 Jun 21 14:05 share

至此,測試ok












來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/26613085/viewspace-1223525/,如需轉載,請註明出處,否則將追究法律責任。

相關文章