基於序列化技術(Protobuf)的socket檔案傳輸

中大黑熊發表於2014-07-03

好像好久都沒更博文了，沒辦法，最近各種倒黴事情，搞到最近真的沒什麼心情，希望之後能夠轉運吧。

言歸正傳，這次我要做的是基於序列化技術的socket檔案傳輸來無聊練一下手。

一.socket檔案傳輸

       之前我所做的伺服器和客戶端的tcp/udp通訊都是以字串流來進行單工的，雙工的傳輸，其實關於檔案傳輸的原理也差不多，我的主要方法是通過檔案迭代器遍歷檔案流，並將其讀取轉化為字串流，然後將字串流從伺服器端傳送，然後客戶端在緩衝區中接收資料流，然後並把它寫入檔案當中，從而實現檔案的傳輸，在下面程式當中，我僅僅是實現由伺服器分發，客戶端除了接收什麼都不做的簡單程式。

伺服器端：

#!/usr/bin/env python
#-*- coding:utf-8 -*-

from socket import *
from time import ctime
import os

HOST = ''
PORT = 21567
BUFSIZ = 4096
ADDR = (HOST,PORT)

tcpSerSock = socket(AF_INET,SOCK_STREAM)
tcpSerSock.bind(ADDR)
tcpSerSock.listen(5)

filename = raw_input('Please input the file name:')

while True:
    print 'waiting for connection...'
    tcpCliSock,addr = tcpSerSock.accept()
    print '...connected from:',addr

    print '向客戶端傳送檔案...'
    f = open(filename,'r')
    
    for eachLine in f:
        tcpCliSock.send('%s' % eachLine)
    f.close()
    tcpCliSock.close()
    print 'Done!'
    
tcpSerSock.close()

客戶端：

#!/usr/bin/env python
#-*- coding:utf-8 -*-
from socket import *

HOST = 'localhost'
PORT = 21567
BUFSIZ = 4096
ADDR = (HOST,PORT)

tcpCliSock = socket(AF_INET,SOCK_STREAM)
tcpCliSock.connect(ADDR)

filename = raw_input('Please input the filename you will write to:')
f = open(filename,'a')  #以追加模式開啟檔案
print '正在寫檔案....'
while True:
    data = tcpCliSock.recv(BUFSIZ)
    if not data:
        break
    f.write(data)
    
f.close()
print 'Done!'
tcpCliSock.close()

 

還有一種比較方便的方法是用框架SocketServer來實現檔案傳輸的功能，相當於重新實現linux的scp命令。

伺服器端：

#!/usr/bin/env python
#-*-coding:utf-8-*-

import SocketServer

addr = ('',21568)

class DocHandler(SocketServer.StreamRequestHandler):
    def handle(self):
        doc_len = ord(self.rfile.read(1))  #檔案的第一個字元代表檔名的長度
        name = self.rfile.read(doc_len)
        print "接收檔案:%s" % name
        f = open('../'+name,'w')
        cont = self.rfile.read(4096)
        while cont:
            f.write(cont)
            cont = self.rfile.read(4096)
        f.close()
        print "Done :%s" % name

server = SocketServer.TCPServer(addr,DocHandler)  #利用tcp傳輸
server.serve_forever()

客戶端：

#!/usr/bin/env python
#-*- coding:utf-8 -*-

from socket import *
import os.path

addr = ('',21568)

def get_header(name):
    n_len = len(name)
    assert n_len < 250
    #assert語句，代表一個肯定的判定語句，如果為false，會丟擲AssertError的異常
    return chr(n_len) + name

def send_file(name):
    basename = os.path.basename(name)
    header = get_header(basename)
    cont = open(name).read()
    s = socket(AF_INET,SOCK_STREAM)
    s.connect(addr)
    s.sendall(header)
    s.sendall(cont)
    s.close()

if __name__ == '__main__':
    filename = raw_input("請輸入你要傳輸的檔名:")
    send_file(filename)

 

二.序列化技術

所謂序列化技術，自己可以google一下。

這次我用的序列化技術的框架是google 的protocol buffer（簡稱protobuf），關於它的詳細介紹，可以看看它的介紹，文件和API，它是一種和平臺無關，語言無關的技術。

這次的程式我本來想用C++寫的，但無奈環境搭建失敗，用不了，只好再用python的。

首先為了避免重複勞動，我使用了它原來example的.proto檔案，做一個關於名片傳輸的程式。

addressbook.proto

// See README.txt for information and build instructions.

package tutorial;

option java_package = "com.example.tutorial";
option java_outer_classname = "AddressBookProtos";

message Person {
  required string name = 1;
  required int32 id = 2;        // Unique ID number for this person.
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

// Our address book file is just one of these.
message AddressBook {
  repeated Person person = 1;
}

輸入指令來生成addressbook_pb2.py：（注：如果是C++，則python換為cpp；如果是Java，則python換為java）

protoc -I=./ --python_out=./ addressbook.proto

然後先嚐試弄個單機版出來吧。 根據官方的demo修改整合出來的。

add_person.py

#! /usr/bin/python

# See README.txt for information and build instructions.

import addressbook_pb2
import sys

# This function fills in a Person message based on user input.
def PromptForAddress(person):
  person.id = int(raw_input("Enter person ID number: "))
  person.name = raw_input("Enter name: ")

  email = raw_input("Enter email address (blank for none): ")
  if email != "":
    person.email = email

  while True:
    number = raw_input("Enter a phone number (or leave blank to finish): ")
    if number == "":
      break

    phone_number = person.phone.add()
    phone_number.number = number

    type = raw_input("Is this a mobile, home, or work phone? ")
    if type == "mobile":
      phone_number.type = addressbook_pb2.Person.MOBILE
    elif type == "home":
      phone_number.type = addressbook_pb2.Person.HOME
    elif type == "work":
      phone_number.type = addressbook_pb2.Person.WORK
    else:
      print "Unknown phone type; leaving as default value."

# Main procedure:  Reads the entire address book from a file,
#   adds one person based on user input, then writes it back out to the same
#   file.

filename = raw_input('Please input the file name:')

address_book = addressbook_pb2.AddressBook()

# Read the existing address book.
try:
  f = open(filename, "rb")
  address_book.ParseFromString(f.read())
  f.close()
except IOError:
  print sys.argv[1] + ": File not found.  Creating a new file."

# Add an address.
PromptForAddress(address_book.person.add())

# Write the new address book back to disk.
f = open(filename, "wb")
f.write(address_book.SerializeToString())
f.close()

list_person.py

#! /usr/bin/python

# See README.txt for information and build instructions.

import addressbook_pb2
import sys

# Iterates though all people in the AddressBook and prints info about them.
def ListPeople(address_book):
  for person in address_book.person:
    print "Person ID:", person.id
    print "  Name:", person.name
    if person.HasField('email'):
      print "  E-mail address:", person.email

    for phone_number in person.phone:
      if phone_number.type == addressbook_pb2.Person.MOBILE:
        print "  Mobile phone #:",
      elif phone_number.type == addressbook_pb2.Person.HOME:
        print "  Home phone #:",
      elif phone_number.type == addressbook_pb2.Person.WORK:
        print "  Work phone #:",
      print phone_number.number

# Main procedure:  Reads the entire address book from a file and prints all
#   the information inside.
filename = raw_input("Please input the filename:")
address_book = addressbook_pb2.AddressBook()

# Read the existing address book.
f = open(filename, "rb")
address_book.ParseFromString(f.read())
f.close()

ListPeople(address_book)

 執行過沒有任何問題。

三.整合

好了，到了最後一步了， 實現的功能時，伺服器端先要求你設定你自己要新增自己的名片資訊，序列化，然後分發給客戶端，客戶端再把它反序列化，輸出個人資訊。

伺服器端：

 

 1 #!/usr/bin/env python
 2 #-*- coding:utf-8 -*-
 3 
 4 from socket import *
 5 from time import ctime
 6 import os
 7 import addressbook_pb2
 8 
 9 HOST = ''
10 PORT = 21567
11 BUFSIZ = 4096
12 ADDR = (HOST,PORT)
13 
14 tcpSerSock = socket(AF_INET,SOCK_STREAM)
15 tcpSerSock.bind(ADDR)
16 tcpSerSock.listen(5)
17 
18 #新增聯絡人函式
19 def PromptForAddress(person):
20   person.id = int(raw_input("Enter person ID number: "))
21   person.name = raw_input("Enter name: ")
22 
23   email = raw_input("Enter email address (blank for none): ")
24   if email != "":
25     person.email = email
26 
27   while True:
28     number = raw_input("Enter a phone number (or leave blank to finish): ")
29     if number == "":
30       break
31 
32     phone_number = person.phone.add()
33     phone_number.number = number
34 
35     type = raw_input("Is this a mobile, home, or work phone? ")
36     if type == "mobile":
37       phone_number.type = addressbook_pb2.Person.MOBILE
38     elif type == "home":
39       phone_number.type = addressbook_pb2.Person.HOME
40     elif type == "work":
41       phone_number.type = addressbook_pb2.Person.WORK
42     else:
43       print "Unknown phone type; leaving as default value."
44 
45 filename = raw_input('Please input the file name:')
46 
47 address_book = addressbook_pb2.AddressBook()
48 
49 try:
50     f = open(filename,"rb")
51     address_book.ParseFromString(f.read())
52     f.close()
53 except IOError:
54     print filename + ": File not found. Creating a new file."
55 
56 #新增聯絡人資訊
57 PromptForAddress(address_book.person.add())
58 
59 #寫進去
60 f = open(filename,"wb")
61 f.write(address_book.SerializeToString())
62 f.close()
63 
64 while True:
65     print 'waiting for connection...'
66     tcpCliSock,addr = tcpSerSock.accept()
67     print '...connected from:',addr
68 
69     print '向客戶端傳送檔案...'
70     f = open(filename,'rb')
71     
72     for eachLine in f:
73         tcpCliSock.send('%s' % eachLine)
74     f.close()
75     tcpCliSock.close()
76     print 'Done!'
77     
78 tcpSerSock.close()

 

客戶端：

 1 #!/usr/bin/env python
 2 #-*- coding:utf-8 -*-
 3 from socket import *
 4 import addressbook_pb2
 5 
 6 HOST = 'localhost'
 7 PORT = 21567
 8 BUFSIZ = 4096
 9 ADDR = (HOST,PORT)
10 
11 tcpCliSock = socket(AF_INET,SOCK_STREAM)
12 tcpCliSock.connect(ADDR)
13 
14 #輸出資訊函式
15 def ListPeople(address_book):
16   for person in address_book.person:
17     print "Person ID:", person.id
18     print "  Name:", person.name
19     if person.HasField('email'):
20       print "  E-mail address:", person.email
21 
22     for phone_number in person.phone:
23       if phone_number.type == addressbook_pb2.Person.MOBILE:
24         print "  Mobile phone #:",
25       elif phone_number.type == addressbook_pb2.Person.HOME:
26         print "  Home phone #:",
27       elif phone_number.type == addressbook_pb2.Person.WORK:
28         print "  Work phone #:",
29       print phone_number.number
30 
31 
32 filename = raw_input('Please input the filename you will write to:')
33 address_book = addressbook_pb2.AddressBook()
34 
35 f = open(filename,'ab')  #以追加模式開啟檔案
36 print '正在寫檔案....'
37 while True:
38     data = tcpCliSock.recv(BUFSIZ)
39     if not data:
40         break
41     f.write(data)
42     
43 f.close()
44 print 'Done!'
45 
46 f = open(filename,"rb")
47 address_book.ParseFromString(f.read())
48 f.close()
49 
50 ListPeople(address_book)
51 
52 tcpCliSock.close()

搞定！請多多指教！

轉載請註明出處：http://www.cnblogs.com/sysu-blackbear/

 

相關文章