Netty 實現HTTP檔案伺服器

weixin_34197488發表於2016-04-07

一,需求

檔案伺服器使用HTTP協議對外提供服務。使用者通過瀏覽器訪問檔案伺服器,首先對URL進行檢查,若失敗返回403錯誤;若通過校驗,以連結的方式開啟當前目錄,每個目錄或檔案都以超連結的形式展現,可遞迴訪問,並下載檔案。

 

二,關鍵實現程式碼

①檔案伺服器啟動類

需要新增的通道處理器如下:

@Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast("http-decoder", new HttpRequestDecoder());
                    ch.pipeline().addLast("http-aggregator", new HttpObjectAggregator(65536));
                    ch.pipeline().addLast("http-encoder", new HttpResponseEncoder());
                    ch.pipeline().addLast("http-chunked", new ChunkedWriteHandler());
                    ch.pipeline().addLast("fileServerHandler", new HttpFileServerHandler(url));
                }

1) HttpRequestDecoder

 Decodes {@link ByteBuf}s into {@link HttpRequest}s and {@link HttpContent}s.

它負責把位元組解碼成Http請求。

2) HttpObjectAggregator

 A {@link ChannelHandler} that aggregates an {@link HttpMessage}  and its following {@link HttpContent}s into a single {@link FullHttpRequest} or {@link FullHttpResponse} (depending on if it used to handle requests or responses)

它負責把多個HttpMessage組裝成一個完整的Http請求或者響應。到底是組裝成請求還是響應,則取決於它所處理的內容是請求的內容,還是響應的內容。這其實可以通過Inbound和Outbound來判斷,對於Server端而言,在Inbound 端接收請求,在Outbound端返回響應。

It is useful when you don't want to take care of HTTP messages whose transfer encoding is 'chunked'.

如果Server向Client返回的資料指定的傳輸編碼是 chunked。則,Server不需要知道傳送給Client的資料總長度是多少,它是通過分塊傳送的,參考分塊傳輸編碼

Be aware that you need to have the {@link HttpResponseEncoder} or {@link HttpRequestEncoder} before the {@link HttpObjectAggregator} in the {@link ChannelPipeline}.

注意,HttpObjectAggregator通道處理器必須放到HttpRequestDecoder或者HttpRequestEncoder後面。

3) HttpResponseEncoder

當Server處理完訊息後,需要向Client傳送響應。那麼需要把響應編碼成位元組,再傳送出去。故新增HttpResponseEncoder處理器。

4)ChunkedWriteHandler

 A {@link ChannelHandler} that adds support for writing a large data stream asynchronously neither spending a lot of memory nor getting {@link OutOfMemoryError}.

該通道處理器主要是為了處理大檔案傳輸的情形。大檔案傳輸時,需要複雜的狀態管理,而ChunkedWriteHandler實現這個功能。

5) HttpFileServerHandler

自定義的通道處理器,其目的是實現檔案伺服器的業務邏輯。

 

通道處理器新增完畢之後,需要啟動伺服器。程式碼如下:

 

ChannelFuture f = b.bind("localhost", port).sync();
f.channel().closeFuture().sync();

 

因為在Netty中所有的事件都是非同步的,因此bind操作是一個非同步操作,通道的關閉也是一個非同步操作。因此使用ChannelFuture來作為一個 palceholder,代表操作執行之後的結果。

最後關閉事件執行緒,程式碼如下:

bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();

 

②檔案處理器類

HttpFileServerHandler.java是自定義的通道處理器,用來實現HTTP檔案伺服器的業務邏輯。從上面新增的Handler可以看出,在HTTP檔案伺服器的實現過程中,Netty已經為我們解決了很多工作,如:HttpRequestDecoder自動幫我們解析HTTP請求(解析byte);再比如:HttpObjectAggregator把多個HTTP請求中的資料組裝成一個,當伺服器傳送的response事先不知道響應的長度時就很有用。

參考:HttpChunkAggregator分析

檔案處理器通過繼承SimpleChannelInboundHandler來實現,程式碼如下:

public class HttpFileServerHandler extends SimpleChannelInboundHandler<FullHttpRequest>{

    private final String url;
    
    public HttpFileServerHandler(String url) {
        this.url = url;
    }
    
    @Override
    protected void messageReceived(ChannelHandlerContext ctx,
            FullHttpRequest request) throws Exception {
        if(!request.decoderResult().isSuccess())
        {
            sendError(ctx, HttpResponseStatus.BAD_REQUEST);
            return;
        }
        if(request.method() != HttpMethod.GET)
        {
            sendError(ctx, HttpResponseStatus.METHOD_NOT_ALLOWED);
            return;
        }

當伺服器接收到訊息時,會自動觸發 messageReceived方法。該方法首先對URL進行判斷,並只接受GET請求。

 

相關的驗證通過後,通過RandomAccessFile類開啟檔案,並構造響應。

        RandomAccessFile randomAccessFile = null;
        try{
            randomAccessFile = new RandomAccessFile(file, "r");
        }catch(FileNotFoundException fnfd){
            sendError(ctx, HttpResponseStatus.NOT_FOUND);
            return;
        }
        
        long fileLength = randomAccessFile.length();
        HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);

 

如果請求中帶有“KEEP-ALIVE”,則不關閉連線。

if(HttpHeaderUtil.isKeepAlive(request)){
            response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE);
        }

 

進行資料的傳送

sendFileFuture = ctx.write(new ChunkedFile(randomAccessFile, 0, fileLength, 8192), ctx.newProgressivePromise());
        sendFileFuture.addListener(new ChannelProgressiveFutureListener() {
            
            @Override
            public void operationComplete(ChannelProgressiveFuture future)
                    throws Exception {
                System.out.println("Transfer complete.");
                
            }
ctx.write(response);

 

當傳送完資料之後,由於採用的是Transfer-Encoding:chunk模式來傳輸資料,因此需要在傳送一個長度為0的chunk用來標記資料傳輸完成。

參考:HTTP協議頭部與Keep-Alive模式詳解

ChannelFuture lastContentFuture = ctx.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT);
        if(!HttpHeaderUtil.isKeepAlive(request))
            lastContentFuture.addListener(ChannelFutureListener.CLOSE);
        
    }

使用Keep-Alive,可以減少HTTP連線建立的次數,在HTTP1.1中該選項是預設開啟的。

Connection: Keep-Alive When the server processes the request and generates a response, it also adds a header to the response: Connection: Keep-Alive When this is done, the socket connection is not closed as before, but kept open after sending the response.  When the client sends another request, it reuses the same connection. The connection will continue to be reused until either the client or the server decides that the conversation is over, and one of them drops the connection.

在使用Keep-Alive的情況下,當Server處理了Client的請求且生成一個response後,在response的頭部新增Connection: Keep-Alive選項,把response返回給client,此時Socket連線並不會關閉。

【若沒有Keep-Alive,一次HTTP請求響應之後,本次Socket連線就關閉了】

由於連線還沒有關閉,當client再傳送另一個請求時,就會重用這個Socket連線,直至其中一方drops the connection.

關於Keep-Alive的討論,參考:

 

整個原始碼參考:

package httpFileServer;

import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpRequestDecoder;
import io.netty.handler.codec.http.HttpRequestEncoder;
import io.netty.handler.codec.http.HttpResponseEncoder;
import io.netty.handler.stream.ChunkedWriteHandler;

public class HttpFileServer {
    private static final String DEFAULT_URL = "/src/";
    
    public void run(final int port, final String url)throws Exception{
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        
        try{
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class)
            .childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast("http-decoder", new HttpRequestDecoder());
                    ch.pipeline().addLast("http-aggregator", new HttpObjectAggregator(65536));
                    ch.pipeline().addLast("http-encoder", new HttpResponseEncoder());
                    ch.pipeline().addLast("http-chunked", new ChunkedWriteHandler());
                    ch.pipeline().addLast("fileServerHandler", new HttpFileServerHandler(url));
                }
            });
            
            ChannelFuture f = b.bind("localhost", port).sync();
            System.out.println("HTTP 檔案伺服器啟動, 地址是: " + "http://localhost:" + port + url);
            f.channel().closeFuture().sync();
            
        }finally{
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
    
    public static void main(String[] args)throws Exception {
        int port = 8888;
        if(args.length > 0)
        {
            try{
                port = Integer.parseInt(args[0]);
            }catch(NumberFormatException e){
                port = 8080;
            }
        }
        
        String url = DEFAULT_URL;
        if(args.length > 1)
            url = args[1];
        new HttpFileServer().run(port, url);
    }
}


package httpFileServer;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.RandomAccessFile;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.regex.Pattern;

import javax.activation.MimetypesFileTypeMap;
import javax.swing.text.html.MinimalHTMLWriter;

import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelProgressiveFuture;
import io.netty.channel.ChannelProgressiveFutureListener;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.handler.codec.http.DefaultFullHttpResponse;
import io.netty.handler.codec.http.DefaultHttpResponse;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpHeaderUtil;
import io.netty.handler.codec.http.HttpHeaderValues;
import io.netty.handler.codec.http.HttpHeaders;
import io.netty.handler.codec.http.HttpMessage;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http.HttpResponse;
import io.netty.handler.codec.http.HttpResponseStatus;
import io.netty.handler.codec.http.HttpVersion;
import io.netty.handler.codec.http.LastHttpContent;
import io.netty.handler.codec.http2.Http2Headers;
import io.netty.handler.stream.ChunkedFile;
import io.netty.util.CharsetUtil;
import io.netty.util.concurrent.Future;
import io.netty.util.concurrent.GenericFutureListener;

public class HttpFileServerHandler extends SimpleChannelInboundHandler<FullHttpRequest>{

    private final String url;
    
    public HttpFileServerHandler(String url) {
        this.url = url;
    }
    
    @Override
    protected void messageReceived(ChannelHandlerContext ctx,
            FullHttpRequest request) throws Exception {
        if(!request.decoderResult().isSuccess())
        {
            sendError(ctx, HttpResponseStatus.BAD_REQUEST);
            return;
        }
        if(request.method() != HttpMethod.GET)
        {
            sendError(ctx, HttpResponseStatus.METHOD_NOT_ALLOWED);
            return;
        }
        
        final String uri = request.uri();
        final String path = sanitizeUri(uri);
        if(path == null)
        {
            sendError(ctx, HttpResponseStatus.FORBIDDEN);
            return;
        }
        
        File file = new File(path);
        if(file.isHidden() || !file.exists())
        {
            sendError(ctx, HttpResponseStatus.NOT_FOUND);
            return;
        }
        if(file.isDirectory())
        {
            if(uri.endsWith("/"))
            {
                sendListing(ctx, file);
            }else{
                sendRedirect(ctx, uri + "/");
            }
            return;
        }
        if(!file.isFile())
        {
            sendError(ctx, HttpResponseStatus.FORBIDDEN);
            return;
        }
        
        RandomAccessFile randomAccessFile = null;
        try{
            randomAccessFile = new RandomAccessFile(file, "r");
        }catch(FileNotFoundException fnfd){
            sendError(ctx, HttpResponseStatus.NOT_FOUND);
            return;
        }
        
        long fileLength = randomAccessFile.length();
        HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
        HttpHeaderUtil.setContentLength(response, fileLength);
//        setContentLength(response, fileLength);
        setContentTypeHeader(response, file);
        
        
        
        if(HttpHeaderUtil.isKeepAlive(request)){
            response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE);
        }
        
        ctx.write(response);
        ChannelFuture sendFileFuture = null;
        sendFileFuture = ctx.write(new ChunkedFile(randomAccessFile, 0, fileLength, 8192), ctx.newProgressivePromise());
        sendFileFuture.addListener(new ChannelProgressiveFutureListener() {
            
            @Override
            public void operationComplete(ChannelProgressiveFuture future)
                    throws Exception {
                System.out.println("Transfer complete.");
                
            }
            
            @Override
            public void operationProgressed(ChannelProgressiveFuture future,
                    long progress, long total) throws Exception {
                if(total < 0)
                    System.err.println("Transfer progress: " + progress);
                else
                    System.err.println("Transfer progress: " + progress + "/" + total);
            }
        });
        
        ChannelFuture lastContentFuture = ctx.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT);
        if(!HttpHeaderUtil.isKeepAlive(request))
            lastContentFuture.addListener(ChannelFutureListener.CLOSE);
        
    }
    
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
            throws Exception {
        cause.printStackTrace();
        if(ctx.channel().isActive())
            sendError(ctx, HttpResponseStatus.INTERNAL_SERVER_ERROR);
    }
    
    private static final Pattern INSECURE_URI = Pattern.compile(".*[<>&\"].*");
    private String sanitizeUri(String uri){
        try{
            uri = URLDecoder.decode(uri, "UTF-8");
        }catch(UnsupportedEncodingException e){
            try{
                uri = URLDecoder.decode(uri, "ISO-8859-1");
            }catch(UnsupportedEncodingException e1){
                throw new Error();
            }
        }
        
        if(!uri.startsWith(url))
            return null;
        if(!uri.startsWith("/"))
            return null;
        
        uri = uri.replace('/', File.separatorChar);
        if(uri.contains(File.separator + '.') || uri.contains('.' + File.separator) || uri.startsWith(".") || uri.endsWith(".") 
                || INSECURE_URI.matcher(uri).matches()){
            return null;
        }
        return System.getProperty("user.dir") + File.separator + uri;
    }
    
    private static final Pattern ALLOWED_FILE_NAME = Pattern.compile("[A-Za-z0-9][-_A-Za-z0-9\\.]*");
    
    private static void sendListing(ChannelHandlerContext ctx, File dir){
        FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
//        response.headers().set("CONNECT_TYPE", "text/html;charset=UTF-8");
        response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/html;charset=UTF-8");
        
        String dirPath = dir.getPath();
        StringBuilder buf = new StringBuilder();
        
        buf.append("<!DOCTYPE html>\r\n");
        buf.append("<html><head><title>");
        buf.append(dirPath);
        buf.append("目錄:");
        buf.append("</title></head><body>\r\n");
        
        buf.append("<h3>");
        buf.append(dirPath).append(" 目錄:");
        buf.append("</h3>\r\n");
        buf.append("<ul>");
        buf.append("<li>連結:<a href=\" ../\")..</a></li>\r\n");
        for (File f : dir.listFiles()) {
            if(f.isHidden() || !f.canRead()) {
                continue;
            }
            String name = f.getName();
            if (!ALLOWED_FILE_NAME.matcher(name).matches()) {
                continue;
            }
            
            buf.append("<li>連結:<a href=\"");
            buf.append(name);
            buf.append("\">");
            buf.append(name);
            buf.append("</a></li>\r\n");
        }
        
        buf.append("</ul></body></html>\r\n");
        
        ByteBuf buffer = Unpooled.copiedBuffer(buf,CharsetUtil.UTF_8);  
        response.content().writeBytes(buffer);  
        buffer.release();  
        ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE); 
    }
    
    
    private static void sendRedirect(ChannelHandlerContext ctx, String newUri){
        FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.FOUND);
//        response.headers().set("LOCATIN", newUri);
        response.headers().set(HttpHeaderNames.LOCATION, newUri);
        ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
    }
    private static void sendError(ChannelHandlerContext ctx, HttpResponseStatus status){
        FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, status, 
                Unpooled.copiedBuffer("Failure: " + status.toString() + "\r\n", CharsetUtil.UTF_8));
        response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/html;charset=UTF-8");
        ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
    }
    private static void setContentTypeHeader(HttpResponse response, File file){
        MimetypesFileTypeMap mimetypesFileTypeMap = new MimetypesFileTypeMap();
        response.headers().set(HttpHeaderNames.CONTENT_TYPE, mimetypesFileTypeMap.getContentType(file.getPath()));
    }
}

 

相關文章