NIO框架之MINA原始碼解析(四):粘包與斷包處理及編碼與解碼

chaofanwei發表於2014-09-11

NIO框架之MINA原始碼解析(一):背景


NIO框架之MINA原始碼解析(二):mina核心引擎


NIO框架之MINA原始碼解析(三):底層通訊與責任鏈模式應用



1、粘包與段包


粘包:指TCP協議中,傳送方傳送的若干包資料到接收方接收時粘成一包,從接收緩衝區看,後一包資料的頭緊接著前一包資料的尾。
造成的可能原因:

    傳送端需要等緩衝區滿才傳送出去,造成粘包

    接收方不及時接收緩衝區的包,造成多個包接收


斷包:也就是資料不全,比如包太大,就把包分解成多個小包,多次傳送,導致每次接收資料都不全。


2、訊息傳輸的格式


訊息長度+訊息頭+訊息體  即前N個位元組用於儲存訊息的長度,用於判斷當前訊息什麼時候結束。

訊息頭+訊息體    即固定長度的訊息,前幾個位元組為訊息頭,後面的是訊息頭。

在MINA中用的是

訊息長度+訊息體 即前4個位元組用於儲存訊息的長度,用於判斷當前訊息什麼時候結束。


3、編碼與解碼


   在網路中,資訊的傳輸都是通過位元組的形式傳輸的,而我們在編寫自己的程式碼時,則都是具體的物件,那麼要想我們的物件能夠在網路中傳輸,就需要編碼與解碼。


   編碼:即把我們的訊息編碼成二進位制形式,能以位元組的形式在網路中傳輸。

   解碼:即把我們收到的位元組解碼成我們程式碼中的物件。

   在MINA中物件的編碼與解碼用的都是JDK提供的ObjectOutputStream來實現的。


4、MINA中訊息的處理實現


訊息的接受處理,我們常用的是TCP協議,而TCP協議會分片的,在下面的程式碼中,具體功能就是迴圈從通道里面讀取資料,直到沒有資料可讀,或者buffer滿了,然後就把接受到的資料發給解碼工廠進行處理。


4.1、訊息的接收


//class AbstractPollingIoProcessor
private void read(S session) {
        IoSessionConfig config = session.getConfig();
        int bufferSize = config.getReadBufferSize();
        IoBuffer buf = IoBuffer.allocate(bufferSize);

        final boolean hasFragmentation = session.getTransportMetadata().hasFragmentation();

        try {
            int readBytes = 0;
            int ret;

            try {
                //是否有分片 tcp傳輸會有分片,即把大訊息分片成多個小訊息再傳輸
                if (hasFragmentation) {
            //read方法非阻塞,沒有讀到資料的時候返回0
                    while ((ret = read(session, buf)) > 0) {
                        readBytes += ret;
                        //buffer 滿了
                        if (!buf.hasRemaining()) {
                            break;
                        }
                    }
                } else {
                    ret = read(session, buf);

                    if (ret > 0) {
                        readBytes = ret;
                    }
                }
            } finally {
                buf.flip();
            }

            if (readBytes > 0) {
                IoFilterChain filterChain = session.getFilterChain();
        //處理訊息
                filterChain.fireMessageReceived(buf);
                buf = null;

                if (hasFragmentation) {
                    if (readBytes << 1 < config.getReadBufferSize()) {
                        session.decreaseReadBufferSize();
                    } else if (readBytes == config.getReadBufferSize()) {
                        session.increaseReadBufferSize();
                    }
                }
            }

            if (ret < 0) {
                scheduleRemove(session);
            }
        } catch (Throwable e) {
            if (e instanceof IOException) {
                if (!(e instanceof PortUnreachableException)
                        || !AbstractDatagramSessionConfig.class.isAssignableFrom(config.getClass())
                        || ((AbstractDatagramSessionConfig) config).isCloseOnPortUnreachable()) {
                    scheduleRemove(session);
                }
            }

            IoFilterChain filterChain = session.getFilterChain();
            filterChain.fireExceptionCaught(e);
        }
    }


4.2、解碼與編碼


//class AbstractIoBuffer
 public Object getObject(final ClassLoader classLoader) throws ClassNotFoundException {
	//首先判斷當前buffer中訊息長度是否完整,不完整的話直接返回
        if (!prefixedDataAvailable(4)) {
            throw new BufferUnderflowException();
        }

	//訊息長度
        int length = getInt();
        if (length <= 4) {
            throw new BufferDataException("Object length should be greater than 4: " + length);
        }

        int oldLimit = limit();
	//limit到訊息結尾處
        limit(position() + length);
        try {
            ObjectInputStream in = new ObjectInputStream(asInputStream()) {
                @Override
                protected ObjectStreamClass readClassDescriptor() throws IOException, ClassNotFoundException {
                    int type = read();
                    if (type < 0) {
                        throw new EOFException();
                    }
                    switch (type) {
                    case 0: // NON-Serializable class or Primitive types
                        return super.readClassDescriptor();
                    case 1: // Serializable class
                        String className = readUTF();
                        Class<?> clazz = Class.forName(className, true, classLoader);
                        return ObjectStreamClass.lookup(clazz);
                    default:
                        throw new StreamCorruptedException("Unexpected class descriptor type: " + type);
                    }
                }

                @Override
                protected Class<?> resolveClass(ObjectStreamClass desc) throws IOException, ClassNotFoundException {
                    String name = desc.getName();
                    try {
                        return Class.forName(name, false, classLoader);
                    } catch (ClassNotFoundException ex) {
                        return super.resolveClass(desc);
                    }
                }
            };
            return in.readObject();
        } catch (IOException e) {
            throw new BufferDataException(e);
        } finally {
            limit(oldLimit);
        }
    }

//判斷當前訊息是否完整 
public boolean prefixedDataAvailable(int prefixLength, int maxDataLength) {
        if (remaining() < prefixLength) {
            return false;
        }

        int dataLength;
        switch (prefixLength) {
        case 1:
            dataLength = getUnsigned(position());
            break;
        case 2:
            dataLength = getUnsignedShort(position());
            break;
        case 4:
            dataLength = getInt(position());
            break;
        default:
            throw new IllegalArgumentException("prefixLength: " + prefixLength);
        }

        if (dataLength < 0 || dataLength > maxDataLength) {
            throw new BufferDataException("dataLength: " + dataLength);
        }
	//判斷當前訊息是否完整 
        return remaining() - prefixLength >= dataLength;
    }

//編碼
 public IoBuffer putObject(Object o) {
        int oldPos = position();
        skip(4); // Make a room for the length field.預留4個位元組用於儲存訊息長度
        try {
            ObjectOutputStream out = new ObjectOutputStream(asOutputStream()) {
                @Override
                protected void writeClassDescriptor(ObjectStreamClass desc) throws IOException {
                    try {
                        Class<?> clz = Class.forName(desc.getName());
                        if (!Serializable.class.isAssignableFrom(clz)) { // NON-Serializable class
                            write(0);
                            super.writeClassDescriptor(desc);
                        } else { // Serializable class
                            write(1);
                            writeUTF(desc.getName());
                        }
                    } catch (ClassNotFoundException ex) { // Primitive types
                        write(0);
                        super.writeClassDescriptor(desc);
                    }
                }
            };
            out.writeObject(o);
            out.flush();
        } catch (IOException e) {
            throw new BufferDataException(e);
        }

        // Fill the length field
        int newPos = position();
        position(oldPos);
	//儲存訊息長度
        putInt(newPos - oldPos - 4);
        position(newPos);
        return this;
    }


4.3、斷包與粘包處理

// class CumulativeProtocolDecoder
 public void decode(IoSession session, IoBuffer in, ProtocolDecoderOutput out) throws Exception {
	//是否有分片,tcp 有分片
        if (!session.getTransportMetadata().hasFragmentation()) {
            while (in.hasRemaining()) {
                if (!doDecode(session, in, out)) {
                    break;
                }
            }

            return;
        }

	// 1、斷包處理
	// 2、處理粘包
        boolean usingSessionBuffer = true;
	//session中是否有斷包情況(上次處理後),斷包儲存在session中
        IoBuffer buf = (IoBuffer) session.getAttribute(BUFFER);
        // If we have a session buffer, append data to that; otherwise
        // use the buffer read from the network directly.
        if (buf != null) {//有斷包,則把當前包拼接到斷包裡面
            boolean appended = false;
            // Make sure that the buffer is auto-expanded.
            if (buf.isAutoExpand()) {
                try {
                    buf.put(in);
                    appended = true;
                } catch (IllegalStateException e) {
                    // A user called derivation method (e.g. slice()),
                    // which disables auto-expansion of the parent buffer.
                } catch (IndexOutOfBoundsException e) {
                    // A user disabled auto-expansion.
                }
            }

            if (appended) {
                buf.flip();
            } else {
                // Reallocate the buffer if append operation failed due to
                // derivation or disabled auto-expansion.
                buf.flip();
                IoBuffer newBuf = IoBuffer.allocate(buf.remaining() + in.remaining()).setAutoExpand(true);
                newBuf.order(buf.order());
                newBuf.put(buf);
                newBuf.put(in);
                newBuf.flip();
                buf = newBuf;

                // Update the session attribute.
                session.setAttribute(BUFFER, buf);
            }
        } else {
            buf = in;
            usingSessionBuffer = false;
        }

	//2 粘包處理,可能buffer中有多個訊息,需要多次處理(解碼)每個訊息,直到訊息處理完,或者剩下的訊息不是一個完整的訊息或者buffer沒有資料了

        for (;;) {
            int oldPos = buf.position();
            boolean decoded = doDecode(session, buf, out);
            if (decoded) {//解碼 成功
                if (buf.position() == oldPos) {
                    throw new IllegalStateException("doDecode() can't return true when buffer is not consumed.");
                }
		//buffer空了
                if (!buf.hasRemaining()) {//buffer沒有資料了
                    break;
                }
            } else {//剩下的訊息不是一個完整的訊息,斷包出現了
                break;
            }
        }

        // if there is any data left that cannot be decoded, we store
        // it in a buffer in the session and next time this decoder is
        // invoked the session buffer gets appended to
        if (buf.hasRemaining()) {//剩下的訊息不是一個完整的訊息,斷包出現了
		//如果斷包已經儲存在session中,則更新buffer,沒有的話,就把剩下的斷包儲存在session中
            if (usingSessionBuffer && buf.isAutoExpand()) {
                buf.compact();
            } else {
                storeRemainingInSession(buf, session);
            }
        } else {
            if (usingSessionBuffer) {
                removeSessionBuffer(session);
            }
        }
    }


//class  ObjectSerializationDecoder
 protected boolean doDecode(IoSession session, IoBuffer in, ProtocolDecoderOutput out) throws Exception {
	//首先判斷當前buffer中訊息長度是否完整,不完整的話直接返回
        if (!in.prefixedDataAvailable(4, maxObjectSize)) {
            return false;
        }

        out.write(in.getObject(classLoader));
        return true;
    }




相關文章