解析XML檔案時,無效的XML 字元 (Unicode: 0x7)異常處理

伏念先生發表於2018-08-06
XML字元Unicode

報錯資訊：

2015-01-29 00:10:22,075  ERROR commonapi.CommonApiAction - errorCode:5000,5000-00；Description:程式異常。Error on line 1 of document  : An invalid XML character (Unicode: 0x19) was found in the CDATA section. Nested exception: An invalid XML character (Unicode: 0x19) was found in the CDATA section.
org.dom4j.DocumentException: Error on line 1 of document  : An invalid XML character (Unicode: 0x19) was found in the CDATA section. Nested exception: An invalid XML character (Unicode: 0x19) was found in the CDATA section.
at org.dom4j.io.SAXReader.read(SAXReader.java:482)
at org.dom4j.DocumentHelper.parseText(DocumentHelper.java:278)
at com.hoodong.engine.commonapi.CommonApiAction.getWapDocsSearchJsonInfo(CommonApiAction.java:1866)
at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)

錯誤原因：

這些無效的字元在一些文件中作為文件處理器的控制編碼（微軟選擇了那些再0x82到0x95之間的字元作為"smart"標點），這些也被Unicode保留作為控制編碼的，並且在XML中是不合法的。這裡的無效字元不是指<,>等不能出現在XML檔案的標籤以外的字元，也不是由於編碼問題引起的亂碼，而是一些超出XML合法字元範圍的不可見字元。根據W3C標準，有一些字元不能出現在XML檔案中：

// Document authors are encouraged to avoid "compatibility characters", as defined in 
// Unicode [Unicode]. The characters defined in the following ranges are also discouraged. // They are either control characters or permanently undefined Unicode characters:

[#x1-#x8], [#xB-#xC], [#xE-#x1F], [#x7F-#x84], [#x86-#x9F], [#xFDD0-#xFDDF],
[#x1FFFE-#x1FFFF], [#x2FFFE-#x2FFFF], [#x3FFFE-#x3FFFF],
[#x4FFFE-#x4FFFF], [#x5FFFE-#x5FFFF], [#x6FFFE-#x6FFFF],
[#x7FFFE-#x7FFFF], [#x8FFFE-#x8FFFF], [#x9FFFE-#x9FFFF],
[#xAFFFE-#xAFFFF], [#xBFFFE-#xBFFFF], [#xCFFFE-#xCFFFF],
[#xDFFFE-#xDFFFF], [#xEFFFE-#xEFFFF], [#xFFFFE-#xFFFFF],
[#x10FFFE-#x10FFFF].

解決辦法：

為了保證常用XML解析工具能將自己生成的XML檔案成功解析，就需要先將檔案中的無效字元過濾掉，或在生成XML檔案時就對字元的有效性進行判斷，拋棄無效字元。

Unicode是國際組織制定的可以容納世界上所有文字和符號的字元編碼方案。目前的Unicode字元分為17組編排，0x0000 至 0x10FFFF，每組稱為平面（Plane），而每平面擁有65536個碼位，共1114112個。然而目前只用了少數平面。UTF-8、UTF-16、UTF-32都是將數字轉換到程式資料的編碼方案。

查了一下W3C中對XML 1.0的定義[1]，其Unicode的合法字元範圍(16進位制)是：

Character Range
[2]   	Char	   ::=   	#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]	
/* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */

方法一：

// 保留合法字元
public String stripNonValidXMLCharacters(String in) {
    StringBuffer out = new StringBuffer(); // Used to hold the output.
    char current; // Used to reference the current character.

    if (in == null || ("".equals(in))) return ""; // vacancy test.
    for (int i = 0; i < in.length(); i++) {
        current = in.charAt(i); // NOTE: No IndexOutOfBoundsException caught here; it should not happen.
        if ((current == 0x9) ||
            (current == 0xA) ||
            (current == 0xD) ||
            ((current >= 0x20) && (current <= 0xD7FF)) ||
            ((current >= 0xE000) && (current <= 0xFFFD)) ||
            ((current >= 0x10000) && (current <= 0x10FFFF)))
            out.append(current);
    }
    return out.toString();
}

方法二：

//過濾非法字元
//注意，以下正規表示式過濾不全面，過濾範圍為
//  0x00 - 0x08
//  0x0b - 0x0c
//  0x0e - 0x1f

public static String stripNonValidXMLChars(String str) {
  if (str == null || "".equals(str)) {
    return str;
  }
  return str.replaceAll("[\\x00-\\x08\\x0b-\\x0c\\x0e-\\x1f]", "");
}

參考：https://www.w3.org/TR/xml/#charsets；https://stackoverflow.com/questions/5742543/an-invalid-xml-character-unicode-0xc-was-found

相關文章