AutoText
智慧文字自動處理工具（Intelligent text automatic processing tool）。
專案地址：https://github.com/jiangnanboy/AutoText
AutoText的功能主要有文字糾錯，圖片ocr以及表格結構識別等。
Guide
文字糾錯
- 文字糾錯部分詳細見jcorrector
- 本專案目前主要包括：
- 基於ngram的糾錯
- 基於深度學習的糾錯
- 基於模板中文語法糾錯
- 成語、專名糾錯
- 具體使用見本專案中的examples/correct部分，或者jcorrector
圖片ocr
-
這部分主要利用paddleocr 中的檢測與識別部分，並將其中模型轉為onnx格式進行呼叫，本專案在識別前對圖片進行了預處理，使得在cpu環境下，平均一張圖10秒左右。
-
具體使用見本專案中的examples/ocr/text/OcrDemo部分
-
PS
- 模型網盤下載
- 提取碼：b5vq
- 模型下載後可放入resources的text_recgo下或其它位置
-
使用
// read image file
String imagePath = "examples\\ocr\\img_test\\text_example.png";
var imageFile = Paths.get(imagePath);
var image = ImageFactory.getInstance().fromFile(imageFile);
// init model
String detectionModelFile = OcrDemo.class.getClassLoader().getResource(PropertiesReader.get("text_recog_det_model_path")).getPath().replaceFirst("/", "");
String recognitionModelFile = OcrDemo.class.getClassLoader().getResource(PropertiesReader.get("text_recog_rec_model_path")).getPath().replaceFirst("/", "");
Path detectionModelPath = Paths.get(detectionModelFile);
Path recognitionModelPath = Paths.get(recognitionModelFile);
OcrApp ocrApp = new OcrApp(detectionModelPath, recognitionModelPath);
ocrApp.init();
// predict result and consume time
var timeInferStart = System.currentTimeMillis();
Pair<List<TextListBox>, Image> imagePair = ocrApp.ocrImage(image, 960);
System.out.println("consume time: " + (System.currentTimeMillis() - timeInferStart)/1000.0 + "s");
for (var result : imagePair.getLeft()) {
System.out.println(result);
}
// save ocr result image
ocrApp.saveImageOcrResult(imagePair, "ocr_result.png", "examples\\ocr\\output");
ocrApp.closeAllModel();
- 結果，為文字及其座標
position: [800.0, 609.0, 877.0, 609.0, 877.0, 645.0, 800.0, 645.0], text: 8.23%
position: [433.0, 607.0, 494.0, 607.0, 494.0, 649.0, 433.0, 649.0], text: 68.4
position: [96.0, 610.0, 316.0, 611.0, 316.0, 641.0, 96.0, 640.0], text: 股東權益比率（%）
position: [624.0, 605.0, 688.0, 605.0, 688.0, 650.0, 624.0, 650.0], text: 63.2
position: [791.0, 570.0, 887.0, 570.0, 887.0, 600.0, 791.0, 600.0], text: -39.64%
position: [625.0, 564.0, 687.0, 564.0, 687.0, 606.0, 625.0, 606.0], text: 49.7
position: [134.0, 568.0, 279.0, 568.0, 279.0, 598.0, 134.0, 598.0], text: 毛利率（%）
......
- 結果圖片展示圖片OCR
表格結構識別
- 基於規則由opencv研發，主要識別的表格型別有：有邊界表格、無邊界表格以及部分有邊界表格。
- 具體使用見本專案中的examples/ocr/table/TableDemo部分
- 使用
public static void borderedRecog() {
String imagePath = "D:\\project\\idea_workspace\\AutoText\\src\\main\\java\\examples\\ocr\\img_test\\bordered_example.png";
Mat imageMat = imread(imagePath);
System.out.println("imageMat : " + imageMat.size().height() + " " + imageMat.size().width() + " ");
List<List<List<Integer>>> resultList = BorderedRecog.recognizeStructure(imageMat);
System.out.println(resultList);
// ImageUtils.imshow("Image", pair.getRight());
}
public static void unBorderedRecog() {
String imagePath = "D:\\project\\idea_workspace\\AutoText\\src\\main\\java\\examples\\ocr\\img_test\\unbordered_example.jpg";
Mat imageMat = imread(imagePath);
System.out.println("imageMat : " + imageMat.size().height() + " " + imageMat.size().width() + " ");
List<List<List<Integer>>> resultList = UnBorderedRecog.recognizeStructure(imageMat);
System.out.println(resultList);
// ImageUtils.imshow("Image", pair.getRight());
}
public static void partiallyBorderedRecog() {
String imagePath = "D:\\project\\idea_workspace\\AutoText\\src\\main\\java\\examples\\ocr\\img_test\\partially_example.jpg";
Mat imageMat = imread(imagePath);
System.out.println("imageMat : " + imageMat.size().height() + " " + imageMat.size().width() + " ");
List<List<List<Integer>>> resultList = PartiallyBorderedRecog.recognizeStructure(imageMat);
System.out.println(resultList);
// ImageUtils.imshow("Image", pair.getRight());
}
- 結果，為表格單元格座標
[[[58, 48, 247, 182], [560, 48, 247, 182], [811, 48, 246, 182], [309, 48, 247, 182], [1312, 48, 247, 182],
[1061, 48, 247, 182]], [[58, 234, 247, 118], [309, 234, 247, 118], [1061, 234, 247, 118], [560, 234, 247, 118],
[811, 234, 246, 118], [1312, 234, 247, 118]], [[58, 356, 247, 118], [309, 356, 247, 118], [560, 356, 247, 118],
[811, 356, 246, 118], [1061, 356, 247, 118], [1312, 356, 247, 118]], [[58, 478, 247, 118], [309, 478, 247, 118],
[560, 478, 247, 118], [811, 478, 246, 118], [1061, 478, 247, 118], [1312, 478, 247, 118]], [[58, 600, 247, 119],
[309, 600, 247, 119], [560, 600, 247, 119], [811, 600, 246, 119], [1061, 600, 247, 119], [1312, 600, 247, 119]],
[[58, 723, 247, 118], [309, 723, 247, 118], [560, 723, 247, 118], [1061, 723, 247, 118], [1312, 723, 247, 118],
[811, 723, 246, 118]], [[58, 845, 247, 118], [309, 845, 247, 118], [560, 845, 247, 118], [811, 845, 246, 118],
[1312, 845, 247, 118], [1061, 845, 247, 118]]]
- 結果圖片展示表格結構
表格結構和OCR
- 這部分將整合表格結構和OCR識別，同時識別出表格單元格和OCR文字。
- 具體使用見本專案中的examples/ocr/table_text/TableTextDemo部分
- 使用
public static void main(String...args) throws IOException, TranslateException {
String imagePath = "D:\\project\\idea_workspace\\AutoText\\src\\main\\java\\examples\\ocr\\img_test\\bordered_example.png";
TableText tableText = new TableText();
/**
* maxSideLen:image resize
*
* borderType:{0:all, 1:bordered(default), 2:unbordered, 3:partiallybordered}
*/
int maxSideLen = -1; // default, no resize
int borderType = 1; // default, bordered
List<TextListBox> listBoxes = tableText.tableTextRecog(imagePath);
for(TextListBox textListBox : listBoxes) {
System.out.print(textListBox);
}
}
- 結果，為表格單元格座標以及單元格內的文字
position: [58.0, 48.0, 305.0, 48.0, 305.0, 230.0, 58.0, 230.0], text: 節次 星期
position: [309.0, 48.0, 556.0, 48.0, 556.0, 230.0, 309.0, 230.0], text: 週一
position: [811.0, 48.0, 1057.0, 48.0, 1057.0, 230.0, 811.0, 230.0], text: 週三
position: [1061.0, 48.0, 1308.0, 48.0, 1308.0, 230.0, 1061.0, 230.0], text: 週四
position: [560.0, 48.0, 807.0, 48.0, 807.0, 230.0, 560.0, 230.0], text: 週二
position: [1312.0, 48.0, 1559.0, 48.0, 1559.0, 230.0, 1312.0, 230.0], text: 週五
position: [58.0, 234.0, 305.0, 234.0, 305.0, 352.0, 58.0, 352.0], text:
position: [309.0, 234.0, 556.0, 234.0, 556.0, 352.0, 309.0, 352.0], text: 語文
position: [811.0, 234.0, 1057.0, 234.0, 1057.0, 352.0, 811.0, 352.0], text: 英語
position: [560.0, 234.0, 807.0, 234.0, 807.0, 352.0, 560.0, 352.0], text: 英語
position: [1061.0, 234.0, 1308.0, 234.0, 1308.0, 352.0, 1061.0, 352.0], text: 自然
position: [1312.0, 234.0, 1559.0, 234.0, 1559.0, 352.0, 1312.0, 352.0], text: 數學
position: [58.0, 356.0, 305.0, 356.0, 305.0, 474.0, 58.0, 474.0], text: 3
position: [560.0, 356.0, 807.0, 356.0, 807.0, 474.0, 560.0, 474.0], text: 英語
position: [309.0, 356.0, 556.0, 356.0, 556.0, 474.0, 309.0, 474.0], text: 語文
position: [811.0, 356.0, 1057.0, 356.0, 1057.0, 474.0, 811.0, 474.0], text: 英語
position: [1312.0, 356.0, 1559.0, 356.0, 1559.0, 474.0, 1312.0, 474.0], text: 數學
position: [1061.0, 356.0, 1308.0, 356.0, 1308.0, 474.0, 1061.0, 474.0], text: 語文
position: [58.0, 478.0, 305.0, 478.0, 305.0, 596.0, 58.0, 596.0], text: 三
position: [309.0, 478.0, 556.0, 478.0, 556.0, 596.0, 309.0, 596.0], text: 數學
position: [560.0, 478.0, 807.0, 478.0, 807.0, 596.0, 560.0, 596.0], text: 語文
position: [811.0, 478.0, 1057.0, 478.0, 1057.0, 596.0, 811.0, 596.0], text: 數學
position: [1312.0, 478.0, 1559.0, 478.0, 1559.0, 596.0, 1312.0, 596.0], text: 英語
position: [1061.0, 478.0, 1308.0, 478.0, 1308.0, 596.0, 1061.0, 596.0], text: 語文
position: [58.0, 600.0, 305.0, 600.0, 305.0, 719.0, 58.0, 719.0], text: 四
position: [309.0, 600.0, 556.0, 600.0, 556.0, 719.0, 309.0, 719.0], text: 數學
position: [811.0, 600.0, 1057.0, 600.0, 1057.0, 719.0, 811.0, 719.0], text: 數學
position: [560.0, 600.0, 807.0, 600.0, 807.0, 719.0, 560.0, 719.0], text: 語文
position: [1061.0, 600.0, 1308.0, 600.0, 1308.0, 719.0, 1061.0, 719.0], text: 體育
position: [1312.0, 600.0, 1559.0, 600.0, 1559.0, 719.0, 1312.0, 719.0], text: 英語
position: [58.0, 723.0, 305.0, 723.0, 305.0, 841.0, 58.0, 841.0], text: 五
position: [560.0, 723.0, 807.0, 723.0, 807.0, 841.0, 560.0, 841.0], text: 思想品德
position: [309.0, 723.0, 556.0, 723.0, 556.0, 841.0, 309.0, 841.0], text: 體育
position: [1061.0, 723.0, 1308.0, 723.0, 1308.0, 841.0, 1061.0, 841.0], text: 數學
position: [1312.0, 723.0, 1559.0, 723.0, 1559.0, 841.0, 1312.0, 841.0], text: 手工
position: [811.0, 723.0, 1057.0, 723.0, 1057.0, 841.0, 811.0, 841.0], text: 語文
position: [58.0, 845.0, 305.0, 845.0, 305.0, 963.0, 58.0, 963.0], text: 六
position: [309.0, 845.0, 556.0, 845.0, 556.0, 963.0, 309.0, 963.0], text: 美術
position: [560.0, 845.0, 807.0, 845.0, 807.0, 963.0, 560.0, 963.0], text: 音樂
position: [1061.0, 845.0, 1308.0, 845.0, 1308.0, 963.0, 1061.0, 963.0], text: 數學
position: [811.0, 845.0, 1057.0, 845.0, 1057.0, 963.0, 811.0, 963.0], text: 語文
position: [1312.0, 845.0, 1559.0, 845.0, 1559.0, 963.0, 1312.0, 963.0], text: 寫字
Todo
- 加入jcorrector文字糾錯，修改部分程式
- 基於paddleocr模型，利用java實現圖片ocr
- 基於規則利用opencv識別表格結構
- 整合規則表格識別與OCR識別