Mejora y reconocimiento de imágenes de documentos en escenarios de bajos recursos: aplicación a cifrados y texto manuscrito

Mohamed Ali Souibgui

Alicia Fornés Yousri Kessentini

dirección de Alicia Fornés, investigadora sénior del Centre de Visió per Computador – CVC; Yo Yousri Kessentini, Centro de Investigación Digital de Sfax (CRNS).

This thesis introduces innovative contributions aimed at advancing the enhancement and recognition of handwritten document images, in particular those that include rare alphabets such as encrypted documents or highly degraded documents. Beyond the scope of technical advances, our research explores the profound impact on the optical character recognition (OCR) industry, recognizing the tangible applications of these innovations. In the first part of the thesis, we focus on solving the challenge of paper degradation, an obstacle for current OCR systems. To solve this, we present end-to-end models for document image enhancement (DIE) using deep learning techniques. This involves deploying generative adversarial networks (cGANs) for tasks such as document cleaning, binarization, blurring, and removing watermarks on paper. In particular, we integrate a text recognition module into the cGAN model, improving the quality of images and making them clearer and more readable. In addition, the introduction of a new encoder-decoder architecture based on vision transformers provides a comprehensive end-to-end solution for enhancing document images, both printed and handwritten. This advancement makes it easier for OCR systems to accurately recognize and convert text from images into editable and searchable digital formats, thus impacting several industrial fields. The second part of the thesis focuses on text recognition in resource-poor scenarios, precisely one of the challenges industries face when there is little labeled data to train deep learning models. Specifically, we propose methods for recognizing rare alphabets with few data, including a system based on few-shot object detection (fOL) and a progressive learning strategy to minimize human annotation efforts while maintaining model performance. In addition, we introduce a data generation technique based on Bayesian Program Learning (BPL) to compensate for the sparse labeled data. Finally, we present a Text Degradation Invariant Autoencoder (Text-DIAE), a self-supervised model that simultaneously addresses text recognition and document image enhancement, requiring substantially fewer data samples to converge, a critical efficiency factor for OCR and machine learning industry applications.

In summary, OCR has been a key component of computer vision (CV), driven by deep learning. The thesis concentrated on enhancing OCR through image improvement and data-efficient training in resource-constrained scenarios. Advancements in image enhancement, employing cGANs and vision transformers, enhanced OCR accuracy and efficiency in diverse sectors. Innovations like few-shot learning, data generation, and self-supervised learning addressed limited training data challenges, extending OCR to less common text images. These developments continue to boost productivity and accessibility across industries, aligning with the evolving digital landscape.

Imágenes de documentos manuscritos; Mejora; Reconocimiento; Alfabetos raros; Documentos cifrados; Documentos altamente degradados; Industria del reconocimiento óptico de caracteres (OCR); Avances técnicos; Aplicaciones tangibles; Degradación del papel; Sistemas OCR; Modelos de extremo a extremo; Mejora de imágenes de documentos (DIE); Técnicas de aprendizaje profundo; Redes generativas antagónicas (cGAN); Limpieza de documentos; Binarización; Desenfoque; Eliminación de marca de agua; Módulo de reconocimiento de texto; Mejora de la calidad de la imagen; Legibilidad; Arquitectura de codificador-decodificador; Transformadores de visión; Documentos impresos; Documentos manuscritos; Campos industriales; Reconocimiento de texto; Escenarios con recursos limitados; Escasez de datos etiquetados; Detección de objetos de pocos disparos (fOL); Estrategia de aprendizaje progresivo; Minimización de esfuerzos de anotación humana; Mantenimiento del rendimiento del modelo; Técnica de generación de datos; Aprendizaje de programa bayesiano (BPL); Compensación de datos etiquetados dispersos; Autocodificador invariante de degradación de texto (Text-DIAE); Modelo autosupervisado; Reconocimiento de texto simultáneo; Mejora simultánea de imágenes de documentos; Muestras de datos reducidas; Factor de eficiencia crítico; Aplicaciones industriales de OCR; Aplicaciones industriales de aprendizaje automático.