Text Extraction and Translation Through Lip Reading using Deep Learning

Abstract: Deep learning has revolutionized industries such as natural language processing and computer vision. This study explores the fusion of these domains by proposing a novel approach for text extraction and translation using lip reading and deep learning. Lip reading, the process of interpreting spoken language by analyzing lip movements, has garnered interest due to its potential applications in noisy environments, silent communication, and accessibility enhancements. This study employs the power of deep learning architectures such as CNNs and RNNs to accurately extract text content from lip movements captured in video sequences. The proposed model consists of multiple stages: lip region detection, feature extraction, text recognition, and translation. Initially, the model identifies and isolates the lip region within video frames using a CNN-based object detection approach. Subsequently, relevant features are extracted from the lip region using CNNs to capture intricate motion patterns and convert these visual features into textual in-formation. The extracted text is further processed and translated into the desired language using machine translation techniques to enable translation.

Sai Teja Krithik Putcha, Yelagandula Sai Venkata Rajam, K. Sugamya and Sushank Gopala. “Text Extraction and Translation Through Lip Reading using Deep Learning”. International Journal of Advanced Computer Science and Applications (IJACSA) 15.6 (2024). http://dx.doi.org/10.14569/IJACSA.2024.01506156

@article{Putcha2024,
title = {Text Extraction and Translation Through Lip Reading using Deep Learning},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2024.01506156},
url = {http://dx.doi.org/10.14569/IJACSA.2024.01506156},
year = {2024},
publisher = {The Science and Information Organization},
volume = {15},
number = {6},
author = {Sai Teja Krithik Putcha and Yelagandula Sai Venkata Rajam and K. Sugamya and Sushank Gopala}
}