Improving Education for Visually Challenged Students through Image Captioning and Automated Reading

Abstract

The integration of advanced technologies into education has emerged as a vital solution to bridge the accessibility gap for visually challenged students. This paper introduces an innovative system that uses image captioning and automated reading to enhance learning experiences for such students. A combination of natural language processing (NLP) along with computer vision has been employed to transform the images into descriptive text, which is further converted to audio for auditory learning. The present work attempts to circumvent the limitations of current image captioning models. A major one include these models being biased on generic real-world images, instead of educational material. A custom dataset has been created with diagrams of various subjects from standard textbooks with an aim to improve classroom comprehension for students, thereby addressing educational content. Optical Character Recognition (OCR) has also been incorporated into the proposed system for reading text embedded in images, thereby providing a comprehensive tool for visually impaired students to access complex visual and textual information.