Skip to content

Frame2Text4LLM

Many videos carry their translation burned into the picture: a series in Mooré with French subtitles, a sermon in Dioula subtitled in English. Frame2Text4LLM reads those subtitles with OCR, finds when each one starts and ends, and cuts the audio underneath. What comes out is a set of pairs, the speech and the text that goes with it, ready for a speech recognition or speech translation dataset.

The pipeline

Install

pip install "frame2text4llm[easyocr]"

The OCR engines are extras, so that you install only the one you use:

extra engine needs
easyocr EasyOCR torch
paddle PaddleOCR 3.x paddlepaddle
vlm vision-language models (Qwen3-VL, Florence-2, InternVL2) torch, transformers
openai OpenAI vision models an API key
all all of the above

Mistral OCR needs only an API key. Audio cuts need ffmpeg on the path.

Where to go next

Frame2Text4LLM was written by Alban Nyantudre and Salif Sawadogo for BurkimbIA, which builds speech and language tools for the languages of Burkina Faso. MIT licence.