Skip to main content

A hybrid transformer-BiLSTM model for punctuation restoration in low-resource languages: Uzbek as a case study

2026, vol.18 , no.4, pp. 79-90

Article [2026-04-08]

Authors
Maksud Sharipov
Hushnudbek Adinaev
Abstract

Automatic punctuation restoration is an important natural language processing task that improves the readability of unpunctuated text and supports automatic speech recognition, machine translation and text editing systems. This study proposes a language-adaptable hybrid Transformer+BiLSTM framework for token-level punctuation restoration in low-resource languages and evaluates it using Uzbek as a case study. The task was formulated as a sequence-labeling problem with eleven classes. RoBERTa, XLM-RoBERTa, BERTbek and TahrirchiBERT were fine-tuned also BiLSTM layers were added to selected transformer encoders. The models were evaluated using accuracy, precision, recall and F1-score. XLM-RoBERTa+BiLSTM achieved the best overall results, with 0.8588 accuracy, 0.8493 precision, 0.8588 recall and 0.8393 F1-score. The findings indicate that bidirectional sequential modelling can complement transformer representations in low-resource punctuation restoration.

Keywords

NLP, transformer models, Punctuation restoration, BiLSTM, low-resource languages

DOI

https://doi.org/10.59035/LJKP6387

Download full article

Citation of this article:

Maksud Sharipov, Hushnudbek Adinaev. A hybrid transformer-BiLSTM model for punctuation restoration in low-resource languages: Uzbek as a case study. International Journal on Information Technologies and Security, vol.18 , no.4, 2026, pp. 79-90. https://doi.org/10.59035/LJKP6387

↑