Noila Matyakubova

The scarcity of token-level verified bilingual word-alignment resources for the Uzbek language limits opportunities for evaluating alignment models and advancing cross-linguistic research. In this study, the UzAlign Gold Standard word-alignment corpus for the English–Uzbek language pair was created; it comprises 1471 parallel sentence pairs sourced from three different origins and is divided into TRAIN, DEV, and TEST sets. A five-stage annotation process was employed to verify token-level correspondences, involving the use of a bilingual dictionary, morphological analysis, identification of compound verbs, verification of low-confidence links, and the recovery of missing alignments. Model-proposed alignments were reviewed by two independent annotators, with disagreements resolved through adjudication. The corpus was enriched with lemmas, UPOS tags, Universal Dependencies morphological features, and morpheme segmentation. Compared with the final adjudicated Gold reference, the initial alignment proposals showed 98.38% Precision, 99.86% Recall, 99.12% F₁, and an AER of 0.88%. These values reflect the degree of agreement between the initial proposals and the human-verified Gold reference and are not interpreted as an independent evaluation of model generalization. Pair-level decision agreement was 97.96% (Cohen’s κ = 0.9579), while link-level agreement over the defined candidate-link universe was 99.88% (Dice = 0.9994). The majority of errors stemmed from the incorrect alignment of English copula units with Uzbek linking elements. UzAlign combines Uzbek–English token alignment with morphological annotation to provide a reusable reference resource for word alignment and other cross-linguistic NLP research.

In cross-lingual natural language processing (NLP) systems, expressing a meaning from one language through an appropriate lexical unit in another language is an important task. However, many semantically and culturally rich Uzbek concepts, including kinship terms, ceremonial vocabulary, forms of address, and expressions related to traditional medicine, do not have direct and concise equivalents in English. In this work, we present UzbekLexLacuna, an Uzbek–English dataset developed to represent such cases at the sense level. The dataset is designed for the systematic description of lexical, cultural, semantic, and pragmatic lacunae. The cleaned release candidate contains 760 TRUE_LACUNA entries and 100 NOT_LACUNA polysemy-control entries, yielding a total of 860 curated sense-level records with predefined benchmark splits. All entries were constructed through a systematic analysis of Uzbek–English equivalence based on the five-volume Explanatory Dictionary of the Uzbek Language. The annotation scheme distinguishes the scope of a lacuna (FULL or PARTIAL) from its source (CULTURAL, SEMANTIC, or PRAGMATIC). This structure supports sense-level lacuna detection and classification, the development of translation strategies, and the creation and evaluation of multi-task NLP models. The dataset is released under the CC BY 4.0 license together with controlled vocabularies, provenance metadata, and benchmark splits.

Automatic extraction of genealogical information from historical archival-genealogical documents in Uzbek is an understudied problem for low-resource languages. Multi-layer NLP benchmarks are not sufficient to automatically identify individuals, family relationships, dates, place names, and archival identifiers in such texts. Also, the same people are mentioned in various forms: full name, pronoun (18.8%), initial, surname-name order, indirect expression (9.4%), and title. Existing NER and relation extraction corpora are mainly focused on high-resource languages or general domain texts and do not sufficiently cover the FAMILY_ROLE signals, historical spelling variants, and fond–opis–delos identifiers specific to Uzbek archival-genealogical texts. Proposed resource: We present the ArchiveGene Corpus, a controlled, fully synthetic, and reproducible five-layer resource consisting of 1000 Uzbek archival-genealogical-style documents, divided into 700 training, 150 validation, and 150 test documents. The corpus contains 8366 named entities, 10,625 person mentions, 2000 coreference chains, and 1000 genealogical relation triples. The dataset was generated using a deterministic template-based pipeline and a lexicon of Uzbek names, and is fully reproducible. Inter-annotator agreement values were 0.847 for NER, 0.793 for coreference, and 0.821 for RE, according to Cohen’s κ. Comparative results are presented with four baseline models (rule-based, BiLSTM-CRF, mBERT, and XLM-RoBERTa). The dataset is openly hosted on the Zenodo platform under the CC BY 4.0 license; concept DOI: 10.5281/zenodo.20670360, v1.1.1 version DOI: 10.5281/zenodo. 21429998. Scientific significance: To the best of our knowledge, ArchiveGene is among the first openly released, controlled synthetic resources for Uzbek that integrates named-entity recognition, person-mention detection, coreference resolution, genealogical relation extraction, and final tuple generation within a single annotation framework. The baseline analysis provides three main conclusions: (1) on the clean synthetic test set, the transformer models already reach 100.00 Micro-F1 for NER and 100.00 Macro-F1 for coreference-aware relation extraction, so coreference aggregation adds little on synthetic data (+2.25 for mBERT and +0.04 for XLM-RoBERTa) but its contribution is expected to grow on real archival text; (2) the rule-based and heuristic baselines lag far behind (Macro-F1 50.28 and 70.73) and fail entirely on spouse_of, showing the limits of lexical rules; and (3) a zero-shot evaluation on a real-document pilot reduces NER Micro-F1 from 100.00 to 22.17, indicating that the synthetic corpus is trivially learnable and that real-archival validation is essential.

The increasing flow of documents in digital libraries, archives and electronic document management systems makes the standardization, adaptation and automation of the process of creating metadata an urgent scientific problem. Metadata directly affects the efficiency of document search, identification, semantic interpretation, long-term storage and intersystem exchange. However, while standardized description based on MARC21, a flexible approach to creating a dynamic field, and intelligent methods based on deep learning, cover these requirements separately, the issue of their full integration into a single methodological system has not been sufficiently resolved. In this study, an integrated hybrid model for describing electronic documents based on standardized, flexible, and intelligent metadata was proposed. A mixed electronic document corpus of 1500 documents was formed for evaluation. The corpus consisted of books, dissertations, scientific articles, archival documents, and heterogeneous electronic documents, with 300 samples selected from each group. Key metadata elements for each document were manually identified and used as ground truth. According to experimental results, the MARC21-based constructor achieved 96.8% structural compatibility and 95.6% metadata completeness, but the average description time was 6.8 min. The dynamic field approach achieved 93.4% structural compatibility and 94.1% metadata completeness, and reduced the description time to 4.1 min. The deep learning-based intelligent module achieved a structural matching score of 91.7%, a metadata extraction score of 93.8% F1, and reduced the processing time to 1.9 min. The proposed hybrid model achieved a structural matching score of 95.9%, a metadata F1 score of 95.1%, and an average description time of 2.3 min. The results showed that the hybrid model is a balanced solution between metadata quality, flexibility, and automation.

This study proposes an integrated software model for automatic document classification and metadata generation based on the Dublin Core standard to address the issue of rapid and consistent management of archival documents in a digital environment. This approach combines the stages of receiving incoming documents, converting them to text using optical character recognition (OCR), image preprocessing (binarization, deskew, noise reduction), and text cleaning and vectorization (TF–IDF) into a single pipeline. In the document classification stage, the Bidirectional Encoder Representations from Transformers (BERT) model with a context-sensitive transformer architecture is used, along with classical machine learning models (Logistic Regression, Naive Bayes, Support Vector Machine) and an ensemble approach (LightGBM), to increase the accuracy by modeling the document content at a deep semantic level. Experiments were conducted on the RVL-CDIP dataset, and the OCR efficiency was evaluated using the Character Error Rate (CER) indicator, and the classification results were evaluated using the accuracy, precision, recall and F1-score metrics. The results confirmed the high stability and generalization ability of the BERT (accuracy, 95.1%; F1, 95.0%) and LightGBM (accuracy, 93.2%; F1, 93.2%) models. In the final stage, OCR, NER, and classification outputs are automatically organized into Dublin Core metadata elements (Title, Creator, Date, Description, Subject, Type, Format, Language) and exported in JSON/XML formats. This automation significantly reduces manual cataloging effort and improves indexing and retrieval efficiency in digital archival systems.