Dmitry Martysyuk

This paper presents a pipeline for structured information extraction and commodity classification from unstructured customs declaration texts using a sparse Mixture-of-Experts large language model with deterministic post-processing and rule-based validation, and integrates the extracted outputs into an interactive recommendation system that supports customs officers in classification and decision-making tasks. A Mixtral-8×7B decoder-only architecture is adapted to the customs domain using Low-Rank Adaptation, enabling parameter-efficient fine-tuning while processing long inputs of up to 32k tokens. On a corpus of 847 Russian-language customs product descriptions, the model produces syntactically valid JSON outputs in 98.1% of cases, with deterministic post-processing reducing the invalid rate to 1.06%. Fine-tuning with LoRA on the original dataset yields validation losses in the range 0.015–0.025, while training on an augmented dataset of approximately 106 records reduces validation loss to below 0.002, at the cost of increased training time. Inference benchmarks show that the sparse MoE model, activating 2 of 8 experts per token, achieves higher scores than dense baselines under comparable inference budgets, with gains of approximately 10–15 percentage points on reasoning-oriented tasks. The results also identify limitations related to semantic correctness, numeric and unit interpretation, routing stability, and uncertainty estimation, indicating directions for further development of language-model-based systems for regulatory and decision-support applications.

This paper presents a pipeline for analyzing Telegram posts and predicting audience reactions in the absence of comment data. Raw messages were processed using both NLP methods and GPT-based structured prompts to extract linguistic, stylistic, semantic, and emotional features. Comment-level responses were independently annotated using a dedicated prompt to generate thread-level labels capturing sentiment, toxicity, stance, humor, and conflict. These labels were then used as targets in a post-to-comment inference task formulated as a multi-task classification problem. XGBoost models achieved up to 0.85 accuracy for emotional attributes (cuteness, conflict) and 0.71 for toxic comment prediction, confirming that post content contains reliable signals of potential audience behavior. SHAP-based explainability showed that LLM-derived features are interpretable and behaviorally meaningful. Overall, the approach demonstrates that content dynamics in comment threads can be anticipated from post features alone, enabling proactive moderation, risk assessment, and socially aware AI support for online communication.

Mapping thematic structure in large scientific corpora enables the systematic analysis of research trends and conceptual organization. This work presents an unsupervised framework that leverages large language models (LLMs) as fixed semantic inference operators guided by structured soft prompts. The framework transforms raw abstracts into normalized semantic representations that reduce stylistic variability while retaining core conceptual content. These representations are embedded into a continuous vector space, where density-based clustering identifies latent research themes without predefining the number of topics. Cluster-level interpretation is performed using LLM-based semantic decoding to generate concise, human-readable descriptions of the discovered themes. Experiments on ICML and ACL 2025 abstracts demonstrate that the method produces coherent clusters reflecting problem formulations, methodological contributions, and empirical contexts. The findings indicate that prompt-driven semantic normalization combined with geometric analysis provides a scalable and model-agnostic approach for unsupervised thematic discovery across large scholarly corpora.