Andrii Nikolaiev
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Andrii Nikolaiev has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Anatoliy V. Anisimov
Taras Shevchenko National University of Kyiv, Kyiv, Ukraine
2 shared publications - Simone Teufel
University of Cambridge
1 shared publication Yiannos Stathopoulos
University of Cambridge
1 shared publication
Latest Publications
- 30 May 2025
- Published by Springer Nature in Cybernetics and Systems Analysis
- Vol. 61 (3) , 354-363
Neural Network Methods for Selecting and Generating Synthetic Variations of Combinatorial Problems
- 30 May 2025
- Published by Springer Nature in Cybernetics and Systems Analysis
- Vol. 61 (3) , 354-363
Neural Network Methods for Selecting and Generating Synthetic Variations of Combinatorial Problems
- 1 January 2025
- Published by V.M. Glushkov Institute of Cybernetics in Kibernetyka ta Systemnyi Analiz
- p. 22-32
NEURAL NETWORK METHODS FOR SELECTING AND GENERATING SYNTHETIC VARIATIONS OF COMBINATORIAL PROBLEMS
This study explores neural network methods for selecting and generating synthetic variations of mathematical combinatorial problems using large language models. We filter 1000 problems from the NuminaMath-CoT dataset and introduce three new variations: fictional, adversarial, and contextual disguise. We propose the Variation Consistency Score to measure the accuracy in the solution generation rate of the variations and their correlation rate with the original problems. Experiments with GPT-4o-mini showed consistent performance in the generation process, demonstrating the highest score on the fictional variation with the greatest average statement length and the lowest score on the contextual disguise variation. The model demonstrates the ability to generate diverse variations while preserving their mathematical core. Keywords: Neural Network Methods; Natural Language Processing; Large Language Models; Synthetic Data Generation; Mathematical Problems.
- 1 January 2025
- Published by V.M. Glushkov Institute of Cybernetics in Kibernetyka ta Systemnyi Analiz
- p. 22-32
NEURAL NETWORK METHODS FOR SELECTING AND GENERATING SYNTHETIC VARIATIONS OF COMBINATORIAL PROBLEMS
This study explores neural network methods for selecting and generating synthetic variations of mathematical combinatorial problems using large language models. We filter 1000 problems from the NuminaMath-CoT dataset and introduce three new variations: fictional, adversarial, and contextual disguise. We propose the Variation Consistency Score to measure the accuracy in the solution generation rate of the variations and their correlation rate with the original problems. Experiments with GPT-4o-mini showed consistent performance in the generation process, demonstrating the highest score on the fictional variation with the greatest average statement length and the lowest score on the contextual disguise variation. The model demonstrates the ability to generate diverse variations while preserving their mathematical core. Keywords: Neural Network Methods; Natural Language Processing; Large Language Models; Synthetic Data Generation; Mathematical Problems.
- Open Access
Can Language Models Rival Mathematics Students? Evaluating Mathematical Reasoning through Textual Manipulation and Human Experiments
In this paper we look at the ability of recent large language models (LLMs) at solving mathematical problems in combinatorics. We compare models LLaMA-2, LLaMA-3.1, GPT-4, and Mixtral against each other and against human pupils and undergraduates with prior experience in mathematical olympiads. To facilitate these comparisons we introduce the Combi-Puzzles dataset, which contains 125 problem variants based on 25 combinatorial reasoning problems. Each problem is presented in one of five distinct forms, created by systematically manipulating the problem statements through adversarial additions, numeric parameter changes, and linguistic obfuscation. Our variations preserve the mathematical core and are designed to measure the generalisability of LLM problem-solving abilities, while also increasing confidence that problems are submitted to LLMs in forms that have not been seen as training instances. We found that a model based on GPT-4 outperformed all other models in producing correct responses, and performed significantly better in the mathematical variation of the problems than humans. We also found that modifications to problem statements significantly impact the LLM's performance, while human performance remains unaffected.
Open AccessCan Language Models Rival Mathematics Students? Evaluating Mathematical Reasoning through Textual Manipulation and Human Experiments
In this paper we look at the ability of recent large language models (LLMs) at solving mathematical problems in combinatorics. We compare models LLaMA-2, LLaMA-3.1, GPT-4, and Mixtral against each other and against human pupils and undergraduates with prior experience in mathematical olympiads. To facilitate these comparisons we introduce the Combi-Puzzles dataset, which contains 125 problem variants based on 25 combinatorial reasoning problems. Each problem is presented in one of five distinct forms, created by systematically manipulating the problem statements through adversarial additions, numeric parameter changes, and linguistic obfuscation. Our variations preserve the mathematical core and are designed to measure the generalisability of LLM problem-solving abilities, while also increasing confidence that problems are submitted to LLMs in forms that have not been seen as training instances. We found that a model based on GPT-4 outperformed all other models in producing correct responses, and performed significantly better in the mathematical variation of the problems than humans. We also found that modifications to problem statements significantly impact the LLM's performance, while human performance remains unaffected.