Ivan Reinaldo Meneghini

Prompt engineering is crucial for unlocking the potential of Large Language Models (LLMs). Still, since manual prompt design is often complex, non-intuitive, and time-consuming, automatic prompt optimization has emerged as a research area. However, a significant challenge in prompt optimization is managing the inherent trade-off between task performance, such as accuracy, and context size. Most existing automated methods focus on a single objective, typically performance, thereby failing to explore the critical spectrum of efficiency and effectiveness. This paper introduces the MOPrompt, a novel Multi-objective Evolutionary Optimization (EMO) framework designed to optimize prompts for both accuracy and context size (measured in tokens) simultaneously. Our framework maps the Pareto front of prompt solutions, presenting practitioners with a set of trade-offs between context size and performance – a crucial tool for deploying Large Language Models (LLMs) in real-world applications. We evaluate MOPrompt on a sentiment analysis task in Portuguese, using Gemma-2B and Sabiazinho-3 as evaluation models. Our findings show that MOPrompt substantially outperforms the baseline framework. For the Sabiazinho model, MOPrompt identifies a prompt that achieves the same peak accuracy (0.97) as the best baseline solution, but with a 31% reduction in token length.

The Analytic Hierarchy Process (AHP) multicriteria method can be cognitively demanding for large-scale decision problems due to the requirement for the decision maker to make pairwise evaluations of all alternatives. To address this issue, this paper presents an interactive method that uses online learning to provide scalability for AHP. The proposed method involves a machine learning algorithm that learns the decision maker’s preferences through evaluations of small subsets of solutions, and guides the search for the optimal solution. The methodology was tested on four optimization problems with different surfaces to validate the results. We conducted a one factor at a time experimentation of each hyperparameter implemented, such as the number of alternatives to query the decision maker, the learner method, and the strategies for solution selection and recommendation. The results demonstrate that the model is able to learn the utility function that characterizes the decision maker in approximately 15 iterations with only a few comparisons, resulting in significant time and cognitive effort savings. The initial subset of solutions can be chosen randomly or from a cluster. The subsequent ones are recommended during the iterative process, with the best selection strategy depending on the problem type. Recommendation based solely on the smallest Euclidean or Cosine distances reveals better results on linear problems. The proposed methodology can also easily incorporate new parameters and multicriteria methods based on pairwise comparisons.

Solving many-objective problems (MaOPs) is still a significant challenge in the multi-objective optimization (MOO) field. One way to measure algorithm performance is through the use of benchmark functions (also called test functions or test suites), which are artificial problems with a well-defined mathematical formulation, known solutions and a variety of features and difficulties. In this paper we propose a parameterized generator of scalable and customizable benchmark problems for MaOPs. It is able to generate problems that reproduce features present in other benchmarks and also problems with some new features. We propose here the concept of generative benchmarking, in which one can generate an infinite number of MOO problems, by varying parameters that control specific features that the problem should have: scalability in the number of variables and objectives, bias, deceptiveness, multimodality, robust and non-robust solutions, shape of the Pareto front, and constraints. The proposed Generalized Position-Distance (GPD) tunable benchmark generator uses the position-distance paradigm, a basic approach to building test functions, used in other benchmarks such as Deb, Thiele, Laumanns and Zitzler (DTLZ), Walking Fish Group (WFG) and others. It includes scalable problems in any number of variables and objectives and it presents Pareto fronts with different characteristics. The resulting functions are easy to understand and visualize, easy to implement, fast to compute and their Pareto optimal solutions are known.