The article is devoted to the study of hybrid encryption methods to increase the reliability of information systems and their optimization. The basic concepts of hybrid encryption are defined, its principle of operation is considered, and the overall process is schematically presented. The views of Ukrainian scientists who investigated the issues of hybrid encryption are analyzed, namely: researcher O. Liashuk, who considers hybrid encryption as a multi-layered approach for data protection in unsecure networks; M. Kalushka, who analyzed and studied the principles of operation of symmetric, asymmetric, and hybrid algorithms; scientist P. M. Lisovskyi, who highlighted the advantages of using a multi-level data encryption model, increasing the efficiency of risk analysis, and ensuring compliance with regulatory requirements. The improvement indicators of the hybrid encryption model are characterized through: the application of multi-layered encryption ( MHED ); conducting cryptanalysis; the use of a modular architecture; implementing automated key management. A diagnostics of the resulting model was performed, and optimization options for the reliability of the information system through the integration of several levels of hybrid encryption were proposed. At the same time, the architecture contains an interconnected modular system, namely: a hybrid cryptographic core, a metaheuristic optimization submodule, a steganographic embedding submodule, and a key management submodule. A detailed study highlights the effectiveness of the resulting model, which utilizes hybrid encryption methods optimized by advanced metaheuristic techniques in order to achieve a higher degree of reliability in information systems. It is expected that further research and practical implementations based on this project will make a significant contribution to the development of secure, high-performance communication systems and serve as reference architectures for both academic research and industrial application.