MGMaria Grigorieva

Maria Grigorieva

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Maria Grigorieva has a total of 71 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Decoding dataset access patterns in high-energy physics through embedding-driven density clustering

    by Maria Grigorieva, Nina Popova, Mikhail Shubin

    High-energy physics (HEP) research relies on the analysis of large datasets representing various physical processes. In most cases, datasets are accessed in groups rather than as individual files, as they are often interrelated and collectively contribute to data production and analysis. However, the relevance of these dataset groups changes over time and requires continuous assessment to optimize storage and computational resource utilization. Modeling dataset access patterns as time series enables the application of clustering algorithms for the detection of dataset groups being accessed together. In this study, we explore the application of embedding-driven density-based clustering techniques for analyzing HEP dataset access patterns collected over multiple years. Our results demonstrate the advantages of embedding-driven density clustering for identifying meaningful dataset categories, providing actionable insights for intelligent data management in large-scale scientific computing environments.

  • NanoporeInspect: An Interactive Tool for Evaluating Nanopore Sequencing Quality and Ligation Efficiency

    by Maria A. Grigoryeva, Maria G. Khrenova, Maria I. Zvereva

    In nanopore sequencing, especially in SELEX-based aptamer discovery, the correct ligation of artificial sequences (primers, adapters, barcodes) is crucial for library quality. Errors at this stage can lead to misidentification of sequences and loss of valuable information. Existing quality control tools lack focused capabilities to assess the positioning and prevalence of these artificial sequences. NanoporeInspect is a web-based tool designed to fill this gap by providing targeted insights into ligation efficacy and systematic biases within sequencing data. NanoporeInspect operates as a user-friendly, web-based platform that leverages a modern software stack with Flask, Celery and Redis to handle scalable and asynchronous task processing, and Plotly to deliver interactive visualizations. Evaluation of NanoporeInspect on various nanopore datasets demonstrated its effectiveness in discerning differences in ligation quality. Libraries with inefficient ligation showed irregular adapter and barcode distributions, indicating preparation issues, while high-quality libraries displayed uniform patterns, reflecting effective ligation.

  • Open Access

    Operational Analytics Studies for ATLAS Distributed Computing: Data Popularity Forecast and Utilization of the WLCG Centers

    by Maria Grigoryeva, Alexei Klimentov, Nina Popova, Mikhail Shubin, Markus Schulz

    Operational analytics is the direction of research related to the analysis of the current state of computing processes and the prediction of future states in order to anticipate imbalances and take timely measures to stabilize a complex system. There are two relevant areas in ATLAS Distributed Computing that are currently the focus of studies: user physics analysis including the forecast of popularity of data samples among users, and evaluating WLCG centers for their readiness to process user analysis payloads. Studying these areas is challenging due to the complexity involved, as it requires a comprehensive understanding of numerous boundary conditions typically found in large-scale distributed computing infrastructures. Forecasts of data popularity are problematic without the categorization of user tasks by their types (data transformation or physics analysis), which do not always appear on the surface but may induce noise, which introduces significant distortions for predictive analysis. Evaluating the WLCG resources by their analysis workloads is also a challenging task as it is necessary to find a balance between the workload of the resource, its performance, the waiting time for jobs on it, as well as the volume of jobs that it processes. This is especially difficult in a heterogeneous computing environment, where legacy resources are used along with modern high-performance machines. We will look at these areas of research in detail and discuss what tools and methods are used in our work, demonstrating results already obtained.


  • Methodology of evaluating ATLAS computing resources in handling user analysis workflows

    by Maria Grigoryeva

    The distributed computing infrastructure of the ATLAS Experiment includes over 170 sites and executes up to 3 million computing jobs daily. PanDA (Production and Distributed Analysis) is the Workload Management System responsible for task and job execution; its key components are the broker and job scheduler that define the mapping of computing jobs to the resources. The optimization of this mapping is crucial for handling the expected computational payloads during the HL-LHC era. Considering the heterogeneity and the distributed structure of the Worldwide LHC Computing Grid (WLCG) infrastructure that provides computing resources for analyzing the data, there is a need for specific approaches for evaluating computing resources according to their ability to process different types of workflows. This evaluation can potentially enhance the efficiency of the Grid by optimally distributing different types of payloads in heterogeneous computing environments. To tackle this challenge, this research proposes a method for evaluating WLCG resources regarding their ability to process user analysis payloads. This evaluation is based on leveraging available information about job execution on PanDA queues within the ATLAS computing environment.


  • Open Access

    TRANSFORMER-BASED MODEL FOR THE SEMANTIC PARSING OF ERROR MESSAGES IN DISTRIBUTED COMPUTING SYSTEMS IN HIGH ENERGY PHYSICS

    by D. Grin, M. Grigorieva

    Large-scale computing centers supporting modern scientific experiments store and analyze vast amounts of data. A noticeable number of computing jobs executed within the complex distributed computing environments ends with errors of some kind, and the amount of error log data generated every day complicates manual analysis by human experts. Moreover, traditional methods such as specifying regular expression patterns to automatically group error messages become impractical in a heterogeneous computing environment without a well-defined structure of error messages. ClusterLogs framework for error message clustering was developed to address this challenge. Theframework can discover common patterns in error messages from various sources and group them together. One of the essential results of this process is the clear automated description of the resulting clusters, which will be used for the analysis. In this research, we propose that interpreting error messages as a natural language allows us to use transformer-based deep learning models such as BERT for this task. A model for extracting the relevant part of messages was trained and integrated into ClusterLogs to represent each cluster as a few actionable items, ensuring better interpretation and validation of the results of clustering.

  • Open Access

    Methods of Data Popularity Evaluation in the ATLAS Experiment at the LHC

    by Thomas Beermann, Olga Chuchuk, Alessandro Di Girolamo, Maria Grigorieva, Alexei Klimentov, Mario Lassnig, Markus Schulz, Andrea Sciaba, Eugeny Tretyakov

    The ATLAS Experiment at the LHC generates petabytes of data that is distributed among 160 computing sites all over the world and is processed continuously by various central production and user analysis tasks. The popularity of data is typically measured as the number of accesses and plays an important role in resolving data management issues: deleting, replicating, moving between tapes, disks and caches. These data management procedures were still carried out in a semi-manual mode and now we have focused our efforts on automating it, making use of the historical knowledge about existing data management strategies. In this study we describe sources of information about data popularity and demonstrate their consistency. Based on the calculated popularity measurements, various distributions were obtained. Auxiliary information about replication and task processing allowed us to evaluate the correspondence between the number of tasks with popular data executed per site and the number of replicas per site. We also examine the popularity of user analysis data that is much less predictable than in the central production and requires more indicators than just the number of accesses.


  • Parallel Version of the Framework for Clustering Error Messages

    by M. Vorobyov, K. Zhukov, M. Grigorieva, S. Korobkov

    Distributed computing environments execute great amount of various computing jobs that can fail or break for some reason. The analysis of the error messages describing the reasons of failures has become one of the most crucial parts of the existing monitoring systems. This analysis is complicated by the presence of a large number of messages, especially in the case of the retrospective analysis. ClusterLogs framework was developed as a modular and flexible tool for the clustering of error messages of computing jobs in distributed computing infrastructures. The general purpose of this tool is to simplify the error messages analysis by grouping together messages that share similar failure reasons and textual patterns. Proposed clustering method includes a set of sequential data processing stages and provides various clustering options: deterministic similarity-based clustering and unsupervised multiple machine learning methods with preliminary vectorization of error messages using the word embedding technique. The performance tests had revealed the most time consuming stages. In this paper we describe the parallelilzing method for these stages and demonstrate how it has allowed the increased performance of the whole clustering pipeline. The performance tests were executed on the HPC system Polus.


  • Open Access

    TRACER (TRACe route ExploRer): A tool to explore OSG/WLCG network route topologies

    by Evgeniy Tretyakov, Alexey Artamonov, Maria Grigorieva, Alexei Klimentov, Shawn McKee, Ilija Vukotic

    The experiments at the Large Hadron Collider (LHC) rely upon a complex distributed computing infrastructure (WLCG) consisting of hundreds of individual sites worldwide at universities and national laboratories, providing about half a billion computing job slots and an exabyte of storage interconnected through high speed networks. Wide Area Networking (WAN) is one of the three pillars (together with computational resources and storage) of LHC computing. More than 5 PB/day are transferred between WLCG sites. Monitoring is one of the crucial components of WAN and experiments operations. In the past years all experiments have invested significant effort to improve monitoring and integrate networking information with data management and workload management systems. All WLCG sites are equipped with perfSONAR servers to collect a wide range of network metrics. We will present the latest development to provide the 3D force directed graph visualization for data collected by perfSONAR. The visualization package allows site admins, network engineers, scientists and network researchers to better understand the topology of our Research and Education networks and it provides the ability to identify nonreliable or/and nonoptimal network paths, such as those with routing loops or rapidly changing routes.


  • Open Access

    Parallel Coordinates Visualization in the ELK Stack

    by Timofei Galkin, Maria Grigorieva

    Modern large-scale distributed computing systems, processing large volumes of data, require mature monitoring systems able to control and track in re-sources, networks, computing tasks, queues and other components. In recent years, the ELK stack has become very popular for the monitoring of computing environment, largely due to the efficiency and flexibility of the Elastic Search storage and wide variety of Kibana visualization tools. The analysis of computing infrastructure metadata often requires the visual exploration of multiple parameters simultaneously on one graphical image. Stacked bar charts, heat maps, radar charts are widely used for the multivariate visual data analysis, but these methods have limitations on the number of parameters. In this research the authors propose to enhance the capacity of Kibana, adding Parallel Coordinates diagram - one of the most powerful method for visual interactive analysis of high-dimensional data. It allows to compare many variables together and observe correlations between them. This work describes the development process of Parallel Coordinates as a Kibana plugin, and demonstrates an example of visual data analysis based on the Nginx logs metadata.