Yagya Raj Pandeya
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Yagya Raj Pandeya has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Joonwhoan Lee
Division of Computer Science and Engineering, Jeonbuk National University, Jeonju 54896, Republic of Korea
7 shared publications - Bhuwan Bhattarai
Center for Advanced Image and Information Technology, Jeonbuk National University, Jeonju 54896, Republic of Korea
6 shared publications Joonwhoan Lee
Division of Computer Science and Engineering, Jeonbuk National University, Jeonju, South Korea
2 shared publications- Arjun Kumar Lamichhane
Central Department of Computer Science and Information Technology, Tribhuvan University, Kirtipur, Nepal
1 shared publication Ashim Gelal
1 shared publication- Bhaskar Bhatt
AI Research Unit DigNep Group Pvt. Ltd Kathmandu Nepal
1 shared publication
Latest Publications
- Open Access
- 31 December 2025
- Published by Nepal Journals Online (JOL) in Api Journal of Science
- Vol. 2, 48-53
Translating Nepal’s National AI Policy into Action: Strategic Roadmap for Education, Industry, Infrastructure, and Digital Diplomacy
Artificial intelligence is fundamentally transforming Nepal's socio-economic landscape, with far- reaching implications across diverse sectors. Recognizing this transformative potential, the Government of Nepal published an AI Concept Paper in2024, followed by the National AI Policy in 2025. However, translating policy frameworks into tangible outcomes requires developing comprehensive short-term and long-term AI strategies that deliver public value. Education, digital infrastructure, industry, and digital diplomacy emerge as pivotal sectors where AI can advance national interests, accelerate sectoral development, and address critical cybersecurity challenges. This study examines AI-driven opportunities and risks across four key domains: industrial value chains, education, digital infrastructure, and digital diplomacy. Each sectoral analysis explores current conditions, potential use cases, associated risks, and emerging opportunities for AI integration. This study employed a multi-method approach combining extensive desk reviews, iterative report evaluations, focused group discussions, and stakeholder consultations. This methodology incorporates diverse perspectives and draws upon global best practices in AI adoption to ensure comprehensive and contextually relevant findings. The study reveals that each sector requires a holistic and adaptive framework that integrates customized AI implementation strategies, proactive risk management strategies, sustainable funding mechanisms, and robust evaluation systems. Critical priorities identified across sectors include enacting data privacy legislation, investing in workforce upskilling programs, developing essential digital infrastructure, implementing AI risk mitigation protocols, strengthening digital diplomacy capabilities, and enhancing cybersecurity measures. These interconnected priorities collectively shape Nepal's evolving position within the global AI ecosystem and determine its capacity to leverage AI for inclusive development. This study provides policymakers with strategic insights, evidence-based analysis, and practical implementation tools to formulate effective AI policies, address critical infrastructure and governance gaps, and foster inclusive capacity building across sectors.
- 31 December 2025
- Published by Nepal Journals Online (JOL) in Api Journal of Science
- Vol. 2, 48-53
Open AccessTranslating Nepal’s National AI Policy into Action: Strategic Roadmap for Education, Industry, Infrastructure, and Digital Diplomacy
Artificial intelligence is fundamentally transforming Nepal's socio-economic landscape, with far- reaching implications across diverse sectors. Recognizing this transformative potential, the Government of Nepal published an AI Concept Paper in2024, followed by the National AI Policy in 2025. However, translating policy frameworks into tangible outcomes requires developing comprehensive short-term and long-term AI strategies that deliver public value. Education, digital infrastructure, industry, and digital diplomacy emerge as pivotal sectors where AI can advance national interests, accelerate sectoral development, and address critical cybersecurity challenges. This study examines AI-driven opportunities and risks across four key domains: industrial value chains, education, digital infrastructure, and digital diplomacy. Each sectoral analysis explores current conditions, potential use cases, associated risks, and emerging opportunities for AI integration. This study employed a multi-method approach combining extensive desk reviews, iterative report evaluations, focused group discussions, and stakeholder consultations. This methodology incorporates diverse perspectives and draws upon global best practices in AI adoption to ensure comprehensive and contextually relevant findings. The study reveals that each sector requires a holistic and adaptive framework that integrates customized AI implementation strategies, proactive risk management strategies, sustainable funding mechanisms, and robust evaluation systems. Critical priorities identified across sectors include enacting data privacy legislation, investing in workforce upskilling programs, developing essential digital infrastructure, implementing AI risk mitigation protocols, strengthening digital diplomacy capabilities, and enhancing cybersecurity measures. These interconnected priorities collectively shape Nepal's evolving position within the global AI ecosystem and determine its capacity to leverage AI for inclusive development. This study provides policymakers with strategic insights, evidence-based analysis, and practical implementation tools to formulate effective AI policies, address critical infrastructure and governance gaps, and foster inclusive capacity building across sectors.
- Open Access
- 1 January 2025
- Published by Wiley in International Journal of Intelligent Systems
- Vol. 2025 (1)
Deep Dive Into Music Videos: Hierarchical Emotion Recognition With Rich Audio and Visual Features
This study aimed to address the challenges of cultural diversity and limited labeled data for music emotion classification. We introduced a benchmark dataset for music videos, featuring hierarchical emotion labels ranging from coarse to fine levels. We considered six established audio and video features, including geometric, spectral, harmonic, temporal, spatiotemporal, and visual attributes, for music emotion classification. We proposed hierarchical music video emotion classification networks and established baseline results using our dataset. Additionally, we presented a pipeline for audio processing using graph neural networks with reduced edge connections. Our convolutional neural network models for 1D, 2D, and 3D audio and video processing outperformed existing methods in various scenarios while requiring minimal training parameters. The study utilizes both quantitative measures and visual analysis to evaluate the results.
- 1 January 2025
- Published by Wiley in International Journal of Intelligent Systems
- Vol. 2025 (1)
Open AccessDeep Dive Into Music Videos: Hierarchical Emotion Recognition With Rich Audio and Visual Features
This study aimed to address the challenges of cultural diversity and limited labeled data for music emotion classification. We introduced a benchmark dataset for music videos, featuring hierarchical emotion labels ranging from coarse to fine levels. We considered six established audio and video features, including geometric, spectral, harmonic, temporal, spatiotemporal, and visual attributes, for music emotion classification. We proposed hierarchical music video emotion classification networks and established baseline results using our dataset. Additionally, we presented a pipeline for audio processing using graph neural networks with reduced edge connections. Our convolutional neural network models for 1D, 2D, and 3D audio and video processing outperformed existing methods in various scenarios while requiring minimal training parameters. The study utilizes both quantitative measures and visual analysis to evaluate the results.
- 15 February 2024
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (30) , 74141-74158
GlocalEmoNet: An optimized neural network for music emotion classification and segmentation using timbre and chroma features
- 15 February 2024
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (30) , 74141-74158
GlocalEmoNet: An optimized neural network for music emotion classification and segmentation using timbre and chroma features
- Open Access
Deep Learning based Tomato Disease Detection and Remedy Suggestions using Mobile Application
We have developed a comprehensive computer system to assist farmers who practice traditional farming methods and have limited access to agricultural experts for addressing crop diseases. Our system utilizes artificial intelligence (AI) to identify and provide remedies for vegetable diseases. To ensure ease of use, we have created a mobile application that offers a user-friendly interface, allowing farmers to inquire about vegetable diseases and receive suitable solutions in their local language. The developed system can be utilized by any farmer with a basic understanding of a smartphone. Specifically, we have designed an AI-enabled mobile application for identifying and suggesting remedies for vegetable diseases, focusing on tomato diseases to benefit the local farming community in Nepal. Our system employs state-of-the-art object detection methodology, namely You Only Look Once (YOLO), to detect tomato diseases. The detected information is then relayed to the mobile application, which provides remedy suggestions guided by domain experts. In order to train our system effectively, we curated a dataset consisting of ten classes of tomato diseases. We utilized various data augmentation methods to address overfitting and trained a YOLOv5 object detector. The proposed method achieved a mean average precision of 0.76 and offers an efficient mobile interface for interacting with the AI system. While our system is currently in the development phase, we are actively working towards enhancing its robustness and real-time usability by accumulating more training samples.
Open AccessDeep Learning based Tomato Disease Detection and Remedy Suggestions using Mobile Application
We have developed a comprehensive computer system to assist farmers who practice traditional farming methods and have limited access to agricultural experts for addressing crop diseases. Our system utilizes artificial intelligence (AI) to identify and provide remedies for vegetable diseases. To ensure ease of use, we have created a mobile application that offers a user-friendly interface, allowing farmers to inquire about vegetable diseases and receive suitable solutions in their local language. The developed system can be utilized by any farmer with a basic understanding of a smartphone. Specifically, we have designed an AI-enabled mobile application for identifying and suggesting remedies for vegetable diseases, focusing on tomato diseases to benefit the local farming community in Nepal. Our system employs state-of-the-art object detection methodology, namely You Only Look Once (YOLO), to detect tomato diseases. The detected information is then relayed to the mobile application, which provides remedy suggestions guided by domain experts. In order to train our system effectively, we curated a dataset consisting of ten classes of tomato diseases. We utilized various data augmentation methods to address overfitting and trained a YOLOv5 object detector. The proposed method achieved a mean average precision of 0.76 and offers an efficient mobile interface for interacting with the AI system. While our system is currently in the development phase, we are actively working towards enhancing its robustness and real-time usability by accumulating more training samples.
- Open Access
- 23 September 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (19) , 9571
Tracking the Rhythm: Pansori Rhythm Segmentation and Classification Methods and Datasets
This paper presents two methods to understand the rhythmic patterns of the voice in Korean traditional music called Pansori. We used semantic segmentation and classification-based structural analysis methods to segment the seven rhythmic categories of Pansori. We propose two datasets; one is for rhythm classification and one is for segmentation. Two classification and two segmentation neural networks are trained and tested in an end-to-end manner. The standard HR network and DeepLabV3+ network are used for rhythm segmentation. A modified HR network and a novel GlocalMuseNet are used for the classification of music rhythm. The GlocalMuseNet outperforms the HR network for Pansori rhythm classification. A novel segmentation model (a modified HR network) is proposed for Pansori rhythm segmentation. The results show that the DeepLabV3+ network is superior to the HR network. The classifier networks are used for time-varying rhythm classification that behaves as the segmentation using overlapping window frames in a spectral representation of audio. Semantic segmentation using the DeepLabV3+ and the HR network shows better results than the classification-based structural analysis methods used in this work; however, the annotation process is relatively time-consuming and costly.
- 23 September 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (19) , 9571
Open AccessTracking the Rhythm: Pansori Rhythm Segmentation and Classification Methods and Datasets
This paper presents two methods to understand the rhythmic patterns of the voice in Korean traditional music called Pansori. We used semantic segmentation and classification-based structural analysis methods to segment the seven rhythmic categories of Pansori. We propose two datasets; one is for rhythm classification and one is for segmentation. Two classification and two segmentation neural networks are trained and tested in an end-to-end manner. The standard HR network and DeepLabV3+ network are used for rhythm segmentation. A modified HR network and a novel GlocalMuseNet are used for the classification of music rhythm. The GlocalMuseNet outperforms the HR network for Pansori rhythm classification. A novel segmentation model (a modified HR network) is proposed for Pansori rhythm segmentation. The results show that the DeepLabV3+ network is superior to the HR network. The classifier networks are used for time-varying rhythm classification that behaves as the segmentation using overlapping window frames in a spectral representation of audio. Semantic segmentation using the DeepLabV3+ and the HR network shows better results than the classification-based structural analysis methods used in this work; however, the annotation process is relatively time-consuming and costly.
- Open Access
- 14 September 2022
- Published by Springer Nature in Circuits, Systems, and Signal Processing
- Vol. 42 (2) , 1083-1104
High-Resolution Representation Learning and Recurrent Neural Network for Singing Voice Separation
Music source separation has traditionally followed the encoder-decoder paradigm (e.g., hourglass, U-Net, DeconvNet, SegNet) to isolate individual music components from mixtures. Such networks, however, result in a loss of location-sensitivity, as low-resolution representation drops the useful harmonic patterns over the temporal dimension. We overcame this problem by performing singing voice separation using a high-resolution representation learning (HRNet) system coupled with a long short-term memory (LSTM) module to retain high-resolution feature map and capture the temporal behavior of the acoustic signal. We called this joint combination of HRNet and LSTM as HR-LSTM. The predicted spectrograms produced by this system are close to ground truth and successfully separate music sources, achieving results superior to those realized by past methods. The proposed network was tested using four datasets (DSD100, MIR-1K, Korean Pansori, and Nepal Idol singing voice). Our experiments confirmed that the proposed HR-LSTM outperforms state-of-the-art networks at singing voice separation when the DSD100 dataset is used, performs comparably to alternative methods when the MIR-1K dataset is used, and separates the voice and accompaniment components well when the Pansori and NISVS datasets are used. In addition to proposing and validating our network, we also developed and shared our Nepal Idol dataset.
- 14 September 2022
- Published by Springer Nature in Circuits, Systems, and Signal Processing
- Vol. 42 (2) , 1083-1104
Open AccessHigh-Resolution Representation Learning and Recurrent Neural Network for Singing Voice Separation
Music source separation has traditionally followed the encoder-decoder paradigm (e.g., hourglass, U-Net, DeconvNet, SegNet) to isolate individual music components from mixtures. Such networks, however, result in a loss of location-sensitivity, as low-resolution representation drops the useful harmonic patterns over the temporal dimension. We overcame this problem by performing singing voice separation using a high-resolution representation learning (HRNet) system coupled with a long short-term memory (LSTM) module to retain high-resolution feature map and capture the temporal behavior of the acoustic signal. We called this joint combination of HRNet and LSTM as HR-LSTM. The predicted spectrograms produced by this system are close to ground truth and successfully separate music sources, achieving results superior to those realized by past methods. The proposed network was tested using four datasets (DSD100, MIR-1K, Korean Pansori, and Nepal Idol singing voice). Our experiments confirmed that the proposed HR-LSTM outperforms state-of-the-art networks at singing voice separation when the DSD100 dataset is used, performs comparably to alternative methods when the MIR-1K dataset is used, and separates the voice and accompaniment components well when the Pansori and NISVS datasets are used. In addition to proposing and validating our network, we also developed and shared our Nepal Idol dataset.
- Open Access
- 6 October 2021
- Published by Springer Nature in Scientific Reports
- Vol. 11 (1) , 1-14
Music video emotion classification using slow–fast audio–video network and unsupervised feature representation
Affective computing has suffered by the precise annotation because the emotions are highly subjective and vague. The music video emotion is complex due to the diverse textual, acoustic, and visual information which can take the form of lyrics, singer voice, sounds from the different instruments, and visual representations. This can be one reason why there has been a limited study in this domain and no standard dataset has been produced before now. In this study, we proposed an unsupervised method for music video emotion analysis using music video contents on the Internet. We also produced a labelled dataset and compared the supervised and unsupervised methods for emotion classification. The music and video information are processed through a multimodal architecture with audio–video information exchange and boosting method. The general 2D and 3D convolution networks compared with the slow–fast network with filter and channel separable convolution in multimodal architecture. Several supervised and unsupervised networks were trained in an end-to-end manner and results were evaluated using various evaluation metrics. The proposed method used a large dataset for unsupervised emotion classification and interpreted the results quantitatively and qualitatively in the music video that had never been applied in the past. The result shows a large increment in classification score using unsupervised features and information sharing techniques on audio and video network. Our best classifier attained 77% accuracy, an f1-score of 0.77, and an area under the curve score of 0.94 with minimum computational cost.
- 6 October 2021
- Published by Springer Nature in Scientific Reports
- Vol. 11 (1) , 1-14
Open AccessMusic video emotion classification using slow–fast audio–video network and unsupervised feature representation
Affective computing has suffered by the precise annotation because the emotions are highly subjective and vague. The music video emotion is complex due to the diverse textual, acoustic, and visual information which can take the form of lyrics, singer voice, sounds from the different instruments, and visual representations. This can be one reason why there has been a limited study in this domain and no standard dataset has been produced before now. In this study, we proposed an unsupervised method for music video emotion analysis using music video contents on the Internet. We also produced a labelled dataset and compared the supervised and unsupervised methods for emotion classification. The music and video information are processed through a multimodal architecture with audio–video information exchange and boosting method. The general 2D and 3D convolution networks compared with the slow–fast network with filter and channel separable convolution in multimodal architecture. Several supervised and unsupervised networks were trained in an end-to-end manner and results were evaluated using various evaluation metrics. The proposed method used a large dataset for unsupervised emotion classification and interpreted the results quantitatively and qualitatively in the music video that had never been applied in the past. The result shows a large increment in classification score using unsupervised features and information sharing techniques on audio and video network. Our best classifier attained 77% accuracy, an f1-score of 0.77, and an area under the curve score of 0.94 with minimum computational cost.
- Top Cited PapersOpen Access
An Instance Segmentation Model for Strawberry Diseases Based on Mask R-CNN
Plant diseases must be identified at the earliest stage for pursuing appropriate treatment procedures and reducing economic and quality losses. There is an indispensable need for low-cost and highly accurate approaches for diagnosing plant diseases. Deep neural networks have achieved state-of-the-art performance in numerous aspects of human life including the agriculture sector. The current state of the literature indicates that there are a limited number of datasets available for autonomous strawberry disease and pest detection that allow fine-grained instance segmentation. To this end, we introduce a novel dataset comprised of 2500 images of seven kinds of strawberry diseases, which allows developing deep learning-based autonomous detection systems to segment strawberry diseases under complex background conditions. As a baseline for future works, we propose a model based on the Mask R-CNN architecture that effectively performs instance segmentation for these seven diseases. We use a ResNet backbone along with following a systematic approach to data augmentation that allows for segmentation of the target diseases under complex environmental conditions, achieving a final mean average precision of 82.43%.
Top Cited PapersOpen AccessAn Instance Segmentation Model for Strawberry Diseases Based on Mask R-CNN
Plant diseases must be identified at the earliest stage for pursuing appropriate treatment procedures and reducing economic and quality losses. There is an indispensable need for low-cost and highly accurate approaches for diagnosing plant diseases. Deep neural networks have achieved state-of-the-art performance in numerous aspects of human life including the agriculture sector. The current state of the literature indicates that there are a limited number of datasets available for autonomous strawberry disease and pest detection that allow fine-grained instance segmentation. To this end, we introduce a novel dataset comprised of 2500 images of seven kinds of strawberry diseases, which allows developing deep learning-based autonomous detection systems to segment strawberry diseases under complex background conditions. As a baseline for future works, we propose a model based on the Mask R-CNN architecture that effectively performs instance segmentation for these seven diseases. We use a ResNet backbone along with following a systematic approach to data augmentation that allows for segmentation of the target diseases under complex environmental conditions, achieving a final mean average precision of 82.43%.
- Top Cited PapersOpen Access
Deep-Learning-Based Multimodal Emotion Classification for Music Videos
Music videos contain a great deal of visual and acoustic information. Each information source within a music video influences the emotions conveyed through the audio and video, suggesting that only a multimodal approach is capable of achieving efficient affective computing. This paper presents an affective computing system that relies on music, video, and facial expression cues, making it useful for emotional analysis. We applied the audio–video information exchange and boosting methods to regularize the training process and reduced the computational costs by using a separable convolution strategy. In sum, our empirical findings are as follows: (1) Multimodal representations efficiently capture all acoustic and visual emotional clues included in each music video, (2) the computational cost of each neural network is significantly reduced by factorizing the standard 2D/3D convolution into separate channels and spatiotemporal interactions, and (3) information-sharing methods incorporated into multimodal representations are helpful in guiding individual information flow and boosting overall performance. We tested our findings across several unimodal and multimodal networks against various evaluation metrics and visual analyzers. Our best classifier attained 74% accuracy, an f1-score of 0.73, and an area under the curve score of 0.926.
Top Cited PapersOpen AccessDeep-Learning-Based Multimodal Emotion Classification for Music Videos
Music videos contain a great deal of visual and acoustic information. Each information source within a music video influences the emotions conveyed through the audio and video, suggesting that only a multimodal approach is capable of achieving efficient affective computing. This paper presents an affective computing system that relies on music, video, and facial expression cues, making it useful for emotional analysis. We applied the audio–video information exchange and boosting methods to regularize the training process and reduced the computational costs by using a separable convolution strategy. In sum, our empirical findings are as follows: (1) Multimodal representations efficiently capture all acoustic and visual emotional clues included in each music video, (2) the computational cost of each neural network is significantly reduced by factorizing the standard 2D/3D convolution into separate channels and spatiotemporal interactions, and (3) information-sharing methods incorporated into multimodal representations are helpful in guiding individual information flow and boosting overall performance. We tested our findings across several unimodal and multimodal networks against various evaluation metrics and visual analyzers. Our best classifier attained 74% accuracy, an f1-score of 0.73, and an area under the curve score of 0.926.
- Open Access
- 12 November 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 206016-206027
Parallel Stacked Hourglass Network for Music Source Separation
Music source separation is one of the old and challenging problems in music information retrieval society. Improvements in deep learning lead to big progress in decomposing music into its constitutive components with a variety of music. This research uses three types of datasets for source separation namely; Korean traditional music Pansori dataset, MIR-1K dataset, and DSD100 dataset. DSD100 dataset includes multiple sound sources and other two datasets has relatively smaller number of sound sources. We synthetically constructed a novel dataset for Pansori music and trained a novel parallel stacked hourglass network (PSHN) with multiple band spectrograms. In comparison with past study, proposed architecture performs the best results in real-world test samples of Pansori music of any length. The network performance was also tested for the public DSD100 and MIR-1K dataset for strength comparison in multiple source data and found comparable quantitative and qualitative outcomes. System performance is evaluated using median value of signal-to-distortion ratio (SDR), source-to-interference ratio (SIR), and source-to-artifacts ratio (SAR) measured in decibels (dB) and visual comparison of prediction results with ground truth. We report better performance in the Pansori dataset and MIR-1K dataset and perform detailed ablation studies based on architecture variation. The proposed system is better applicable for separating the music source with voices and single or fewer musical instruments.
- 12 November 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 206016-206027
Open AccessParallel Stacked Hourglass Network for Music Source Separation
Music source separation is one of the old and challenging problems in music information retrieval society. Improvements in deep learning lead to big progress in decomposing music into its constitutive components with a variety of music. This research uses three types of datasets for source separation namely; Korean traditional music Pansori dataset, MIR-1K dataset, and DSD100 dataset. DSD100 dataset includes multiple sound sources and other two datasets has relatively smaller number of sound sources. We synthetically constructed a novel dataset for Pansori music and trained a novel parallel stacked hourglass network (PSHN) with multiple band spectrograms. In comparison with past study, proposed architecture performs the best results in real-world test samples of Pansori music of any length. The network performance was also tested for the public DSD100 and MIR-1K dataset for strength comparison in multiple source data and found comparable quantitative and qualitative outcomes. System performance is evaluated using median value of signal-to-distortion ratio (SDR), source-to-interference ratio (SIR), and source-to-artifacts ratio (SAR) measured in decibels (dB) and visual comparison of prediction results with ground truth. We report better performance in the Pansori dataset and MIR-1K dataset and perform detailed ablation studies based on architecture variation. The proposed system is better applicable for separating the music source with voices and single or fewer musical instruments.